Merge pull request #73 from salopensource/django1.10

Sal 3.0

.gitignore

@@ -46,8 +46,6 @@ sal_env/
 # node stuff
 node_modules/
 
-server/management
-
 *.db copy
 
 *.db
@@ -60,5 +58,8 @@ sal.dump
 config.codekit
 wip_plugins/
 sal/metadata.xml
+test-pg-db
 
 sal.log
+
+.idea

Dockerfile

@@ -14,48 +14,56 @@ ENV DOCKER_SAL_LANG en_GB
 ENV DOCKER_SAL_DISPLAY_NAME Sal
 ENV DOCKER_SAL_DEBUG false
 
-ADD / $APP_DIR
 RUN apt-get update && \
     apt-get install -y libc-bin && \
     apt-get install -y software-properties-common && \
     apt-get -y update && \
     add-apt-repository -y ppa:nginx/stable && \
     apt-get -y install \
     git \
-    python-setuptools \
     nginx \
+    python-setuptools \
     postgresql \
     postgresql-contrib \
     libpq-dev \
     python-dev \
+    wget \
     supervisor \
     libffi-dev && \
     apt-get clean && \
     rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
+ADD setup/requirements.txt /requirements.txt
 RUN easy_install pip && \
-    pip install -r $APP_DIR/setup/requirements.txt && \
-    pip install psycopg2==2.5.3 && \
-    pip install gunicorn && \
-    pip install setproctitle
-ADD docker/nginx/nginx-env.conf /etc/nginx/main.d/
-ADD docker/nginx/sal.conf /etc/nginx/sites-enabled/sal.conf
-ADD docker/nginx/nginx.conf /etc/nginx/nginx.conf
+    pip install -r /requirements.txt && \
+    pip install psycopg2==2.6.2 && \
+    pip install gunicorn==19.6.0 && \
+    pip install setproctitle && \
+    rm /requirements.txt && \
+    update-rc.d -f postgresql remove && \
+    mkdir -p /home/app && \
+    mkdir -p /home/backup
+ADD / $APP_DIR
 ADD docker/settings.py $APP_DIR/sal/
+ADD docker/supervisord.conf $APP_DIR/supervisord.conf
 ADD docker/settings_import.py $APP_DIR/sal/
 ADD docker/brute_settings.py $APP_DIR/sal/
 ADD docker/wsgi.py $APP_DIR/
 ADD docker/gunicorn_config.py $APP_DIR/
-ADD docker/django/management/ $APP_DIR/sal/management/
 ADD docker/run.sh /run.sh
-ADD docker/supervisord.conf $APP_DIR/supervisord.conf
+ADD docker/nginx/nginx-env.conf /etc/nginx/main.d/
+ADD docker/nginx/sal.conf /etc/nginx/sites-enabled/sal.conf
+ADD docker/nginx/nginx.conf /etc/nginx/nginx.conf
+ADD docker/crontab /etc/cron.d/search-maint
+ADD docker/search_maint.sh /usr/local/bin/search_maint.sh
 
-RUN update-rc.d -f postgresql remove && \
+RUN chmod 755 /run.sh && \
     update-rc.d -f nginx remove && \
     rm -f /etc/nginx/sites-enabled/default && \
-    mkdir -p /home/app && \
-    mkdir -p /home/backup && \
-    ln -s $APP_DIR /home/app/sal
+    ln -s $APP_DIR /home/app/sal && \
+    chmod 644 /etc/cron.d/search-maint &&\
+    chmod 755 /usr/local/bin/search_maint.sh
 
+WORKDIR $APP_DIR
 EXPOSE 8000
 
 CMD ["/run.sh"]

README.md

@@ -4,7 +4,9 @@ Sal is a multi-tenanted reporting dashboard for [Munki](https://github.com/munki
 
 With Sal, you are able to allow access to reports on certain sets of machines to certain people - for example, giving a manager access to the reports on the machines in their department.
 
-![Sal](assets/Sal.png)
+Sal also features powerful search capabilities and application inventory and support for Munki's license tracking.
+
+![Sal](https://github.com/salopensource/sal/raw/master/assets/Sal.png)
 
 ## Getting Started
 
@@ -18,6 +20,8 @@ If you would like a demo of setting up Sal along with some of the features pleas
 
 Sal has full search across machines, Facts and Munki conditions. For more information, see [it's documentation](https://github.com/salopensource/sal/wiki/Search).
 
+![Search](https://github.com/salopensource/sal/raw/master/assets/Built Search.png)
+
 ## Plugins
 
 You can enable, disable and re-order your plugins from the Settings page, under the 'person' menu in the main menu bar. For more information on using and installing your own plugins, visit the [Using Plugins](https://github.com/salopensource/sal/wiki/Installing-and-using-plugins) page.
@@ -28,28 +32,18 @@ After re-ordering and hiding plugins from some screens, you might even want to m
 
 Sal has built in brute force protection. See it's [documentation](https://github.com/salopensource/sal/wiki/Brute-force-protection) for more details or if you're using Docker, see how to turn it on in the [Readme](https://github.com/salopensource/sal/blob/master/docker/README.md).
 
-## LDAP Authentication
+## External Authentication
 
-There is a variant of Sal that supports LDAP authenctication. At this time, only using the [sal-ldap Docker image](https://hub.docker.com/r/macadmins/sal-ldap) is supported.
+There are variants of Sal that support both [SAML](https://hub.docker.com/r/macadmins/sal-saml/) and [LDAP](https://hub.docker.com/r/macadmins/sal-ldap/) authentication.
 
 ## Having problems?
 
-You should check out the [troubleshooting](https://github.com/salopensource/sal/wiki/Troubleshooting) page.
+You should check out the [troubleshooting](https://github.com/salopensource/sal/wiki/Troubleshooting) page, consider getting in touch via the [Google group](http://groups.google.com/group/sal-discuss), or heading over the the #slack channel on the [macadmins.org Slack](http://macadmins.org).
 
 ## API
 
 There is a simple API available for Sal. Documentation can be found at [docs/Api.md](https://github.com/salopensource/sal/wiki/API)
 
-## Discussion
-
-Discussion on the use and development of Sal is [available on the Google Group](http://groups.google.com/group/sal-discuss).
-
 ## Why Sal?
 
 It's the Internet's fault! I asked on Twitter what I should call it, and Peter Bukowinski ([@pmbuko](https://twitter.com/pmbuko)) [suggested the name](https://twitter.com/pmbuko/status/377155523726290944), based on a Monkey puppet called [Sal Minella](http://muppet.wikia.com/wiki/Sal_Minella).
-
-## Thank yous
-
-First off, thanks to Greg Neagle and the rest of the Munki Project. Munki is an amazing product, and managing OS X at any scale would be miserable without it.
-
-Thanks to Puppet Labs for basically giving away the crown jewels for nothing.

api/serializers.py

@@ -38,11 +38,16 @@ class Meta:
         model = PendingUpdate
         exclude = ('machine',)
 
+class FullMachineSerializer(serializers.ModelSerializer):
+    class Meta:
+        model = Machine
+
 class MachineSerializer(serializers.ModelSerializer):
-    facts = FactSerializer(many=True, required=False)
-    conditions = ConditionSerializer(many=True, required=False)
-    pending_apple_updates = PendingAppleUpdateSerializer(many=True, required=False)
-    pending_updates = PendingUpdateSerializer(many=True, required=False)
+    # facts = FactSerializer(many=True, required=False)
+    # conditions = ConditionSerializer(many=True, required=False)
+    # pending_apple_updates = PendingAppleUpdateSerializer(many=True, required=False)
+    # pending_updates = PendingUpdateSerializer(many=True, required=False)
 
     class Meta:
         model = Machine
+        exclude = ('report','install_log','install_log_hash')

api/urls.py

@@ -4,9 +4,12 @@
 urlpatterns = [
     url(r'^machines/(?P<serial>.+)/inventory/$', views.machine_inventory),
     url(r'^machines/(?P<serial>.+)/$', views.machine_detail),
+    url(r'^machines/(?P<serial>.+)/full/$', views.machine_full_detail),
     url(r'^machines/$', views.machine_list),
     url(r'^facts/(?P<serial>.+)/$', views.facts),
     url(r'^conditions/(?P<serial>.+)/$', views.conditions),
+    url(r'^pending_apple_updates/(?P<serial>.+)/$', views.pending_apple_updates),
+    url(r'^pending_updates/(?P<serial>.+)/$', views.pending_updates),
     url(r'^business_units/(?P<pk>.+)/$', views.business_unit),
     url(r'^business_units/$', views.business_unit_list),
     url(r'^machine_groups/(?P<pk>.+)/$', views.machine_group),

api/views.py

@@ -53,7 +53,35 @@ def machine_detail(request, serial):
         machine.delete()
         return HttpResponse(status=204)
 
+@csrf_exempt
 @validate_api_key
+def machine_full_detail(request, serial):
+    """
+    Retrieve, update or delete a machine.
+    """
+    try:
+        machine = Machine.objects.get(serial=serial)
+    except Machine.DoesNotExist:
+        return HttpResponse(status=404)
+
+    if request.method == 'GET':
+        serializer = FullMachineSerializer(machine)
+        return JSONResponse(serializer.data)
+
+    elif request.method == 'PUT':
+        data = JSONParser().parse(request)
+        serializer = FullMachineSerializer(machine, data=data)
+        if serializer.is_valid():
+            serializer.save()
+            return JSONResponse(serializer.data)
+        return JSONResponse(serializer.errors, status=400)
+
+    elif request.method == 'DELETE':
+        machine.delete()
+        return HttpResponse(status=204)
+
+@validate_api_key
+@csrf_exempt
 def machine_inventory(request, serial):
     """
     Retrieve machine inventory.
@@ -66,6 +94,38 @@ def machine_inventory(request, serial):
     serializer = InventoryItemSerializer(machine.inventoryitem_set.all(), many=True)
     return JSONResponse(serializer.data)
 
+@validate_api_key
+@csrf_exempt
+def pending_apple_updates(request, serial):
+    """
+    Retrieves pending apple updates for a given machine.
+    """
+    try:
+        machine = Machine.objects.get(serial=serial)
+    except Machine.DoesNotExist:
+        return HttpResponse(status=404)
+
+    updates = PendingAppleUpdate.objects.filter(machine=machine)
+    if request.method == 'GET':
+        serializer = PendingAppleUpdateSerializer(updates, many=True)
+        return JSONResponse(serializer.data)
+
+@validate_api_key
+@csrf_exempt
+def pending_updates(request, serial):
+    """
+    Retrieves pending updates for a given machine.
+    """
+    try:
+        machine = Machine.objects.get(serial=serial)
+    except Machine.DoesNotExist:
+        return HttpResponse(status=404)
+
+    updates = PendingUpdate.objects.filter(machine=machine)
+    if request.method == 'GET':
+        serializer = PendingpdateSerializer(updates, many=True)
+        return JSONResponse(serializer.data)
+
 @validate_api_key
 @csrf_exempt
 def facts(request, serial):

catalog/views.py

@@ -1,7 +1,7 @@
 from django.http import HttpResponse, HttpRequest, HttpResponseRedirect
 from django.template import RequestContext, Template, Context
-from django.shortcuts import render_to_response
-from django.core.context_processors import csrf
+from django.shortcuts import render
+from django.template.context_processors import csrf
 from django.views.decorators.csrf import csrf_exempt
 from django.core.urlresolvers import reverse
 from django.http import Http404
@@ -12,32 +12,36 @@
 from django.db.models import Q
 from django.db.models import Count
 from server import utils
-from django.shortcuts import render_to_response, get_object_or_404, redirect
+from django.shortcuts import get_object_or_404, redirect
 import plistlib
 import hashlib
+import base64
+import bz2
 
 from models import *
 from server.models import *
+from sal.decorators import *
 
 # Create your views here.
 
 def decode_to_string(base64bz2data):
     '''Decodes an inventory submission, which is a plist-encoded
     list, compressed via bz2 and base64 encoded.'''
     try:
-        bz2data = base64.b64decode(base64bz2data)
-        return bz2.decompress(bz2data)
+        return bz2.decompress(base64.b64decode(base64bz2data))
     except Exception:
         return ''
 
 @csrf_exempt
+@key_auth_required
 def submit_catalog(request):
     if request.method != 'POST':
         raise Http404
 
     submission = request.POST
     key = submission.get('key')
     name = submission.get('name')
+    sha = submission.get('sha256hash')
     machine_group = None
     if key:
         try:
@@ -46,10 +50,11 @@ def submit_catalog(request):
             raise Http404
 
         compressed_catalog = submission.get('base64bz2catalog')
+        # print compressed_catalog
         if compressed_catalog:
-            compressed_catalog = compressed_catalog.replace(" ", "+")
+            # compressed_catalog = compressed_catalog.replace(" ", "+")
             catalog_str = decode_to_string(compressed_catalog)
-
+            print catalog_str
             try:
                 catalog_plist = plistlib.readPlistFromString(catalog_str)
             except Exception:
@@ -59,12 +64,13 @@ def submit_catalog(request):
                     catalog = Catalog.objects.get(name=name, machine_group=machine_group)
                 except Catalog.DoesNotExist:
                     catalog = Catalog(name=name, machine_group=machine_group)
-                catalog.sha256hash = \
-                    hashlib.sha256(catalog_str).hexdigest()
+                catalog.sha256hash = sha
                 catalog.content = catalog_str
                 catalog.save()
     return HttpResponse("Catalogs submitted.")
+
 @csrf_exempt
+@key_auth_required
 def catalog_hash(request):
     if request.method != 'POST':
         print 'method not post'
@@ -80,19 +86,19 @@ def catalog_hash(request):
         except MachineGroup.DoesNotExist:
             raise Http404
     if catalogs:
-        catalogs = catalogs.replace(" ", "+")
         catalogs = decode_to_string(catalogs)
         try:
             catalogs_plist = plistlib.readPlistFromString(catalogs)
         except Exception:
             catalogs_plist = None
-        for item in catalogs_plist:
-            name = item['name']
-            sha256hash = item['sha256hash']
-            try:
-                found_catalog = Catalog.objects.get(name=name, machine_group=machine_group)
-                output.append({'name': name, 'sha256hash':found_catalog.sha256hash})
-            except Catalog.DoesNotExist:
-                output.append({'name': name, 'sha256hash': ''})
+        if catalogs_plist:
+            for item in catalogs_plist:
+                name = item['name']
+                sha256hash = item['sha256hash']
+                try:
+                    found_catalog = Catalog.objects.get(name=name, machine_group=machine_group)
+                    output.append({'name': name, 'sha256hash':found_catalog.sha256hash})
+                except Catalog.DoesNotExist:
+                    output.append({'name': name, 'sha256hash': 'NOT FOUND'})
 
     return HttpResponse(plistlib.writePlistToString(output))

datatableview/__init__.py

@@ -0,0 +1,14 @@
+# -*- coding: utf-8 -*-
+
+from .datatables import Datatable, ValuesDatatable, LegacyDatatable
+from .columns import (Column, TextColumn, DateColumn, DateTimeColumn, BooleanColumn, IntegerColumn,
+                      FloatColumn, DisplayColumn, CompoundColumn)
+from .exceptions import SkipRecord
+
+__name__ = 'datatableview'
+__author__ = 'Tim Valenta'
+__version_info__ = (0, 9, 0)
+__version__ = '.'.join(map(str, __version_info__))
+__date__ = '2013/11/14 2:00:00 PM'
+__credits__ = ['Tim Valenta', 'Steven Klass']
+__license__ = 'See the file LICENSE.txt for licensing information.'