README

Last update 21 MAY. 2015

Installation
=================================================================
Complete Guide: http://trac.myslice.info/wiki/Manifold/Install

Recommended OS
===============
Debian GNU/Linux 7.5 (wheezy) x64

PYTHON DEPENDENCIES
=====================
sudo apt-get install python-pip or sudo easy_install pip==1.4.1
sudo apt-get install python-dev (for paramiko and pyOpenSSL)
sudo apt-get install libffi-dev (for pyOpenSSL)

$ pip install django=="1.5.2"
$ apt-get install python-django-south
$ pip install requests
$ pip install djangorestframework
$ pip install django-celery
$ pip install geopy
$ pip install paramiko
$ pip install pyparsing
$ pip install python-dateutil
$ pip instal pyOpenSSL
$ pip install xmltodict

MYSLICE
=======
git clone ssh://yourlogin@git.onelab.eu/git/myslice.git
cd myslice
git checkout onelab

edit/create myslice/myslice.ini and enter the details of your manifold backend

$ ./manage.py syncdb
$ ./manage.py migrate
$ ./manage.py collectstatic

Run the development server
$ ./devel/server-loop.sh

MANIFOLD
==========
git clone git://git.onelab.eu/manifold.git
cd manifold
git checkout devel
make && make install

SFA
===
$ git clone -b geni-v3 git://git.onelab.eu/sfa.git
$ cd sfa 
$ git checkout geni-v3

$ make version
$ python ./setup.py install

=====================================================================

## Whenever doing a git pull the following operations are recommended:

$ make static # will refresh static/ from all the pieces in the project
$ make templates # same, for templates
$ make redo-static # clears up (rm -rf) static/ first, then make static
$ make redo-templates # ditto for templates
$ make redo == make redo-static redo-templates


* run a local server:
$ ./manage.py runserver 0.0.0.0:8000
-- or -- my advice:
$ devel/server-loop.sh
when you just need to hit ^C yourself when your static files need to be refreshed - see below

* use it from your browser 
(See more notes on using the development server below)

* install dependencies 
$ pip install -r path/to/requirements/file.txt
Note. not quite sure what this is about, I could not spot this file..

=====APACHE Config=====
if are running apache please do:

$ chown www-data:www-data /var/myslice-f4f

otherwise you may get the following error:
"DatabaseError: unable to open database file"

==================== Status

*** Authentication ***

Should be mostly fine
Not quite sure if/how the user gets proper notifications when
. his session has expired (i.e. his frontend is not logged into the backend any longer)
. his credentials have expired (i.e. the uploaded credentials, e.g. SFA delegated cred)
  expired and she needs to run e.g. sfi myslice again

Hard-coded accounts (from a very early stage) are gone


*** Packaging ***

I've done a very rough attempt at packaging for rpm.
The logic seems about right but needs more work, in particular in terms of installing myslice.conf
in the httpd conf.d directory. 
It seems like our app won't work on f14 as is because Django is only 1.3.1 on f14
Plan is to target f18 but I lack a test machine.
Also of course I'll try to tackle debian/ubunti at some point.

There also is a working packaging for debian(s) and ubuntu(s) that we use 
on an almost daily basis to upgrade manifold.pl.sophia.inria.fr


==================== 

Third party tools shipped:

* jquery
* datatables
* spin
* bootstrap

Others are added as we build the system when they become needed
Look in third-party/ for a more detailed list

As a rule of thumb, please try to keep in mind that these will need to
be upgraded over time I've tried to keep track of the version I picked
and to have an easy upgrade path (depending on the way the original
package is published)

==================== Contents: 1st level subdirs

========== code from git

* myslice/
  this is the django 'project', where to look for
  . settings.py
  . urls.py

* manifold/
  the code for dealing with queries, sending them to the backend, and offering the /manifold/proxy/ URL

* unfold/
  the code for building / rendering plugins 

* plugins/
  the actual code for plugins

* auth/ 
  a django 'app' that deals with authentication; see especially
  auth.backend.MyCustomBackend 
  for how to use a separate authentication system, 
  as well as settings.py for how to enable it

* ui/
  provides building blocks for the UI, especially layouts (1 or 2 columns) as
  well as the topmenu widget
  + some global static files (css, js, images..)

* portal/
  this is where the first implementation of myslice, with complete
  user-management including registration, is taking place

* trash/
  rough/preliminary scaffolding views are in here
  as the name suggests this is temporary

* insert_above: 
  a third-party django app for adding on-the-fly mentions to css or js files that need to go in the header

* third-party/
   * third party javascript and css stuff (bootstrapfs, jquery, this kind of things)
     see more about that below too

* devel:
  no code in there, only various notes and other scripts useful for developers

========== automatically generated 

* static/: (generated by collectstatic, see above, do not source-control)
  $ manage.py [ --noinput ] collectstatic

* templates/

* myslice.sqlite3
  this is where django stores its own stuff, as per settings.py

==================== conventions for templates & static files
==================== and NOTES on using the development server

. first off, running manage.py runserver is provided by django as a development convenience but
  SHOULD NOT be used in production

. second, when you do use it for developement purposes, please be aware that:

.. the recommended layout for the various files and pieces (py, html, js and css) with django is e.g.
      plugins/quickfilter/___init__.py, 
      plugins/quickfilter/templates/quickfilter.html,
      plugins/quickfilter/static/js/quickfilter.js 
      plugins/quickfilter/static/css/quickfilter.css
      plugins/quickfilter/static/img/some-image.png

.. the files actually used by the development server are the ones located in
 static/
 templates/

you can and should use the following make targets to refresh the
contents of these directories when running a developement server
$ make static	  	      to refresh static/
$ make redo-static	      to clean up static/ and then refresh its contents
$ make templates	      to refresh templates/
$ make redo-templates	      to clean up templates/ and then refresh its contents
$ make redo		      equivalent to make redo-static redo-templates

.. as far as possible, please make sure to use third-party to store
any javascript tool or utility that your plugin depends upon

also we have the convention that all material in third-party should be
tagged with a version number, with a symlink pointing to the version
being used, like this

~/git/myslice/third-party $ ls -ld spin*
lrwxr-xr-x  1 parmentelat  staff   10 Sep  6 17:55 spin -> spin-1.3.0
drwxr-xr-x  7 parmentelat  staff  238 Sep  6 17:55 spin-1.2.8
drwxr-xr-x  7 parmentelat  staff  238 Sep  6 17:55 spin-1.3.0

finally, as far as possible we keep track of the urls used to pull
stuff in the first place so that upgrades are easier

. third, be careful when importing third party material, to stay away from demo-oriented material

e.g. at some point we were using demo_page.css and demo_table.css from the datatables demo and sample pages
unfortunately these are not tailored for production use as they are setting styles on a very wide scope 
that breaks a lot of stuff, so please refrain from using these altogether


======== update django database to reflect changes in existing models without any migration system (e.g., south) =========

# older version
$python manage.py reset <your_app>

#Django 1.5.1 or later
$python manage.py flush

This will update the database tables for your app, but will completely destroy any data that existed in those tables. 
If the changes you made to your app model do not break your old schema (for instance, you added a new, optional field) 
you can simply dump the data before and reload it afterwards, like so:

$python manage.py syncdb
$python manage.py dumpdata <your_app> > temp_data.json
$python manage.py flush
$python manage.py loaddata temp_data.json

If your changes break your old schema this won't work - in which case tools like south or django evolution are great.


Add a new model to the DB

$python manage.py schemamigration <your_app> --auto
$python manage.py migrate

======== update django database to reflect changes in existing models with migration system (e.g., south) =========

As south is already installed , you just have to do:

# ./manage.py schemamigration portal --initial
# sqlite3 /var/unfold/unfold.sqlite3
	sqlite> DROP TABLE "portal_institution";
	sqlite> DROP TABLE "portal_pendinguser";
	sqlite> DROP TABLE "portal_pendingauthority";
	sqlite> DROP TABLE "portal_pendingslice";
	sqlite> .quit
# ./manage.py migrate portal