Merge pull request #49 from ver228/development

Development

.gitignore

@@ -1,9 +1,7 @@
 
-create_binaries/*.app
-create_binaries/build/*
-create_binaries/dist/*
-tests/data/*
-debug_tests/*.txt
+_old/create_binaries/*.app
+_old/create_binaries/build/*
+_old/create_binaries/dist/*
 
 *.hdf5
 *.pyc
@@ -32,3 +30,5 @@ tierpsy/analysis/ske_create/segWormPython/cython_files/cleanWorm_cython.c
 tierpsy/analysis/stage_aligment/get_mask_diff_var.c
 tierpsy/analysis/ske_create/segWormPython/cython_files/curvspace.c
 tierpsy/analysis/ske_create/segWormPython/cython_files/circCurvature.c
+
+tierpsy/tests/data/*

README.md

@@ -1,10 +1,14 @@
 # Tierpsy Tracker
-### *Double-click executables are available for Windows (7 or latest) and OSX (Yosemite or latest) can be found in the [releases page](https://github.com/ver228/tierpsy-tracker/releases).*
-
 <img src="https://user-images.githubusercontent.com/8364368/41229372-a9e8fcfa-6d73-11e8-877c-de306be55526.gif" width="800">
 
 Tierpsy Tracker is a multi-animal tracker developed in the [MRC-LMS](http://lms.mrc.ac.uk/) [Behavioural Genomics Group](http://behave.csc.mrc.ac.uk/) at Imperial College London. The project combines the throughput of multiworm tracking with the resolution of single worm tracking, which means you can extract detailed phenotypic fingerprints from more animals.  A large field of view also makes it possible to analyse collective behaviours that depend on animal interactions.  The code is open source and extensible and we have ongoing projects using the tracker to analyse fish, *Drosophila* larvae, and fluorescently labelled worms.
 
+The easiest way to install Tierpsy Tracker is to use the conda-forge as:
+```
+conda install tierpsy -c conda-forge
+```
+For more details see the [installation](docs/INSTALLATION.md) instructions section.
+
 ## [Installation Instructions](docs/INSTALLATION.md)
 ## [How to Use](docs/HOWTO.md)
 ## [Algorithm Explanation](docs/EXPLANATION.md)

_old/scripts/tierpsy_gui.py

@@ -0,0 +1,5 @@
+from tierpsy.gui.SelectApp import tierpsy_gui
+import sys
+
+if __name__ == '__main__':
+    sys.exit(tierpsy_gui())

_old/scripts/tierpsy_gui_simple.py

@@ -0,0 +1,6 @@
+import sys
+
+from tierpsy.gui.HDF5VideoPlayer import tierpsy_gui_simple
+
+if __name__ == '__main__':
+    sys.exit(tierpsy_gui_simple())

_old/scripts/tierpsy_process.py

@@ -0,0 +1,10 @@
+# -*- coding: utf-8 -*-
+"""
+Created on Tue Jun  9 15:12:48 2015
+
+@author: ajaver
+"""
+from tierpsy.processing.processMultipleFilesFun import tierpsy_process
+
+if __name__ == '__main__':
+	tierpsy_process()

_old/scripts/tierpsy_tests.py

@@ -0,0 +1,10 @@
+# -*- coding: utf-8 -*-
+"""
+Created on Tue Jun  9 15:12:48 2015
+
+@author: ajaver
+"""
+from tierpsy.tests.run_tests import tierpsy_tests
+
+if __name__ == '__main__':
+	tierpsy_tests()

docs/HOWTO.md

@@ -18,22 +18,9 @@ The processing times for in MacBook Pro (15-inch, 2017) were 04:31 minutes for t
 
 ## Getting Started
 
-If you are using any of the [binary executables](https://github.com/ver228/tierpsy-tracker/releases) you only have to double click the file and the program should start after a few seconds.
-
-If you [installed tierpsy tracker from the source](https://github.com/ver228/tierpsy-tracker/blob/master/docs/INSTALLATION.md) in OSX or Windows there should be a clickable executable named `TierpsyTracker` on the Desktop. If the executable is missing you can re-create it by running:
-
-```bash
-#(OSX/Linux)
-installation/installation_script.sh --link_desktop 
-
-#(Windows)
-installation/installation_script.bat --link_desktop
-```  
-
-Alternatively open a terminal, go to the directory where Tierpsy Tracker is installed and type: 
-
+Follow the installation [instuctions](INSTALLATION.md) and open a terminal or an Anaconda prompt (Windows) and type:
 ```bash
-python3 cmd_scripts/TierpsyTrackerConsole.py
+tierpsy_gui
 ```
 
 The main widget should look like the one below:

docs/INSTALLATION.md

@@ -1,63 +1,52 @@
-*The instructions below are to install Tierpsy Tracker from the source code. I would recommend to do this only if you are using Linux or want to run the development version, otherwise use the double-click executables available for Windows (7 or latest) and OSX (Yosemite or latest) in the [releases page](https://github.com/ver228/tierpsy-tracker/releases).*
+# Installation Instructions
 
-# System Requirements 
-- Python 3.6 (I would recommend to use [miniconda](https://conda.io/miniconda.html)).
-- Optional [ffmpeg](https://ffmpeg.org/download.html): ffprobe must be accessible from the command line to calculate the video timestamps.
-- [C compiler compatible with cython](http://cython.readthedocs.io/en/latest/src/quickstart/install.html). In Windows, you can use [Visual C++ 2015 Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/). In OSX, if you install [homebrew](https://brew.sh/) it will setup the C compiler without the need to download XCode from the appstore. 
-- [Git](https://git-scm.com/). [Here](https://gist.github.com/derhuerst/1b15ff4652a867391f03) are some instructions to install it.
+## Installation from precompiled packages
+- Download python 3.6>= using [anaconda](https://www.anaconda.com/download/) or [miniconda](https://conda.io/miniconda.html) if you prefer a lighter installation.
+- Open a Terminal in OSX or Linux. In Windows you need to open the Anaconda Prompt.
+- [Optional] I would recommend to create and activate an [enviroment](https://conda.io/docs/user-guide/tasks/manage-environments.html) as:
 
-# Installation
-
-1. Install dependencies:
-
-Using pip:
 ```bash
-pip install numpy matplotlib tables pandas pyqt5 h5py scipy scikit-learn scikit-image seaborn xlrd cython statsmodels numba keras==2.1.5 tensorflow opencv-python-headless
-```
+#Windows
+conda create -n tierpsy 
+conda activate tierpsy 
 
-Using anaconda:
-```bash
-conda install --channel conda-forge numpy matplotlib pytables pandas gitpython pyqt h5py \
-scipy scikit-learn scikit-image seaborn xlrd cython statsmodels numba keras=2.1.5 opencv tensorflow 
+#OSX or Linux
+source create -n tierpsy 
+source activate tierpsy 
 ```
-
-2. Clone this repository either using the [Github Desktop](https://desktop.github.com/) or from the command line as:
-
+- Finally, donwload the package from conda-forge
 ```bash
-git clone https://github.com/ver228/tierpsy-tracker
+conda install tierpsy -c conda-forge
 ```
+- After you can start tierpsy tracker by typing:
+```bash
+tierpsy_gui
+``` 
+On OSX the first time tierpsy is intialized it will create a file in the Desktop called tierpsy_gui.command. By double-cliking on this file tierpsy can be started without having to open a terminal.
 
+Do not forget to activate the enviroment every time you start a new session.
 
-3. Install the rest of the modules. Firts move to the tierpsy-tracker root folder (the folder with the cloned repository) as:
-```bash
-cd tierpsy-tracker
-```
-Then install the rest of the modules using the following command according to your operative system.
+Note: it seems that there might be some problems with the opencv version available through `conda`. If you have problems reading video files or encounter error related with `import cv2`, then you can try to install opencv using pip as:
 ```bash
-bash installation/installation_script.sh #OSX or Linux
-
-installation/installation_script.bat #Windows
+pip install opencv-python-headless
 ```
 
-## Possible Issues
-
-
-- The most common problem in the installation is OpenCV (error in import cv2). Try a fresh miniconda installation (or a fresh enviroment) and make sure your are using the [conda-forge](https://conda-forge.org/) packages. It this does not work I am afraid you would have to solve the problem by yourself (Google is your friend).
-
-- You do not need to install manually the [Open Worm Analysis Toolbox](https://github.com/openworm/open-worm-analysis-toolbox). However if you do (and I do not recommend it), be aware that there is a bug with the pip installer: it is missing some dependencies and it will create a corrupt [.egg](https://stackoverflow.com/questions/2051192/what-is-a-python-egg) in your packages folder. Manually delete the .egg (use the error traceback to find the its location) and re-run `installation_script.sh`. The script will download the Open Worm Analysis Toolbox repository and install it using `python setup.py develop`. 
-
-
-# Test Data
-On Mac OSX or Linux, some test data can be downloaded by running 
+## Installation from source
+- Download python 3.6>= using [anaconda](https://www.anaconda.com/download/)) or [miniconda](https://conda.io/miniconda.html)).
+- Install [git](https://git-scm.com/). [Here](https://gist.github.com/derhuerst/1b15ff4652a867391f03) are some instructions to install it.
+- Install a [C compiler compatible with cython](http://cython.readthedocs.io/en/latest/src/quickstart/install.html). In Windows, you can use [Visual C++ 2015 Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/). In OSX, if you install [homebrew](https://brew.sh/) it will setup the C compiler without the need to download XCode from the appstore. 
 
+- Open a Terminal or Anaconda prompt and type:
 ```bash
-installation/installation_script.sh --download_examples
+git clone https://github.com/ver228/tierpsy-tracker
+cd tracker
+source create -n tierpsy #[optional]
+conda install --file requirements.txt
+pip install -e .
 ```
 
-The tests can also be manually downloaded using [this link](https://imperiallondon-my.sharepoint.com/personal/ajaver_ic_ac_uk/_layouts/15/guestaccess.aspx?guestaccesstoken=ldZ18fLY%2bzlu7XuO9mbKVdyiKoH4naiesqiLXWU4vGQ%3d&docid=0cec4e52f4ccf4d5b8bb3a737020fc12f&rev=1). Uncompress the data and save it in the main repository folder `tests/data` .
-
-You can then run the tests by running: 
+# Batch processing from the command line.
+The script `TierpsyTrackerConsole.py` was deprecated in favor of using the command `tierpsy_process`. Type `tierpsy_process -h` for help.
 
-```bash
-python tests/run_tests.py
-```
+# Tests
+After installing you can run the testing scripts using the command `tierpsy_tests` on the terminal. Type `tierpsy_tests -h` for help. I would recommed to run one test at the time since there is not currently a way to summarize the results of all the tests.

recipe/bld.bat

@@ -0,0 +1,2 @@
+"%PYTHON%" setup.py install --single-version-externally-managed --record=record.txt
+if errorlevel 1 exit 1

recipe/build.sh

@@ -0,0 +1,2 @@
+#!/bin/bash
+$PYTHON setup.py install --single-version-externally-managed --record=record.txt  # Python command to install the script.

recipe/meta.yaml

@@ -0,0 +1,56 @@
+package:
+  name: tierpsy
+  version: "1.5.1-beta"
+
+source:
+  path: ..
+  #git_rev: "v1.5.0"
+  #git_url: "https://github.com/ver228/tierpsy-tracker"
+
+build:
+  osx_is_app: True
+
+requirements:
+  build:
+    - python
+    - numpy >=1.14
+    - setuptools
+    - cython
+
+  run:
+    - python
+    - keras=2.1.5 
+    - opencv 
+    - tensorflow 
+    - numpy >=1.14
+    - matplotlib 
+    - pytables 
+    - pandas 
+    - pyqt
+    - scipy 
+    - scikit-learn 
+    - scikit-image
+    - statsmodels 
+    - numba
+    - tqdm
+    - requests
+
+app:
+  entry: tierpsy_gui
+  summary: Tierpsy Tracker Multi-Worm Tracker
+  type: desk
+
+test:
+  imports:
+    - tierpsy
+    - tierpsy.features.tierpsy_features
+    - tierpsy.features.open_worm_analysis_toolbox
+  commands:
+    - tierpsy_process -h
+    - tierpsy_tests -h
+
+about:
+  home: https://github.com/ver228/tierpsy-tracker
+  license: MIT
+  license_file: LICENSE
+  summary: Tierpsy Tracker Multi-Worm Tracker

requirements.txt

@@ -0,0 +1,16 @@
+cython
+keras=2.1.5 
+opencv 
+tensorflow 
+numpy >=1.14
+matplotlib 
+pytables 
+pandas 
+pyqt
+scipy 
+scikit-learn 
+scikit-image
+statsmodels 
+numba
+tqdm
+requests

setup.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 
-from setuptools import setup
+from setuptools import setup, find_packages
 from distutils.extension import Extension
 from Cython.Build import cythonize
 from Cython.Distutils import build_ext
@@ -43,18 +43,36 @@ def _get_mod_path(name):
   			   for name, files in ext_files.items()]
   return ext_modules
 
+PKG_DATA = [
+  'extras/*',
+  'extras/param_files/*',
+  'features/tierpsy_features/extras/*',
+  'features/open_worm_analysis_toolbox/features/master_eigen_worms_N2.mat',
+  'features/open_worm_analysis_toolbox/features/feature_metadata/features_list.csv'
+]
 
 
 #install setup
-setup(name=MODULE_NAME,
-   version=VERSION,
-   description=DESCRIPTION,
-   author=AUTHOR,
-   author_email=AUTHOR_EMAIL,
-   url=URL,
-   packages=['tierpsy'],
-   cmdclass={'build_ext': build_ext},
-   ext_modules=_get_ext_modules(),
-   include_dirs=[numpy.get_include()],
-   include_package_data=True
+setup(name = MODULE_NAME,
+   version = VERSION,
+   description = DESCRIPTION,
+   author = AUTHOR,
+   author_email = AUTHOR_EMAIL,
+   url = URL,
+   packages = find_packages(),
+   cmdclass = {'build_ext': build_ext},
+   ext_modules = _get_ext_modules(),
+   include_dirs = [numpy.get_include()],
+   package_data = {'tierpsy': PKG_DATA},
+   entry_points= {
+          'gui_scripts': [
+              'tierpsy_gui = tierpsy.gui.SelectApp:tierpsy_gui',
+              'tierpsy_gui_simple = tierpsy.gui.HDF5VideoPlayer:tierpsy_gui_simple'
+          ],
+          'console_scripts': [
+              'tierpsy_process = tierpsy.processing.processMultipleFilesFun:tierpsy_process',
+              'tierpsy_tests = tierpsy.tests.run_tests:tierpsy_tests'
+          ]
+      }
    )
+

tierpsy/__init__.py

@@ -9,6 +9,12 @@
 import warnings
 from .version import __version__
 
+#this is an anaconda program that should not cause problems.  https://github.com/ContinuumIO/anaconda-issues/issues/6678
+warnings.filterwarnings('ignore', message='numpy.dtype size changed, may indicate binary incompatibility.*')
+def warning_on_one_line(message, category, filename, lineno, file=None, line=''):
+    return '{}:{}: {}:{}\n'.format(filename, lineno, category.__name__, message)
+warnings.formatwarning = warning_on_one_line
+
 #I want to be sure tierpsy loads tensorflow flow backend
 os.environ['KERAS_BACKEND']='tensorflow'