v5: Refactor

Start From Scratch

A complete re-write of Appia from scratch. Appia started as several different scripts I wrote for convenience with no web UI, and I slowly added more and more. Eventually, the web UI became the most powerful and useful part of the entire thing, but it was held back by my old ideas about what the package was. So I re-wrote everything from scratch.

Processing is now faster and much easier to understand, docker installation actually works and is much more secure, and the web UI includes some powerful new features such as re-normalization of HPLC on a focused area.

New

• You can renormalize traces from the web interface.
  • Highlight the region of interest in the top, unnormalized trace and click the new "Renormalize HPLC" button.
  • After a few moments, the bottom, normalized trace will be renormalized setting 1 to the maximum over that region. 0 is still set globally, as rarely do you want to strictly normalize over a region.
• Normalized and view regions can be shared.
  • The new version of Appia makes extensive use of the URL and query strings, meaning you can now send not only a combined trace but also a zoomed in region of that trace, or a trace normalized over a particular region.
• The Appia server is now entirely dockerized, meaning it is completely portable.

Major Changes

• Processing has been streamlined.
  • Instead of using different commands for HPLC and FPLC data, everything is combined under appia.py process.
  • The options have been combined and slightly changed. The main difference is that instead of providing a directory of HPLC traces, you instead provide a UNIX-style glob, i.e. (traces/.arw) or several, i.e. traces/.arw traces/.asc traces/.csv) to process several types of data or data locations at once.
  • The combination of processing now means names and data movement are slightly different. Appia will prefer to pull an experiment name from a Waters Sample Set name, then from a Shimadzu run name, then from the file name of your SEC .csv, whichever is present first. If you provide your own experiment name using --id, then obviously that name will be used.
• When your data are processed, CSV files are saved at full resolution but downsampled to approximately 1000 points per sample per channel automatically before uploading to the database. If you wish to prevent this from happening, you must use the --reduce option and set it to a higher number, but I would encourage you not to. 1000 points per sample per channel still gives you quite a few points per second, and makes the web interface much faster.
• By default, Appia no longer uploads new experiments to the database. You must use the -d argument and tell Appia where your config file is.
  The format of config files has changed from a python dict to a JSON file.

Removed Functionality

• Appia no longer processes 3D HPLC data (no plans to bring this back)

Automated Processing

Using powershell's FileSystemWatcher, data processing is automated.

Docker installation

You can finally install using docker. No need to set up your networking or couchdb. The defaults are in the example templates, but you can set up whichever ports you like.

HPLC and FPLC combined web interface

HPLC and FPLC processing can both create and update experiments, which are uploaded and displayed using plotly dash. Experiment data structure went through bottom-up re-write from v2.0, should be more flexible in the future. Web interface was also completely re-written.