Export data to files with Suitcase.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Export data to files with Suitcase\n",
    "\n",
    "In this notebook you will:\n",
    "\n",
    "* Export saved data to CSV and TIFF.\n",
    "* See a preview of how exporting data live during a scan works.\n",
    "\n",
    "## Configuration\n",
    "Below, we will connect to EPICS IOC(s) controlling simulated hardware in lieu of actual motors, detectors. The IOCs should already be running in the background. Run this command to verify that they are running: it should produce output with RUNNING on each line. In the event of a problem, edit this command to replace `status` with `restart all` and run again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!supervisorctl -c supervisor/supervisord.conf status"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%run scripts/beamline_configuration.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scalar data to CSV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "RE(scan([ph], motor_ph, -1, 1, 5), sample_comp='Ni', barcode='12345')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import suitcase.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "suitcase.csv.export(db[-1].documents(fill=True), 'my_exported_files/')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "suitcase.csv.export(db[-1].documents(fill=True), 'my_exported_files/', file_prefix='{start[sample_comp]}-{start[barcode]}-')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "suitcase.csv.export(db[-1].documents(fill=True), 'my_exported_files/', file_prefix='{start[sample_comp]}/{start[barcode]}-')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Image data to TIFF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "RE(scan([spot], mtr_spotx, -1, 1, 5), sample_comp='Ni', barcode='12345')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TIFF stack, a 3D block of images in one file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import suitcase.tiff_stack"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "suitcase.tiff_stack.export(db[-1].documents(fill=True), 'my_exported_files/', file_prefix='{start[sample_comp]}/{start[barcode]}-')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TIFF series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import suitcase.tiff_series"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unlike CSV or TIFF stack, this exporter creates separate file per row (Event document). We can include information from a specific Event in the filename. Here, we include the motor position."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "suitcase.tiff_series.export(db[-1].documents(fill=True), 'my_exported_files/', file_prefix='{start[sample_comp]}/{start[barcode]}/{event[data][motor_spotx]}-')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## \"Beta\" feature: Set up live exporting\n",
    "\n",
    "*We plan to streamline this in the future, but here is a demonstration that it is possible.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Execute this once to configure RE to send data to an exporter.\n",
    "# At the beamline, this would normally be done in a script automatically at startup.\n",
    "\n",
    "from event_model import RunRouter\n",
    "from databroker.core import discover_handlers\n",
    "\n",
    "def factory(name, start_doc):\n",
    "\n",
    "    serializer = suitcase.tiff_series.Serializer(\n",
    "        'my_exported_files/', file_prefix='{start[sample_comp]}/{start[barcode]}-{start[uid]:.8}-')\n",
    "\n",
    "    def cb(name, doc):\n",
    "        serializer(name, doc)\n",
    "\n",
    "    return [cb], []\n",
    "\n",
    "\n",
    "rr = RunRouter([factory], discover_handlers())\n",
    "RE.subscribe(rr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "RE(count([spot], num=30, delay=1), sample_comp='Lab6', barcode='12345')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}