Tiled with Bluesky.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "738520ca",
   "metadata": {},
   "source": [
    "# Tiled with Data from Bluesky"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95dbcbcc",
   "metadata": {},
   "source": [
    "## Connect to a Tiled Service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6a7c680e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from tiled.client import from_uri"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f05f98d0",
   "metadata": {},
   "outputs": [],
   "source": [
    "c = from_uri(\"https://tiled-demo.blueskyproject.io/api\")\n",
    "c"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcc47ee2",
   "metadata": {},
   "source": [
    "## Navigate to a Catalog of Bluesky Runs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "abfdb430",
   "metadata": {},
   "outputs": [],
   "source": [
    "bmm = c['bmm']['raw']\n",
    "bmm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d837ab96",
   "metadata": {},
   "source": [
    "Tiled has a mechanism to communicate that this data has a certain \"specification\" (in this case, a catalog of Bluesky runs) and we use this to display useful information, provide certain conveniences, and achieve speed-ups."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0b9be09",
   "metadata": {},
   "source": [
    "**Aside:** For users who have used Databroker before, compare this to what the \"old way\" would show:\n",
    "\n",
    "```python\n",
    ">>> db\n",
    "<databroker.v1.Broker at 0x7fb66f1cbd50>  # <-- Not as useful!\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34e9584b",
   "metadata": {},
   "source": [
    "## Access a specific Bluesky Run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a044fea7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from databroker import Broker\n",
    "db = Broker(bmm)\n",
    "repr(db[22521])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4cc60262",
   "metadata": {},
   "outputs": [],
   "source": [
    "bmm[22521]  # Access (latest) scan_id 22521."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9ebd83f",
   "metadata": {},
   "source": [
    "**Aside:** For users who have used Databroker before, compare this to what the \"old way\" would show:\n",
    "\n",
    "```python\n",
    ">>> db[22521]\n",
    "<databroker.v1.Header object at 0x7fb66f1e3ed0>  # <-- Not as useful!\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21d13c91",
   "metadata": {},
   "source": [
    "## Drill in to get data of interest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3aac1b0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "bmm[22521]['primary']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf363782",
   "metadata": {},
   "outputs": [],
   "source": [
    "bmm[22521]['primary']['data']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f32961c",
   "metadata": {},
   "outputs": [],
   "source": [
    "bmm[22521]['primary']['data'].read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5576ecd",
   "metadata": {},
   "outputs": [],
   "source": [
    "bmm[22521]['primary']['data'][\"I0\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c28831d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "bmm[22521]['primary']['data'].read([\"dcm_energy\", \"I0\", \"It\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "edee3cc5",
   "metadata": {},
   "outputs": [],
   "source": [
    "bmm[22521]['primary']['data'].export(\"data.csv\", variables=[\"dcm_energy\", \"I0\", \"It\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c82acf1",
   "metadata": {},
   "source": [
    "## Small convenience functions are your friend!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec541aae",
   "metadata": {},
   "outputs": [],
   "source": [
    "def load_scan(scan_id):\n",
    "    return bmm[scan_id]['primary']['data'].read([\"I0\", \"It\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87ae52ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "load_scan(22521)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6180da15",
   "metadata": {},
   "source": [
    "## Access metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20fe5fb5",
   "metadata": {},
   "outputs": [],
   "source": [
    "bmm[22521].metadata  # Spoiler alert: there is a lot!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d6183d1",
   "metadata": {},
   "source": [
    "## Search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5038cdba",
   "metadata": {},
   "outputs": [],
   "source": [
    "bmm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f58b92d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from databroker.queries import TimeRange, FullText\n",
    "\n",
    "bmm.search(TimeRange(since=\"2020-4-1\", until=\"2021-10-31\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a68e6dd5",
   "metadata": {},
   "source": [
    "Does that feel verbose? Again, convenience functions are your friend."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a27a9ce4",
   "metadata": {},
   "outputs": [],
   "source": [
    "def Since(since):\n",
    "    return TimeRange(since=since)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d0e76e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "bmm.search(Since(\"2020-4-1\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "288a1842",
   "metadata": {},
   "source": [
    "### Chain searches to progressively narrow results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "457c8f82",
   "metadata": {},
   "outputs": [],
   "source": [
    "bmm.search(Since(\"2020\")).search(FullText(\"Ni\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17b7c783",
   "metadata": {},
   "source": [
    "**Aside:** For users who have used Databroker before, compare this to what the \"old way\" would return:\n",
    "\n",
    "```python\n",
    ">>> db(\"Ni\")\n",
    "<databroker.v1.Results at 0x7fb66e8b8610>  # <-- Not as useful!\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eebd2e61",
   "metadata": {},
   "source": [
    "### Count results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7e1c024a",
   "metadata": {},
   "outputs": [],
   "source": [
    "results = bmm.search(Since(\"2020\"))\n",
    "\n",
    "len(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3a5d4ff",
   "metadata": {},
   "source": [
    "### Access the first result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7179be61",
   "metadata": {},
   "outputs": [],
   "source": [
    "results.values_indexer[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a442c8eb",
   "metadata": {},
   "source": [
    "### Access the first N results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5d86014c",
   "metadata": {},
   "outputs": [],
   "source": [
    "results.values_indexer[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d215f70",
   "metadata": {},
   "source": [
    "### Loop over search results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4dab112",
   "metadata": {},
   "outputs": [],
   "source": [
    "for result in results.values():\n",
    "    # Do something useful here\n",
    "    print(result)\n",
    "    ...\n",
    "    break  # Remove this line to actually run the full loop."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71a356cf",
   "metadata": {},
   "source": [
    "## Access large array (e.g. image time series) data\n",
    "\n",
    "There are a couple ways to do this. One is to use \"Dask\" to defer I/O (and, if you like, computation)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "becb413a",
   "metadata": {},
   "outputs": [],
   "source": [
    "c = from_uri(\"https://tiled-demo.blueskyproject.io/api\", \"dask\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ef88895",
   "metadata": {},
   "outputs": [],
   "source": [
    "c[\"fxi\"][\"raw\"][62611][\"primary\"][\"data\"][\"Andor_image\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d455954",
   "metadata": {},
   "outputs": [],
   "source": [
    "da = c[\"fxi\"][\"raw\"][62611][\"primary\"][\"data\"][\"Andor_image\"][22, 10]\n",
    "da"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43cb75f5",
   "metadata": {},
   "outputs": [],
   "source": [
    "img = da.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f612347e",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib widget\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "plt.imshow(img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a764e4ed",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}