update to rbid2 exgal

extragalactic_fields/README.md

@@ -2,6 +2,8 @@
 
 Tools for analyzing the DECam DDF extragalactic fields' data, COSMOS and ELAIS.
 
+In Version 2 of this repository, we use the retrained R/B score (RB ID = 2).
+
 To do science investigations with the publicly-available "probably-real" (see below) candidates, using either their individual-exposure photometry or combined nightly-epoch lightcurves, see the `science_with_candidates/` directory.
 
 The notebooks in the `science_with_objects/` use the non-public databases and are still in development.
@@ -11,7 +13,7 @@ Note:
  * "object" refers to a single detection in a difference image
  * "candidate" refers to associated detections at a given sky location
  * "probably-real" extragalactic candidates means >= 10 detections, mean R/B >= 0.4
-   * cuts shown to be reasonable in `extragalactic_fields/source_detection_summary_figures.ipynb/`
+   * cuts shown to be reasonable in `extragalactic_fields/source_detection_summary_figures.ipynb`
    * cuts also discussed in Section 3 of Graham et al. (in prep)
 
 
@@ -34,21 +36,15 @@ Note:
  * Create plots to explore object detection and R/B scores vs. image quality.
  * Identify **"probably-real" candidates (>= 10 detections, mean R/B >= 0.4)** and create data files just for these candidates.
  * Identify potential flare stars as candidates with > 1 detections in g-band, a mean R/B >=0.6, and a time span of <0.5 days (i.e., detected in one night only).
- * Write two naive filters to test methods for identifying transients and fast-risers with alerts. Find they're too simple to work well.
-   * 'Transients' as candidates with >=5 objects in two consecutive nights in any filter (118 candidates)
-   * 'Fast Risers' as candidates with >=5 objects in the first night in any filter, all brightening (0 candidates)
 
 **Inputs/Outputs**<br>
  * inputs are Rob's databases, requires password
  * output plots are in `source_detection_summary_figures/`
  * output files of **"probably-real"** candidate data are in `source_detection_summary_files/`
    * candidates.dat (summary statistics for "probably-real" candidates, like number of detections)
-   * candidate_objects_(field).dat (all objects for all "probably-real" candidates)
-   * exposures_(field).dat (data for all exposures, e.g., limiting magnitudes, seeing)
- * output files for further study of the low-Cobj, high mean R/B score candidates are in:
-   * `source_detection_summary_files/MLG_lowCobj_highMRB/`
- * output files for further study of potential flare stars are in:
-   * `source_detection_summary_files/potential_flare_stars/`
+   * candidate_objects.dat (all objects for all "probably-real" candidates)
+   * exposures.dat (data for all exposures, e.g., limiting magnitudes, seeing)
+ * output files for further study of potential flare stars in `source_detection_summary_files/potential_flare_stars/`
    * contains a candidates.dat and candidate_objects.dat
 
 ### candidate_nightly_epochs.ipynb

extragalactic_fields/candidate_nightly_epochs_summary.ipynb

@@ -3,7 +3,9 @@
   {
    "cell_type": "markdown",
    "id": "1a06816f-764a-4648-8fc5-682a54d8e3e8",
-   "metadata": {},
+   "metadata": {
+    "tags": []
+   },
    "source": [
     "# Candidate Nightly Epochs Summary\n",
     "\n",
@@ -455,7 +457,7 @@
    "id": "08c6be4c-0863-4df0-aa58-cdec2ea60edd",
    "metadata": {},
    "source": [
-    "### The 25 brightest in any filter"
+    "### The 20 brightest in any filter"
    ]
   },
   {
@@ -467,10 +469,10 @@
    "source": [
     "sx = np.argsort( paramLC_minmag )\n",
     "\n",
-    "fig, ax = plt.subplots( 5, 5, figsize=(16,16), sharex=False, sharey=False )\n",
+    "fig, ax = plt.subplots( 4, 5, figsize=(15,10), sharex=False, sharey=False )\n",
     "\n",
     "s = 0\n",
-    "for i in range(5):\n",
+    "for i in range(4):\n",
     "    for j in range(5):\n",
     "        for f,filt in enumerate(all_filters):\n",
     "            cx = np.where( (candLC_candid == paramLC_candid[sx[s]]) & (candLC_filter == filt) )[0]\n",
@@ -486,6 +488,7 @@
     "                     verticalalignment='bottom', horizontalalignment='left',  transform=ax[i,j].transAxes)\n",
     "        s += 1\n",
     "\n",
+    "plt.tight_layout()\n",
     "plt.show()\n",
     "del sx,s"
    ]
@@ -495,7 +498,7 @@
    "id": "98b5efe5-41ea-4807-9ad0-f0493cf086f5",
    "metadata": {},
    "source": [
-    "### The 25 largest amplitude in the _g_ filter\n",
+    "### The 20 largest amplitude in the _g_ filter\n",
     "\n",
     "**Warning**: use amplitude _in a filter_ as the demo here, because \"the amplitude in _any_ filter\" can be due more to color than change in brightness."
    ]
@@ -509,10 +512,10 @@
    "source": [
     "sx = np.flip( np.argsort( paramLC_amp_g ) )\n",
     "\n",
-    "fig, ax = plt.subplots( 5, 5, figsize=(16,16), sharex=False, sharey=False )\n",
+    "fig, ax = plt.subplots( 4, 5, figsize=(15,10), sharex=False, sharey=False )\n",
     "\n",
     "s = 0\n",
-    "for i in range(5):\n",
+    "for i in range(4):\n",
     "    for j in range(5):\n",
     "        for f,filt in enumerate(all_filters):\n",
     "            cx = np.where( (candLC_candid == paramLC_candid[sx[s]]) & (candLC_filter == filt) )[0]\n",
@@ -528,6 +531,7 @@
     "                     verticalalignment='bottom', horizontalalignment='left',  transform=ax[i,j].transAxes)\n",
     "        s += 1\n",
     "\n",
+    "plt.tight_layout()\n",
     "plt.show()\n",
     "del sx,s"
    ]
@@ -537,7 +541,7 @@
    "id": "48928185-f327-495f-8e6b-27013959469a",
    "metadata": {},
    "source": [
-    "### The 25 best-sampled in any filter\n",
+    "### The 20 best-sampled in any filter\n",
     "\n",
     "They're all in COSMOS, which recieved more epochs so far."
    ]
@@ -551,10 +555,10 @@
    "source": [
     "sx = np.flip( np.argsort( paramLC_neps ) )\n",
     "\n",
-    "fig, ax = plt.subplots( 5, 5, figsize=(16,16), sharex=False, sharey=False )\n",
+    "fig, ax = plt.subplots( 4, 5, figsize=(15,10), sharex=False, sharey=False )\n",
     "\n",
     "s = 0\n",
-    "for i in range(5):\n",
+    "for i in range(4):\n",
     "    for j in range(5):\n",
     "        for f,filt in enumerate(all_filters):\n",
     "            cx = np.where( (candLC_candid == paramLC_candid[sx[s]]) & (candLC_filter == filt) )[0]\n",
@@ -570,6 +574,7 @@
     "                     verticalalignment='bottom', horizontalalignment='left',  transform=ax[i,j].transAxes)\n",
     "        s += 1\n",
     "\n",
+    "plt.tight_layout()\n",
     "plt.show()\n",
     "del sx,s"
    ]
@@ -579,9 +584,9 @@
    "id": "b12030d1-f04e-4426-b063-5c7e4402bbca",
    "metadata": {},
    "source": [
-    "### The 19 most \"SN-Like\"\n",
+    "### A random sample of 20 \"transient-Like\"\n",
     "\n",
-    "A time span of 30-150 days, and an amplitude of at least 0.5 mag, in _every_ filter."
+    "A time span of 20-120 days, and an amplitude of at least 0.5 mag, in _every_ filter."
    ]
   },
   {
@@ -591,12 +596,12 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "tx = np.where( (paramLC_tspan_g > 30) & (paramLC_tspan_r > 30) & (paramLC_tspan_i > 30) &\n",
-    "               (paramLC_tspan_g < 200) & (paramLC_tspan_r < 200) & (paramLC_tspan_i < 200) &\n",
+    "tx = np.where( (paramLC_tspan_g > 20) & (paramLC_tspan_r > 20) & (paramLC_tspan_i > 20) &\n",
+    "               (paramLC_tspan_g < 120) & (paramLC_tspan_r < 120) & (paramLC_tspan_i < 120) &\n",
     "               (paramLC_amp_g > 0.5) & (paramLC_amp_r > 0.5) & (paramLC_amp_i > 0.5) )[0]\n",
     "print(len(tx))\n",
     "\n",
-    "fig, ax = plt.subplots( 4, 5, figsize=(16,12), sharex=False, sharey=False )\n",
+    "fig, ax = plt.subplots( 4, 5, figsize=(15,10), sharex=False, sharey=False )\n",
     "\n",
     "t = 0\n",
     "for i in range(4):\n",
@@ -616,6 +621,7 @@
     "                         verticalalignment='bottom', horizontalalignment='left',  transform=ax[i,j].transAxes)\n",
     "            t += 1\n",
     "\n",
+    "plt.tight_layout()\n",
     "plt.show()\n",
     "del tx,t"
    ]
@@ -625,7 +631,7 @@
    "id": "1f502756-37af-4983-be9f-bedfcf5de03a",
    "metadata": {},
    "source": [
-    "### Short-duration bright evolving phenomena\n",
+    "### A random sample of 20 short-duration bright evolving phenomena\n",
     "\n",
     "Short timespan (<20 days), reach a peak of at least 22 mag, and an amplitude of at least 0.5 mag.\n",
     "\n",
@@ -644,7 +650,7 @@
     "               ((paramLC_amp_g >= 0.5) | (paramLC_amp_r >= 0.5) | (paramLC_amp_i >= 0.5)) )[0]\n",
     "print(len(tx))\n",
     "\n",
-    "fig, ax = plt.subplots( 4, 5, figsize=(16,12), sharex=False, sharey=False )\n",
+    "fig, ax = plt.subplots( 4, 5, figsize=(15,10), sharex=False, sharey=False )\n",
     "\n",
     "t = 0\n",
     "for i in range(4):\n",
@@ -668,6 +674,7 @@
     "                         verticalalignment='bottom', horizontalalignment='left',  transform=ax[i,j].transAxes)\n",
     "            t += 1\n",
     "\n",
+    "plt.tight_layout()\n",
     "plt.show()\n",
     "del tx,t"
    ]

extragalactic_fields/image_processing_summary.ipynb

@@ -3,14 +3,18 @@
   {
    "cell_type": "markdown",
    "id": "bfb20ca1-dc1c-4aa7-a60d-4bade917a5c7",
-   "metadata": {},
+   "metadata": {
+    "tags": []
+   },
    "source": [
     "# Image Processing Summary\n",
     "Author: Melissa\n",
     "\n",
+    "This NB uses the retrained R/B score (RB ID = 2).\n",
+    "\n",
     "This NB uses the file archive_image_list.txt, which was generated for all images from both 2021A-0113 and 2021B-0149 in the NOIRLab Data Archive.\n",
     "\n",
-    "**Class:** MLG has assigned all archive images a \"class\", as follows.<br>\n",
+    "**Archive Image Class:** MLG has assigned all archive images a \"class\", as follows.<br>\n",
     "Some images were obtained with a different exposure time and are not \"standard\".<br>\n",
     "We don't want to bother with those or with any other exposures.\n",
     "\n",

extragalactic_fields/science_with_candidates/01_demo_candidates.ipynb

@@ -14,7 +14,7 @@
     "Recall:\n",
     " * \"object\" refers to a single detection in a difference image\n",
     " * \"candidate\" refers to associated detections at a given sky location\n",
-    " * \"probably-real\" candidates means >10 detections, mean R/B>0.4\n",
+    " * \"probably-real\" candidates means >10 detections, mean R/B>0.6\n",
     "   * cuts shown to be reasonable in `extragalactic_fields/source_detection_summary_figures.ipynb`\n",
     "   * cuts also discussed in Section 3 of Graham et al. (in prep)"
    ]
@@ -160,7 +160,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "1852\n"
+      "4384\n"
      ]
     }
    ],
@@ -212,7 +212,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "119416\n"
+      "314769\n"
      ]
     }
    ],
@@ -269,7 +269,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "240408\n"
+      "578808\n"
      ]
     }
    ],
@@ -324,7 +324,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "1852\n"
+      "4384\n"
      ]
     }
    ],
@@ -550,7 +550,7 @@
    "id": "d1243561-069d-4f49-8e59-627b5fa67940",
    "metadata": {},
    "source": [
-    "### For one rapidly evolving candidate, DC21ksmv\n",
+    "### For one short-duration candidate, DC21ksmv\n",
     "\n",
     "This candidate was selected from the `candidate_nightly_epochs.ipynb` plot for short-duration bright evolving phenomena.\n",
     "\n",

extragalactic_fields/science_with_candidates/README.md

@@ -52,10 +52,4 @@ Is saved in an executed state as a quicklook demo.
 
 ## Notebooks (In Progress)
 
-### MLG_lowCobj_highMRB_investigation.ipynb
-
-**Description**<br>
- * Investigate the nature of the low-Cobj, high mean R/B score candidates
-
-**Inputs/Outputs**<br>
- * inputs are `../source_detection_summary_files/MLG_lowCobj_highMRB/`
+None right now.