Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIGHT/EXPID/TILEID = 20241012/257708/26055 has some invalid values in the exposures-daily file. #232

Open
weaverba137 opened this issue Oct 14, 2024 · 2 comments
Assignees

Comments

@weaverba137
Copy link
Member

This exposure has some missing values for GFA quantities. These should have been replaced by zeros in desi_tsnr_afterburner.

 NIGHT   EXPID  TILEID  TILERA TILEDEC       MJD       ...  SKY_MAG_Z_SPEC  EFFTIME_GFA EFFTIME_DARK_GFA EFFTIME_BRIGHT_GFA EFFTIME_BACKUP_GFA
 int32   int32  int32  float64 float64     float64     ...     float64        float64       float64           float64            float64      
-------- ------ ------ ------- ------- --------------- ... ---------------- ----------- ---------------- ------------------ ------------------
20241012 257708  26055 131.683  27.238  60596.52421959 ... 17.5417679294026          --               --                 --                 --

This exposure was not processed by the gfa_reduce pipeline:

INFO:gfa_single_night.py:190:_gfa_single_night: Found file /global/cfs/cdirs/desi/spectro/data/20241012/00257708/desi-00257708.fits.fz on the server.
INFO:gfa_single_night.py:201:_gfa_single_night: Spectro file has no corresponding guide cube : /global/cfs/cdirs/desi/spectro/data/20241012/00257708/desi-00257708.fits.fz.

I'm not sure which file is the guide cube, but these two files do not exist: guide-rois-00257708.fits.fz and guide-00257708-0000.fits.fz, both of which do exist for earlier exposures, such as 257707 which was processed successfully.

So I don't think there's much mystery as to why it was not in the GFA summary file. However, we do need to better understand the circumstances in which exposures with bad GFA data have that data set to zero.

@weaverba137 weaverba137 self-assigned this Oct 14, 2024
@schlafly schlafly transferred this issue from desihub/desispec Oct 15, 2024
@schlafly
Copy link
Contributor

@araichoor , let's mark observation as unsure. I'd propose that we mark this as bad as well; it's only 28 effective seconds on a bright tile, so it's not a big loss, and the fact that we're missing guide files is weird enough to be suspect.

@araichoor
Copy link
Contributor

TILEID=26055 got further observations on 20241013 (one exposure with efftime=111s, and another one with efftime=0s).
it has not be QA'ed yet as it has so far efftime=139.1s, below the 180*0.85=153s to show up in the qa validation process.
so maybe we just want the data team to mark the exposure as "bad", and re-process the tiles/cumulative files? (ie nothing to do on the surveyops side).

if useful, few additional infos:

it was the last science exposure of the night; the night log does mention some issue with the guide files:

05:34 [12:34] Exp. 257708- Good - Tile 26055, transparency 97.3%, seeing 0.76 arcsec, survey speed 16%, positioner and guiding ok. Data looks good. (SO)
Tile 26055, Exptime: 580.07, Airmass: 1.13, Sequence: DESI, Flavor: science, Program: BRIGHT

05:38 [12:38] Warning from GUIDE 5 readout error due to incorrect image size. (LO)

05:48 [12:48] Stop last exposure. Run park robots (LO)

don t know if that s related, but there were some hexapod warning few exposures ealier:

04:44 [11:44] Got hexapod warning - "attempt to use AOS data failed because of missing AOS data (last lut None, doz None). Switching to temperature offset mode. Disabling autofocus tracker" (LO)

04:45 [11:45] Got a warning that I need to have autofocus enabled for DESI sequences so re-enable it. (LO)

04:48 [11:48] Exp. 257705- Good - Tile 5059, transparency 94.6%, seeing 0.98 arcsec, survey speed 115.6%, positioner and guiding ok. Data looks good. (SO)
Tile 5059, Exptime: 1113.66, Airmass: 1.21, Sequence: DESI, Flavor: science, Program: DARK

04:51 [11:51] Hexapod warning was on Exposure 257704 (LO)

there actually is one guide file, but not the "usual three files":

ll /global/cfs/cdirs/desi/spectro/data/20241012/00257708/
total 796M
-r--r----- 1 desi desi 5.0K Oct 13 05:31 request-00257708.json
-r--r----- 1 desi desi 5.0M Oct 13 05:31 fiberassign-026055.fits.gz
-r--r----- 1 desi desi  13M Oct 13 05:33 guide-00257708-0000.fits.fz
-r--r----- 1 desi desi  59K Oct 13 05:34 etc-00257708.png
-r--r----- 1 desi desi 1.1M Oct 13 05:35 pm-00257708.fits
-r--r----- 1 desi desi  40M Oct 13 05:35 pm-00257708-logs.tar
-r--r----- 1 desi desi  61M Oct 13 05:35 fvc-00257708.fits.fz
-r--r----- 1 desi desi 105K Oct 13 05:44 centroids-00257708.json
-r--r----- 1 desi desi  52K Oct 13 05:44 etc-00257708.json
-r--r----- 1 desi desi  93M Oct 13 05:44 focus-00257708.fits.fz
-r--r----- 1 desi desi  76M Oct 13 05:44 sky-00257708.fits.fz
-r--r----- 1 desi desi 507M Oct 13 05:46 desi-00257708.fits.fz
-r--r----- 1 desi desi 3.9M Oct 13 05:46 coordinates-00257708.fits
-r--r----- 1 desi desi 1.2K Oct 13 05:46 checksum-00257708.sha256sum

for comparison, here s the "usual" sequence of files from the previous exposure:

ll /global/cfs/cdirs/desi/spectro/data/20241012/00257707/
total 1.1G
-r--r----- 1 desi desi 5.0K Oct 13 05:22 request-00257707.json
-r--r----- 1 desi desi 4.8M Oct 13 05:22 fiberassign-020955.fits.gz
-r--r----- 1 desi desi  13M Oct 13 05:24 guide-00257707-0000.fits.fz
-r--r----- 1 desi desi  67K Oct 13 05:25 etc-00257707.png
-r--r----- 1 desi desi 1.1M Oct 13 05:26 pm-00257707.fits
-r--r----- 1 desi desi  40M Oct 13 05:26 pm-00257707-logs.tar
-r--r----- 1 desi desi  61M Oct 13 05:26 fvc-00257707.fits.fz
-r--r----- 1 desi desi 2.0M Oct 13 05:31 guide-rois-00257707.fits.fz
-r--r----- 1 desi desi  56K Oct 13 05:31 etc-00257707.json
-r--r----- 1 desi desi 272K Oct 13 05:31 centroids-00257707.json
-r--r----- 1 desi desi  51M Oct 13 05:31 focus-00257707.fits.fz
-r--r----- 1 desi desi 475M Oct 13 05:31 guide-00257707.fits.fz
-r--r----- 1 desi desi  43M Oct 13 05:31 sky-00257707.fits.fz
-r--r----- 1 desi desi 375M Oct 13 05:33 desi-00257707.fits.fz
-r--r----- 1 desi desi 3.9M Oct 13 05:33 coordinates-00257707.fits
-r--r----- 1 desi desi 1.3K Oct 13 05:33 checksum-00257707.sha256sum

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants