Pulling kbmod-tno-scripts over to the corresponding tasks. #18

drewoldag · 2024-07-18T18:28:54Z

This PR is composed of several rough commits to pull over the code, including:

Initial commit of ic_to_wu task. Copied script over, peeled away what was not needed.
Pulling over the reprojection portion of the old ic_to_wu script.
Included kbmod search script
General clean up of function parameter names, and first pass over cleaning up the logging messages.

… was not needed.

…ous messages that are logged.

src/kbmod_wf/task_impls/kbmod_search.py

src/kbmod_wf/task_impls/reproject_wu.py

drewoldag · 2024-07-18T22:49:22Z

src/kbmod_wf/task_impls/uri_to_ic.py

-def no_uri_to_ic(target_uris_file_path=None, uris_base_dir=None, ic_output_file_path=None, logger=None):
+def uri_to_ic(
+    uris_filepath: str = None,
+    uris_base_dir: str = None,


@wilsonbb I have an open question about uris_base_dir. I don't think that we really need this as an input parameter, but before I yanked it out, I wanted to check to see if I was missing something here. I also have an open issue #8 asking this same question since I noticed it a while ago.

If I recall correctly, this was added because Colin was running the scripts both on klone and on another cluster where he had copied that target uris over (I believe on Monsoon). So he could then run the same script on the same target uris files but then specify a different base directory (the folder where he had copied the data to)

I don't think it's essential to keep in, but maybe not remove it until we settle whether we will be changing the .lst files after Dino's work

@ColinOrionChandler could you confirm this? Was this parameter in place so that you could run on multiple clusters?

src/kbmod_wf/workflow.py

src/kbmod_wf/task_impls/ic_to_wu.py

src/kbmod_wf/task_impls/kbmod_search.py

wilsonbb

Thanks Drew! Left a few comments for some additional docstrings, but overall this looks correct to me.

While there are still items to clean up, I'm also fine with merging as we are so we can begin testing the current workflow on klone and see how parsl fares with our actual workloads before fiddling further.

src/kbmod_wf/workflow.py

wilsonbb · 2024-07-19T04:19:48Z

src/kbmod_wf/task_impls/uri_to_ic.py

-def no_uri_to_ic(target_uris_file_path=None, uris_base_dir=None, ic_output_file_path=None, logger=None):
+def uri_to_ic(
+    uris_filepath: str = None,
+    uris_base_dir: str = None,


If I recall correctly, this was added because Colin was running the scripts both on klone and on another cluster where he had copied that target uris over (I believe on Monsoon). So he could then run the same script on the same target uris files but then specify a different base directory (the folder where he had copied the data to)

I don't think it's essential to keep in, but maybe not remove it until we settle whether we will be changing the .lst files after Dino's work

src/kbmod_wf/task_impls/reproject_wu.py

src/kbmod_wf/task_impls/kbmod_search.py

wilsonbb

Thanks LGTM!

Initial commit of ic_to_wu task. Copied script over, peeled away what…

1efe0d6

… was not needed.

drewoldag requested a review from wilsonbb July 18, 2024 18:28

drewoldag self-assigned this Jul 18, 2024

drewoldag added 3 commits July 18, 2024 13:00

Porting over the reprojection part of the workflow.

b82919b

Copied over the kbmod_search portion of the TNO scripts.

3fdd0c1

Unifying the parameter names, attempting a quick clean up of the vari…

adb4d20

…ous messages that are logged.