Save task results to host filesystem #229
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
commit 51efbe8 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed May 10 12:39:24 2023 -0400 Bump django from 3.2.18 to 3.2.19 in /src (#241) * Bump django from 3.2.18 to 3.2.19 in /src Bumps [django](https://github.com/django/django) from 3.2.18 to 3.2.19. - [Commits](django/django@3.2.18...3.2.19) --- updated-dependencies: - dependency-name: django dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update Django in requirements.in * pip-compile requirements --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Anthony Romaniello <[email protected]> commit ae1ddbd Merge: 09dbf86 6288c58 Author: Justin Haze <[email protected]> Date: Fri Apr 28 10:11:04 2023 -0600 Merge pull request #239 from NTIA/scheduler_failure_recovery Scheduler failure recovery commit 6288c58 Author: Doug Boulware <[email protected]> Date: Thu Apr 27 13:50:47 2023 -0600 Add MAX_FAILURES to all settings to fix tests. commit 840e0d3 Author: Doug Boulware <[email protected]> Date: Thu Apr 27 13:41:56 2023 -0600 ran pre-commit commit 105c334 Author: Doug Boulware <[email protected]> Date: Wed Apr 26 15:38:12 2023 -0600 Decrease default MAX_FAILURES to 2. commit fd271f8 Author: Doug Boulware <[email protected]> Date: Tue Apr 25 15:53:34 2023 -0600 Add MAX_FAILURES into runtime_settings commit 09dbf86 Author: Anthony Romaniello <[email protected]> Date: Tue Apr 25 13:37:23 2023 -0600 Update Browsable API Screenshots in README (#219) * Update browsable API images in README * remove EXIF data * increase image quality * Squashed commit of the following: commit 49512a0 Merge: c890b3a 208922d Author: Anthony Romaniello <[email protected]> Date: Thu Jan 19 14:20:32 2023 -0700 Merge pull request #220 from NTIA/nasctn-sea Consolidate NASCTN SEA changes commit 208922d Author: Anthony Romaniello <[email protected]> Date: Thu Jan 19 11:37:52 2023 -0700 Recompile requirements commit 4b98ef7 Author: Anthony Romaniello <[email protected]> Date: Thu Jan 19 11:16:29 2023 -0700 Update pre-commit hooks commit fe658a5 Author: Anthony Romaniello <[email protected]> Date: Thu Jan 19 11:12:44 2023 -0700 Delete potentially dangerous rebuild script commit 2cf32d0 Author: Anthony Romaniello <[email protected]> Date: Thu Jan 19 11:12:24 2023 -0700 Format empty preselector config commit 997fdb7 Author: Anthony Romaniello <[email protected]> Date: Thu Jan 19 11:12:13 2023 -0700 Remove calibration files commit 8cfeca4 Merge: 2aa2169 c890b3a Author: Anthony Romaniello <[email protected]> Date: Thu Jan 19 10:53:37 2023 -0700 Merge branch 'master' into nasctn-sea commit c890b3a Author: Justin Haze <[email protected]> Date: Fri Jan 13 15:55:25 2023 -0700 Change Default Configuration, Rename Signal (#218) * requirements * tekrsa_api_wrap version * add scos-tekrsa to logging config. * removed debug_toolbar and django_extensions * Update to changes in action interface. * Renamed add_metadata_decorators * Added default calibration files. * updated defaul cal files. * updated defaul cal files. * requirements * migrations * Trying to fix capabilities endpoint crashing. Moved getting actions into init. * Moved logger action to scos-actions. * add actions back in * remove logger import. * update version of tekrsa wrapper. * Switch actions branch to dsp-refactor * Switched scos-tekrsa branch * Update base image for scos-tekrsa * Fix requirements - remove Windows-specific packages * Updated requirements * Updated requirements * Python 3.7 --> 3.8 * Updated api_entrypoint.sh to use python 3.8. Dont reload modules to get calibration time. Ran autoformat. * Updated requirements * update requirements * Initial SEA configs * Updated base image version * Update configs * Updated requirements * Update preselector package to 2.0.0 * requirements update. * requirements update. * Update scos-actions and its-preselector * Temporary fix for dynamic status updates * Add shared memory for Ray in SEA data product action * change monitor handler to health check handler, update scos-actions and scos-usrp branches, format files * fix log message * update scos-actions version to 5.0.0 * update scos-usrp version * test scos-tekrsa health check * decrease healthcheck interval * update requirements * revert to "master" version of actions import * autoformat * change scos-actions branch * update scos-tekrsa and scos-actions versions * update scos-actions version * fix warning message for django admin view * update scos-tekrsa version * update preselector config * update sensor definition * empty preselector config * Delete calibration files * remove rebuild script * Remove redundant scos_tekrsa entry * autoformat files, fix task result duration description, update README for shm_size Co-authored-by: dboulware <[email protected]> Co-authored-by: Anthony Romaniello <[email protected]> Co-authored-by: Anthony Romaniello <[email protected]> commit 2aa2169 Merge: 43cdb58 ed5bb23 Author: Anthony Romaniello <[email protected]> Date: Thu Jan 12 11:22:34 2023 -0700 Merge pull request #208 from NTIA/dynamic_status_nasctn_sea Dynamic status and fix action/sigan loading commit ed5bb23 Author: Doug Boulware <[email protected]> Date: Thu Jan 12 11:12:41 2023 -0700 nasctn-sea dependencies. commit 2db6495 Author: Doug Boulware <[email protected]> Date: Thu Jan 12 11:11:01 2023 -0700 switch to scos-actions nasctn-sea. commit 001d533 Merge: fcc1bf6 43cdb58 Author: Doug Boulware <[email protected]> Date: Thu Jan 12 11:08:14 2023 -0700 Merge branch 'nasctn-sea' of https://github.com/NTIA/scos-sensor into dynamic_status_nasctn_sea commit 43cdb58 Merge: 7a160e5 2d68505 Author: Anthony Romaniello <[email protected]> Date: Thu Jan 12 10:30:23 2023 -0700 Merge branch 'master' into nasctn-sea commit fcc1bf6 Author: Doug Boulware <[email protected]> Date: Thu Jan 12 09:38:32 2023 -0700 change libusb-1.0.0-dev to libusb-1.0.0 commit 9cd94ad Merge: 60cbd16 d8237dc Author: dboulware <[email protected]> Date: Wed Dec 7 10:06:14 2022 -0700 Merge pull request #213 from NTIA/fix-data-download-script Update data archive path in download script commit d8237dc Author: Anthony Romaniello <[email protected]> Date: Thu Oct 6 12:50:12 2022 -0600 Update data archive path commit 60cbd16 Author: dboulware <[email protected]> Date: Tue Sep 13 14:16:23 2022 -0600 put on tagged scos repos. commit a37b0a9 Author: dboulware <[email protected]> Date: Fri Sep 9 12:44:53 2022 -0600 Turn off debug commit 7e190e5 Author: dboulware <[email protected]> Date: Fri Sep 9 10:49:01 2022 -0600 add libusb-1.0.0-dev to docker container. commit ea295e6 Author: dboulware <[email protected]> Date: Fri Sep 9 09:33:44 2022 -0600 Removed preselector from sensor definition so tests would work. commit a897c69 Merge: 9fdae2f 7a160e5 Author: dboulware <[email protected]> Date: Fri Sep 9 09:21:34 2022 -0600 Merge branch 'nasctn-sea' of https://www.github.com/ntia/scos-sensor into dynamic_status � Conflicts: � configs/preselector_config.json � docker/Dockerfile-api � entrypoints/api_entrypoint.sh � src/.isort.cfg � src/actions/__init__.py � src/authentication/migrations/0002_alter_user_first_name.py � src/capabilities/__init__.py � src/capabilities/views.py � src/requirements-dev.in � src/requirements-dev.txt � src/requirements.in � src/requirements.txt � src/schedule/migrations/0002_alter_scheduleentry_action.py � src/schedule/models/schedule_entry.py � src/sensor/settings.py � src/status/__init__.py � src/status/views.py commit 7a160e5 Author: Anthony Romaniello <[email protected]> Date: Thu Sep 8 19:46:34 2022 +0000 Add shared memory for Ray in SEA data product action commit c8d60f3 Author: Anthony Romaniello <[email protected]> Date: Thu Sep 8 19:46:02 2022 +0000 Temporary fix for dynamic status updates commit 5ab68ad Author: Anthony Romaniello <[email protected]> Date: Thu Sep 8 19:14:41 2022 +0000 Update scos-actions and its-preselector commit 5e287d8 Author: dboulware <[email protected]> Date: Thu Sep 8 10:08:20 2022 -0600 requirements update. commit 6ad4486 Author: dboulware <[email protected]> Date: Thu Sep 8 09:47:50 2022 -0600 requirements update. commit f7a25c1 Author: Anthony Romaniello <[email protected]> Date: Wed Aug 31 18:02:51 2022 +0000 Update preselector package to 2.0.0 commit fd6f56c Author: Anthony Romaniello <[email protected]> Date: Tue Aug 23 17:55:59 2022 +0000 Updated requirements commit 1490c85 Merge: 40ae477 60335cf Author: Anthony Romaniello <[email protected]> Date: Tue Aug 23 17:50:28 2022 +0000 Merge branch 'nasctn-sea' of https://github.com/ntia/scos-sensor into nasctn-sea commit 40ae477 Author: Anthony Romaniello <[email protected]> Date: Tue Aug 23 17:48:54 2022 +0000 Update configs for seadog03 commit 60335cf Author: Anthony Romaniello <[email protected]> Date: Wed Aug 17 21:43:48 2022 +0000 Updated base image version commit d096512 Author: Anthony Romaniello <[email protected]> Date: Wed Aug 17 16:27:19 2022 +0000 Initial SEA configs commit 60ec290 Author: Anthony Romaniello <[email protected]> Date: Wed Aug 10 21:09:07 2022 +0000 update requirements commit 524902a Author: Anthony Romaniello <[email protected]> Date: Wed Aug 10 15:19:38 2022 +0000 Updated requirements commit 607fa0f Author: dboulware <[email protected]> Date: Mon Aug 8 14:33:07 2022 -0600 Updated api_entrypoint.sh to use python 3.8. Dont reload modules to get calibration time. Ran autoformat. commit 63837e6 Author: Anthony Romaniello <[email protected]> Date: Mon Aug 8 12:36:39 2022 -0600 Python 3.7 --> 3.8 commit dcf43cc Author: Anthony Romaniello <[email protected]> Date: Tue Jul 26 14:27:39 2022 -0600 Updated requirements commit 52946bd Author: Anthony Romaniello <[email protected]> Date: Tue Jul 26 11:41:00 2022 -0600 Updated requirements commit 4759b0d Author: Anthony Romaniello <[email protected]> Date: Tue Jul 26 11:20:37 2022 -0600 Fix requirements - remove Windows-specific packages commit dc9bc29 Author: Anthony Romaniello <[email protected]> Date: Mon Jul 25 15:21:52 2022 -0500 Update base image for scos-tekrsa commit ddc309c Author: Anthony Romaniello <[email protected]> Date: Wed Jul 13 15:50:44 2022 -0600 Switched scos-tekrsa branch commit 27a1a23 Author: Anthony Romaniello <[email protected]> Date: Wed Jul 13 14:46:09 2022 -0600 Switch actions branch to dsp-refactor commit b03e64c Author: dboulware <[email protected]> Date: Tue Jun 28 12:48:00 2022 -0600 update version of tekrsa wrapper. commit a5af68c Author: dboulware <[email protected]> Date: Fri Jun 17 10:10:04 2022 -0600 remove logger import. commit 9e81980 Author: dboulware <[email protected]> Date: Fri Jun 17 09:48:28 2022 -0600 add actions back in commit 5bbd2e1 Author: dboulware <[email protected]> Date: Fri Jun 17 09:14:36 2022 -0600 Moved logger action to scos-actions. commit bc97349 Author: dboulware <[email protected]> Date: Thu Jun 16 13:05:55 2022 -0600 Trying to fix capabilities endpoint crashing. Moved getting actions into init. commit e624e20 Author: dboulware <[email protected]> Date: Thu Jun 16 11:07:37 2022 -0600 migrations commit 219cd43 Author: dboulware <[email protected]> Date: Thu Jun 16 11:04:48 2022 -0600 requirements commit ea3874e Author: Doug Boulware <[email protected]> Date: Thu Jun 16 10:24:43 2022 -0600 updated defaul cal files. commit 1c9edbb Author: Doug Boulware <[email protected]> Date: Thu Jun 16 10:23:27 2022 -0600 updated defaul cal files. commit adffc70 Author: Doug Boulware <[email protected]> Date: Thu Jun 16 09:47:06 2022 -0600 Added default calibration files. commit 8717576 Author: Doug Boulware <[email protected]> Date: Wed Jun 15 13:24:31 2022 -0600 Renamed add_metadata_decorators commit 63365a8 Author: Doug Boulware <[email protected]> Date: Fri May 20 12:47:42 2022 -0600 Update to changes in action interface. commit 8541e27 Author: Doug Boulware <[email protected]> Date: Thu May 19 14:12:12 2022 -0600 removed debug_toolbar and django_extensions commit 3bf3bf6 Author: Doug Boulware <[email protected]> Date: Fri May 13 13:21:08 2022 -0600 add scos-tekrsa to logging config. commit 5aa7879 Author: Doug Boulware <[email protected]> Date: Fri May 13 07:34:11 2022 -0600 tekrsa_api_wrap version commit 24ad1c5 Author: dboulware <[email protected]> Date: Fri May 6 13:31:37 2022 -0600 requirements * Revert "Squashed commit of the following:" This reverts commit 033352f. commit df9938d Author: Doug Boulware <[email protected]> Date: Tue Apr 25 13:30:03 2023 -0600 fix consecutive_failures references. commit 28b2f32 Author: Doug Boulware <[email protected]> Date: Tue Apr 25 13:08:00 2023 -0600 Restart container upon multiple consecutive failed actions. commit 81d0653 Author: Anthony Romaniello <[email protected]> Date: Tue Apr 25 11:28:38 2023 -0600 Fix scheduling future actions (#237) * Fix scheduling future actions * add unit tests for sensor.utils datetime manipulations
commit eb4be4d Author: Anthony Romaniello <[email protected]> Date: Fri Jun 30 09:01:08 2023 -0600 Improve debugging tools for `ray` and segmentation faults (#245) * testing branches for scos-actions and scos-tekrsa * record Ray reference creation sites in debug mode * start ray dashboard in debug mode * use faulthandler in debug mode * Compile deps * remove ray init from scheduler * decrease ray version * Update Ray to 2.4.0 * tagged versions of actions and tekrsa * increase sleep time before CI container health check * try 60s wait for API container test * do not use grep quiet option using "-q", grep always has an exit status of 0. Without this flag, exit status is only 0 if a line is selected by grep * update pre-commit hooks * run pre-commit hooks on all files * use sea-prototype-v0.4.1 tags * recompile requirements * update markdownlint hook * revert unnecessary GH actions workflow changes * Bump minimum ray debug version to match scos-actions * Increase wait for container healthy time in GH Action * update scos-actions branch for testing * update scos-tekrsa version for testing * update requirements tagged scos plugin versions, other minor updates commit f2de439 Author: Anthony Romaniello <[email protected]> Date: Fri Jun 23 10:07:04 2023 -0600 Update SCOS Actions for SigMF-NS-NTIA updates (#244) * testing update-sigmf-ns-ntia * Update sensor_definition.json * remove pywin dev requirement for linux testing * update requirements (add msgspec) * update pre-commit version * Update measurement_handler.py * revert attribute check to key check * do not overwrite sensor ID with FQDN * Remove old, unused schemas * remove sample count check this is no longer possible from the metadata, and the sha512 check makes it redundant * update requirements tagged releases for scos-actions and scos-tekrsa * increase wait time for container healthy in CI tests commit 78732f2 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Jun 6 15:53:03 2023 -0400 Bump cryptography from 39.0.1 to 41.0.0 in /src (#243) * Bump cryptography from 39.0.1 to 41.0.0 in /src Bumps [cryptography](https://github.com/pyca/cryptography) from 39.0.1 to 41.0.0. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](pyca/cryptography@39.0.1...41.0.0) --- updated-dependencies: - dependency-name: cryptography dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Update requirements.in and run pip-compile --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Anthony Romaniello <[email protected]>
dboulware
approved these changes
Aug 29, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Use a Docker named volume to store task result files to the host filesystem. The default mapping places files in the
filesdirectory in the SCOS Sensor repository root directory on the host filesystem, and this mapping is configurable by editing the volume indocker-compose.yml