From fa2208b8b0d90b45091e41b5a2348252e828d7cf Mon Sep 17 00:00:00 2001 From: Joshua Garde Date: Wed, 21 Aug 2024 14:12:30 -0700 Subject: [PATCH] Update README.md --- README.md | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index b19f914..62345a1 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,26 @@ # swodlr-ingest-to-sds -swodlr-ingest-to-sds is a system for ingesting input products to on-demand SDS + +A SWODLR microservice for ingesting granules from a CNM-R SNS topic to a SDS + +## How it works + +`ingest-to-sds` subscribes to a SNS topic which receives CNM-R messages from the +PO.DAAC Cumulus pipeline which processes the PIXC, PIXCVec, and Orbit granules +that SWODLR requires as input granules. `ingest-to-sds` then requests that the +SDS powering SWODLR perform an ingest on the products within those CNM-R +messages. `ingest-to-sds` will continue to track the ingest job as it proceeds +through the SDS in order to monitor for its completion. This data is tracked +within a DynamoDB table for later monitoring by operations/development teams. +Errors that result from ingestion will also be cataloged within that table +for quick mass troubleshooting/diagnostics. + +`ingest-to-sds` is a purely internal SWODLR service and is not part of any user +request pipeline. `ingest-to-sds` also operates as a "best-effort" service. +Because `raster-create` can perform its own ingest process for required granules +via its Preflight process, `ingest-to-sds` is simply a service to help bring +down latencies for `raster-create` by preloading the granule database on the +SDS. + +`ingest-to-sds` is ***NOT*** a critical SWODLR service and occasional errors +that occur during its operation should ***NOT*** be treated with high priority +**unless occuring with unusually high frequency** \ No newline at end of file