-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
25 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,26 @@ | ||
# swodlr-ingest-to-sds | ||
swodlr-ingest-to-sds is a system for ingesting input products to on-demand SDS | ||
|
||
A SWODLR microservice for ingesting granules from a CNM-R SNS topic to a SDS | ||
|
||
## How it works | ||
|
||
`ingest-to-sds` subscribes to a SNS topic which receives CNM-R messages from the | ||
PO.DAAC Cumulus pipeline which processes the PIXC, PIXCVec, and Orbit granules | ||
that SWODLR requires as input granules. `ingest-to-sds` then requests that the | ||
SDS powering SWODLR perform an ingest on the products within those CNM-R | ||
messages. `ingest-to-sds` will continue to track the ingest job as it proceeds | ||
through the SDS in order to monitor for its completion. This data is tracked | ||
within a DynamoDB table for later monitoring by operations/development teams. | ||
Errors that result from ingestion will also be cataloged within that table | ||
for quick mass troubleshooting/diagnostics. | ||
|
||
`ingest-to-sds` is a purely internal SWODLR service and is not part of any user | ||
request pipeline. `ingest-to-sds` also operates as a "best-effort" service. | ||
Because `raster-create` can perform its own ingest process for required granules | ||
via its Preflight process, `ingest-to-sds` is simply a service to help bring | ||
down latencies for `raster-create` by preloading the granule database on the | ||
SDS. | ||
|
||
`ingest-to-sds` is ***NOT*** a critical SWODLR service and occasional errors | ||
that occur during its operation should ***NOT*** be treated with high priority | ||
**unless occuring with unusually high frequency** |