Skip to content

Mikedorfman/cumulus 4384#4241

Open
mikedorfman wants to merge 40 commits intomasterfrom
mikedorfman/CUMULUS-4384
Open

Mikedorfman/cumulus 4384#4241
mikedorfman wants to merge 40 commits intomasterfrom
mikedorfman/CUMULUS-4384

Conversation

@mikedorfman
Copy link
Contributor

@mikedorfman mikedorfman commented Feb 2, 2026

Summary: Summary of changes

Addresses CUMULUS-4384: Add workflow to example deployment for rolling archive

Changes

  • Added a rolling-archive workflow deployment to the example stack
  • Added a rolling-archive integration tests covering the three criteria in granule-invalidator
  • Added a rolling-archive cookbook entry

Fixed the following:

  • package.sh was not appropriately packaging dependencies. Reverted back to uv pip install with a target platform specified.
  • The lambda handler was incorrect - this was updated
  • Added a missing PRIVATE_API_LAMBDA_ARN env var in the lambda and appropriate terraform vars in cumulus/archive, cumulus/ingest and the task Terraform to provide this arn.
  • Updated the granule-invalidator bulk delete input to provide a list of granule IDs based on changes to the Cumulus API

PR Checklist

  • Update CHANGELOG
  • Unit tests
  • Ad-hoc testing - Deploy changes and test manually
  • Integration tests

📝 Note:
For most pull requests, please Squash and merge to maintain a clean and readable commit history.

@mikedorfman mikedorfman marked this pull request as draft February 2, 2026 21:50
@mikedorfman mikedorfman marked this pull request as ready for review February 4, 2026 17:15
"collectionId": f"{collection}___{version}",
}
for invalid_granule in invalid_granules
invalid_granule["granuleId"] for invalid_granule in invalid_granules
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change was based on recent Cumulus updates which accept only a list of granule IDs in a bulk delete.

Comment on lines -7 to +9
filename = "${path.module}/../dist/lambda.zip"
source_code_hash = filebase64sha256("${path.module}/../dist/lambda.zip")
handler = "main.handler"
filename = "${path.module}/../dist/final/lambda.zip"
source_code_hash = filebase64sha256("${path.module}/../dist/final/lambda.zip")
handler = "main.lambda_handler"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Packaging updates result in a different path for the file

"Type": "Choice",
"Choices": [
{
"Variable": "$deletion_count",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"Variable": "$deletion_count",
"Variable": "$.deletion_count",

Small missing "." to help with the reference

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is actually correct without the . - I've used a StepFunction variable to assign a value to deletion_count. Unlike referencing input, vars are referenced with just the $ at the beginning.

"Type": "Choice",
"Choices": [
{
"Variable": "$deletion_count",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"Variable": "$deletion_count",
"Variable": "$.deletion_count",

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is actually correct without the . - I've used a StepFunction variable to assign a value to deletion_count. Unlike referencing input, vars are referenced with just the $ at the beginning.

expect(workflowExecution.status).toEqual('completed');
});

it('ingestDateDate rolloff configuration is honored', async () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you intend for "ingestDateDate", or just "ingestDate"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely me stuttering - I'll fix this here and in a few other spots.

}
)).toBeRejectedWithError(CumulusApiClientError, /404/);

const afteringestDateDateTimeCutoffGranule = await getGranule(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe its intentional, I might be missing something

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants