feat(etl): Update subscription deletion ETL scripts#9216
Conversation
BenWu
left a comment
BenWu
left a comment
There was a problem hiding this comment.
Just some comments on column descriptions but othwerwise LGTM
| time_partitioning: | ||
| type: day | ||
| field: submission_timestamp | ||
| require_partition_filter: false |
There was a problem hiding this comment.
The expectation is that new tables have column descriptions in the schema.yaml and this should be set
| require_partition_filter: false | |
| require_partition_filter: false | |
| require_column_descriptions: true |
Creating tables via the cli will add this but can get missed when copy/pasting
There was a problem hiding this comment.
Updated to add require_column_description: true
| fields: | ||
| - name: submission_timestamp | ||
| type: TIMESTAMP | ||
| mode: NULLABLE |
There was a problem hiding this comment.
A description is useful here because the meaning of submission_timestamp here doesn't match the meaning we're used to having everywhere else (logged event vs. telemetry ping). I know the delete_events_v1 table doesn't have this but this would be an improvement
| mode: NULLABLE | |
| mode: NULLABLE | |
| description: ... |
There was a problem hiding this comment.
Updated to add description: Earliest timestamp of the account deletion log entry for this nimbus_user_id on the given date and description: Pre-derived Nimbus user identifier logged by payments-api at account deletion time
Because: * Current glean deletion ETL query duplicates the nimbus_user_id derivation logic that lives in the fxa repository This commit: * Updates the script to match the paired [PR](mozilla/fxa#20381), while still supporting the existing approach Closes #PAY-3438
BenWu
left a comment
BenWu
left a comment
There was a problem hiding this comment.
I'll merge this when CI finishes
2 participants
Because:
This commit:
Closes #PAY-3438