Slack messages for derived/primary datasets

[gesinaphillips]

Major thing to review

• Updates to utils.pythonop_set_dataset_state and how it is called in DAGs.

Changes to utils.pythonop_set_dataset_state

• Adds params to make MessageManager aware of when a processing pipeline is involved.
  • parent_dataset_uuid_callable: add parent ID so we can message based on the primary dataset when a pipeline fails
  • pipeline_name: we can determine whether MessageManager was triggered by a pipeline; plus now we have richer messaging data
• Calls call_message_managers when there is no UUID passed (because creation of derived dataset failed--this skips StatusChanger) to trigger messages for primary dataset.
• Updates processing DAGs with new args.

Ergonomic changes

• MessageManager now takes a messages dict param where before it expected several different parameters. This removes several messaging-only params from StatusChanger while allowing messaging data to still be passed in explicitly.
  • The base MessageManager class parses the messages dict and turns expected optional values into properties.
  • Changes StatusChanger signature to use messages param.
  • Updates StatusChanger calls.
• Adds status_utils.py > call_message_managers to allow use of messaging classes outside of StatusChanger.
• Statuses can now be passed to MessageManager as strings.
• Adds status_utils.py > get_status_enum to status utils for easier conversion between metadata status and Statuses enum.

SlackMessage

• Addition of run_id and processing_pipeline params.
• Added helper methods for dealing with primary dataset info.
• New messaging classes:
  • dataset_error_derived - error after creating derived dataset
  • dataset_error_primery_pipeline - error on the primary dataset occurred while running processing pipeline
  • dataset_new - base class, does nothing
  • dataset_new_derived - a new derived dataset has been created
  • dataset_qa_derived - a derived dataset has reached QA

Tests

• Added 'em.