-
Notifications
You must be signed in to change notification settings - Fork 1
Description
Test instances that are missing mandatory information (broadly defined), like a test case name, or a failure_type (if passed is false) should never make it into the database and should be spat back to the submitter while accepting the rest. Might also spit back unrecognized keys since they are likely wrong, or at least as a reminder to the developer to make them do something on the backend.
Background
mesa handles creating the YAML files, and mesa_test handles submitting them. In principle, mesa_test could check the YAML files for consistency so it never makes it to the server, but that would require mesa_test to be "smart", and more importantly, updated. Kicking this up to the testhub means that mesa_test can remain a blameless messenger between the testing scripts of mesa and the database in the testhub.