feat: allow disabling tests enabled tests using checkin #1707
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist
Description
This diff allows disabling default enabled tests in the engine using the checkin api. The summary of changes are:
if a nettest is enabled by the engine but disabled by checkin and the web_connectivity experiment was run in the last 24 hours, the nettest would throw an error and not run
if the checkin cache has expired and web_connectivity was not called in the last 24 hours, the nettest will run and switch to default behaviour
Apart from this, the running should WAI i.e., if the nettest is disabled by default in the engine, we will check the
OONI_FORCE_ENABLE_EXPERIMENTenv variable and also the checkin cache. However, if a checkin cache is not found at this point, the test will fail to run and throw an error. It is only when we find an explicit permission from the checkin cache that we allow a disabled test to run