Skip to content

Conversation

@jpnurmi
Copy link
Collaborator

@jpnurmi jpnurmi commented Jan 26, 2026

How about automatically re-running the flaky test_shutdown_timeout test to mitigate frequent CI failures?

# the timings here are:
# * the process waits 2s for the background thread to shut down, which fails
# * it then dumps everything and waits another 1s before terminating the process
# * the python runner waits for 2.4s in total to close the request, which
# will cleanly terminate the background worker.
# the assumption here is that 2s < 2.4s < 2s+1s. but since those timers
# run in different processes, this has the potential of being flaky

Copy link
Member

@JoshuaMoelans JoshuaMoelans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL that this mark.flaky exists! Nice QOL addition 👍

@JoshuaMoelans
Copy link
Member

@getsentry/dev-infra license compliance seems to fail because of us adding pytest-rerunfailures ; anything we can do to clear this up?

@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Jan 26, 2026

Luckily, there are multiple ready-made pytest extensions for dealing with flaky tests. Let's try this instead: https://pypi.org/project/flaky/ 😄

@jpnurmi jpnurmi merged commit a7fee67 into master Jan 26, 2026
42 checks passed
@jpnurmi jpnurmi deleted the test/rerun-flaky-test branch January 26, 2026 13:29
@jpnurmi
Copy link
Collaborator Author

jpnurmi commented Jan 26, 2026

Interesting how manual re-runs often pass, but this didn't seem to help... 🙁

https://github.com/getsentry/sentry-native/actions/runs/21359409357/job/61474580850

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants