[parallel_testsuite.py] Output testing results in a single line #25752
Conversation
Like many modern tests runners we can output all results on a single line a by re-writing the some line on the TTY. We only do this under certain circumstances: 1. stderr is a TTY 2. The `--buffer` options us used (test output hidden unless fail) This mode is still not the default yet since `--buffer` is not yet the default, but the output is so much nicer I think we should do that at some point.
|
Small things, but this change I think will be huge qualify of life improvement for those of us that run the test suite on the command line a lot! |
This is now what it looks like when I run the entire core test suite of 1176 tests! |
|
|
||
| def clearline(): | ||
| assert sys.stderr.isatty() | ||
| sys.stdout.write('\r\033[K') |
There was a problem hiding this comment.
This might warrant a comment on what this code is?
| self.failing_and_slow_first = options.failing_and_slow_first | ||
| self.progress_counter = 0 | ||
| self.use_single_line_output = options.buffer and sys.stderr.isatty() | ||
| if self.use_single_line_output: |
There was a problem hiding this comment.
Maybe instead of use_single_line_output, a more precise name could be something like collapse_test_success_prints?
| self.progress_counter = 0 | ||
| self.use_single_line_output = options.buffer and sys.stderr.isatty() | ||
| if self.use_single_line_output: | ||
| self.terminal_width = shutil.get_terminal_size()[0] |
There was a problem hiding this comment.
How expensive is getting this size? Caching this size here sounds like things will break if one then resizes the window size mid-run?
It might be quite common for user to start a test run, then mid-run realize "the logs are not fitting on one line, let me grow the terminal window a bit."
| msg = 'skipped' | ||
| else: | ||
| reason = res.skipped[0][1] | ||
| msg = f"skipped '{reason}'" |
There was a problem hiding this comment.
What is this block doing? In single line output mode, the skip reasons are not printed? Why does that matter if the skip reasons get cleared anyways?
| if self.use_single_line_output: | ||
| min_len = len(progress) + len(msg) + 5 | ||
| max_name = self.terminal_width - min_len | ||
| test_name = str(res.test)[:max_name] |
There was a problem hiding this comment.
What is happening here? Some kind of padding? Maybe a comment would help.
| def writeln(x, arg=None): | ||
| if arg: | ||
| x.write(arg) | ||
| x.write('\n') |
|
Looks good, though needs Windows and macOS testing before landing. We still have the previous Windows 10 TTY/coloring regression piled up, I hope this won't interact with that. |
Was that not fixed by #25529? What is the regression you are referring to? |
This test runner does a few things differ the base TextTestRunner: 1. It improves the behviour of `--buffer` by also buffering/redirecting logging output that occurs during the test run. 2. It displays all results on a single line, each result erasing the contents of the line before re-drawing it. 3. It uses ANSI colors to the show the results. 4. It should the progress as each results is displayed so its easy to see how far you are through the test suite "[XX/YY]" I also updated parallel_testsuite.py use the same "XX/YY" progress rather than a percent. See emscripten-core#25752, which implements similar thing in the parallel_runner.
This test runner does a few things differ the base TextTestRunner: 1. It improves the behviour of `--buffer` by also buffering/redirecting logging output that occurs during the test run. 2. It displays all results on a single line, each result erasing the contents of the line before re-drawing it. 3. It uses ANSI colors to the show the results. 4. It should the progress as each results is displayed so its easy to see how far you are through the test suite "[XX/YY]" I also updated parallel_testsuite.py use the same "XX/YY" progress rather than a percent. See emscripten-core#25752, which implements similar thing in the parallel_runner.
This test runner does a few things differ the base TextTestRunner: 1. It improves the behviour of `--buffer` by also buffering/redirecting logging output that occurs during the test run. 2. It displays all results on a single line, each result erasing the contents of the line before re-drawing it. 3. It uses ANSI colors to the show the results. 4. It should the progress as each results is displayed so its easy to see how far you are through the test suite "[XX/YY]" I also updated parallel_testsuite.py use the same "XX/YY" progress rather than a percent. See emscripten-core#25752, which implements similar thing in the parallel_runner.
2 participants
Like many modern tests runners we can output all results on a single line a by re-writing the some line on the TTY.
We only do this under certain circumstances:
--bufferoptions us used (test output hidden unless fail)This mode is still not the default yet since
--bufferis not yet the default, but the output is so much nicer I think we should do that at some point.