Skip to content

Conversation

@nmeum
Copy link

@nmeum nmeum commented Jan 17, 2023

This adds a Python module which is suitable for running the benchmarks within a simulator that doesn't not support a GDB stub and allows for a rough performance estimation of said simulator. The module is based on the existing run_native.py module and has the same deficiencies (i.e. inaccurate measurements through the use of the time(1) command). An alternative approach would be modifying the run_native.py module in away that a wrapper could be passed via an additional command-line flag (e.g. ./benchmark_speed.py --target-module run_native --wrapper 'my-fancy-simulator ...') the passed string would then also be inserted into the time(1) command as proposed here.

Usage example of the proposed run_simulator module:

$ ./benchmark_speed.py --target-module run_simulator --simulator 'my-simulator <additional args for the simulator>'

Fixes #106

nmeum added 2 commits January 17, 2023 20:59
Otherwise, this function does not handle multiple line output correctly.
For example, if a simulator writes text to standard error for some
reason.
@jeremybennett
Copy link
Collaborator

Hi @nmeum

Thanks for the PR. A useful addition. Could you update the documentation in doc/README.md. I suggest a new section on how to run using your run_simulator module instead of the approach detailed would be sufficient.

Thanks for the contribution.

@nmeum
Copy link
Author

nmeum commented Mar 10, 2023

Hi, good suggestion! I can also add documentation for run_simulator. Do you have any hints where specifically in doc/README.md I should add a section? Essentially, run_simulator is no different then the existing run scripts. Unfortunately, I wasn't able to find any detailed documentation for the existing run scripts in doc/README.md and hence I am unsure where to best add documentation for run_simulator.

I also just pushed an additional improvement to also use multiline regex search in decode_results when looking for the timing information. This is analog to 5d05d18 and necessary when the simulator writes data to stderr itself for some reason.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Queries on Embench benchmarks- How to config my simulator(target)

2 participants