Use psrecord and Plotly.js to profile a Mantid workflow.
It monitors CPU and RAM usage, and reports the time span of each algorithm (currently, only algorithms are recorded).
direnv
automatically activates the pixi environment as soon as you enter the root directory of the repository,
by parsing an .envrc file that you create there.
We recommend the following contents for the .envrc file:
watch_file pixi.lock # invokes the shell-hook whenever pixi.lock changes
eval "$(pixi shell-hook --change-ps1 false)"
eval "$(register-python-argcomplete mantidprofiler)" # register argcomplete
To profile the SNSPowderReduction.py workflow:
python SNSPowderReduction.py & mantidprofiler $!
Executable mantidprofiler is available in the pixi environment, but sometimes
you need to run the script in a different environment, one supporting Mantid.
For the "mantid-developer" conda environment:
conda activate mantid-developer
(mantid-developer) python SNSPowderReduction.py & pixi run --manifest-path /path/to/repository/mantid-profiler/pyproject.toml mantidprofiler $!
For the pixi environment:
pixi run --manifest-path /path/to/repo/mantid/pixi.toml SNSPowderReduction.py & pixi run --manifest-path /path/to/repo/mantid-profiler/pyproject.toml mantidprofiler $!
The script attaches to the last spawned process, so you can also use the profiler if you are working with MantidPlot:
./MantidPlot & mantidprofiler $!
psutil- You need to build Mantid with the
-DPROFILE_ALGORITHM_LINUX=ONCMakeflag to get the timing output from the algorithms.
After running on the SNSPowderReduction.py workflow, the profiler produces a profile.html file to be viewed with an internet browser.
You can interact with a demo profile here.
Controls:
- Mouse wheel to zoom (horizontal zoom only)
- Left click to select zoom region (horizontal zoom only)
- Double-click to reset axes
- Hold shift and mouse click to pan
-h,--helpshow this help message and exit--outfile OUTFILEname of output html file (default:profile.html)--infile INFILEname of input file containing algorithm timings (default:algotimeregister.out)--logfile LOGFILEname of output file containing process monitor data (default:mantidprofile.txt)--diskfile DISKFILEname of output file containing process disk usage data (default:mantiddisk.txt)--interval INTERVALhow long to wait between each sample (in seconds). By default the process is sampled as often as possible. (default: None)--nocleanremove files upon successful completion (default: False)--height HEIGHTheight for html plot (default: 800)--bytesReport disk speed in GBps rather than Gbps (default: False)--mintime MINTIMEminimum duration for an algorithm to appear inthe profiling graph (in seconds). (default: 0.1)
The version number for releases is stored in pyproject.toml and everything else reads this information.
To change the version number for a release, either edit the file by hand or pixi project version minor to bump the minor version number.
viztracer creates similar information for generic python software
- Mantid's AlgoTimeRegister explains the under-the-hood of the timing mechanism.
- Mantid developer profiler documentation provides additional information as to how it works with mantid.
- Mantid User Properties File has some fields to be noted.
Neil Vaytet, European Spallation Source
