AutoPrimenet Performance

[jcoxdco]

I'd like to take a run at a performance optimization by hoisting a few regex patterns up to the module level and trying to cut down on per line regex work by adding string guard or different if/then logic if you haven't tried that already.

If so, I'd like to know if you have a few gnarly mlucas and gpuowl log files I could use in testing. I don't have any left on my machine.

[tdulcet]

Yes, that code for parsing the log files likely could be significantly optimized. Thanks for the offer. I have previously made minor optimizations, such as compiling the regexs, but I have not done any experimentation or benchmarked the code. Likely the biggest performance win would be reading the log files starting from the end (like the Linux tail command), as we typically only need the last few lines, but Python does not support this natively and would be tricky to implement correctly and performantly. The GpuOwl code is also somewhat tricky, since it is targeting the GpuOwl v6 branch as well as all of the later 7.x versions, some of which used different log file formats.

Anyway, I attached a ZIP file with several log files. The GpuOwl log files are what I originally used when I wrote the parsing code. The Mlucas logs files are from the last month or so on my production system. Mlucas uses a different log file for each exponent, so they do not get too gnarly unless one restarts a test multiple times or there is a rollback due to a fatal roundoff or GEC error.

GIMPS log files.zip

If you need more log files, my Linux install scripts should make it easy to get setup with Mlucas and/or GpuOwl. If you are interested in adding support for a new GIMPS program, AutoPrimeNet does not currently support assignment progress reporting for CUDAPm1, which would require parsing the CUDAPm1 log/output files.