Long time to generate web pages

[pbitzer]

Depending on the logging frequency and the number log files, it can take a long time (~10 minutes) for Sindri to generate the website.

[CAM-Gerlach]

Mitigations to implement (from low-hanging fruit to more complex tuning)

• Add customizable limit to look back when rebuilding the overall data, and the CSVs the first run
• If not the first run, only rebuild CSVs that have been changed since last run
• Only update plot data, archive table, etc every n runs (default 5, i.,e. every 5 minutes)
• More efficiently filter old data filer earlier and more aggressively?

[CAM-Gerlach]

Right now, for 0.2.9 we can at least implement the first point, its basically making what you already did into a customizable setting

[CAM-Gerlach]

Part of this was implemented with #9 ; moving generation to the server in v0.3.0 should improve things further.