I stated a benchdiff run, and then realized I had gotten it wrong and stopped it with ^C.
When I reran it, it said "skipping benchmark for ref "HEAD^" because output file exists" although the file in the cache was just a couple lines from the first benchmark.
Instead, the output should go to a temporary file and only get moved to the cache upon success.