Tamp v2 Discussion #243
Description
Open notes about upcoming Tamp v2:
- Tamp V2 will be able to read Tamp V1 compressed files.
- Tamp V2 has a bit set in the header that will cause Tamp V1 decompressors to error out during initialization (good).
- Tamp V2 will have a little bit higher compression ratios by 2 main methods: Extended-Pattern-Matching, and Run-Length-Encoding. For typical messages without long patterns or runs, compression will be the same as Tamp V1.
- Memory usage/execution speed should remain about the same.
Benchmarks:
| file | raw | tamp | tamp v2 | zlib | heatshrink |
|---|---|---|---|---|---|
| enwik8 | 100,000,000 | 51,635,633 | 51,060,117 | 56,205,166 | 56,110,394 |
| build/silesia/dickens | 10,192,446 | 5,546,761 | 5,538,308 | 6,049,169 | 6,155,768 |
| build/silesia/mozilla | 51,220,480 | 25,121,385 | 24,424,111 | 25,104,966 | 25,435,908 |
| build/silesia/mr | 9,970,564 | 5,027,032 | 4,516,210 | 4,864,734 | 5,442,180 |
| build/silesia/nci | 33,553,445 | 8,643,610 | 6,880,294 | 5,765,521 | 8,247,487 |
| build/silesia/ooffice | 6,152,192 | 3,814,938 | 3,775,766 | 4,077,277 | 3,994,589 |
| build/silesia/osdb | 10,085,684 | 8,520,835 | 8,466,376 | 8,625,159 | 8,747,527 |
| build/silesia/reymont | 6,627,202 | 2,847,981 | 2,821,601 | 2,897,661 | 2,910,251 |
| build/silesia/samba | 21,606,400 | 9,102,594 | 8,390,175 | 8,862,423 | 9,223,827 |
| build/silesia/sao | 7,251,944 | 6,137,755 | 6,136,095 | 6,506,417 | 6,400,926 |
| build/silesia/webster | 41,458,703 | 18,694,172 | 18,148,495 | 20,212,235 | 19,942,817 |
| build/silesia/x-ray | 8,474,240 | 7,510,606 | 7,508,813 | 7,351,750 | 8,059,723 |
| build/silesia/xml | 5,345,280 | 1,681,687 | 1,471,884 | 1,586,985 | 1,665,179 |
| build/RPI_PICO-20250415-v1.25.0.uf2 | 667,648 | 331,310 | 290,008 | 303,763 | 338,685 |
I'm behind a bit on projects, but currently i have the pure python (prototype) implementation done. I need to write the C implementation before finalizing anything (sometimes we can make small compression tradeoffs for better computation/memory). I've been pretty busy with various projects/life, so no ETA on release as of right now.
Please post below if you have any thoughts/techniques/features to improve Tamp V2!