Improve Compression from Generator #59
Conversation
I'm a big fan of the Code Generator tool, but I noticed the all-in-one pages serialize the game as a JSON array, which can get kind of chunky. This PR swaps the JSON for a base64 string, bringing a `53.35mb` .html down to `21.85mb`. This should help with storage and load-times. Zipping the .html afterwards brings the whole thing down to within 10% of the original game size. bytes <-> base64 conversions adopted from examples on this page: https://developer.mozilla.org/en-US/docs/Web/API/Window/btoa
This PR adds compression to the generated .html files using the browser's built-in gzip compression/decompression routines. This brings what would have been a `53.35mb` .html on master (or a `21.85mb` .html with EmulatorJS#58 ) down to `16.32mb`. This further compresses down to `12mb` if zipped. Due to the async call needed for this, I made this a separate PR since the structure of the generated .html has changed to only load the emulator after the game data has been decompressed. This is incompatible with the non-single-file-packing code, so it has been removed.
✅ Deploy Preview for emulatorjs-org ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
| async function CompressBlob(blob) { | ||
| const cs = new CompressionStream("gzip"); | ||
| const compressedStream = blob.stream().pipeThrough(cs); | ||
| return await new Response(compressedStream).blob(); | ||
| } |
There was a problem hiding this comment.
Compression using CompressionStream should fallback if its unsupported, since this api is still relatively new
There was a problem hiding this comment.
https://developer.mozilla.org/en-US/docs/Web/API/Compression_Streams_API#browser_compatibility
It looks like most browsers got it in 2020, and the stragglers got it in 2023... so I think the odds are pretty low...
Since it's also used on the decompression side... should the baked .html outputs optionally import pako if it can't find Compression/Decompression Streams?
There was a problem hiding this comment.
I think the odds are pretty low
You'd be surprised :). I had several issues a while back (a few months ago) when I broke support for a GPU that hasn't had updates for 5 years so it's definitely needed.
I would rather not pull in dependencies with the output code, though if you relied on zip compression then EmulatorJS does have the ability to detect and decompress that
There was a problem hiding this comment.
Hmm, I can switch the compression for deflate, and check to see if EmulatorJS can handle that directly... is there anything I need to do to let it know that it's compressed? Perhaps a MIME type in the blob URL?
There was a problem hiding this comment.
Bah, CompressionStreams can natively produce .gz files, but not .zip files (which can be understood by EmulatorJS).
Luckily, it turns out there's a way to make JSZip actually run compression 😅 https://stackoverflow.com/a/41614764
Hopefully it's not too much slower... May want to try littlezipper if it is...
So perhaps I'll discard this PR in exchange for using the existing libraries...
2 participants
This PR adds compression to the generated .html files using the browser's built-in
gzipcompression/decompression routines.This brings what would have been a
53.35mb.html on main (or a21.85mb.html with #58 ) down to16.32mb. This further compresses down to12mbif zipped.Due to the async call needed for this, I made this a separate PR from #58 since the structure of the generated .html has changed to only load the emulator after the game data has been decompressed. This is incompatible with the non-single-file-packing code, so it has been removed.
@allancoding