Documentation for how to use IPC RecordBatch compression

[kylebarron]

I think I understand this after reading through the code, but it would be great to have some documentation included in the repo about how to use this feature. Documentation is not a strength of Arrow JS.

• If you call registerCompressionCodecs and no codecs have been set, then it'll automatically register lz4js and fetch zstd-codec? That might be unexpected, especially since zstd-codec is probably pretty large, right? To turn that off, you have to set the compressionRegistry for ZSTD to anything other than null? And then if set to something "fake", you just have to make sure you don't try to load ZSTD files?
• lz4js is always bundled with arrow-js now it looks like? So there's no penalty to always registering lz4 compression. But there is a penalty to registering zstd compression because that'll trigger the fetch of the codec, right?
• Oh wait, that registerCompressionCodecs is just in the tests..? So users always have to copy that implementation? That's fine, just trying to understand.

Originally posted by @kylebarron in #14 (comment)

[Djjanks]

There are no default codecs automatically registered. Users have to explicitly register their own compression codecs if they need.

Regarding the current implementation:

1. Codec Registration: Users need to manually register compression codecs using compressionRegistry.set(). The code you saw in tests (registerCompressionCodecs) is indeed just for testing purposes and serves as an example that users can adapt.

2. Error Handling Logic:

• When users register a codec with encoder, we validate it by checking the magic number, message length, and structure compressed data
• Errors will only be thrown when users attempt to:
  • Write compressed data without a registered codec
  • Read compressed data without the appropriate codec available

[kylebarron]

Can we add this documentation to the docstring of the IPC reading functions?