[FEATURE] Arrow Flight

[pixie79]

Have you considered adding an Arrow gRPC option in addition to the Proto gRPC transmission method. It would be good if Zerobus service could handle Arrow Flight as an IPC method as an alternative option to gRPC.

I think on many of the producer sides that would increase throughput options as many data products now offer Apache Arrow based Dataframes due to its standard and zerocopy methods. In theory it may also scale giving a better request per second transfer rate?

[zacdav-db]

I had the exact same idea, did a search, and found that you're just a day earlier!

This would be a killer feature.

[elenagaljak-db]

Hey, @pixie79, thank you for opening the first issue on the Zerobus Rust SDK!!

We really appreciate your suggestion. Support for Arrow Flight (in addition to the current Proto gRPC transmission) is something we’ve discussed internally a few times. It’s not currently on our roadmap, but it’s definitely something we’d like to explore in the future.

We’ll keep this issue open to track interest and future progress on it.

[benodiwal]

Hey @elenagaljak-db, I would to work on this issue, can i get assigned. Thanks

[teodordelibasic-db]

Hey @pixie79 @zacdav-db @benodiwal, this is close to be finished by our team, both from server and SDK side. On SDK side the newest release already has the implementation but it is off by default since it's still disabled on server-side. We have a bit more testing/polishing to do, then we will enable it on both sides by default.