V0.10.1 rdkafka-ruby with librdkafka along for the ride

.gitignore

@@ -1,6 +1,3 @@
-# Ignore bundler config.
-/.bundle
-
 Gemfile.lock
 ext/ports
 ext/tmp
@@ -9,6 +6,3 @@ ext/librdkafka.*
 .yardoc
 doc
 coverage
-vendor
-.idea/
-out/

CHANGELOG.md

@@ -1,198 +1,99 @@
-# Rdkafka Changelog
-
-## 0.17.0 (Unreleased)
-- [Enhancement] Update `librdkafka` to `2.4.0`
-- [Feature] Add `#seek_by` to be able to seek for a message by topic, partition and offset (zinahia)
-- [Fix] Switch to local release of librdkafka to mitigate its unavailability.
-
-## 0.16.1 (2024-07-10)
-- [Fix] Switch to local release of librdkafka to mitigate its unavailability.
-
-## 0.16.0 (2024-06-13)
-- **[Breaking]** Retire support for Ruby 2.7.
-- **[Breaking]** Messages without headers returned by `#poll` contain frozen empty hash.
-- **[Breaking]** `HashWithSymbolKeysTreatedLikeStrings` has been removed so headers are regular hashes with string keys.
-- **[Feature]** Support incremental config describe + alter API.
-- **[Feature]** Oauthbearer token refresh callback (bruce-szalwinski-he)
-- **[Feature]** Provide ability to use topic config on a producer for custom behaviors per dispatch.
-- [Enhancement] Use topic config reference cache for messages production to prevent topic objects allocation with each message.
-- [Enhancement] Provide `Rrdkafka::Admin#describe_errors` to get errors descriptions (mensfeld)
-- [Enhancement] Replace time poll based wait engine with an event based to improve response times on blocking operations and wait (nijikon + mensfeld)
-- [Enhancement] Allow for usage of the second regex engine of librdkafka by setting `RDKAFKA_DISABLE_REGEX_EXT` during build (mensfeld)
-- [Enhancement] name polling Thread as `rdkafka.native_kafka#<name>` (nijikon)
-- [Enhancement] Save two objects on message produced and lower CPU usage on message produced with small improvements.
-- [Change] Allow for native kafka thread operations deferring and manual start for consumer, producer and admin.
-- [Change] The `wait_timeout` argument in `AbstractHandle.wait` method is deprecated and will be removed in future versions without replacement. We don't rely on it's value anymore (nijikon)
-- [Fix] Background logger stops working after forking causing memory leaks (mensfeld)
-- [Fix] Fix bogus case/when syntax. Levels 1, 2, and 6 previously defaulted to UNKNOWN (jjowdy)
-
-## 0.15.2 (2024-07-10)
-- [Fix] Switch to local release of librdkafka to mitigate its unavailability.
-
-## 0.15.1 (2024-01-30)
-- [Enhancement] Provide support for Nix OS (alexandriainfantino)
-- [Enhancement] Replace `rd_kafka_offset_store` with `rd_kafka_offsets_store` (mensfeld)
-- [Enhancement] Alias `topic_name` as `topic` in the delivery report (mensfeld)
-- [Enhancement] Provide `label` producer handler and report reference for improved traceability (mensfeld)
-- [Enhancement] Include the error when invoking `create_result` on producer handle (mensfeld)
-- [Enhancement] Skip intermediate array creation on delivery report callback execution (one per message) (mensfeld).
-- [Enhancement] Report `-1` instead of `nil` in case `partition_count` failure (mensfeld).
-- [Fix] Fix return type on `#rd_kafka_poll` (mensfeld)
-- [Fix] `uint8_t` does not exist on Apple Silicon (mensfeld)
-- [Fix] Missing ACL `RD_KAFKA_RESOURCE_BROKER` constant reference (mensfeld)
-- [Fix] Partition cache caches invalid nil result for `PARTITIONS_COUNT_TTL` (mensfeld)
-- [Change] Rename `matching_acl_pattern_type` to `matching_acl_resource_pattern_type` to align the whole API (mensfeld)
-
-## 0.15.0 (2023-12-03)
-- **[Feature]** Add `Admin#metadata` (mensfeld)
-- **[Feature]** Add `Admin#create_partitions` (mensfeld)
-- **[Feature]** Add `Admin#delete_group` utility (piotaixr)
-- **[Feature]** Add Create and Delete ACL Feature To Admin Functions (vgnanasekaran)
-- **[Feature]** Support `#assignment_lost?` on a consumer to check for involuntary assignment revocation (mensfeld)
-- [Enhancement] Expose alternative way of managing consumer events via a separate queue (mensfeld)
-- [Enhancement] **Bump** librdkafka to 2.3.0 (mensfeld)
-- [Enhancement] Increase the `#lag` and `#query_watermark_offsets` default timeouts from 100ms to 1000ms. This will compensate for network glitches and remote clusters operations (mensfeld)
-- [Change] Use `SecureRandom.uuid` instead of `random` for test consumer groups (mensfeld)
-
-## 0.14.1 (2024-07-10)
+# 0.10.1
 - [Fix] Switch to local release of librdkafka to mitigate its unavailability.
 
-## 0.14.0 (2023-11-21)
-- [Enhancement] Add `raise_response_error` flag to the `Rdkafka::AbstractHandle`.
-- [Enhancement] Allow for setting `statistics_callback` as nil to reset predefined settings configured by a different gem (mensfeld)
-- [Enhancement] Get consumer position (thijsc & mensfeld)
-- [Enhancement] Provide `#purge` to remove any outstanding requests from the producer (mensfeld)
-- [Enhancement] Update `librdkafka` to `2.2.0` (mensfeld)
-- [Enhancement] Introduce producer partitions count metadata cache (mensfeld)
-- [Enhancement] Increase metadata timeout request from `250 ms` to `2000 ms` default to allow for remote cluster operations via `rdkafka-ruby` (mensfeld)
-- [Enhancement] Introduce `#name` for producers and consumers (mensfeld)
-- [Enhancement] Include backtrace in non-raised binded errors (mensfeld)
-- [Fix] Reference to Opaque is not released when Admin, Consumer or Producer is closed (mensfeld)
-- [Fix] Trigger `#poll` on native kafka creation to handle oauthbearer cb (mensfeld)
-- [Fix] `#flush` does not handle the timeouts errors by making it return `true` if all flushed or `false` if failed. We do **not** raise an exception here to keep it backwards compatible (mensfeld)
-- [Change] Remove support for Ruby 2.6 due to it being EOL and WeakMap incompatibilities (mensfeld)
-- [Change] Update Kafka Docker with Confluent KRaft (mensfeld)
-- [Change] Update librdkafka repo reference from edenhill to confluentinc (mensfeld)
-
-## 0.13.0 (2023-07-24)
-- Support cooperative sticky partition assignment in the rebalance callback (methodmissing)
-- Support both string and symbol header keys (ColinDKelley)
-- Handle tombstone messages properly (kgalieva)
-- Add topic name to delivery report (maeve)
-- Allow string partitioner config (mollyegibson)
-- Fix documented type for DeliveryReport#error (jimmydo)
-- Bump librdkafka to 2.0.2 (lmaia)
-- Use finalizers to cleanly exit producer and admin (thijsc)
-- Lock access to the native kafka client (thijsc)
-- Fix potential race condition in multi-threaded producer (mensfeld)
-- Fix leaking FFI resources in specs (mensfeld)
-- Improve specs stability (mensfeld)
-- Make metadata request timeout configurable (mensfeld)
-- call_on_partitions_assigned and call_on_partitions_revoked only get a tpl passed in (thijsc)
-
-## 0.12.0 (2022-06-17)
-- Bumps librdkafka to 1.9.0
-- Fix crash on empty partition key (mensfeld)
-- Pass the delivery handle to the callback (gvisokinskas)
-
-## 0.11.0 (2021-11-17)
-- Upgrade librdkafka to 1.8.2
-- Bump supported minimum Ruby version to 2.6
-- Better homebrew path detection
-
-## 0.10.0 (2021-09-07)
-- Upgrade librdkafka to 1.5.0
-- Add error callback config
-
-## 0.9.0 (2021-06-23)
-- Fixes for Ruby 3.0
-- Allow any callable object for callbacks (gremerritt)
-- Reduce memory allocations in Rdkafka::Producer#produce (jturkel)
-- Use queue as log callback to avoid unsafe calls from trap context (breunigs)
-- Allow passing in topic configuration on create_topic (dezka)
-- Add each_batch method to consumer (mgrosso)
-
-## 0.8.1 (2020-12-07)
-- Fix topic_flag behaviour and add tests for Metadata (geoff2k)
-- Add topic admin interface (geoff2k)
-- Raise an exception if @native_kafka is nil (geoff2k)
-- Option to use zstd compression (jasonmartens)
-
-## 0.8.0 (2020-06-02)
-- Upgrade librdkafka to 1.4.0
-- Integrate librdkafka metadata API and add partition_key (by Adithya-copart)
-- Ruby 2.7 compatibility fix (by Geoff Thé)A
-- Add error to delivery report (by Alex Stanovsky)
-- Don't override CPPFLAGS and LDFLAGS if already set on Mac (by Hiroshi Hatake)
-- Allow use of Rake 13.x and up (by Tomasz Pajor)
-
-## 0.7.0 (2019-09-21)
-- Bump librdkafka to 1.2.0 (by rob-as)
-- Allow customizing the wait time for delivery report availability (by mensfeld)
-
-## 0.6.0 (2019-07-23)
-- Bump librdkafka to 1.1.0 (by Chris Gaffney)
-- Implement seek (by breunigs)
-
-## 0.5.0 (2019-04-11)
-- Bump librdkafka to 1.0.0 (by breunigs)
-- Add cluster and member information (by dmexe)
-- Support message headers for consumer & producer (by dmexe)
-- Add consumer rebalance listener (by dmexe)
-- Implement pause/resume partitions (by dmexe)
-
-## 0.4.2 (2019-01-12)
-- Delivery callback for producer
-- Document list param of commit method
-- Use default Homebrew openssl location if present
-- Consumer lag handles empty topics
-- End iteration in consumer when it is closed
-- Add support for storing message offsets
-- Add missing runtime dependency to rake
-
-## 0.4.1 (2018-10-19)
-- Bump librdkafka to 0.11.6
-
-## 0.4.0 (2018-09-24)
-- Improvements in librdkafka archive download
-- Add global statistics callback
-- Use Time for timestamps, potentially breaking change if you
+# 0.10.0
+* Upgrade librdkafka to 1.5.0
+* Add error callback config
+
+# 0.9.0
+* Fixes for Ruby 3.0
+* Allow any callable object for callbacks (gremerritt)
+* Reduce memory allocations in Rdkafka::Producer#produce (jturkel)
+* Use queue as log callback to avoid unsafe calls from trap context (breunigs)
+* Allow passing in topic configuration on create_topic (dezka)
+* Add each_batch method to consumer (mgrosso)
+
+# 0.8.1
+* Fix topic_flag behaviour and add tests for Metadata (geoff2k)
+* Add topic admin interface (geoff2k)
+* Raise an exception if @native_kafka is nil (geoff2k)
+* Option to use zstd compression (jasonmartens)
+
+# 0.8.0
+* Upgrade librdkafka to 1.4.0
+* Integrate librdkafka metadata API and add partition_key (by Adithya-copart)
+* Ruby 2.7 compatibility fix (by Geoff Thé)A
+* Add error to delivery report (by Alex Stanovsky)
+* Don't override CPPFLAGS and LDFLAGS if already set on Mac (by Hiroshi Hatake)
+* Allow use of Rake 13.x and up (by Tomasz Pajor)
+
+# 0.7.0
+* Bump librdkafka to 1.2.0 (by rob-as)
+* Allow customizing the wait time for delivery report availability (by mensfeld)
+
+# 0.6.0
+* Bump librdkafka to 1.1.0 (by Chris Gaffney)
+* Implement seek (by breunigs)
+
+# 0.5.0
+* Bump librdkafka to 1.0.0 (by breunigs)
+* Add cluster and member information (by dmexe)
+* Support message headers for consumer & producer (by dmexe)
+* Add consumer rebalance listener (by dmexe)
+* Implement pause/resume partitions (by dmexe)
+
+# 0.4.2
+* Delivery callback for producer
+* Document list param of commit method
+* Use default Homebrew openssl location if present
+* Consumer lag handles empty topics
+* End iteration in consumer when it is closed
+* Add suport for storing message offsets
+* Add missing runtime dependency to rake
+
+# 0.4.1
+* Bump librdkafka to 0.11.6
+
+# 0.4.0
+* Improvements in librdkafka archive download
+* Add global statistics callback
+* Use Time for timestamps, potentially breaking change if you
   rely on the previous behavior where it returns an integer with
   the number of milliseconds.
-- Bump librdkafka to 0.11.5
-- Implement TopicPartitionList in Ruby so we don't have to keep
+* Bump librdkafka to 0.11.5
+* Implement TopicPartitionList in Ruby so we don't have to keep
   track of native objects.
-- Support committing a topic partition list
-- Add consumer assignment method
+* Support committing a topic partition list
+* Add consumer assignment method
 
-## 0.3.5 (2018-01-17)
-- Fix crash when not waiting for delivery handles
-- Run specs on Ruby 2.5
+# 0.3.5
+* Fix crash when not waiting for delivery handles
+* Run specs on Ruby 2.5
 
-## 0.3.4 (2017-12-05)
-- Bump librdkafka to 0.11.3
+# 0.3.4
+* Bump librdkafka to 0.11.3
 
-## 0.3.3 (2017-10-27)
-- Fix bug that prevent display of `RdkafkaError` message
+# 0.3.3
+* Fix bug that prevent display of `RdkafkaError` message
 
-## 0.3.2 (2017-10-25)
-- `add_topic` now supports using a partition count
-- Add way to make errors clearer with an extra message
-- Show topics in subscribe error message
-- Show partition and topic in query watermark offsets error message
+# 0.3.2
+* `add_topic` now supports using a partition count
+* Add way to make errors clearer with an extra message
+* Show topics in subscribe error message
+* Show partition and topic in query watermark offsets error message
 
-## 0.3.1 (2017-10-23)
-- Bump librdkafka to 0.11.1
-- Officially support ranges in `add_topic` for topic partition list.
-- Add consumer lag calculator
+# 0.3.1
+* Bump librdkafka to 0.11.1
+* Officially support ranges in `add_topic` for topic partition list.
+* Add consumer lag calculator
 
-## 0.3.0 (2017-10-17)
-- Move both add topic methods to one `add_topic` in `TopicPartitionList`
-- Add committed offsets to consumer
-- Add query watermark offset to consumer
+# 0.3.0
+* Move both add topic methods to one `add_topic` in `TopicPartitionList`
+* Add committed offsets to consumer
+* Add query watermark offset to consumer
 
-## 0.2.0 (2017-10-13)
-- Some refactoring and add inline documentation
+# 0.2.0
+* Some refactoring and add inline documentation
 
-## 0.1.x (2017-09-10)
-- Initial working version including producing and consuming
+# 0.1.x
+* Initial working version including producing and consuming

Gemfile

@@ -1,5 +1,3 @@
-# frozen_string_literal: true
-
 source "https://rubygems.org"
 
 gemspec

MIT-LICENSE → LICENSE

@@ -1,7 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2017-2023 Thijs Cadier
-              2023, Maciej Mensfeld
+Copyright (c) 2017 Thijs Cadier
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal