Apache Kafka® 3.6 is here! On behalf of the Kafka community, Danica Fine highlights key release updates, with KIPs from Kafka Core, Kafka Streams, and Kafka Connect.

Check out the following blog for more information about the latest Apache Kafka release: https://cnfl.io/apache-kafka-3-6-blog

Kafka Core:

• KIP-405 introduces tiered storage to Kafka. It allows for seamless extension of storage to remote objects with minimal operational changes. Note that this is an early access release and some limitations apply.
• KIP 890 provides a way to address hanging transactions by verifying partition additions.
• KIP 797 supports having both IPv4 and IPv6 on the same listener port. This change only applies to listeners, as advertised listeners already have this feature.
• KIP 902 updates the bundled Apache ZooKeeper version in Kafka to version 3.8.2, which includes several improvements and security fixes. This is the last planned update in ZooKeeper until its removal in version 4.0.
• KIP 868 improves overall durability of the KRaft metadata layer. Records can now be committed atomically to the KRaft layer in multiple batches.
• KIP 919 enables admin clients to communicate with controllers directly to more easily perform administrative operations or troubleshoot issues related to metadata.
• KIP 937 improves the validation logic for message timestamps by rejecting messages with future timestamps and providing a descriptive exception. It also helps improve data integrity and prevent potential pitfalls caused by inaccurate timestamp handling.
• KIP 726 makes the “cooperative-sticky, range” as the new default assignor. Existing applications that don’t set an assignor can safely upgrade using a normal rolling bounce, and will automatically transition to cooperative rebalancing.

Kafka Streams:

• KIP 923 adds a grace period to the stream-table join. It improves table-side out-of-order data handling, and delays processing stream-record (i.e., delayed table-lookup)
• KIP 925 adds rack-aware assignment of active tasks.
• KIP 941 allows to pass `null` into `withRange(...)` for `lower/upper` bounds to specify a full or half-open range.

Kafka Connect:

• KIP 898 modernizes Connect plugin discovery. Connect workers now read from ServiceLoader manifests and module info directly during startup for more efficient plugin class discovery.

EPISODE LINKS
► See release notes for AK 3.6: https://archive.apache.org/dist/kafka/3.6.0/RELEASE_NOTES.html
► Download AK 3.6: https://kafka.apache.org/downloads
► Read the blog: https://cnfl.io/apache-kafka-3-6-blog
► Get started with AK 3.6: https://kafka.apache.org/documentation/#quickstart

CHAPTERS

00:00 - Intro

00:22 - KIP 405: Kafka Tiered Storage (Early Access)

01:15 - KIP 890: Transactions Server Side Defense (Part 1)

01:33 - KIP 797: Accept duplicate listener on port for IPv4/IPv6

01:55 - KIP 902: Upgrade Zookeeper to 3.8.X

02:17 - KIP 868: Metadata Transactions

02:54 - KIP 919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration

03:33 - KIP 937: Improve Message Timestamp Validation

03:58 - KIP 726: Make the "cooperative-sticky, range" as the default assignor

04:35 - KIP 923: Add a grace-period to stream-table join

05:00 - KIP 925: Add rack-aware task assignment

05:31 - KIP 941: Accept `null` for range-query bounds

06:00 - KIP 898: Modernize Connect Plugin Discovery

ABOUT CONFLUENT
Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion – designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io.

#apachekafka #kafka #confluent