Learn how replication works in Apache Kafka®. Deep dive into its critical aspects, including:

• The challenges of distributed systems and replication
• Kafka's approach to replication and how it ensures data durability
• How to monitor replication and handle failures
• Practical experiments and configurations using Docker and Jupyter notebooks, featuring a demo available on GitHub

For more information and resources, checkout Confluent Developer: https://cnfl.io/3VYWxow

Whether you're a systems architect, developer, or just curious about Kafka, this video provides valuable insights and hands-on examples. Don't forget to check out our GitHub repo to get all of the code used in the demo, and to contribute your own enhancements.

RELATED RESOURCES
► GitHub repo to run the examples - https://github.com/confluentinc/learn-monitoring-troubleshooting-exercises/tree/main/02.replicas.in.sync
► Confluent Co-Founder Neha Narkhede’s blog on replication - https://cnfl.io/47WGJqE
► Apache Kafka documentation on Replication - https://kafka.apache.org/documentation/#replication
► Apache Kafka documentation on Metrics - https://kafka.apache.org/documentation/#monitoring
► Data Durability: https://cnfl.io/3XQ3JTO
► Confluent documentation on Replication - https://cnfl.io/3BxcESE
► Current 2023 Talk: Seek and Destroy: Kafka Under Replication - https://cnfl.io/3NcRkUZ
► Uber’s Kafka - https://www.uber.com/blog/kafka/

CHAPTERS

00:00 - Intro

00:37 - Why is Replication So Hard?

02:28 - Replication in Kafka

03:29 - In Sync Replicas (ISR)

04:46 - Producer Acks Setting

06:01 - Key Metrics for Replication

06:55 - Demo Setup

08:18 - Demo Broker Failure

11:05 - Transient Network Failures

13:52 - Troubleshooting Summary

—

ABOUT CONFLUENT Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion – designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit Confluent | Apache Kafka® Reinvented for the Cloud.

#streamprocessing #kafka #apachekafka #confluent