Systems | Development | Analytics | API | Testing

Welcome back to the third installment of our blog series where we’re diving into the beautiful black box that is Apache Kafka to better understand how we interact with the cluster through producer and consumer clients. Earlier in the series, we took a look at the Kafka producer to see how the client works before following a produce request as it’s processed by the cluster.

Throughout my career in enterprise technology, I've witnessed numerous transformations play out across the Asia-Pacific (APAC) region. But the shift we're seeing now with data streaming is truly unprecedented. What was once a supportive technology is rapidly becoming the very foundation of modern business in our region.

We are now in the final chapter of Apache Kafka’s multi-year journey to remove Apache ZooKeeper and fully transition to self-managed metadata in KRaft. Many Kafka users and customers are beginning to migrate to KRaft and are eager to understand its performance characteristics in production environments.

Alright, I’m back. Time for part 2. In the first part, I covered how we handle bad data in batch processing. In particular, cutting out the bad data, replacing it, and running it again. But this strategy doesn’t work for immutable event streams as they are, well, immutable. You can’t cut out and replace bad data like you would in batch processed data sets.

Imagine getting into your car to head to work on a hot day. Your car already knows and sets the temperature, the ambient lighting, and the music you prefer. Not only that, it optimizes your route, and with Level 3 autonomy, it can even drive you there. But what does the automotive industry have to do on the backend in order to achieve this kind of personalization?

After a short break, we’re back with Part 2 of this series on Spring Framework, Confluent Cloud, and the Kotlin language. Many organizations that write applications and microservices for the JVM have chosen Spring Framework, leveraging the many libraries available for features such as REST services, persisting data to a variety of datastores, and integration with messaging. These organizations have existing investments in building, testing, deploying, and monitoring applications using Spring.