Real-time Data Warehousing With Streaming Pipelines
See how to build a streaming data pipeline to a cloud data warehouse for real-time analytics. Leverage Confluent Data Streaming Platform to.
Systems | Development | Analytics | API | Testing
Modern Software Development For Today's Systems & Applications
How to Use Confluent for Kubernetes to Manage Resources Outside of Kubernetes
Apache Kafka® cluster administrators often need to solve problems like how to onboard new teams, manage resources like topics or connectors, and maintain permission control over these resources. In this post, we will demonstrate how to use Confluent for Kubernetes (CfK) to enable GitOps with a CI/CD pipeline and delegate resource creation to groups of people without distributing admin permission passwords to other people in the organization.
What is the Transactional Outbox Pattern? | Designing Event-Driven Microservices
The transactional outbox pattern leverages database transactions to update a microservice's state and an outbox table. Events in the outbox will be sent to an external messaging platform such as Apache Kafka. This technique is used to overcome the dual-write problem which occurs when you have to write data to two separate systems such as a database and Apache Kafka. The database transactions can be used to ensure atomic writes between the two tables. From there, a separate process can consume the outbox and update the external system as required.
How To Improve Data Quality with Domain Validation Rules | Data Quality Rules
Dive into streaming data quality with Gilles Philippart! Learn to keep bad data out with Confluent's Data Quality Rules.
What is the Dual Write Problem? | Designing Event-Driven Microservices
The dual write problem occurs when you try to write to two separate systems and need them to be atomic. If one write fails, and the other succeeds, you can end up with inconsistent state. This is an easy trap to fall into, and it can be difficult to avoid. We'll explore what causes the dual-write problem and explore both valid and invalid solutions to it.
Confluent's Customer Zero: Building a Real-Time Alerting System With Confluent Cloud and Slack
We talk a lot about how customers can use Confluent as the data backbone for event streaming applications and enable a new class of event-driven microservices by completely decoupling services from one another. With Confluent, organizations can rapidly build and deploy business applications with greater flexibility, support larger scale, and be more responsive to customer demands. But we don’t just talk about it, we do it ourselves as Confluent’s “Customer Zero”!
How To Build Scalable and Resilient Microservices | Microservices 101
Building scalable and resilient microservices requires an approach that eliminates the need to treat them as special. They should be treated as easily replaceable building blocks. This means eliminating bottlenecks and single points of failure but it can also mean changing from a pull-based approach to a push-based approach. CHAPTERS.
Point-to-Point vs Publish/Subscribe | Microservices 101
Communication between microservices can be broadly categorized as either point-to-point or publish/subscribe. Point-to-point is often used synchronously, while publish/subscribe tends to be asynchronous. Each of these techniques can have a place in a modern microservices platform, but it is important to understand the role each one plays so that they can be used effectively. CHAPTERS.
Extending the Confluent CLI With Custom Plugins
A good command line interface is essential for developer productivity. If you look at any of the major cloud providers, they all have a robust CLI API that enables you to achieve high productivity. The key benefits of a CLI include: Confluent offers a powerful CLI that lets you quickly create and manage Apache Kafka® clusters and Apache Flink® compute pools and all associated operations with both.
Getting Started with OAuth for Confluent Cloud Using Azure AD DS
Released in December 2022, OAuth support on Confluent Cloud allows Confluent Cloud users to integrate their own third-party identity provider (IdP) with Confluent Cloud, centralizing account management across all of their cloud services. This article explains how to configure Azure Active Directory DS (Azure AD DS) and Confluent Cloud so that the Azure Directory can be used to authenticate and authorize applications to use Confluent Cloud clusters.