Buy 10 sandwiches, get 1 free. Classic punch cards (and fishing for them in your wallet or occasionally misplacing one) have become a thing of the past, as today's digital landscape demands more innovative solutions. Today’s customer loyalty programs are increasingly sophisticated—evolving, proliferating, and diversifying across every industry from retail, travel, and hospitality to healthcare (e.g., a discount for paying within 30 days of a hospital visit).

In part one of this series, we walked through how to use Streamlit, Apache Kafka, and Apache Flink to create a live data-driven user interface for a market data application to select a stock (e.g., SPY) and discussed the structure of the app at a high level. First, data with information on stock bid prices is moved via an Alpaca websocket, then, it’s produced to a Kafka topic in Confluent Cloud where it is also processed with Flink SQL.

Businesses that are best able to leverage data have a significant competitive advantage. This is especially true in financial services, an industry in which leading organizations are in constant competition to develop the most responsive, personalized customer experiences. Often, however, legacy infrastructure, data silos, and batch systems introduce significant technical hurdles.

Market data analytics has always been a classic use case for Apache Kafka. However, new technologies have been developed since Kafka was born. Apache Flink has grown in popularity for stateful processing with low latency output. Streamlit, a popular open source component library and deployment platform, has emerged, providing a familiar Python framework for crafting powerful and interactive data visualizations. Acquired by Snowflake in 2022, Streamlit remains agnostic with respect to data sources.

The dual-write problem occurs when two external systems must be updated in an atomic fashion. A classic example is updating an application’s database while pushing an event into a messaging system like Apache Kafka. If the database update succeeds but the write to Kafka fails, the system ends up in an inconsistent state. However, the dual-write problem isn’t unique to event-driven systems or Kafka. It occurs in many situations involving different technologies and architectures.

The Data Streaming Awards is back for its third year! Designed to bring the data streaming community together, this one-of-a-kind industry award event recognizes organizations that are harnessing the power of this revolutionary technology to drive business and customer experience transformation. If you know a company (even your own team) that is using data streaming technology to transform their business and provide amazing value to their customers and communities, the time is now to submit a nomination.

Managing Confluent Cloud infrastructure efficiently poses challenges due to the complexities involved in deploying and maintaining various components like environments, clusters, topics, and authorizations. Without proper tooling and practices, teams struggle with manual configuration errors, lack of consistency, and potential security risks. The Confluent Terraform.

The Defense Information Systems Agency (DISA) launched an ambitious program with a name as intimidating as the effort to re-imagine the security architecture of the Department of Defense (DoD). Thunderdome is DISA’s zero trust network access and application security architecture.

In large organizations, Confluent Cloud is often simultaneously accessed by many different users along with business-critical applications, potentially across different lines of business. With so many individual pieces working together, the risk of an individual outage, error, or incident affecting other services increases. An incident could be constituted by a user clicking a wrong button, an application’s misconfiguration, or just a bug—you name it.