Deploying Lenses.io for AWS MSK
Automate your deployment of Lenses.io for your #AWS Managed Streaming for #ApacheKafka (#MSK) directly into your AWS VPC through portal.lenses.io. You'll be practicing #DataOps in minutes.
Systems | Development | Analytics | API | Testing
Modern Software Development For Today's Systems & Applications
Explore Apache Kafka with a Lenses.io DataOps demo environment
Explore Apache Kafka & Lenses.io through a demo environment packed with sample data and data flows.
Kafka Office Hours: Load Testing your Apache Kafka Cluster
David Esposito, a Solutions Architect from #Aiven, explores load testing in Kafka Office Hours, a recurring forum for #ApacheKafka thought-sharing hosted by our partner Lenses.io.
On the importance of load testing Kafka
Socrates preached, “To know thyself is the beginning of wisdom.” This ancient Greek anecdote applies to your modern Apache Kafka project: developers, go forth and load test your real-time application to understand the capacity and limitations of your project before deployment. Failure to do so will cost you time and money (e.g. Robinhood’s outage on a historic trading day). Load testing your real-time applications has three main objectives.
Get your GitOps for real-time apps on Apache Kafka & Kubernetes
Infrastructure as code has been an important practice of DevOps for years. Anyone running an Apache Kafka data infrastructure and running on Kubernetes, the chances are you’ve probably nailed defining your infrastructure this way. If you’re running on Kubernetes, you’re likely using operators as part of your CI/CD toolchain to automate your deployments.
Explore & Process Protobuf Data in Apache Kafka
Walkthrough of how to create custom Serde classes that deserialise #Protobuf data in #ApacheKafka to explore and process in Lenses.io with #SQL.
Archive data from to S3 with the new Kafka Connect connector
The new open-source #ApacheKafka Connect sink connector for #S3 gives you full control on how to sink data to S3 and save money on long term storage costs in #Kafka. The connector has the ability to flush data out in a number of different formats including #AVRO, #JSON, #Parquet and #Binary as well as ability to create S3 buckets based on partitions, metadata fields and value fields.
Manage Kafka Connect connectors with SQL, Secret Management & RBAC
Demo of how to manage your #KafkaConnect connectors in Lenses.io with #SQL support for mapping fields between source & target systems, #RBAC and secret management to avoid exposing credentials in your configuration.
Monitoring & Data Operations for AWS MSK with Lenses.io
Walkthrough of getting visibility and self-service governance over your #AWS Apache Kafka Managed Streaming for Apache Kafka (#MSK) and AWS EKS data platform.
Introducing the Apache Kafka App Catalog
Working with Apache Kafka and real-time applications comes with challenges. Visibility into the deployed applications and their dependency on what we call the “data fabric” is one of them (For the sake of this blog, it means Kafka and all its state and configuration). If you’ve built a multi-tenant real-time data platform with Kafka, where teams are deploying applications outside your jurisdiction, this is where the pain is particularly acute. It goes something like this.