Inside @WhatIfMediaGroup's Massive #Kafka Migration to #Kubernetes | Interview with Ryan Anguiano
In this episode, Drew Oetzel sits down with Ryan Anguiano, Staff Architect at @WhatIfMediaGroup to discuss their massive migration of #apachekafka from legacy EC2 instances to #kubernetes using the @Strimzi operator.
Ryan shares deep technical insights into how they optimized their data streaming architecture, including their use of #aws EKS, EBS storage striping, and why the 12-Factor App methodology was the key to migrating over 100 services in just a few months.
Key topics covered:
- Transitioning from manual EC2/Ansible/Terraform provisioning to Kubernetes.
- Why Strimzi is the go-to operator for Kafka on K8s.
- Storage optimization: Configuring multiple EBS volumes per node for high performance.
- Implementing the 12-Factor App methodology for cloud-native success.
- How Lenses provides an easy on-ramp for developers to visualize Kafka data and consumer lag.
- Avoiding migration pitfalls: The danger of hidden legacy systems.
#Kafka #Kubernetes #Strimzi #AWS #DataStreaming #SoftwareArchitecture #12FactorApp #Helm #Lenses #WhatIfMedia
00:00 - Intro: Managing Disparate Tech Stacks
01:07 - Welcome to the Lenses Pod with Ryan Aguiano
01:40 - The Role of a Systems Architect at What If Media
02:34 - Migrating Kafka from EC2 to Kubernetes
03:21 - Why Strimzi? Benefits of Open-Source Kafka Operators
04:52 - Handling High Volume Data & Compaction in Kafka
06:05 - Storage Optimization: Multiple EBS Volumes on EKS
07:28 - Empowering Developers with Lenses & Async Communication
09:23 - The 'One Helm Chart to Rule Them All' Project
13:36 - The Power of the 12-Factor App Methodology
15:15 - Production Config & Local Development Best Practices
16:26 - Migration Pitfalls: The Mystery of Hidden Legacy Systems
19:00 - Career Journey: From Flash & PHP to Systems Architecture
20:30 - Beyond Tech: Music, Italy, and Retro Gaming