In another life, I taught the Book of Genesis to high school students, including The Tower of Babel excerpt. It struck me ironic that God’s wrath strikes down the tower, cofounds the universal language and scatters humans around the globe to teach King Nimrod a lesson in hubris; meanwhile, the boys in my class were texting their girlfriends across the country and playing video games with friends in Europe and Asia.

Many in the community have been asking us to develop a new Kafka to S3 connector for some time. So we’re pleased to announce it's now available. It’s been designed to deliver a number of benefits over existing S3 connectors. Like our other Stream Reactors, the connector extends the standard connect config adding a parameter for a SQL command (Lenses Kafka Connect Query Language or “KCQL”). This defines how to map data from the source (in this case Kafka) to the target (S3).

In June, Snowflake announced the public preview of the external functions feature with support for calling external APIs via AWS API Gateway. With external functions, you can easily extend your data pipelines by calling out to external services, third-party libraries, or even your own custom logic, enabling exciting new use cases. For example, you can use external functions for external tokenization, geocoding, scoring data using pre-trained machine learning models, and much more.

Two of the most useful tools for making software development run smoothly are CI & CD. Over the years our stack has evolved from a humble Jenkins box into a cloud-native platform. In this post, we’ll share our learnings from this journey.

React Hooks is a new addition to React which enables you to use state and other features of the library without having to create a class. By ‘hooking into’ React’s existing features, you can significantly reduce the number of concepts you have to grapple with, and you can create custom hooks to share all kinds of non-visual logic, making your code much more reusable.