38% of data teams spend between 20% and 40% of their time fixing data pipelines¹. Combating these data failures is a costly and stressful activity for those looking to deliver reliable data to end users. Organizations using Azure Data Factory can now benefit from the integration with Kensu to expedite this process. Their data teams can now observe data within their Azure Data Factory pipelines and receive valuable insights into data lineage, schema changes, and performance metrics.

If you ever tried to go global, you have probably faced a reality check. A whole new set of issues starts to appear when you start to operate a workload over multiple locations across the globe: So it looks like a great idea in theory, but in practice, all of this complexity multiplies the number of failure scenarios to consider!

Learn key strategies for migrating legacy systems to cloud-based platforms to modernize.

We previously explored how we built our own Serverless Engine and a multi-region networking layer based on Nomad, Firecracker, and Kuma. But what about the architecture of the engine that orchestrates these components across the world? This is an interesting topic to work on and we thought it could be useful to share some internals out there. Put on your scuba equipment, this is a deep dive into our architecture and the story of how we built our own global deployment engine.