Apache Spark is the leading technology for big data processing, on-premises and in the cloud. Spark powers advanced analytics, AI, machine learning, and more. Spark provides a unified infrastructure for all kinds of professionals to work together to achieve outstanding results.

Amazon EMR is growing in popularity, and is emerging as the leading platform for big data processing on AWS. EMR is the preferred platform for “lift and shift” migration of existing Hadoop and Spark workloads to the cloud, with minimal refactoring. You get better control, enhanced flexibility, and greater responsiveness.

In this session from Data+AI Summit 2021 (formerly Spark+AI Summit) Shivnath Babu, CTO/Adjunct Professor, Unravel Data Systems/Duke University presents, "Jeeves Grows Up: An AI Chatbot for Performance and Quality."

Amazon EMR is a go-to platform for those who want all the power of Hadoop and Spark in the cloud. However, cost and performance trade-offs can reduce the advantages of EMR over alternatives. Lack of visibility into the root cause of problems, right-sizing options, and cost allocation can add confusion and frustration for EMR users. Unravel Data gives you visibility into the minute-to-minute operations of your workloads on EMR. Get root cause analysis (RCA) of workload breakdowns and slowdowns; AI-powered recommendations; and proactive fixes for many problems. With Unravel Data, you can meet and beat your SLAs, saving thousands - even millions - of dollars per year in the process.

In this webinar, Unravel CDO and VP Engineering Sandeep Uttamchandani describes the fourth and final step for any large, data-driven project: the Operationalize phase. You've found your data (Discover phase), readied it for processing (Prep phase), and built out your processing logic and machine learning model(s) (Build phase). Now you need to Operationalize all your work to data as a live project, in production.

Organizations are moving big data from on-premises to the cloud, using best-of-breed technologies like Databricks, Amazon EMR, Azure HDI, and Cloudera, to name a few. However, many cloud migrations fail. Why? And, how can you overcome the barriers and succeed? Join Chris Santiago, Director of Solution Engineering, as he describes the biggest pain points and how you can avoid them, and make your move to the cloud a success.

Databricks is a great solution for customers looking to unlock the powerful use cases that Spark enables, with the high performance of Databricks and the convenience of a managed service. Databricks is available in AWS, Microsoft Azure, and GCP clouds. If you are already a Databricks customer, you want to get the most out of your investment - and if you're considering Databricks, you'll be wondering how hard it is to move to the platform, and how to optimize your investment once you get there.

Observability of Apache Airflow presented by Ry Walker, Founder & CTO at Astronomer. Apache Airflow has become an important tool in the modern data stack. We will explore the current state of observability of Airflow, common pitfalls if you haven't planned for observability, and chart a course for where we can take it going forward.