Note: This is part 2 of the Make the Leap New Year’s Resolution series. For part 1 please go here. When we introduced Cloudera Data Engineering (CDE) in the Public Cloud in 2020 it was a culmination of many years of working alongside companies as they deployed Apache Spark based ETL workloads at scale.
In recent years, Ethical AI has become an area of increased importance to organisations. Advances in the development and application of Machine Learning (ML) and Deep Learning (DL) algorithms, require greater care to ensure that the ethics embedded in previous rule-based systems are not lost. This has led to Ethical AI being an increasingly popular search term and the subject of many industry analyst reports and papers.
During the information age, and throughout the 4th industrial revolution, technology, data and information were in abundance. But, soon came the realization that technology and access to all of the data and information, by itself, was not the solution.
As a modern, data-driven organization, you are likely pulling data from a multitude of diverse sources. There’s consumer data from marketing programs, CRM, and point of sale systems, plus financial data from accounting software and banking services. Finally, there is product data from user logs and web applications. With so much data pouring in every day, it feels like you should have everything you need to answer any question that could arise. And yet, so many times you don’t.
After the launch of Cloudera DataFlow for the Public Cloud (CDF-PC) on AWS a few months ago, we are thrilled to announce that CDF-PC is now generally available on Microsoft Azure, allowing NiFi users on Azure to run their data flows in a cloud-native runtime. With CDF-PC, NiFi users can import their existing data flows into a central catalog from where they can be deployed to a Kubernetes based runtime through a simple flow deployment wizard or with a single CLI command.
Electricity is fundamental to our society. As climate change becomes more severe and demand for clean energy increases, the future is the electrification of everything and along with it, the need for reliable energy. The U.S. infrastructure spans over a vast 200,000 miles and inspecting all of it is a time-consuming and high-risk process that often calls for hanging from helicopters or climbing tall towers. It is inefficient, costly, and dangerous.
Are you storing your data in BigQuery and interested in using that data to train and deploy models? Or maybe you’re already building ML workflows in Vertex AI, but looking to do more complex analysis of your model’s predictions? In this post, we’ll show you five integrations between Vertex AI and BigQuery, so you can store and ingest your data; build, train and deploy your ML models; and manage models at scale with built-in MLOps, all within one platform. Let’s get started!
At Wayfair, we have several unique challenges relating to the scale of our product catalog, our global delivery network, and our position in a multi-sided marketplace that supports both our customers and suppliers. To give you a sense of our scale we have a team of more than 3,000 engineers with tens of millions of customers. We supply more than 20 Million items using more than 16,000 supplier partners.
Heading into a new year filled with myriad crosscurrents, this much is certain: more organizations will find smarter ways to use data as they realize the benefits of digitally transforming their operations. We’re seeing this trend toward data-driven decision making already play out in different industries around the world as companies modernize their infrastructures.
