As DataOps moves along the maturity curve, many organizations are deciphering how to best balance the success of running critical jobs with optimized time and cost governance. Watch the fireside chat from Data Teams Summit where Mark Sear, Head of Data Platform Optimization for Maersk, shares how his team is driving towards enabling strong engineering practices, design tenets, and culture at one of the largest shipping and logistics companies in the world.

Once in a while I stumble upon Spark code that looks like it has been written by a Java developer and it never fails to make me wince because it is a missed opportunity to write elegant and efficient code: it is verbose, difficult to read, and full of distributed processing anti-patterns. One such occurrence happened a few weeks ago when one of my colleagues was trying to make some churn analysis code downloaded from GitHub work.

A modern data infrastructure is essential for retailers looking to stay competitive today. Companies are abandoning more traditional, on-premises IT infrastructures and moving to more centralized “as a service” (XaaS) models of delivery enabled by cloud technologies, according to McKinsey. Aging on-premises infrastructures are unable to meet demands for agility and innovation, eating up too much time and too many resources for teams trying to maintain them.

Financial inclusion, defined as the availability and accessibility of financial services to underserved communities, is a critical issue facing the banking industry today. According to the World Bank, 1.7 billion adults around the world do not have access to formal financial services, meaning that they cannot open a bank account or access credit, insurance, or other financial products.

How to preprocess data using BigQuery ML.