A quick and easy way to speed up small queries in BigQuery (such as to populate interactive applications or dashboards) is to use BI Engine. The New York Times, for example, uses the SQL interface to BI Engine to speed up their Data Reporting Engine. To Illustrate, I’ll use three representative queries on tables between 100 MB and 3 GB — tables that are typically considered smallish by BigQuery standards.
Those who use data wisely have competitive advantages and more profits. As a result, companies are increasing their focus on improving their data literacy. For example, the importance of data has led companies like AppNexus1 and Chevron2 to conduct internal data science competitions to identify and hone analytical talent. But, as noted in the kickoff blog post to our series on data-driven organizations, merely having data does not ensure you have a useful interpretation of that data.
Working in the engineering field means navigating a variety of needs. Those range from meeting various local and national regulatory statutes, to measuring and monitoring delivery of essential outputs like drinking water and power supply, to understanding the data surrounding regional operations on both the supply and demand side. Organizations that serve this market operate behind the scenes, yet impact our daily life in the United States.
When looking at data, business decision makers are often blocked by an intermediate question of "What should I take away from this data?" Beyond putting together the numbers and building the results, data analysts and data scientists play a critical role in helping answer this question. Organizations big and small depend on data analysts and data scientists to help “translate from words to numbers, and then back to words” as sports analytics pioneer Dean Oliver once said.
Spring is here. Clocks move forward. The Sakura (cherry blossom) festival in Japan marks the celebration of the new season. In India, the holi festival of colors ushers in the new harvest season. It’s a time for renewal and new ways of doing things. This month, we are pleased to debut our newest set of SQL features in BigQuery to help our analysts and data engineers spring forward.
CDP Public Cloud is now available on Google Cloud. The addition of support for Google Cloud enables Cloudera to deliver on its promise to offer its enterprise data platform at a global scale. CDP Public Cloud is already available on Amazon Web Services and Microsoft Azure. With the addition of Google Cloud, we deliver on our vision of providing a hybrid and multi-cloud architecture to support our customer’s analytics needs regardless of deployment platform.
Since Snowflake’s inception we’ve had the needs of our customers as our North Star, with a clear focus on security and governance of data, continuous advances on performance and reduction of latencies, and relentless capabilities innovation. As part of our product development, we constantly evaluate where open standards, open formats, and open source can help or hinder our progress towards those goals. In short, it is a matter of degree.
Have you ever looked at clouds with a friend and seen patterns, shapes, and animals that they just can’t make out? The images appear so obviously to you but no matter how hard you try to show them, your friend sees an entirely different shape, or perhaps nothing at all. Building product features can be a similar exercise in frustration. As designers and builders, sometimes we see our own “cloud shapes” in the product.
In October 2020, Cloudera made a strategic acquisition of a company called Eventador. This was primarily to augment our streaming capabilities within Cloudera DataFlow. Eventador was adept at simplifying the process of building streaming applications. Their flagship product, SQL Stream Builder, made access to real-time data streams easily possible with just SQL (Structured Query Language).
By 2025, Gartner predicts data stories will be the most widespread way of consuming analytics. It’s far from a fad - it’s a part of modern analytics you can’t afford to overlook. However, are you still unsure of what data storytelling is, or why there's a sudden buzz?
