Systems | Development | Analytics | API | Testing

When I was working at Google back in the mid 2000’s, we dealt with tens of billions of ad impressions a day, trained several machine learning models on years worth of historic data, and used frequently-updated models in ranking ads. The whole system was an amazing feat of engineering and there was no system out there that was even close to handling this much data. It took us years and hundreds of engineers to make this happen, today, the same scale can be achieved in any enterprise.

A little over a year ago, I found myself feeling stuck in my role as a data engineer. I had majored in business in college and was looking to connect more with that side of things. I enjoyed my tasks as a data engineer but I wanted more flexibility and creativity. I wanted to be involved in business decisions rather than my tasks already being decided for me.

Two years ago, we announced our Databricks partnership—including the launch of ThoughtSpot for Databricks, which gives joint customers the ability to run ThoughtSpot search queries directly on the Databricks Lakehouse without the need to move any data. Since then, we’ve empowered teams at companies like Johnson & Johnson, NASDAQ, and Flyr to safely self-serve business-critical insights on governed and reliable data.

The most successful organizations today know they need to use business analytics to make decisions and drive outcomes. Often, however, these decisions must be driven by insights that can remain hidden in data. That’s where data mining comes into play. Data mining is a powerful tool to help extract meaningful insights from even the largest, most complex data sets.

We’ve established that we’re living in the defining decade of data. Data underpins the seismic technology shifts of the past few years, transforming the way we buy, work, make business decisions, even value our companies. As ThoughtSpot’s co-founder Ajeet Singh said, “Once in a generation, the opportunities to create a legacy increase massively. It happens when truly tectonic shifts happen in the ecosystem. We’re living through one of those times.”