In the past, most software applications were all about “data processing.” In the parlance of old-school management information systems, that meant an almost exclusive focus on keeping accurate transactional records alongside any master data necessary to complete that mission. Transaction processing is important, of course, but in today’s world, applications are expected to deliver a lot more than that.
Most commonly, data teams have worked with structured data. Unstructured data, which includes images, documents, and videos, will account for up to 80 percent of data by 2025. However, organizations currently use only a small percentage of this data to derive useful insights. One of main ways to extract value from unstructured data is by applying ML to the data.
Our mission at Google Cloud is to help our customers fuel data driven transformations. As a step towards this, BigQuery is removing its limit as a SQL-only interface and providing new developer extensions for workloads that require programming beyond SQL. These flexible programming extensions are all offered without the limitations of running virtual servers.
I’ve been following the Streamlit framework for a while, since Snowflake announced that they would acquire it to enable data engineers to quick spin up data apps. I decided to play around with it and see how we could leverage the speed of creating an app along with the benefits that ThoughtSpot provides, especially around the ability to use NLP for search terms. Streamlit is built in Python.
