As the business world continues to integrate AI and machine learning to better manage big data processes, one area that arguably has benefitted the most is business monitoring. From IT management to business intelligence, the last few years have seen a drastic shift in how companies are monitoring their data.
When it comes to data, let’s start with the obvious. Averages suck. As developers, we all know that percentiles are much more useful. Metrics like P90, P95, P99 give us a much better indication of how our software is performing. The challenge, historically, is how to track the underlying data and calculate the percentiles. Today I will show you how amazingly easy it is to aggregate and create SQL based percentile rollups with Postgresql and t-digest histograms!
We are delighted to announce that Iguazio has been named a sample vendor in the 2020 Gartner Hype Cycle for Data Science and Machine Learning, as well as four additional Gartner Hype Cycles for Infrastructure Strategies, Compute Infrastructure, Hybrid Infrastructure Services, and Analytics and Business Intelligence, among industry leaders such as DataRobot, Amazon Web Services, Google Cloud Platform, IBM and Microsoft Azure (some of whom are also close partners of ours).
Earlier this year, we launched a unique partnership with Fortune Magazine, with the first-ever data analytics site supporting the publication of the annual Fortune 500 list. Today, we extended that partnership with the debut of the “History of the Fortune Global 500,” our interactive data analytics site timed with the publication of the 30th anniversary of the Fortune Global 500 list.
Imagine going to work only to find that your inbox is flooded with customers telling you how happy they are with your software. People are in such a hurry to download your app, you need to scale your servers to meet the demand before the infrastructure crashes. Your phone rings: it’s a tech journalist trying to book an interview with you about your company's growth. This is the dream for every business owner and entrepreneur. But the reality is often in stark contrast to the scenario above.
We are all familiar with this scenario, you work on your training code, fix “all” of the bugs (the ones you know about), wait for a few iterations, see that batch size wasn’t wrong and nothing blows up, and then you happily go home. However, when you come back into the office the next day look at your loss and test accuracy you’re horrified to find that the experiment crashed on the first test cycle because you pointed your test set in the wrong folder 🙁
