Intricacies in Spark 3.0 Partition Pruning
In this blog post, I’ll set up and run a couple of experiments to demonstrate the effects of different kinds of partition pruning in Spark.
In this blog post, I’ll set up and run a couple of experiments to demonstrate the effects of different kinds of partition pruning in Spark.
Over the past several years, there has been an explosion of different terms related to the world of IT operations. Not long ago, it was standard practice to separate business functions from IT operations. But those days are a distant memory now, and for good reason.
Since the start of the pandemic nearly a year ago, there's been one word on the lips of every business leader, analyst, and investor around the world: cloud. COVID-19 fundamentally changed the way businesses operate. In response, organizations went all in on cloud, betting on the unmatched scale, speed, and security of SaaS applications to help them weather the storm. Nowhere was this shift more pronounced that in our own data and analytics industry.
A quote from Tim Cook says- “When you care about people’s happiness and productivity, you give them what brings out the best in them and their creativity. And if you give them a choice, they’ll say, ‘I want an iPhone,’ or ‘I want a Mac.’ We think we can win a lot of corporate decisions at that level.” This says a lot about the love for the iPhone we all have!
Azure API Management (APIM) is a powerful platform that enables you to publish and scale APIs while ensuring they are secured. One of the great features of Azure APIM is that you can add plugins and transforms to your APIs without any code change or restarts. These capabilities are deployed using XML Policies which are a collection of statements.