Systems | Development | Analytics | API | Testing

Bringing your own libraries to run a Spark job on a shared YARN cluster can be a huge pain. In the past, you had to install the dependencies independently on each host or use different Python package management softwares. Nowadays Docker provides a much simpler way of packaging and managing dependencies so users can easily share a cluster without running into each other, or waiting for central IT to install packages on every node.

A new year is upon us and that means it’s time to look ahead to what’s coming next. In cloud computing, organizations are going to be making adjustments in 2020 – to accommodate overstrained budgets, new regulations, and shifting technologies. It will be a year of identifying what’s not working and moving toward the right solutions. Let’s take a look at three trends that will impact cloud computing across all industries in the coming year.