Recently, I've seen a lot of articles and conversation around DevOps. DevOps is a set of practices that automates the processes between software development and IT teams, in order to enable be shorter development cycles, increased deployment frequency, and more dependable releases all in conjunction with business needs.

Are you familiar with Apache Beam? If not, don’t be ashamed, as one of the latest projects developed by the Apache Software Foundation and first released in June 2016, Apache Beam is still relatively new in the data processing world. As a matter of fact, it wasn’t until recently when I started to work closely with Apache Beam, that I loved to learn and learned to love everything about it.

In this blog, we are going to take a look at Apache Spark performance and tuning. This a common discussion among almost everyone that uses Apache Spark, even outside of Talend. When developing and running your first Spark jobs there are always the following questions that come to mind.

Formed in 2016, Paddy Power Betfair (PPB) is the world’s largest publicly quoted sports betting and gaming company, bringing “excitement to life” for five million customers worldwide. The merger of Paddy Power and Betfair in 2016 created an additional data challenge for an already highly data-driven organization. The merged company had to bring together 70TB of data, from dozens of sources, into an integrated platform.

Recently I found myself in a predicament that many of you can relate to, trying to update an aging application that has become too difficult to manage and too costly to continue operating. As we started to talk about what to do, we concluded it was time to start decomposing that application into smaller more manageable pieces.

Companies don’t gain business value just by gathering lots of data. They don’t even necessarily gain value from analyzing the data. The true keys to success are choosing the right data to focus on, knowing what to do with it, and determining the best ways to apply analytics to solve business problems or address market opportunities.

Artificial Intelligence (AI) emerged as a hot topic in 2017. Although scientists have been working on the technology and heralding its numerous anticipated benefits for more than four decades, it’s only in the past few years that society’s AI dreams have come to fruition.