When doing data matching with large sets of data, consideration should be given to the combinations that can be generated, and it’s associated effects on performance. This has an effect when using Talend’s Data Integration Matching and Data Quality components. Matching routines do not scale in a linear fashion.

We know that data is a key driver of success in today data-driven world. Often, companies struggle to efficiently integrate and process enterprise data for fast and reliable analytics, due to reliance on legacy ETL solutions and data silos. To solve this problem, companies are adopting cloud platforms like Microsoft Azure to modernize their IT infrastructure.

Azure SQL Data Warehouse (DW) has quickly become one of the most important elements of the Azure Data Services landscape. Customers are flocking to Azure SQL DW to take advantage of its rich functionality, broad availability and ease-of-use. As a result, Talend’s world-class capabilities in data integration, data quality and preparation, and data governance are a natural fit with Azure SQL DW.

A common perspective that I see amongst software designers and developers is that Machine Learning and Artificial Intelligence (AI) are technologies which are only meant for an elite group. However, if a particular technology is to truly succeed and scale, it should be friendly with the common man (in this case a normal software developer).

Since the release of Talend 7.1 users can build Talend jobs as Docker images and publish them to Docker registries. In this blog post, I am going to run through the steps to publish to the major cloud provider container registries (AWS, Azure and Google Cloud). Before I dig into publishing container images to registries, I am going to remind you the basics of building Talend Jobs in Docker images from Talend Studio as well as point out the difference between a local build and a remote build.

In my last post, I gave you the first six Do’s and Don’ts of Data Governance and promised to bring together an additional six to consider when making a data governance plan for your organization. Here are six more dos and don'ts when building your data governance framework.

Today, we are in the information age with a tremendous amount of data being created (as much as 90% of data being created in the last two years alone). This data comes from a wide range of sources and takes many different forms: human-generated documents and social media communications; transactional data that we use to run our businesses; and there is an ever-increasing proliferation of sensors producing streams of data.