Systems | Development | Analytics | API | Testing

Latest News

How to Develop a Data Processing Job Using Apache Beam - Streaming Pipelines

In our last blog, we talked about developing data processing jobs using Apache Beam. This time we are going to talk about one of the most demanded things in modern Big Data world nowadays – processing of Streaming data. The principal difference between Batch and Streaming is the type of input data source. When your data set is limited (even if it’s huge in terms of size) and it is not being updated along the time of processing, then you would likely use a batching pipeline.

Talend and Splunk: Aggregate, Analyze and Get Answers from Your Data Integration Jobs

Log management solutions play a crucial role in an enterprise's layered security framework— without them, firms have little visibility into the actions and events occurring inside their infrastructures that could either lead to data breaches or signify a security compromise in progress. Splunk is the “Google for log files” heavyset enterprise tool that was the first log analysis software and has been the market leader ever since.

Talend & Apache Spark: Debugging & Logging

So far, our journey on using Apache Spark with Talend has been a fun and exciting one. The first three posts on my series provided an overview of how Talend works with Apache Spark, some similarities between Talend and Spark Submit, the configuration options available for Spark jobs in Talend and how to tune Spark jobs for performance. If you haven’t already read them you should do so before getting started here.

Yellowfin Bytes: Text Replacement and Regular Expressions

The third edition of Yellowfin Bytes brings you an exciting inclusion made to the Data Transformation module. Users have been exploring Yellowfin’s very own lite version of an ETL tool in the form of this module, ever since its introduction in Yellowfin 7.4 last year. We have since been enhancing this functionality by adding new steps, calculations, configurations, and more.

Bridging the gap between data and insights

Today, we want to share a number of updates that will make data analytics easier and more accessible to all businesses. Our goal is to help you focus on data analysis instead of infrastructure management, give you the freedom to orchestrate workloads across clouds, use machine-learning in a way that's integrated with your data analytics operations, and take advantage of open source data processing innovation.

[Step-by-step] Using Talend for cloud-to-cloud deployments and faster analytics in Snowflake

For the past two years, Snowflake and Talend have joined forces developing deep integration capabilities and high-performance connectors so that companies can easily move legacy on-premises data to a built-for-the-cloud data warehouse. Snowflake, which runs on Amazon Web Services (AWS), is a modern data-warehouse-as-a-service built from the ground up for the cloud, for all an enterprise’s data, and all their users.