BigQuery is Google Cloud’s fully managed serverless data platform that supports querying using ANSI SQL. BigQuery also has a data lake storage engine that unifies SQL queries with other open source processing frameworks such as Apache Spark, Tensorflow, and Dask. BigQuery storage provides an API layer for OSS engines to process data. This API enables mixing and matching programming in languages like Python with structured SQL in the same data platform.

In part 1 of this blog we discussed how Cloudera DataFlow for the Public Cloud (CDF-PC), the universal data distribution service powered by Apache NiFi, can make it easy to acquire data from wherever it originates and move it efficiently to make it available to other applications in a streaming fashion.

Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. Administrators, developers, and data engineers who use Kafka clusters struggle to understand what is happening in their Kafka implementations.

It is not unusual for customers to load very large data sets into their enterprise data warehouse. Whether you are doing an initial data ingestion with hundreds of TB of data or incrementally loading from your systems of record, performance of bulk inserts is key to quicker insights from the data. The most common architecture for batch data loads uses Google Cloud Storage(Object storage) as the staging area for all bulk loads.

Performance considerations for loading data into BigQuery for various file types.

We all want better business cash flow and we want it yesterday. You can’t plan for emergencies, geopolitics, or sudden problems that you have no control over. But you can mitigate risks of business cash flow problems by having the right tools at your side. Business cash flow planning can get you out of a jam and save your company. Take a look at our ultimate guide to business cash flow planning highlighting.

We’re back with another Job of the Week! The title is admittedly growing on me... This week’s job takes us into the world of data scraping. First, let’s answer two common FAQ’s.

Innovate or die. It’s one of the few universal rules of business, and it’s one of the main reasons we continue to invest so heavily in data. Only through data can we get the key insights we need to innovate faster, smarter, better and keep ahead of the market. And yet, the vast majority of data initiatives are doomed to fail. Nearly nine out of 10 data science projects never make it to production.

Businesses today adapt to change with breathtaking speed. Lines of business now follow their products out the door digitally and continue tracking usage throughout the lifecycle. This gives companies valuable insights into how customers use and interact with their products, enabling them to analyze and evolve solutions while nurturing customer relationships.

In this article we discuss the various methods to replicate HBase data and explore why Replication Manager is the best choice for the job with the help of a use case. Cloudera Replication Manager is a key Cloudera Data Platform (CDP) service, designed to copy and migrate data between environments and infrastructures across hybrid clouds.