Apache Spark unifies batch processing, real-time processing, stream analytics, machine learning, and interactive query in one-platform. While Apache Spark provides a lot of capabilities to support diversified use cases, it comes with additional complexity and high maintenance costs for cluster administrators. Let’s look at some of the high-level requirements for the underlying resource orchestrator to empower Spark as a one-platform.

Customers of B2B companies rely on insights from applications to grow their business, secure their infrastructure, make business decisions, and more. Unless your B2B company offers a rich set of analytics within its product, your customers likely demand nightly data dumps from your application so they can analyze application data with their own BI stack.

Virtually every marketing organization is taking steps to become more data-driven, but there are considerable gaps between vision and reality. According to a 2018 Salesforce report, only 47% of marketers have a completely unified view of customer data sources. Meanwhile, customer data complexity is only increasing. According to Salesforce’s 2020 “State of Marketing” study, the median number of data sources leveraged by marketers is projected to jump by 50% between 2019 and 2021.

This spring, Sauce Labs announced the Sauce Testrunner Toolkit (beta) to expand developer-first capabilities and support for native JavaScript frameworks. The Testrunner Toolkit makes setting up, writing, and running web tests easier and faster for developers during early pipeline testing. First it supported Puppeteer, followed by Cypress, TestCafe, and Playwright to provide the flexibility to test the way you want, along with Sauce Labs insights, at scale.

Automation testing has become a recognized domain in the world of software testing. As the name implies, automation testing involves the use of automated tools to carry out test cases with minimal human intervention, then comparing various outcomes and generating test reports. Automated testing is a crucial part of every Agile team to keep up with the demands for fast but high-quality software projects.

Moesif had the privilege of picking the brain of the expert who literally wrote the book on API Product Management. In a wide-ranging discussion with Dr. Amancio Bouza we found out what you should really care about as an API product manager, and it’s not what you may think (hint: see the title). We also cover pricing your API, how to become a better product manager and what books should be on your nightstand.

Logs are an important part of your application and logging shouldn’t be one of the last things you think of. You should configure your log system, formatter, and style as soon as you start the development of your app. Also, do your best to document the process and share how it works with the rest of your team. In this article, we’re going to demonstrate how logs work in Elixir. We’ll jump into Elixir’s Logger module, which brings a lot of power to logging features.

The COVID-19 global pandemic has added new urgency to the quest for digital transformation. The pandemic disrupted business processes and displaced and disconnected people. Organizations across the globe witnessed first-hand that the speed with which they could get new automations and applications in place could literally make or break the company. The world quickly learned what some industry leaders already knew: the ability to develop an application and quickly bring it to market is crucial.

Tapping into more compute power is the next frontier of data science. Data scientists need it to complete increasingly complex machine learning (ML) and deep learning (DL) tasks without it taking forever. Otherwise, faced with a long wait for compute jobs to finish, data scientists give in to the temptation to test smaller datasets or run fewer iterations in order to produce results more quickly.