Systems | Development | Analytics | API | Testing

Suppose that you work for the infosec department of a government agency in charge of tax collection. You recently noticed that some tax fraud incident records went missing from a certain Apache Kafka topic. You panic. It is a common requirement for business applications to maintain some form of audit log, i.e. a persistent trail of all the changes to the application’s data. But for Kafka in particular, this can prove challenging.

Suppose that you work for a government tax agency. You recently noticed that some tax fraud incident records have been leaked on the darknet. This information is held in a Kafka Topic. The incident response team wants to know who has accessed this data over the last six months. You panic. It is a common requirement for business applications to maintain some form of audit log, i.e. a persistent trail of all the changes to the application’s data to respond to this kind of situation.

For our latest specialist interview in our series speaking to technology leaders from around the world, we’ve welcomed James Kaplan CEO and Co-Founder of MeetKai. He founded the startup with his Co-Founder and Chairwoman, Weili Dai, after becoming frustrated with the limitations of current automated assistants. Kaplan has had a true passion for AI and coding since he was six. He wrote his first bot at only nine years old and wrote the first original Pokemon Go bot.

With hackers now working overtime to expose business data or implant ransomware processes, data security is largely IT managers’ top priority. And if data security tops IT concerns, data governance should be their second priority. Not only is it critical to protect data, but data governance is also the foundation for data-driven businesses and maximizing value from data analytics. Requirements, however, have changed significantly in recent years.

Historically, maintenance has been driven by a preventative schedule. Today, preventative maintenance, where actions are performed regardless of actual condition, is giving way to Predictive, or Condition-Based, maintenance, where actions are based on actual, real-time insights into operating conditions. While both are far superior to traditional Corrective maintenance (action only after a piece of equipment fails), Predictive is by far the most effective.

In traditional data warehouses, specific types of data are stored using a predefined database structure. Due to this “schema on write” approach, prior to all data sources being consolidated into one warehouse, there needs to be a significant transformation effort. From there, data lakes emerge!

Over the past two decades, marketers have faced an uphill battle in trying to turn marketing into a fully data-driven discipline. Our challenge is not that we don’t have enough data but that data has been difficult to access and use. Marketing, sales, and product data is scattered across different systems, and we can’t get a complete picture of what is going on in our businesses.