Like many other people, I used time over the recent holidays to clean out and organize my digital files. In that process, I finally trashed the speaking notes for a panel I participated in at SMA’s (Strategy Meets Action) first summit in 2012 when I worked at a large global insurer. During that session, a gentleman in the audience asked me what I thought about “big data” and its implications for Insurance.

One of the core reasons that organizations invest in analytic solutions is because they want to get everyone in their organization on the same page. They want everyone to understand what's happening and why it's happening so that individuals know what they need to do to be successful and drive outcomes for the organization.

Let’s try to figure out what happens with the application when the source file is much bigger than the available memory. The memory in the below tests is limited to 900MB […]. Naively we could think that a file bigger than available memory will fail the processing with OOM memory error. And this supposition is true.

In this post, we’ll look into tried and true methods of improving Rails view performance. Specifically, I will focus on database efficiency, view manipulation, and caching. I think the phrase “premature optimization is the root of all evil” has been taken a little out of context. I’ve often heard developers use this during code reviews when simple optimization techniques are pointed out.

A long time ago (in April 2019), in a galaxy far far away (in Tel Aviv, Planet Earth) we launched a self-serve option. Users could now sign up to Rookout without having to contact us first. We invested significant resources in online promotion and -- just like any startup at our stage -- we hoped to soon see legions of users signing up and using the tool to fetch data directly from their code.

Many of us have experienced the feeling of hopelessly digging through log files on multiple servers to fix a critical production issue. We can probably all agree that this is far from ideal. Locating and searching log files is even more challenging when dealing with real-time processing applications where the debugging process itself can be extremely time-sensitive.

The opportunity to create new economic, social and environmental value by unlocking the “good” in data is immense. While the problems we face as a society may be getting harder to solve, the advances we can make when we break down the silos between the physical and digital worlds are profound.

Once upon a time there was only one way to use Apache Spark but support for additional programming languages and APIs have been introduced in recent times. A novice can be confused by the different options that have become available since Spark 1.6 and intimidated by the idea of setting up a project to explore these APIs.