News

ScaleOut Touts Live Hadoop Analysis

• By David Ramel
• April 16, 2013

ScaleOut Software today announced an in-memory data grid that lets developers access live, fast-changing data with Hadoop-based programs for instant analysis.

Called ScaleOut hServer, the product stores the live data so it can be directly updated by applications using its open source API libraries at the same time Hadoop programs are analyzing the data.

"Unlike conventional Hadoop usage, which analyzes static data sets, the ability to continuously perform Hadoop MapReduce analysis on live data enables important trends to be spotted as they occur," ScaleOut said.

ScaleOut hServer stores Hadoop Distributed File System (HDFS) data as key/value pairs in its in-memory data grid, allowing programs to access the data in a distributed cache without having to deal with the HDFS data. "The [in-memory data grid] key/value storage and Java APIs match the object-oriented architecture of your Hadoop application," the company said.

ScaleOut referred to a survey it commissioned that found 93 percent of respondents believed their organization needed or would benefit from such real-time analytics.

Such real-time data analysis could result in new duties for traditional Hadoop developers, according to David Brinker, chief operating officer of ScaleOut. "We have observed that the need for real-time analytics is typically based on fast-changing live data," Brinker told this site. "Generally, this is outside the purview of BI and data warehouse developers who tend to develop programs to analyze lakes of static data. We expect to see the emergence of developers who are focused on writing apps for analyzing live data to enable very fast business decisions."

Examples of users that might benefit from this real-time data analysis include financial services companies tracking frequently changing stock prices or shareholdings, and power companies with "smart grids" that could more efficiently deliver electricity, for example, or instantly respond to potential problems in the system.

The company said ScaleOut hServer comes in a free community edition and several commercial editions, licensed with an annual subscription pricing model that varies with the size of deployment. A "programmer's guide" is also available in PDF format. The company said ScaleOut hServer is the first of a series of products aiming to bring real-time analysis performance to Hadoop, and while the community edition is available for download to run on Red Hat Enterprise Linux 5, the company is assessing customer demand for ScaleOut hServer on Windows.