News

Hazelcast Jet 0.5 Appeals to Wider Java Audience with Pipeline API

• By John K. Waters
• November 29, 2017

The latest release of Hazelcast Jet, an application-embeddable, distributed computing platform for processing of Big Data sets, comes with new functionality designed to appeal to "a wider Java audience," the company announced last week. That new functionality: a Pipeline API for general purpose programming of batch and stream processing.

The Pipeline API included with the Jet 0.5 release is the platform's primary programming interface for batch and stream processing -- wherein lies the appeal to even more Java jocks, the company says. This is a major enhancement to the low-level Core API, which uses directed acyclic graphs (DAG) to model data flow, allowing detailed DAG assembly of processing jobs. The new Pipeline API is easier to use and provides developers with tools to compose batch computations from building blocks, such as filters, aggregators and joiners.

Hazelcast Jet 0.5 also supports the popular Java 8 Stream API, which supports functional-style operations on streams of elements. "The key point is that any Java developer will find the new Pipeline API familiar and productive," the company said.

Since the company launched Jet in February, it has been marketing the platform to developers of applications that require a near real-time experience, such as sensor updates in Internet of Things (IoT) architectures (think smart-house thermostats and lighting systems), as well as in-store e-commerce systems and social media platforms. Hazelcast CEO Greg Luck described Jet in an earlier interview as "a super-fast, low-latency, next-generation DAG Engine for Big Data processing."

"Since its first release Jet has put the Fast in Fast Big Data with performance up to 15 times faster than Spark and Flink," Luck said last week. "In this release we have been working on bringing Hazelcast's legendary programming simplicity to Jet, which we think we have now achieved with the Pipeline API. Programmers, start your Jet engines."

The new release also supports fault tolerance using snapshotting with the integrated Hazelcast IMDG, an open source product distributed under an Apache 2 license that is designed to allow developers to include the grid in their applications. Hazelcast Enterprise is the commercially supported version.

In-memory data grid (IMDG) specialist Hazelcast has roots that run deep in the Java language and platform (its IMDG is written in Java), but the company has spread its support over the years, thanks in part to the efforts of its open source community, to include several clients and programming languages.