Pervasive's Java Framework for Multicore
- By John K. Waters
- September 11, 2007
Pervasive, which has been developing database and data integration software for about 20 years, is set to release the Beta 2 version of the 100-percent Java DataRush framework sometime next month. DataRush is a hyperparallel framework that provides Java developers with a simple model for building highly parallel data processing applications for multicore hardware.
Pervasive's announcement comes amidst AMD's launch this week of its new Opteron quad-core x86 microprocessor family, formerly code-named "Barcelona."
For Java developers, multicore processors signify challenging times ahead, according to Jim Falgout, systems architect for Pervasive Software's DataRush.
"Multicore processors are changing the software development landscape," he said. "Every customer we talk to today is acutely aware of the power and potential of multicore machines. They all have them. But they're also aware that their software really isn't responding to it very well."
Falgout summed up the problem for Java developers in creating software for multicore systems. Essentially, developers must change their implementation methodologies before the disconnect between hardware capabilities and software design leads to massively underutilized computing power in the datacenter. The reason for this disconnect is that the processors in multicore systems aren't individually as powerful as their single-core predecessors, he said.
"If you're not multithreading your code, it's actually slowing down," Falgout said. "Java is a great language. It has good libraries for doing multithreading work, but it's still pretty low-level. You have to understand some advanced concepts to use those libraries. We feel that there's a need for a new framework to help Java programmers, especially in building saleable applications that will take advantage of multicore."
DataRush is designed to eliminate the need to deal with threading libraries, deadlock detection algorithms, or concurrent process design issues, Falgout explained. It comes with a library of out-of-the-box Java components that can be assembled into a series of data-flow operations using simple XML scripting. There's also a software development kit for quickly building and extending the DataRush applications.
The product is aimed at developers working on data-intensive applications, such as those used in financial services, healthcare, government and insurance. In those industries, people are trying to crunch staggering amounts of data in shrinking time windows, Falgout said.
"The amount of data that we can store increases year by year, almost exponentially," he explained. "As the amount of data increases, so does the pressure to do something productive with it. You want to be able to run more complex algorithms and analytics against that data to produce information that helps you run your business better. Software has to take advantage of multicore processors to do that job."
The DataRush framework allows developers to plug any Java code into a process, as long as they abide by an unrestrictive contract and a few simple conventions. This means, Falgout said, that developers can link existing applications by the data they communicate to one another or, even better, pull existing code apart to adapt it to run as a dataflow application. The code within a process may even be threaded in itself, allowing programmers to fine-tune parallelism at all levels of the application. (In a dataflow application, a collection of computations are joined to one another by channels transmitting data in one direction. The engine at the core of DataRush is based on dataflow principles.)
The company appears to be "eating its own dog food." For two years, Pervasive has been shipping a product called Pervasive Data Profiler, which was built entirely on DataRush.
Pervasive is set to announce a Beta 2 release of DataRush at the end of September. Pricing has not been finalized, but the software development kit will be free and free community support will be "fostered" on the company's developer portal. More information and a download are available on that site.
About the Author
John K. Waters is the editor in chief of a number of Converge360.com sites, with a focus on high-end development, AI and future tech. He's been writing about cutting-edge technologies and culture of Silicon Valley for more than two decades, and he's written more than a dozen books. He also co-scripted the documentary film Silicon Valley: A 100 Year Renaissance, which aired on PBS. He can be reached at [email protected].