News

HPE Looks to Open Sourcers for Help with The Machine

• By John K. Waters
• June 13, 2016

Hewlett Packard Enterprise (HPE) is reaching out to open source software (OSS) developers for help with its ambitious effort to rethink computing from the ground up, known as The Machine. The company just announced that it's providing open access to developer tools and libraries to encourage collaboration between HPE and the open source community on the development of software for this new architecture.

The Machine is being developed by Hewlett Packard Labs, HPE's advanced research arm. The project emerged about two years ago from a group of R&D projects poised to go from the lab into production at about the same time. Specifically, it was the combination of very high-capacity, low-cost, low-power, high-density and non-volatile memory systems with chip-level photonic interfaces that allowed HPE to seriously consider breaking the computing model that has been in place for the past 50 years.

Combined, these technologies support what the company calls "memory-driven computing." Instead of processors, memory and data lie at the core of this computing model. Memory and storage are collapsed into a single, vast pool called "universal memory." Memory and processing nodes are connected using light instead of electricity with a photonic fabric.

"This is about making a fundamental change in the underlying hardware infrastructure, moving away from a processor-centric model of computing to a memory-driven model," explained Bdale Garbee, HPE Fellow and Office of the CTO at Hewlett Packard Labs. "Think of it this way: If you can build a very large, byte-addressable, non-volatile array and connect to it with a very fast chip-level photonics interconnect, all of a sudden you can start designing systems around the memory. There's the potential here for developers to think in a qualitatively, not just in a quantitatively, different way about how they build computing solutions."

The Machine is HP Labs' biggest project to date. HP CEO Meg Whitman introduced it to the world in her HP Discover 2014 keynote speech in Las Vegas. "We've been talking about many of the component technologies for some time," she said. "Now we're bringing them together in a single project to make a revolutionary compute architecture available by the end of the decade."

HPE has invited open source developers to write and contribute code to The Machine project relatively early in its evolution, and says its own development team will work transparently with open sourcers going forward. The company plans to start that process within familiar environments, such as Linux, and Portable Operating System Interface APIs and programming languages like C/C++ and Java. The idea is to make the performance advantages of massive memory on fabrics available to developers in a way that makes them productive quickly, the company has said.

"We're hoping to get developers involved in the technology early, so they can start thinking about how to build next generation applications for it," Garbee said. "And, frankly, we're looking forward to getting their help sorting among the ideas and potential research paths we might take with this technology."

The project is making four tools available initially, but promises to add more as the project evolves. Those tools are:

• Fast optimistic engine for data unification services: A completely new database engine that speeds up applications by taking advantage of a large number of CPU cores and non-volatile memory (NVM).
• Fault-tolerant programming model for non-volatile memory: Adapts existing multi-threaded code to store and use data directly in persistent memory, provides simple, efficient fault-tolerance in the event of power failures or program crashes.
• Fabric Attached Memory Emulation: An environment designed to allow users to explore the new architectural paradigm of The Machine.
• Performance emulation for non-volatile memory latency and bandwidth: A DRAM-based performance emulation platform that leverages features available in commodity hardware to emulate different latency and bandwidth characteristics of future byte-addressable NVM technologies.

"In the coming months," the company states, "HPE intends to enhance this code as well as release additional contributions, focused on everything from changes to Linux that enable it to run on The Machine to sample applications that demonstrate how The Machine can significantly improve application scale and performance."