News

IBM Named Hadoop Leader, Launches Big Data Mainframe

• By David Ramel
• January 15, 2015

IBM's huge investments in Big Data initiatives seem to be paying off, as it was recently named a leader in Hadoop accessibility by a research firm and this week launched a new Hadoop-capable mainframe computer it describes as "one of the most sophisticated computer systems ever built."

"In a survey of more than 1,000 Big Data developers, analyst firm Evans Data Corp. found that IBM is the leading provider of Hadoop among developers, with more than 25 percent of respondents identifying IBM's Hadoop as their principle distribution," the company announced Tuesday. "The survey also focused on key growth areas such as machine learning and streaming analytics, where 18 percent of developers cited IBM InfoSphere Streams as their preferred application for machine learning, making it the second most popular choice in the category."

IBM told this site it couldn't release more survey data, contained in a for-sale report by the software development research firm.

On the same day, the z13 mainframe supercomputer was unveiled, giving new meaning to the concept of Big Data transaction processing. "If you plan to be a computer engineer or software programmer, consider this amazing system as the platform for your career," said exec Ross Mauri in a blog post.

Specializing in mobile transactions, the z13 is reportedly the first system capable of processing up to 2.5 billion transactions per day, which IBM said was the equivalent to handling the traffic generated by 100 "cyber Mondays" (the busiest online shopping day of the year) every day for a full year.

"The z13 includes new support for Hadoop, enabling unstructured data to be analyzed in the system. Other analytics advances include faster acceleration of queries by adding DB2 BLU for Linux providing an in-memory database, enhancements to the IBM DB2 analytics accelerator, and vastly improved performance for mathematically intense analytics workloads," the company said.

IBM said the z13 was five years in the making at a cost of $1 billion, leveraging more than 500 new patents and incorporating the work of some 60 client partners. The scope of this project exemplifies the vast resources IBM has ploughed into the mobile/cloud/Big Data space.

The mainframe is reportedly the first to feature practical real-time encryption of any amount of mobile transactions and first to come with embedded analytics to glean insights from ongoing transactions. That's all made possible by the world's fastest microprocessor and huge amounts of memory and bandwidth.

"We designed the z13 machine from the ground up with mobility in mind," Mauri said. "Over the past decade, the world has witnessed an explosion of data -- from electronic commerce, social media, business systems, Web sites and the Internet of Things. Today, our interactions with data and with each other are increasingly going mobile. As a result, we're consuming and creating data all the time, every day. And you can expect 100 times more data to flow 2-3 years from now. The z13, with 141 of the industry's fastest microprocessors (with 5 GHz performance) on board, is the only computer that's purpose-built to handle this mobile data tsunami."

The company also had some software news, unveiling a preview of new z/OS software for advanced analytic and data-serving capabilities. IBM said when it's available it will expand the ability of the new mainframe to process in-memory analytics and analyze mobile transactions.

IBM also announced favorable benchmark testing results for its SQL-on-Hadoop offering, Big SQL; noted that its online Big Data University is used regularly by more than 200,000 developers; and said almost 1,000 coders have signed on for its Big Data for Social Good Hadoop Challenge to solve civil and social problems.