News

Analysts: Hadoop Faces Cloud Disruption

• By David Ramel
• March 2, 2017

Apache Hadoop is headed for the same place as mainframes and data warehouses, Forrester Research Inc. said yesterday, though it has a few years of strong growth left.

The main reasons for that perhaps-surprising conclusion, Forrester said, are Hadoop's complexity and a Big Data analytics shift to public clouds, for which Hadoop wasn't designed. The analysts don't say Hadoop will fade away, but rather that the cloud will change what it is.

Cloud vendors have already begun this process, the company said, noting the advent of services such as AWS Athena, providing a serverless approach to run SQL queries against Big Data in the Amazon Web Services Inc. (AWS) cloud. AWS and other public cloud vendors, the thinking goes, are abstracting notoriously complex on-premises implementations of full-fledged Hadoop distributions -- or replacing Hadoop altogether.

"As more firms get tired of Hadoop's on-premises complexity and shift to the public cloud, they will look to shift their Hadoop stacks there," Forrester said in a news release publicizing its new "The Cloudy Future Of Hadoop" report. "This means that the Hadoop vendors will start to see their revenue shift from on-premises to the cloud.

"But serverless and Hadoop alternatives in the public cloud will gain traction, undercutting Hadoop revenue, unless the Hadoop community can give the industry a compelling reason to keep using all of Hadoop."

Another contributing factor to the cloudy demise of Hadoop as we know it is the advent of deep learning and artificial intelligence (AI).

"Just like Hadoop wasn't designed for the cloud, it wasn't designed to do the matrix math that deep learning requires," Forrester said. "And the cloud crew is busy creating specialized AI-friendly environments, which means Hadoop vendors have even more work to do to keep their software relevant. Will they make Hadoop a platform for AI? Probably not."

Forrester noted that prominent vendors of Hadoop distributions such as Cloudera Inc., Hortonworks Inc. and MapR Technologies Inc. all have different strategies to adjust to the shift to cloud computing and keep Hadoop relevant.

Through those efforts and other developments, Forrester believes Hadoop will survive, just in a different form.

"Do not read this wrong," Forrester concluded. "We think Hadoop has a future and will have strong growth for at least two to three more years, but ultimately it will take its place alongside data warehouses and mainframes. The pace of digital change in the age of the customer is simply too fast for any one technology to keep up for long."