News

Study Reveals what Drives Hadoop Adoption: Not Much, Actually

• By David Ramel
• May 13, 2015

Actual real-world adoption of Hadoop-based Big Data technologies lags behind the hype, a new study from Gartner Inc. indicates.

The primary challenge remains the same as identified in previous studies: lack of specialized skills.

That's still the No. 1 impediment to Hadoop adoption despite many remediations, including: numerous attempts to "democratize" Big Data analytics; increased education and training opportunities; specialized and supported Hadoop distributions; and myriad reports about the monetary benefits of learning Hadoop. They may be working, but not yet.

"Despite substantial hype and reported successes for early adopters, over half of respondents (54 percent) report no plans to invest at this time," the report states. "Additionally, only 18 percent have plans to invest in Hadoop over the next two years."

The report -- "Survey Analysis: Hadoop Adoption Drivers and Challenges" -- was authored by analysts Nick Heudecker and Merv Adrian, based on a February/March survey of 284 members of Gartner's Research Circle, a panel comprising Gartner-managed IT and business leaders.

Heudecker expounded upon the main finding in a news release yeesterday. "The early adopters don't appear to be championing for substantial Hadoop adoption over the next 24 months," he said. "In fact, there are fewer who plan to begin in the next two years than already have."

Despite all the efforts to seed the industry with more Hadoop-skilled workers, Gartner estimates it will be two to three years before the skills gap is closed.

The second-most-cited reason for not implementing Hadoop -- reported by 49 percent of respondents -- is the failure to demonstrate the value it would bring to organizations. Rounding out the responses that garnered at least a 40 percent response were: integrating Big Data technology with existing infrastructure (44 percent); risk and governance issues (40 percent); and integrating multiple data sources (40 percent).

Organization who actually have adopted Hadoop have yet to put the technology in the hands of many users.

"Those respondents piloting or deploying Hadoop report small numbers of users accessing the cluster," the survey stated. "Of these, 70 percent of respondents have between one to 20 users accessing Hadoop. Surprisingly, 4 percent report zero users."

Other survey highlights include:

• Technology executives overwhelmingly initiate Hadoop adoption, and 68 percent of adoption is initiated within the C-suite.
• The main reasons for not investing in Hadoop are: no need; use of another system; no business case or management support; and no prioritization for the technology.
• Hadoop lags substantially when compared with general Big Data adoption.
• With no acceleration of demand, the best bet for Hadoop vendors appears to be moving existing customers to larger deployments.

The research firm's advice to execs considering Hadoop in their organizations includes:

• Avoid deploying Hadoop from the fear of being left behind. Hadoop adoption remains at the early adopter phase, where skills and success are still rare.
• Start with small projects backed by a business stakeholder if Hadoop is necessary to address business problems. This allows organizations to develop skills and establish a record of success before tackling larger projects.
• Focus on who will actually use the software -- include application plans or tool enablement to drive consumption in planning -- or risk expensive systems used by very few constituents.