News

Survey: 'Big Data Paralysis' Is Holding Companies Back

• By David Ramel
• May 26, 2015

Just a week after a report from research firm Gartner Inc. found that investment in Hadoop-based Big Data technologies "remains tentative" with poor adoption rates, a survey from Software AG identified "a Big Data paralysis" that's keeping enterprises from realizing the promised benefits of the analytics craze.

"While decision makers understand that greater visibility over internal processes and real-time data analysis could lead to operational improvements, they still lack the support and solutions needed to make it a reality," Software AG said last week in a news release about its new study.

Is Big Data in the famed "trough of disillusionment" phase of the "hype cycle" popularized by Gartner itself? If so, the analysts may have contributed to the effect themselves with is survey report released earlier this month that highlighted the challenges to Hadoop adoption.

"Despite substantial hype and reported successes for early adopters, over half of respondents (54 percent) report no plans to invest at this time," the Gartner report states. "Additionally, only 18 percent have plans to invest in Hadoop over the next two years."

In its own survey, Software AG commissioned Vanson Bource to poll big companies to determine "How Effective Use of Data Improves Operational Processes."

The results mirrored some of the negative observations reached by Gartner.

"Challenges are holding organizations back from making the improvements that they need," the survey report said. "The majority of those surveyed report that their organization finds analyzing large amounts of data (73 percent) and acting on data in real-time (65 percent) as very real obstacles."

But unlike the Gartner report, the oft-cited Big Data skills shortage is not to blame in the Software AG version of Big Data truth. Rather than lack of skills, the big obstacles identified in the Software AG report focus on IT operations, as one of the stated goals of the survey was to "evaluate if organizations are monitoring operational processes and how these processes are prioritized."

In its news release about the new report, the company coincidentally noted: "Software AG recently launched the first Digital Business Platform, which provides a holistic approach that can manage and govern IT assets and automated business processes, on-premise with cloud integration and advanced analytics, based on an in-memory and event driven architecture."

With that focus on operations, the Software AG report identified the major challenge of successfully collecting and analyzing data: keeping the data secure. In that regard, the findings equate with those of Gartner, which listed security as the No. 2 challenge behind lack of applicable skills.

• 74 percent of respondents agree that more visibility over internal processes could lead to operational improvements.
• 65 percent say that it will also decrease the chances of falling behind their competitors.
• Organizations are most likely to be using quantitative data over other sources to prioritize processes.
• Around three-quarters (73 percent) agree that analyzing a lot of data is a challenge for their organization.
• Acting on data in real-time is seen as a challenge for 65 percent.
• Over eight in 10 (83 percent) state that their operational processes take too long.
• The majority of respondents' organizations cannot act on (57 percent), mine (61 percent) or use (68 percent) real-time data.
• 56 percent of respondents' organizations are using manual processes to collect and analyze data.

Although the back-to-back reports may serve to dampen some of the enthusiasm surrounding Hadoop and Big Data, other industry sources are keeping the hype cycle going upward.

Jeff Kelly, an analyst at Wikibon, earlier this month wrote a blog post about his leaving the research firm, titled "Big Data: From Promise to Reality."

"Big Data is not just hype anymore," Kelly said. "Real companies, from large, established firms to small, nimble startups -- are using Big Data technology to reinvent themselves and upend established markets. Most importantly, they are delivering real value for real people."

SiliconANGLE just yesterday followed up on the Wikibon rebutal in an article titled, "Wikibon Analyst Says Hadoop Pessimism is Part of Natural Adoption Cycle."

"Gartner's rather sombre outlook doesn't mean that Hadoop has failed to live up to the hype," the article states. "On the contrary, the figures are simply a reflection of what it takes for technologies as complex as Hadoop to mature to a point where they can be easily accepted by mainstream users, argues Wikibon's Big Data and analytics analyst George Gilbert."

And if you're a Hadoop/Big Data enthusiast, there's plenty more fodder to support your view, including articles such as:

• Booming Big Data Market Headed for $60B
• Hadoop Numbers Suggest the Best Is Yet to Come
• New Enterprise Survey Reveals Strong Growth for Data, BI and Hadoop in the Cloud
• Survey Indicates Apache Spark Gaining Developer Adoption as Big Data's Projects Require Processing Speed
• Hadoop Analytics Is Finding Favor with More CIOs, Deutsche Bank Says

Which just goes to show: Take your surveys with a grain of salt.