News

Data Inflexibility Wrecks Analytics Projects, Survey Indicates

• By David Ramel
• September 26, 2016

A new survey of data analytics frontline pros and executives finds many have experienced project failures, with data inflexibility topping the list of challenges faced by enterprise teams.

The survey, conducted by Dimensional Research on behalf of Snowflake Computing, polled 376 people with responsibility for data initiatives in order to understand their experiences, challenges and trends surrounding data analytics initiatives.

"Few organizations have been fully successful rolling out their data analytics initiatives," says the report, titled Data Analytics: Beyond the Hype (free download upon providing registration info). "The vast majority (88 percent), have had 'failed' projects, where they have faced serious issues with adoption or use." Those failed projects came in various forms, such as dissatisfied users, low adoption rate, budget overruns or high costs in relation to received value, and even "zombie projects" that were nearly completed but never deemed quite ready for users.

As far as the cause of these failures, "Topping the list was the inflexibility of existing infrastructure (51 percent) which prevented these data teams from achieving success with their projects," the survey report said. "Also reported frequently were finding expertise (47 percent), taking too long to access additional data sources (42 percent) and inadequate budgets (42 percent)."

The inability to find appropriate expertise reflects a longrunning skills shortage in the Big Data and data science spaces, which ADTMag and other media have been reporting for years.

The No. 1 type of technical expertise that enterprises are finding hard to come by is database tuning (45 percent), followed by data science (44 percent), data engineering (39 percent), Hadoop (33 percent), distributed programming (32 percent) and SQL (27 percent). A minority of respondents (15 percent), reported not facing any challenges in finding technical expertise.

The perennial skills issue, however, took second place to data infrastructure inflexibility in this report, though the seriousness of that problem varies with the type of respondent.

"When we compared the responses of senior executives with the frontline staff who do the work, we see an interesting difference emerge," the report said. "Just over a third (36 percent) of frontline people who work with the infrastructure on a day-to-day basis reported that inflexible infrastructure was a barrier. At the same time, almost 6 in 10 (59 percent) of executives saw infrastructure complexity as a challenge. This would appear to indicate that the frontline people are more willing to accept a complex architecture and try to make it work even though it is less than desirable. It's the executives, who are skilled at understanding the big picture impacts, that really grasp the challenges presented by complex and inflexible infrastructure."

The survey report analyzed the results to come up with three key findings: data initiatives are important, but have serious issues; inflexibility of data infrastructure is the underlying cause of many challenges; and cloud-based analytics could help address barriers to data analytics.

Additional data points that support these premises listed in the report include:

• 100 percent of respondents said data analytics is important.
• 48 percent of executives characterized data analytics as "critically important."
• 88 percent have faced "failures" with recent data initiatives.
• 59 percent of executives said their existing analytics infrastructure is too inflexible.
• 75 percent of executives are prevented from acting on business requests because their data infrastructure is too inflexible.
• 99 percent find potential benefits of cloud analytics to be compelling.
• 92 percent would try more things in a pay-as-you-go model licensing model.
• 74 percent of those who have adopted cloud analytics intend to grow use in the coming year.

The online survey of 376 individuals, including 104 VP- or C-level execs, was conducted this summer at the behest of Snowflake Computing, a 2014 start-up that provides a cloud-based data warehouse and data analytics.