News

Survey: DevOps Teams Face Challenges Deploying Machine Learning Apps

• By Richard Seeley
• October 17, 2018

Data science and machine learning teams are not doing enough training and iterating models because they may be bogged down with infrastructure, deployment and engineering issues, according to a survey released this week of 523 data scientists and machine learning by professionals Seattle-based Algorithmia Inc.

When it comes to developing and deploying machine learning, the 2018 "State of Enterprise Machine Learning" report found that larger companies, defined as having 2,500+ employees, are happier with how things are going than smaller companies with 500 employees or less.

The ability of big businesses to hire data science talent and bankroll major projects appears to make a big difference in the successful implementation of machine learning.

"In 2018, large enterprise companies have an advantage when it comes to machine learning because they have access to more data, can continue to invest in big R&D efforts, and have many problems that machine learning technology can solve cost-effectively," Diego Oppenheimer, CEO at Algorithmia, was quoted as saying in a press release announcing the survey. "And yet, even in the largest companies, productionizing and managing machine learning models remains a challenge."

Big tech-based companies including Google, Facebook and Uber are creating a new ML-based infrastructure, which Algorithmia dubs the "AI Layer." An AI Layer manages compute loads, automates deployment of machine learning models, and propagates machine learning throughout the company, according to Algorithmia.

But as Oppenheimer noted the machine learning adoption and application is not without challenges, even for tech giants.

The Algorithmia survey, which focused on organizations in North America, found that while ML is getting a big boost from "massive investments of time, money and focus," human intelligence appears to be dogging artificial intelligence efforts. "For example, data science and machine learning teams are spending too much time on infrastructure, deployment and engineering, and not nearly enough (less than 25 percent) on training and iterating models," the report on the survey results concludes.

To show where the survey respondents see ML roadblocks, Algorithmia cited the following results:

• 30 percent of respondents reported challenges in supporting different programming languages and training frameworks. Machine learning models often are created using a number of different programming languages and training frameworks. This adds an additional level of complexity because several models, written in distinct languages and frameworks, must be pipelined together for a given task. Larger companies, like Facebook and Uber, have solved some of these issues by building large internal tools such as FBLearner and Michelangelo, respectively.
• 38 percent of respondents reported difficulty in deploying models to the needed scale. Anecdotally, the reasons include: DevOps and IT teams not having sufficient resources; data scientists being expected to build the infrastructure to put their models into production; and a lack of existing infrastructure within the organization to support the needs of running ML models at scale.
• 30 percent reported challenges in model management tasks such as versioning and reproducibility.

Despite these challenges Oppenheimer sees a bright future for ML in large and small enterprises: "In general, larger companies have more machine learning use-cases in production than smaller companies. But across the board, all companies are getting smarter about where and how to apply ML technology. We expect to see big leaps in productionized machine learning next year as data scientists can more easily deploy and manage their models."