Qualcomm Using AI to Turn IoT Cameras into Videographers -- ADTmag

Qualcomm Using AI to Turn IoT Cameras into Videographers

By Richard Seeley
May 4, 2018

Bad guys beware. Security cameras are about to get a whole lot sharper thanks to Qualcomm Vision Intelligence Platform using advanced artificial intelligence (AI) technology.

The days are numbered for the low-resolution, black-and-white security camera videos of bank robbers that often accompany TV news reports. You know the ones where you look at a shaky shadowy evildoer and say: "That could be anyone." So you laugh when the news anchor intones: "If you recognize this person, call the police at the number on your screen."

With cutting edge AI, the security footage will make it look like a videographer was on hand as the guy held up the neighborhood convenience store. Beyond crime fighting, the technology can be used for "sports cameras, wearable cameras, virtual reality (VR) 360 and 180 cameras, robotics, smart displays, and more," Qualcomm noted in its announcement. Applications of this AI technology may include mountain bikers and skiers making cinematic videos of their rides and runs. With the virtual reality capability, friends could join in the fun without leaving their living rooms.

The new platform's ability to provide "extraordinary" image quality includes "4K video resolution at 60 frames per second (fps), or 5.7K at 30 fps, as well as multiple concurrent video streams at lower resolutions," according to the announcement. Qualcomm says it is "the most powerful camera processor" it has ever made.

Qualcomm is touting the Vision Intelligence Platform as its "first family of system-on-chips (SoCs) purpose-built for the Internet of Things (IoT) using advanced 10nm FinFET process technology."

The announcement explains: "The QCS605 and QCS603 SoCs are engineered to deliver powerful computing for on-device camera processing and machine learning, with exceptional power and thermal efficiency, across a wide range of IoT applications. The SoCs integrate Qualcomm Technologies' most advanced image signal processor (ISP) to date and the Qualcomm Artificial Intelligence (AI) Engine, along with a heterogeneous compute architecture including a cutting-edge ARM-based multicore CPU, vector processor and GPU."

"Our goal is to make IoT devices significantly smarter as we help customers bring powerful on-device intelligence, camera processing and security" according to Joseph Bousaba, vice president, product management, at Qualcomm, who was quoted in the product release. "AI is already enabling cameras with object detection, tracking, classification and facial recognition, robots that avoid obstacles autonomously, and action cameras that learn and generate a video summary of your latest adventure, but this is really just the beginning."

The new platform is designed to be a launch pad for the development of new applications for IoT devices, he said.

KEDACOM, which develops AI solutions for surveillance and law enforcement applications, and Ricoh THETA, which offers "a camera that shoots everything in 360 degrees," are planning to develop products based on the Qualcomm technology, the company notes.

Technology in the platform includes hardware and software for accelerated on-device AI, the company noted. It includes Qualcomm Snapdragon Neural Processing Engine (NPE) that offers tools for analysis, optimization, and debugging of applications based on "Tensorflow, Caffe and Caffe2 frameworks, Open Neural Network Exchange interchange format, as well as Android Neural Networks API and Qualcomm Hexagon Neural Network library -- all of which are designed to allow developers and OEMs to easily port trained networks into the platform."

Additional technical and support details for developers are available.