News

Microsoft Pitches Surface to Developers at VSLive!

• By John K. Waters
• February 27, 2009

SAN FRANCISCO -- Microsoft officials this week tried to convince developers to build applications for its Surface -- the company's multi-user, multi-touch technology that turns a specially designed table top into an interactive multi-touch interface.

The two officials, Robert Levy and Doug Kramer, made their pitch Wednesday at the VSLive! conference here. Unveiled last April, Surface is entirely touch-based: no mouse, keyboard or joystick. As a surface computing platform, it provides access to applications and data via "natural gestures, touch and physical objects." Applications developed for this system currently run only on Microsoft-built hardware: a 30-inch diagonal display -- basically, a table -- that sells for around $12,000 to $15,000 a pop.

Despite its hefty price tag, the company has sold "hundreds" of Surface systems, including to Harrah's iBar in Las Vegas, some Sheraton hotels, MSNBC (which used a Surface app during its coverage of the recent presidential election), and some AT&T retail outlets, said Kramer, who is lead for the Surface developer experience. It is among several rich client technologies showcased this week at VSLive!

While the Surface has narrow appeal to developers, Kramer said it is the harbinger of things to come in terns of user interface design -- from graphical user interface (GUI) to a "natural user interface."

"We believe that by the year 2013, this [NUI world] will be a multibillion dollar ecosystem with plenty of opportunity for developers," he said.

Levy, who led the design and development of the APIs and controls in the Suface SDK, said the learning curve is not all that high. "The technology from the GUI world has become so advanced that it applies and carries over," Levy said. "You still have the same languages, with extended visual frameworks to accommodate the nuances of natural user interface."

The key challenge for developers working with this technology, Levy said, lies in the way they think about the design of their applications. "You have to start thinking about interaction paradigms that you've never considered before," he said. "You have to start taking into account multiple, simultaneous pieces of input. You may have 10 fingers and three or four objects interacting with your application at the same time, and you have to accommodate that."

The Surface system is currently optimized for 52 simultaneous contacts on the surface, though that number is theoretically limitless. "We figured four users, which is 40 fingers," Levy explained. "Add three physical objects per user and you get to 52."

The SDK group is providing Surface developers with tools designed to help them adapt their Windows Presentation Foundation (WPF) skills. "We take all the fancy Surface input and we report it into WPF applications the same way that mouse, stylus and keyboard input is reported," Levy said. "That means it will automatically do hit testing, and we'll have routed events and attached properties, just as you get with a mouse and stylus."

The APIs the group is providing look and feel a lot like WPF input APIs, Levy said, and the controls feel a lot like WPF controls. "That should mean that it's really easy to take an existing application and port it over to Surface," he said.

The SDK group has also developed three sets of controls that map WPF to Surface, including Surface versions of the most used WPF controls: button, listbox, scrollbar; plus Surface-specific versions of common controls that don't make much sense outside of Surface (multiple menus that open simultaneously, flicking, elasticity, virtual keyboard); and a set of controls that allow developers to use base classes for building their own Surface-specific controls.

Levy and Kramer highlighted two new controls for Surface: ScatterView control, which is the first they developed and one "you'll see in pretty much every Surface application." The ScatterView control is designed to enable applications to take any type of content and set it up for direct manipulation and interaction through gestures "in a 360 degree type of way," he said. Levy likened this control to the Items control class in WPF, from which are derived listbox, preview, listview, menu and others.

The second control is based on the Surface system's ability to recognize physical objects via patterns of dots applied to the bottom of objects. Called "Tags," these dot patterns come in two types: "byte tags," which can hold 256 unique values; and "identity tag," which is effectively a GUID that accommodates billions and billions of values encoded in a square inch of dots. The control designed for these tags is the TagVisualizer.

The Surface SDK is built on WPF 3.5 and Windows Vista SP1, and includes a bundle of multi-touch APIs and special controls.

The current Surface release is 1.0; Surface 2.0 is schedule for release with .NET 4.0, Kramer said, and when Windows 7 is released, the new operating system will come with several multi-touch features developed in the Surface group, including native and common APIs for multi-touch. "As a .NET developer, you'll be able to work with the common multi-touch API to do some common interop to build basic multi-touch applications," he said.

"We're also taking lots of the thinking in the Surface SDK -- how we do manipulations, gestures and controls -- and we're in the process of moving those things into WPF 4.0, so that you can use the exact same concepts and code on multi-touch Windows 7 and on Surface," Levy said.

Does that mean that Surface will be absorbed into Windows 7?  "Surface isn't going away," Levy said, "because of the unique things you can do with it that you can't do with standard multi-touch."

Levy said that, despite the small market for Surface (developers have to buy the whole system to develop for it -- and the Surface system isn't yet supported by MSDN), Microsoft is investing big in the technology. The SDK group has grown to a staff of about 200, he said.

"What we're saying is this is the wave of the future," Levy said. "And if you're a developer who wants to ride that wave, now's the time to jump on."