News

Netflix Releases Open Data-Fetching Developer Preview

• By David Ramel
• August 18, 2015

Following an increasingly popular pattern of giving back source code to the open community, Netflix Inc. has released a developer preview of its in-house data-fetching tool used to provide immediate responsiveness in mobile and desktop apps via client-side caching.

Falcor is a home-brewed JavaScript library used to retrieve remote data and keep it ready for use, simulating the experience of having it already stored on a client, something of special importance to mobile devices with limited storage.

"Developers strive to create the illusion that all of their application's data is sitting right there on the user's device just waiting to be displayed," Netflix wrote in a post on its tech blog yesterday. "To make that experience a reality, data must be efficiently retrieved from the network and intelligently cached on the client."

Developers at the movie-streaming company use the tool to present data stored in the popular JSON data-interchange format. In this case, it's presented as a JSON Graph. "Falcor makes it easy to access as much or as little of your model as you want, when you want it," the company said. "You retrieve your data using familiar JavaScript operations like get, set and call. If you know your data, you know your API."

In its documentation, Netflix explains that a JSON Graph is a convention used to model data as a single JSON object. While often used in Web apps, Netflix said, map-based JSON presents problems by sometimes duplicating data because it uses a tree model, while most modern apps are in fact graphs. Thus, in the process of serialization, duplicate copies of the same information can be created, slowing down data retrieval and sometimes resulting in "stale data" through unpropagated changes to entities.

Developers can try to remove duplicates via client cache integration, through use of unique identifiers before caching. "Unfortunately as most object identifiers are not globally unique, but rather are unique among other entities of the same type, custom code must often be written for each new type added to the system," the company said. To circumvent that problem and remove duplicates, individual entities are inserted into a globally unique location in a graph object using a unique identifier.

"You code the same way no matter where the data is, whether in memory on the client or over the network on the server," Netflix said. "Falcor keeps your data in a single, coherent cache and manages stale data and cache pruning for you. Falcor automatically traverses references in your graph and makes requests as needed. It transparently handles all network communications, opportunistically batching and de-duping requests."

The company promised an upcoming roadmap for the project, which for now has its own main Netflix site and is also available on GitHub and as a private npm (Node package manager) costing $7 per month. The preview technology uses a Falcor Router not yet in production use, so it's included as a Node version for developers interested in trying it out. Also on GitHub is a Falcor express middleware demo, showing "how to create a stateless Virtual JSON Graph object on your application server using a Falcor Router and some express middleware."

While giving back to the community, Netflix is also enlisting the community for help with the project. "We are looking for community help to track down and fix bugs," the GitHub project states. "We are also looking for help integrating with existing MVC frameworks, as well as ports to other platforms."