News

Visual Studio Code Update Overloads npm Service, Forces Version Rollback

• By David Ramel
• November 3, 2016

Microsoft said its Visual Studio Code 1.7 release overloaded the npmjs.org JavaScript package management service for Node.js, forcing a rollback to version 1.6.1.

The overload was caused by a feature in VS Code that automatically acquires "typing files," which supply information about data types in JavaScript code in order to power the IntelliSense code-completion feature in VS Code.

Microsoft's TypeScript guidance includes this explanation of the static typing provided by TypeScript: "This is where type definition files come into play. They allow you to provide type information for JavaScript code that is by itself (by its very nature) not statically typed. The file extension for such a file is .d.ts, where d stands for definition. Type definition files make it possible to enjoy the benefits of type checking, autocompletion, and member documentation."

"The feature was so great that we started to overload the npmjs.org service," Microsoft said in a blog post yesterday explaining the rollback. "The right thing to do in the short term was to revert the release."

"If you had already upgraded to 1.7, you would have been prompted to install another update and you should now be at 1.6.1 again," the post continued. "We apologize for this inconvenience and we're working hard to make both short and long term fixes to bring you a great editing experience in VS Code. You should expect to see a 1.7.1 release early next week."

On the Hacker News social coding site, a user with the handle "seldo," identified as exec Laure Voss at npm Inc., which runs the JavaScript package manager, posted this message:

I'd just like to say on behalf of npm that Microsoft's handling of this incident was A+. As soon as we alerted them to the issue they were all hands on deck and did a rollback.

We've been really pleased that Microsoft chose to put their @types packages into the npm registry rather than a separate, closed system, and in general happy with Microsoft's support of node and npm. We're confident we can make the new features of VSCode work, we just need to work with Microsoft to tweak the implementation a little.

This was an honest mistake on their part, and we caught it in time that there was very little impact visible to any npm users.

Fun fact: at its peak, VSCode users around the world were sending roughly as many requests to the registry as the entire nation of India.

Meanwhile, the 1.6.1 Recovery Build was pushed out to VS code users. "We are releasing a 1.6.1 recovery build to add the final 1.6 translations and fix several important issues," the blog post said.

The October release -- VS Code is updated monthly -- contains significant updates to do with the TypeScript 2.0 language, code editor functionality, the Workbench, extensions, debugging and more. (Note: This article earlier incorrectly referred to the build as a September release).

When TypeScript 2.0 was announced in September, Microsoft highlighted a new feature that provides simplified the language's type definitions, specifically concerning "Declaration File (.d.ts) Acquisition."

Microsoft said at the time, "Typings and tsd have been fantastic tools for the TypeScript ecosystem. Up until now, these package managers helped users get .d.ts files from DefinitelyTyped to their projects as fast as possible. Despite these tools, one of the biggest pain points for new users has been learning how to acquire and manage declaration file dependencies from these package managers. Getting and using declaration files in 2.0 is much easier." There was no indication this new functionality contributed to the npm overload -- rather, the issue appears to relate to getting type data for non-TypeScript code.

In another Hacker News post, Voss provided more information on the issue: "CDNs don't usually cache 404s. VSCode was looking for @types packages for any and every npm package its users were using. Packages that had a type description caused no issue, but most packages don't, so we had a > 1000 percent spike in 404s. Our workaround before MS did the rollback was to cache 404s for @types packages specifically, and it was effective enough that the registry never really went down."

For those who want to delve into the nitty-gritty details of the problem, here's another explanation from Voss:

A VSCode person can (and probably will) answer in more detail, but at heart it's simple: if you want to add type-checking goodness to a library that isn't itself written in TypeScript, you can create a thing called a declaration file: https://github.com/DefinitelyTyped/DefinitelyTyped.

Microsoft publishes a list of known good declaration files for popular npm packages to npm, under the scope @types: https://www.npmjs.com/~types.

The 1.7 release of VSCode helpfully tries to automatically load type declarations for any npm package you use by requesting the equivalent declaration package under @types. When the package exists this is fine, because it's cached in our CDN.

What they forgot to consider is that most CDNs don't cache 404 responses, and since there are 350,000 packages and less than 5000 type declarations, the overwhelming majority of requests from VSCode to the registry were 404s. This hammered the hell out of our servers until we put caching in place for 404s under the @types scope.

We didn't start caching 404s for every package, and don't plan to, because that creates annoying race conditions for fresh publishes, which is why most CDNs don't cache 404s in the first place.

There are any number of ways to fix this, and we'll work with Microsoft to find the best one, but fundamentally you just need a more network-efficient way of finding out which type declarations exist. At the moment there are few enough that they could fetch a list of all of them and cache it (the public registry lacks a documented API for doing that right now, but we can certainly provide one).