News

Facebook Leads Effort to Improve MySQL for Large-Scale Use

• By David Ramel
• March 27, 2014

Facebook today announced an alliance with Web powerhouses Google Inc., LinkedIn and Twitter Inc. to improve the open source MySQL database for large-scale applications.

Called WebScaleSQL, the new collaboration is based on version 5.6 of MySQL, the open source database managed by Oracle Corp. and one of the leading relational databases used in Big Data systems.

"Our goal in launching WebScaleSQL is to enable the scale-oriented members of the MySQL community to work more closely together in order to prioritize the aspects that are most important to us," wrote Facebook engineer Steaphan Greene in an introductory blog post. "We aim to create a more integrated system of knowledge-sharing to help companies leverage the great features already found in MySQL 5.6, while building and adding more features that are specific to deployments in large-scale environments."

The project code on GitHub was updated two days ago, but Greene started a Facebook group page for the project on Oct. 17. The page currently reports 109 people in the group.

The work done so far on the collaboration has resulted in improvements such as an automated framework that will run tests on proposed changes and publish the results from the MySQL built-in test system; a new set of stress tests and an automated performance testing system prototype; plus other testing and performance enhancements, according to an FAQ.

No binaries will be produced, as the stated goal is to develop a common set of code changes that interested parties can use as a reference point to develop customized code for specific use cases. Thus, the collaboration is referred to as a branch of the code, rather than a fork of the MySQL Community release.

Explaining the choice of MySQL rather than other databases such as MariaDB, Percona Server, Drizzle or others, the FAQ stated: "We reached a consensus that MySQL-5.6 was the right choice for this, as it has the production-ready features we need to operate at scale, and the features planned for MySQL-5.7 seem like a fitting path forward for us. We will continue to revisit this decision as the ecosystem evolves." Oracle claims MySQL is "the world's most popular open source database."

The alliance members have set up a system to collaborate by suggesting changes, reviewing code and reporting bugs. Greene said that after a team engineer proposes a change, it will be reviewed by an engineer from another participating company. "If both engineers agree the change makes sense and is functional, it will be pushed into the WebScaleSQL branch for everyone to use," Greene said. "Beyond this, each organization may further customize WebScaleSQL to suit its own needs, just as we all do today."

At Facebook, work is reportedly underway to contribute to an asynchronous MySQL client and preparing to move some "production-tested versions of table, user, and compression statistics" and other components into WebScaleSQL. Greene said the non-blocking client is being reviewing by other WebScaleSQL teams after several months of use at Facebook.

"We're excited to expand our existing work on WebScaleSQL, and we think that this collaboration represents an opportunity for the scale-oriented members of the MySQL community to work together in a more efficient and transparent way that will benefit us all," Greene said.