Oracle Offers Solution for sun.misc.Unsafe in Java 9

What to do with sun.misc.Unsafe in Java 9? One side says it's simply an awful hack from the bad old days that should be gotten rid of; the other side says its heavy use is responsible for the rise of Java in the infrastructure space and popular tools still need it. The problem is, both sides are right. This week, Mark Reinhold, chief architect of Oracle's Java Platform Group, offered a solution.

Writing on the OpenJDK mailing list, Reinhold proposed encapsulating unsupported, internal APIs, including sun.misc.Unsafe, within modules that define and use them. That proposal is now a formal Java Enhancement Proposals (JEP). Posted this week, JEP 260 ("Encapsulate Most Internal APIs") aims to "make most of the JDK's internal APIs inaccessible by default, but leave a few critical, widely used internal APIs accessible, until supported replacements exist for all or most of their functionality." JEPs are similar to Java Specification Requests (JSRs), which are submitted to the Java Community Process (JCP).

"It's well-known that some popular libraries make use of a few of these internal APIs, such as sun.misc.Unsafe, to invoke methods that would be difficult, if not impossible, to implement outside of the JDK," Reinhold wrote, adding that the encapsulation scheme will, in the long run "reduce the costs borne by the maintainers of the JDK itself and by the maintainers of libraries and applications that, knowingly or not, make use of these non-standard, unstable and unsupported internal APIs."

When word got around a few months ago that sun.misc.Unsafe might be removed or hidden in Java 9, howls of protest echoed across the public network. The plan was "an absolute disaster in the making," declared one blogger. Cooler heads organized a working group to develop a document to raise awareness of the problems ditching sun.misc.Unsafe would create. Although still a draft document, "What to do about sun.misc.Unsafe?" is well worth reading. It includes a clear explanation of the uses to which sun.misc.Unsafe has been put over the years, suggestions for what should be done about it now, and a surprisingly (to me, anyway) long list of products that use it (JRuby, Grails, Scala, Akka, Hazelcast, Neo4j, Apache Spark and XRebel, to name a few).

Greg Luck, CTO at Hazelcast and co-author of the JCache spec (and JCP Executive Committee member), is a member of the working group. He learned that Oracle was considering removing or hiding sun.misc.Unsafe in Java 9 in June. So-called unsafe code is sometimes required for low-level Java programming, Luck explained, where developers need to modify platform functionality for a specific purpose. Open source projects in particular use sun.misc.Unsafe as a Java Native Interface (JNI) workaround.

"It's not meant to be a standard part of Java, and yet it's built into every JDK, and everybody uses it," he said. "It's a genie that got out of the bottle."

Martijn Verburg, CEO of jClarity and co-leader of the London Java Users Group, is another member of the working group. The reason sun.misc.Unsafe can't simply be dumped, he told me in an e-mail, is that it provides a number of functionalities that aren't available through any of the standard classes in OpenJDK.

"[sun.misc.Unsafe] should be cleaned up and the safe parts should get standardized," Verburg said. "The rest should be removed! If you want to perform dangerous manual memory allocations, there are other languages for that."

Both Verburg and Luck praised Oracle's proposal to encapsulate unsupported, internal APIs, including sun.misc.Unsafe. Verburg called JEP 260 "a fantastic pragmatic compromise" that "clearly shows [that the] OpenJDK leadership and Oracle are willing to listen to the needs of the ecosystem."

The community seems to be heading toward a solution to the sun.misc.Unsafe problem, and I'm sure it's due, at least in part, to the efforts of Verburg, Luck, and their colleagues in the working group. But this internecine dustup also raises a question that has been lurking in the background since the formation of OpenJDK: Who really makes the decisions about the future of Java? OpenJDK is an open-source community, but unlike the JCP (and organizations like the Apache and Eclipse foundations), it's not vendor neutral. The main goal of the JEP Process, according to the OpenJDK Web site, is "to produce a regularly updated list of proposals to serve as the long-term Roadmap for JDK Release Projects and related efforts." The JEPs allow Oracle to develop small, targeted features for the Java language and virtual machine outside the JCP.

"Who's in charge of Java? That's a very complex [question]," said Verburg. "The reality is that Oracle has the loudest voice, but it's a heavy collaboration .... For the parts of OpenJDK that make up the Reference Implementation of Java, the JCP still has to approve."

A list of the internal APIs Oracle has proposed to remain accessible in JDK 9 are listed on the JEP 260 page. Oracle is welcoming suggested additions to the list "justified by real-world use cases and estimates of developer and end-user impact."

BTW: Another great source for understanding sun.misc.Unsafe is Rafael Winterhaulter's January 2014 blog post, "Understanding sun.misc.Unsafe."

Posted by John K. Waters on August 7, 2015