Oracle CPU Does Little to Fix Serialization Vulnerability

Oracle's latest Critical Patch Update, the first of 2017, left Java security maven and Waratek CTO John Matthew Holt scratching his head about Big O's fix for a particular vulnerability: CVE 2017-3241, which affects Java SE, Java SE Embedded, and JRockit, and earned a CVSS score of 9.0 out of 10.0 (very bad).

Holt considers this CVE -- a deserialization vulnerability inside the core Java Remote Method Invocation (RMI) APIs -- especially serious, and he lamented Oracle's fix: a user-configurable whitelist/blacklist filter. He called deserialization an "application security nightmare," and Oracle's latest fix "easy, obvious and simple," but not especially effective.

"I'm really not knocking Oracle," he told me. "They did address the vulnerability, but it's a primitive form of defense, and far from what is considered to be the best way of dealing with it."

Serialization is the process of converting an object into a stream of bytes for transport and storage. Deserialization reverses the process when the data is received. Security researchers discovered that the RMI registry and DCG implementations in the RMI component of OpenJDK performed deserialization of untrusted inputs. A remote attacker could use this vulnerability to execute arbitrary code with the privileges of RMI registry or a Java RMI application.

In its security advisory, Oracle explained it this way: "This vulnerability applies to Java deployments, typically in clients running sandboxed Java Web Start applications or sandboxed Java applets, that load and run untrusted code (e.g., code that comes from the internet) and rely on the Java sandbox for security. This vulnerability does not apply to Java deployments, typically in servers, that load and run only trusted code (e.g., code installed by an administrator)."

High-profile attacks that exploit deserialization vulnerabilities abound, Holt said. To give me a local example, he cited last year's deserialization attack on the San Francisco Metropolitan Transit Agency's Municipal Rail (MUNI to locals). As security researcher Brian Krebs reported in November, the ransomware attack caused fare station terminals to carry the message, "You are Hacked. ALL Data Encrypted," and fare payment machines to read, "OUT OF SERVICE" in red LED letters. An SFMTA Web-facing server was probably compromised by the deserialization attack after it was identified by a vulnerability scan. A similar vulnerability was exploited to attack Med Star, the Maryland health system, last spring, and deprive multiple hospitals of access to critical systems.

The problem with the Oracle fix, Holt said, is fourfold:

1. Any heuristic or "hands-on" security technique based on whitelists/blacklists almost always requires intricate manual configuration and tuning to operate correctly.
2. Anything requiring manual human configuration and tuning requires special domain-specific and app-specific knowledge to configure, is expensive to test and validate, and prone to mistakes and human errors.
3. Incomplete or incorrect configuration/tuning virtually guarantees false-positive results, which will break application operation and service.
4. Most security professionals know nothing about the serialization mechanics or dependencies of the applications they're tasked with securing, making the job of configuring whitelists/blacklists virtually impossible.

"The consequences of requiring an application owner to know everything about the application -- to be God -- which this approach does, is that you are essentially putting the entire burden of the effectiveness of that security system on the end user, instead of the system," Holt said. "If an attack is successful, it's not the system's fault, but the user's. And that's just not an effective approach."

Waratek has found a way to circumvent this heuristic approach, and in the process, established a niche in a market overflowing with large enterprises that continue to run custom-developed, mission-critical applications on out-of-date versions. The Dublin-based company's virtualization-based AppSecurity product is designed to protect the Java Platform attack surface from known and unknown vulnerabilities by virtually applying critical patch updates and security policies at runtime. Waratek uses a proprietary Smart Compartment to process deserialization without relying on black or white lists.

"Walk into any really large organization and ask them how many Java applications they've got, and they'll tell you, oh we have 500 server-side applications, or 1,000 or 5,000," Holt said. "This is something we hear regularly. No human being is going to sit down in those organizations and manually create individual deserialization lists for those applications."

One of the knottier challenges facing organizations when it comes to application security -- and one Oracle can't solve with a CPU -- is the typical skill sets of the people charged with it, Holt said. "If you look at the security team in any major organization," he said, "most of the people have backgrounds in network security or systems administration. That's traditional. But application security, which is a relatively new problem space, involves thinking about the security inside the application, and these skill sets are inherently outside the application."

Adding to that challenge: App security can be frustratingly language-specific, he said. "The way deserialization vulnerabilities work and are exploitable on Java are entirely different from the way serialization vulnerabilities are exploitable on .NET or Ruby."

Posted by John K. Waters on January 24, 2017