Limitations of reflective method lookup
- By Paul Holser
- July 24, 2001
REFLECTION IS A powerful and intriguing feature of Java. Using the facilities of the class
java.lang.Class and the Reflection API (
package java.lang.reflect), programmers can gather information about classes in the JVM at runtime, including references to methods and constructors. After obtaining these references in the form of
Method and
Constructor objects, developers can invoke them with parameters (via the
invoke() method of
Method and the
newInstance() method of
Constructor) and manipulate the results.
We use the getMethod() and getConstructor() methods of Class to obtain Method and Constructor handles for reflective invocation. Ideally, these methods would be able to make their selections based on a name and the types of the actual arguments to be used in the reflective invocation, just as Java compilers do at compile time on direct invocations in source. Unfortunately, contrary to what many programmers believe, they aren't that flexible.
To illustrate, let us adapt an example from Java in a Nutshell.1 It shows a class called UniversalActionListener (see Listing 1), which implements the interface java.awt.event.ActionListener so that instances can be notified of GUI actions. The motivation behind this class is that listeners of this sort are often very simple, merely invoking a method on some object, possibly passing parameters. Rather than writing numerous classes that implement ActionListener, each calling a known method on its own type of target object, you can just create many instances of UniversalActionListener, each of which knows a target object, the name of a method to invoke on the target when notified, and the parameters to pass to the method.
You construct an instance of UniversalActionListener using the Object to invoke on (target), the name of the method to invoke on the Object (methodName), and the parameter to pass to the method (arg). If arg is null, we assume methodName names a zero-parameter method; otherwise, we assume it names a one-parameter method. If the constructor cannot find the method, it throws NoSuchMethodException. As written, UniversalActionListener cannot invoke methods that take more than one parameter, though simple modifications would enable this.
The constructor sets the stage for the actionPerformed() method to invoke a method reflectively. First, it stores the target Object and the parameter as member data. Then it seeks the desired method by obtaining the Class object (using getClass()) associated with the target, and calling getMethod() on that Class to obtain a Method object for reflective invocation. Because a method's name and parameter types make it unique within its class, the constructor calls getMethod() with the given method name, and a one-element Class array representing the desired type of the method's formal parameter (or a zero-element Class array if we seek a zero-parameter method).
If we have two javax.swing.JButton objects, b1 and b2, and we want the text of the first to change when the second is clicked, we can do this:
b2.addActionListener(
new UniversalActionListener(b1, "setText", "bang"));
When you construct the
UniversalActionListener, it will find the method
javax.swing.AbstractButton.setText(String), and invoke it on
b1 whenever
b2 is activated.
Now suppose we have two (contrived) simple classes, Blah and SubBlah, one extending the other (see Listings 2 and 3). Further, suppose we have the following instances of Blah and SubBlah:
Blah blah1 = new Blah(33);
Blah blah2 = new Blah(44);
Blah sub = new SubBlah(55);
Now set up a
UniversalActionListener to invoke
foo() on
blah1 with the parameter
blah2 whenever
b1 is clicked:
b1.addActionListener(
new UniversalActionListener(
blah1, "foo", blah2));
No problem. What about:
b1.addActionListener(
new UniversalActionListener(
blah1, "foo", sub));
It should be OK, right? Directly, we can invoke
foo() on an instance of
Blah with a
SubBlah parameter, since
SubBlah is a subclass of
Blah. (Hooray for inheritance!)
Reflectively, we can do the same because
Method.invoke() is smart enough to perform a widening conversion to treat the
SubBlah parameter as a
Blah. However, constructing this instance throws a
NoSuchMethodException. What about this?
b1.addActionListener(
new UniversalActionListener(
blah, "bar", new Short("2")));
It should be OK, right? Directly, we can invoke
bar() on an instance of
Blah using a short parameter because the
short can be widened to an
int.
Reflectively, we can invoke
bar() by passing a
Short in our array of parameters to the invocation because
Method.invoke() is smart enough to perform unwrapping of primitive-wrapper
Objects to their primitive formal parameter counterparts and do widening conversions as necessary. Constructing this instance, however, also throws a
NoSuchMethodException.
Surprised? I was. If the purpose of obtaining a Method is to invoke() it later with actual parameters of certain types, why aren't those types useful for obtaining the method using Class.getMethod()?
Here is the first limitation: Class.getMethod() looks for a method with the given name and formal parameters of exactly the types you indicate. In many cases, this suffices because you often know the exact signature of the desired method in advance. However, in our example, we're building up the Class array for Class.getMethod() by using the type of the object to pass in the reflective invocation. This type may not be the exact type in the signature of the method, but instead may be a subtype or a type that is otherwise compatible for reflective invocation. This is the case in our previous demonstration when we wanted to use a SubBlah to call foo() and a short to call bar().
We need the ability to find not an exact match in method signatures, but merely a compatible match. We want to find a method for which reflective invocation is possible, using a set of parameters of particular runtime types. We only need a method with a given name accepting some number of formal parameters, whose types are compatible with the runtime types of the actual parameters we expect to use on the invocation. This ability is the runtime equivalent of the work Java compilers do to resolve method calls at compile time.
This need is more prevalent than you might think at first. For example, ORBs receive requests to invoke methods on resident objects as a byte stream. This byte stream is unmarshalled and interpreted as a reflective method invocation on a target object. The request might contain a reference or tag to allow the ORB to locate the target object, the name of the method to invoke, and the serialized bytes of the objects to pass as parameters to the method. Once the ORB locates the target object and unmarshals the parameters, it must invoke the method reflectively. But to find the right method, calling getMethod() on the Class of the target object won't work in the scenarios we saw previously.
Or perhaps you need to write a simple scripting language in Java that can import Java classes, create Java objects, and invoke methods on them. (Several such languages exist, like Jython.) The language's parser must interpret the text of the script. Suppose we have a script like this (using Perl syntax):
$a = java.util.ArrayList->new();
$a->add(5);
An interpretation of this script might be to:
- Obtain the Class object for java.util.ArrayList
- Call newInstance() on the Class object, assigning the result to $a
- Ask the Class instance getMethod("add", new Class[] {Integer.class})
- invoke() the resultant Method on $a with parameter new Object[]{new Integer(5)}
getMethod() wouldn't work, though, since the one-parameter method
ArrayList.add() accepts an
Object, not an
Integer (the type of "5" in our script). Remember, even though an
Integer is a kind of
Object,
Class.getMethod() looks for exact matches in the parameter types, so it would not find the compatible method.
You may also develop a class with a generic interface, as Lowell Kaplan described in his Java Report column2:
Object callMethod(String methodName, Object[] params)
throws InvocationTargetException, NoSuchMethodException,
IllegalAccessException
You'll need a way to locate the desired method based on the method name and the runtime types of the parameters. Once again,
getMethod() will fail under certain conditions. In short, to find methods based on the runtime types of objects that are to be used in the reflective invocation, you must replace
Class.getMethod().
In his column, Kaplan describes a MethodFinder class (see Listing 4 in our code section). This class's constructor accepts a Class object, and its findMethod() returns a Method on that Class that matches a name and the runtime types of some set of Object parameters. Sounds promising.
MethodFinder would suit our purposes, except that:
- Its handling of primitives is lacking. When matching parameters, MethodFinder knows to match a primitive-wrapper actual parameter against a corresponding primitive formal parameter in a method's signature (e.g., a Character matches char). This is because the reflective invocation would unwrap the primitive-wrapper to the primitive. But it doesn't recognize that Method.invoke() will widen a primitive-wrapper actual parameter if necessary, so that, for example, a Float matches a double. Instead, if the primitive-wrapper match fails, MethodFinder resorts to checking that the Class of the formal parameter isAssignableFrom() the Class of the actual parameter. This works for matching subclass actual parameters against superclass formal parameters. However, the Class of a primitive is never assignable from an instance of any Object, much less a primitive wrapper. When invoked on the Class of a primitive type (e.g., Integer.TYPE), isAssignableFrom() returns true only if the Class passed to it is the same as the receiving Class. Primitive widening also seems to be left out of the picture. To illustrate, try this code:
boolean b =
Integer.TYPE.isAssignableFrom(Short.TYPE);
b will contain false, even though you can assign a short to an int. So, though there might be a compatible match with primitive formal parameters in the target class, MethodFinder may not find it.
- It assumes at most one compatible match; in our scenarios, there may be more. Imagine the following script from our scripting language:
$a = Math.max(2, 3);
Here, supposing our scripting language implements integers using java.lang.Integer, Math.max(int,int) and Math.max(long,long) would be compatible matches, since an int promotes to a long.
The problem is not so much that MethodFinder returns one compatible match, but that: a) because of item #1, it might not find that one match; and b) if we make modifications so that it can recognize more than one compatible match, it's not clear which one it would return. Class.getMethods() loads MethodFinder's method caches, and Class.getMethods() does not specify the ordering of the list it returns.
- It cannot handle actual parameters that are null. null is assignable to an Object reference or an array reference, but not to a primitive. MethodFinder does not address this possibility.
- It may return methods that cannot be invoked for accessibility reasons. For example:
List list = new ArrayList();
MethodFinder f =
new MethodFinder(list.iterator().getClass());
Method m = f.findMethod("hasNext", new Object[0]);
This code finds the method
hasNext() on the
Iterator returned from
List.iterator(). However, doing an
invoke() on
m throws
IllegalAccessException. Why? Because the true class of
list.iterator() is an anonymous inner class, inaccessible to us. We should be able to invoke these methods. We can, but we must do still more work.
Listing 5 (see the code section) shows a solution that overcomes some of the limitations I've discussed. This BetterMethodFinder class improves on its predecessor by accounting for subclassing and widening of primitives when searching for methods, as well as addressing the issues surrounding null and accessibility. These improvements require performing significant code gymnastics. In addition to finding methods, this version can also find constructors using the same principles.
You construct a BetterMethodFinder with the target Class for looking up methods and constructors. If you pass null or a Class that represents a primitive or an array type, the constructor throws IllegalArgumentException. Otherwise, the constructor stores the target Class as a member datum and calls two private methods, loadConstructors() and loadMethods(). These query the target Class for its accessible constructors and methods, and caches them for quick lookup.
Note the target Class might be too private to allow invocation of its methods, but some of those methods might override from a superclass or implemented interface. If necessary, loadMethods() looks for the method using ClassUtilities.getAccessibleMethodFrom(). This searches the target Class's hierarchy (superclass and interfaces, and their hierarchies recursively) until it finds the desired method in an accessible class or interface.
Once you've constructed your BetterMethodFinder, you call findMethod() or findConstructor() to get a Method or Constructor suitable for reflective invocation. Both methods defer to a private method findMemberIn(), which operates on a List of Members (the superclass of Method and Constructor). Like Class.getMethod(), findMethod()'s parameters are the name of the desired method as a String, and an array of Class objects representing the types of the objects to be used in the reflective invocation.
findMethod() passes to findMemberIn() a list of methods in the target class with the desired name, and the desired parameter types as a Class array. If there are no such methods, it throws a NoSuchMethodException. Otherwise, findMemberIn() iterates over the methods. If there is an exact signature match, it is returned. Otherwise, findMemberIn() looks for those methods whose formal parameter types are compatible with the desired types for reflective invocation. ClassUtilities.compatibleClasses() performs this check. (See Listing 6 in our code section.)
compatibleClasses() compares the contents of two Class arrays. For reflective invocation, the types are compatible if the lengths of the arrays are equal, and if each type in the "right-hand-side" array is compatible with its corresponding "left-hand-side" type. For each "right-hand side," compatibleClasses() checks Class.isAssignableFrom() against its corresponding "left-hand-side" type. If that fails, it might be because primitives are involved (remember the limitation I described). Now it must check whether the primitive equivalent of the "right-hand-side" type can be widened to the primitive equivalent of the "left-hand-side" type. The method ClassUtilities.primitiveIsAssignableFrom() does this check. primitiveIsAssignableFrom() uses a mapping from a primitive type to the primitive types it can be widened to. For example, an int can be widened to long, float, or double.
If a "right-hand-side" type is null or Void.TYPE, it is compatible with any Object or array "left-hand-side" type, but not with a primitive. compatibleClasses() treats both null and Void.TYPE as stand-ins for a null actual parameter for the reflective invocation. This is because null is an acceptable value for an Object or array reference, but not for a primitive. Void.TYPE is typically used to determine whether a Method's return type is void, but it is used here as an alternate placeholder for null.
Once it has a set of compatible methods, findMemberIn() must determine which one to return. It could simplify things by assuming that each compatible method is equally good, and return an arbitrary member of the set. Or, it could avoid dealing with ambiguity entirely, and throw an exception if there are many compatible methods.
BetterMethodFinder takes its cue from The Java Language Specification3 instead. In section 15.12.2, the authors describe how to choose between overloaded methods that could be invoked with some set of parameters. They outline what it means for a method to be the most specific in a set of overloaded methods. This part of the specification has compile-time resolution in mind, but I adopt this strategy for choosing among methods for reflective invocation—we want to choose the most specific method, if one exists. We would like the procedure for finding a compatible method at runtime to adhere as closely as possible to the compile-time procedure.
So, once findMemberIn() has its set of compatible methods, findMostSpecificMemberIn() iterates over each of them, deciding whether it is more specific than the most specific method found so far. Informally, a method signature is more specific than another if all of the types in its parameter list can be converted to the corresponding types in the other's parameter list. For example, Foo.bar(byte,short) is more specific than Foo.bar(int,int), and Foo.baz(Derived) is more specific than Foo.baz(Base). The private method memberIsMoreSpecific() serves this purpose.
If a given pair of methods is mutually unspecific, then there is an ambiguity until findMostSpecificMemberIn() finds another method that is more specific than both of these. If it has exhausted the list of compatible methods and has not broken the ambiguity, findMemberIn() throws a NoSuchMethodException. Otherwise, it has found the most specific compatible method for returning.
Now we can modify UniversalActionListener, replacing the call to Class.getMethod() with the creation of a BetterMethodFinder and a call to findMethod():
this.m =
new BetterMethodFinder(clazz)
.findMethod(methodName, parameterTypes);
Also, because
BetterMethodFinder() creates and throws its own
NoSuchMethodExceptions with its own messages, we can eliminate the
try/catch construct, letting those exceptions propagate out of
UniversalActionListener's constructor.
As a convenience for programmers, BetterMethodFinder provides two static methods named getParameterTypesFrom(). One accepts an Object array and the other accepts a String array. Each returns an array of Class objects, corresponding either to the runtime types of the elements of the Object array, or to the classes and primitives named by the Strings in the String array. This Class array is suitable for passing to findMethod() or findConstructor(). A null Object, a null String, the empty string, the string "null", and the string "void" each yield the class Void.TYPE. getParameterTypesFrom() would be useful for UniversalActionListener's constructor if we wanted to modify it to accommodate methods that accept more than one parameter.
Unlike Class.forName(), getParameterTypesFrom(String[]) can retrieve the classes for primitive types. Simply specify the primitive's name, e.g., "char" and "float." For array types, use the names of array classes as described in the API documentation for Class.getName(), e.g., "[I" for an integer array and "[[Ljava.lang.Object;" for a two-dimensional array of Objects.
It is worth noting that BetterMethodFinder finds only public constructors and methods in the desired class. This is because it calls getConstructors() and getMethods() on the target Class in the implementation. If we had used getDeclaredConstructors() and getDeclaredMethods(), we would have also attempted access to protected, package-private, and private constructors and methods. However, doing so would have meant forcing the users of BetterMethodFinder to worry about SecurityExceptions that might result if a security manager is in place, and excluding inherited methods from the searches. getDeclaredMethods() returns only methods declared in the target Class.
Acknowledgements
Thanks to Keith Robertson, Steve Morgan, Paul Jakubik, Dave Howard, and Michael Harreld for helpful suggestions. Thanks to Kent Beck and Erich Gamma for the greatness of JUnit, which was used to test the code described here.
References
- Flanagan, David. Java in a Nutshell, 2nd ed. O'Reilly and Associates, 1997.
- Kaplan, Lowell. "More programming with generic interfaces." Java Report, Vol. 5, No. 2, Feb. 2000, pp. 70–74.
- Gosling, James, et al. The Java Language Specification, 2nd ed. Addison–Wesley, 2000.