Jesse Khunert pointed out correctly, that I was not properly testing OGNL's new bytecode enhancer due to my ignorance of the API.
Indeed, OGNL 2.7 would appear to be faster than MVEL in terms of pure bytecode generation. But this is not the entire story. If we take a look at MVEL's reflection-based performance vs. OGNL's reflection-based performance, it's no contest.
Let's take a look at some test source code (using latest OGNL and latest MVEL 1.2 beta):
--snip--snip---
// Expression we'll test.
String expression = "foo.bar.name";
// Number of iterations
int iterations = 100000;
Base base = new Base();
// Compile expression in MVEL
Serializable mvelCompiled = MVEL.compileExpression(expression);
// Disable MVEL's JIT by making the default optimizer the Reflective optimizer.
OptimizerFactory.setDefaultOptimizer(OptimizerFactory.SAFE_REFLECTIVE);
// Compile OGNL AST
Object ognlCompiled = Ognl.parseExpression(expression);
// We loop twice, once to warm up HotSpot.
for (int repeat = 0; repeat < 2; repeat++) {
long tm = System.currentTimeMillis();
for (int i = 0; i < iterations; i++) {
MVEL.executeExpression(mvelCompiled, base);
}
// Let's not report the results the first time around, HotSpot needs to warm up
if (repeat != 0) System.out.println("MVEL : " + ((System.currentTimeMillis() - tm)) + "ms");
tm = System.currentTimeMillis();
for (int i = 0; i < iterations; i++) {
Ognl.getValue(ognlCompiled, base);
}
// See above.
if (repeat != 0) System.out.println("OGNL : " + ((System.currentTimeMillis() - tm)) + "ms");
}
Full source here
In the above test we put MVEL and OGNL on equal footing. We kill MVEL's internal JIT and we let OGNL and MVEL fight it out using pure reflection. So what do the results look like?
MVEL : 56ms
OGNL : 615ms
Pretty big difference. MVEL is 10 times faster in reflection mode. And you might say: so what? I'm just going to use the JIT from now until forever.
That sounds like a great idea until you befallen the great caveat of on-the-fly code generation in Java: classes don't get garbage collected until their ClassLoader is garbage collected.
Early on in development, we realized the problem created with thousands of bytecode optimizers being generated on-the-fly in large systems when we started running into JVM crashes due to an overflow of classes in the permanent generation. While work-arounds exist, such as the use of wacky JVM options (which often have wacky caveats like breaking singletons) and one-classloader-per-class schemes (a horrible idea) we found that it was impossible to provide a consistent, out-of-the-box safe integration experience for web frameworks and other systems which might find themselves using MVEL as a binding language.
Instead of wait for the Java world to catch up to the world of code generation, we decided to keep our eye on the ball of our reflection-based performance, and as such MVEL allows for parallel and hybrid compilation of both reflective accessors and bytecode generated accessors.
But why is MVEL's bytecode still around 1.2 to 1.5 times slower than OGNL generated bytecode?
MVEL, as a dynamically typed language (with optional static typing) still requires callbacks to the MVEL runtime in order to perform expression egress type narrowing (I'll explain that later) as well as providing consistent type coercion. In fact, unlike OGNL's bytecode compiler, which performs a static type analysis for method calls and accessors, MVEL provides inline dynamic de-optimization points to allow the same compiled expression to be used with two unrelated types. For example:
class Foo {
private String name;
public String getName() { return name; }
}
class Bar {
private CharSequence name;
public CharSequence getName() { return name; }
}
Say we initialize both classes Foo and Bar, and then compile the expression name. Then we apply that compiled expression against each object. Observe the ClassCastException in OGNL 2.7 while MVEL re-optimizes and hums along :)
