Thursday, November 15, 2007

JUEL vs. MVEL

Some people recently asked me what the advantages of MVEL over, say, JUEL were. After all, JUEL implements a JSR standard, and it is supposed to be very fast. So I thought I would have to put JUEL through it's paces.

I designed MVEL to be super-easy to integrate, negate the need to provide factory caching, and integrating with sophisticated interfaces and APIs.

Even thought we provide a very simple facade to MVEL, we provide fully extensible external resolvers and extendability of the coercion support, etc. But we also provide a very trivial and bullet-proof integration API to hide the complexity for those who don't need it.

Certainly, when we use the "simple approach" to MVEL, we compromise some performance by requiring that the MVEL convenience methods provide lightweight wrappers to inject variables from Map's, etc. When using this convenient method, MVEL must be slower than taking advantage of the performance boosting power of factory caching in EL implementations, right? Well, let's see.

Following the integration instructions, I faithfully tried to perform a "fair" test between MVEL and JUEL.

Firstly, I started by defining a simple POJO class: Foo.java

public class Foo {
private String name = "Foo";
private Bar bar = new Bar();

public String getName() {
return name;
}

public void setName(String name) {
this.name = name;
}

public Bar getBar() {
return bar;
}

public void setBar(Bar bar) {
this.bar = bar;
}
}


Then I defined two simple tests to run side-by side. Here are the individual tests, but you can view the full file here.

public void runJUEL() {
ExpressionFactory factory = new de.odysseus.el.ExpressionFactoryImpl();
de.odysseus.el.util.SimpleContext context = new de.odysseus.el.util.SimpleContext();
context.setVariable("foo", factory.createValueExpression(foo, Foo.class));
ValueExpression v = factory.createValueExpression(context, "${foo.name}", String.class);

for (int i = 0; i <>
          if (!"Foo".equals(v.getValue(context))) throw new RuntimeException("invalid value returned");        
        }     
    }     
    public void runMVEL() {         
       Serializable s = MVEL.compileExpression("foo.name");          
       // inject variables into MVEL via a Map (for convenience)         
       Map map = new HashMap(1);         
       map.put("foo", foo);          
       for (int i = 0; i <>
          if (!"Foo".equals(MVEL.executeExpression(s, map)))                 
              throw new RuntimeException("invalid value returned");         
          }    
       }
    }

I elected to run each of these tests 3 times, plus 1 unmeasured HotSpot warmup run, for 100,000 iterations. So, what were the results?

Test #MVELJUEL
16.0ms34.0ms
25.6ms33.6ms
36.0ms34.6ms
Avg.5.8ms34.2ms

With no cached factory, no cached context, and using a simple Map to insert variables into MVEL's VariableFactory, we observe that MVEL yields performance nearly 6 times faster than JUEL.

Let's make it even more interesting.  

What if we removed all caching before the iteration? What if we cold-start JUEL and MVEL for each individual iteration?  What if we have to setup the JUEL factories and contexts for each execution, and what if we just use MVEL's eval() method without any pre-compile on every execution?  Both JUEL and MVEL's results will surely worsen, but by how much?  Here's the results:

Test #MVELJUEL
194.3ms2426.0ms
294.0ms2421.6ms
396.0ms3400.6ms
Avg.94.7ms2749.4ms

When we take away the advantage of resource-reuse from both MVEL and JUEL, MVEL slows down by a factor of 16, and JUEL slows down by a factor 72.  Boo-ya!

1 comment:

Jason said...

Not the best location for this question, but can not find another. Is there any documentation of MVEL beyond what is in language and integration guides? I want to assess for use but am not sure where to start - I want to use capacity to limit language features. Also on checking out from SVN I cannot build due to absence of nb_project file so have not yet built API docs.