Regarding Programming

Always Do The Math!

2009-04-08T12:11:00.000-07:00

I just spent the last three weeks futzing with proof of concept code, trying to get a client/server setup to support the performance requirements of the client. I was researching all kinds of tcp kernel parameters, reading up on the network stack, java.nio.Selectors, asynchronous I/O, TCP receive windows, doing everything I could to get the numbers to what the client was asking for. Finally, after I had gotten the number to about 4x less than what the client wanted, I did a simple math calculation to see how close to the theoretical bandwidth of GigE I was getting. Turns out I was already using about .8Gbps!! Did the math on the performance that the client wants, it would require 4Gbps ethernet!! Could have saved myself and the client three weeks of time if I had simply done that calculation (correctly) up front!!!

Java 5 Static Import "Feature"

2008-04-04T01:17:00.000-07:00

Wow, so first the people at Java 5 give us the convoluted generics which makes code as illegible as C++ templates, without any of the power of C++ templates. At least that provides the ability to avoid casting (but not really). But a new feature I haven't heard advertised much is the ability to import all the static methods of a class so that you don't have to prepend the name of the class when calling the method.

So, if you have a class:


public class StringUtils {
   public static String parseString ( String a, String b ) { ... }
}

Whereas before Java 5 in order to call parseString() you would always have to write:


StringUtils.parseString( ... );

Now in Java 5 you can "statically" import all the static methods of the class, then use the method directly:


import static StringUtils.*;

parseString( ... );  // Calls StringUtils.parseString( );

Now, this basically flies in the face of everything that we have been told from the beginning of Java, which is that having static methods be attached to Classes was important for namespacing. Here we have a situation where finding what parseString() does without the help of an IDE is practically impossible. It looks like a local method call! If I saw this line in a piece of code and needed to figure out what parseString did, I would first look in the class, then start hunting through all of its superclasses (which in Java is usually quite a number). Then I would be thoroughly confused. Only after wasting a large amount of time would I think to inspect the import statements! Yet another language feature that is completely unuseable without the help of an IDE.

But the best part yet is the rationale for Sun's designers putting this "feature" into the language: it's to prevent a really dumb "antipattern". So, basically, they have tried to prevent an antipattern by introducing another one. Brilliant.

"Hey, instead of shooting someone with a gun, how about just stabbing them with a knife instead?" What a great idea. It's bad enough when Sun makes poor design decisions in Java. It's even worse when they follow that up with even worse band-aids to the original mistakes.

Here's a better solution: Developers should just type out the name of the class when using static methods! Since everyone in Javaland relies on IDE's to do everything anyway, it's just a series of mouse-clicks to call a static method anyway. At least that way everything stays true to Sun's original intentions for the language.

Javadoc: What a Joke

2006-10-03T17:54:00.000-07:00

The idea of javadocs have always been pretty cool: a standardized way to document code and then view that documentation. Sun did a pretty good job enabling Java code to be documented in a consistent way.

What Sun didn't do a good job of was producing a useful javadoc tool. The javadoc binary has always been a pretty crappy tool, and today it has shown itself to be useless. The fact that every single class/package had to be listed on the command line in order to be "javadoc'd" made absolutely no sense. Nobody javadocs one file at a time, people javadoc entire projects all at once. Multiple packages, hundreds of files. This is the norm. Yet through over a decade of Java development Sun apparently still has no idea that this is how real Java developers write and document code (maybe this is why they come up with such poor technology specs? Do I need to mention EJB's? JPA is starting to look like garbage, too, a real step BACKWARDS from Hibernate.)

And today, I tried to fun javadoc on my entire project, passing in every single Java file since Javadoc insists on that, and I got this:

Building index for all the packages and classes...
Generating html/overview-tree.html...
Generating html/index-all.html...
java.lang.ClassCastException: com.sun.tools.javadoc.ClassDocImpl

Are you kidding me?! Is this for real?? A ClassCastException?? Who puts out production code like this? This is on JDK 1.5_06. Absolutely ridiculous.

Automatic Overriding of Parameterized Methods in Java 5

2006-08-09T18:57:00.000-07:00

I thought I was going crazy for a while because I seemed to be getting all kinds of odd behaviors, both at compile-time and at run-time, while trying to play with generics. Thanks to the help of the "javap" command, I think I've figured out a key part of how parameterized methods get overloaded versus how normal methods get overloaded.

Consider the following two classes:


class CompareAgainst<T>
{
   public int compareAgainst ( T a ) { return 1; }
}

public class   Main2  extends CompareAgainst<Main2>
{
   public static int testClassComparable ( Object[] a )
   {
      Main2 x = (Main2)a[0];
      Object y = a[1];
      return x.compareAgainst( y );  // Compiler error!
   }

   public static void main (String[] argv)
   {
      Main2 m1 = new Main2( );
      Main2 m2 = new Main2( );
      Main2[] mArray = new Main2[] { m1, m2 };
      System.out.println( testClassComparable( mArray ) );
   }
}

This seems to be a pretty straight-forward piece of code that demonstrates the correct use of generics. We have a base class CompareAgainst that contains a method compareAgainst() that takes a parameterized type as its argument. Just as we'd expect, when we have Main2 extend CompareAgainst and fill in the parameterized type as Main2, the method compareAgainst(T a) effectively becomes compareAgainst(Main2 a), and so in testClassComparable when we force one of the classes to be an Object, the compiler checks to see if compareAgainst(Object a) exists; it doesn't, so there's a compiler error:


   Main2.java:23: compareAgainst(Main2) in CompareAgainst cannot be applied to (java.lang.Object)

However, if we change the implementation of the testClassComparable method so that instead of casting the first parameter to Main2, it instead casts it to a CompareAgainst instance, surprisingly the code compiles and runs just fine!

public class   Main2  extends CompareAgainst<Main2>
{
   public static int testClassComparable ( Object[] a )
   {
      CompareAgainst x = (CompareAgainst)a[0]; // Change cast
      Object y = a[1];
      return x.compareAgainst( y );  // works fine!
   }

   public static void main (String[] argv)
   {
      Main2 m1 = new Main2( );
      Main2 m2 = new Main2( );
      Main2[] mArray = new Main2[] { m1, m2 };
      System.out.println( testClassComparable( mArray ) );
   }
}

You might be wondering, as I was, how the compiler suddenly found compareAgainst(Object a) whereas previously it thought it didn't exist. The key is to understand that the compiler can only use the type that we specify. In the first case, we specified that the type of the class was Main2. What methods does Main2 contain? In this first case, the compiler looks at what methods Main2 has and sees only compareAgainst(Main2 a), because we parameterized the compareAgainst(T a) that was defined in the superclass CompareAgainst. So the compiler throws an error.

In the second case though, inside the testCompareAgainst() method we told the compiler that the type of the object was CompareAgainst. The compiler has no choice but to listen to us and go look at CompareAgainst to see what methods are available. You might be wondering what the compiler sees when it looks at CompareAgainst, since the class defines compareAgainst() to take a parameterized type T. But "javap" will settle that question pretty quickly:


javap CompareAgainst
class CompareAgainst extends java.lang.Object{
 public int compareAgainst(java.lang.Object);
}

It looks like compareAgainst takes an Object as its parameter! This is our good old friend type erasure rearing its head. The compiler first compiles the CompareAgainst class down to binary form, and there it performs the type erasure, leaving just Object as the method parameter. So when the compiler is trying to compile Main2, it checks the binary version of CompareAgainst and sees that compareAgainst(Object a) exists, and thus compiles the code. This code will also run just fine.

All of this makes sense so far once you understand type erasure and how the compiler resolves methods in other classes during compilation. What's really surprising though is if you then overload compareAgainst() in Main2:


public class   Main2  extends CompareAgainst<>
{
   public int compareAgainst ( Main2 m ) { return -1; }

   public static int testClassComparable ( Object[] a )
   {
      CompareAgainst x = (CompareAgainst)a[0];
      Object y = a[1];
      return x.compareAgainst( y ); // Still looks like CompareAgainst.compareAgainst(Object)
   }

   public static void main (String[] argv)
   {
      Main2 m1 = new Main2( );
      Main2 m2 = new Main2( );
      Main2[] mArray = new Main2[] { m1, m2 };
      System.out.println( testClassComparable( mArray ) );
   }
}

Here we have explicitly defined a compareAgainst( Main2 a ) method in Main2. We might assume that this is a case of method overloading, since we know that in the base class it's really a compareAgainst(Object o) that's defined. In that case, we know that method overloading is determined at compile-time based on the types that we have told the compiler about. In this case, the compiler still thinks it has a CompareAgainst instance, not a Main2 instance, and the parameter being sent in still looks like an Object, not a Main2 instance, so the compiler should still call the superclass's compareAgainst method, not our newly defined method. If this was regular method overloading, this is exactly the behavior we would get.

However, this is not the way the compiler sees the situation. From the compiler's point of view, when we specialized the base class with Main2, we created a compareAgainst(Main2) method automatically IN THE BASE CLASS. Again, this is not what the byte code says, but rather what the compiler thinks. When we then define our own compareAgainst(Main2) method, the compiler assumes we want to override the superclass's implementation. Thus, it proceeds to hide the superclass's implementation and make our explicitly defined implementation the one that is called. Running the code above confirms that this is what happens, our new compareAgainst(Main2) method is called, not the version in the base class.

But how does the compiler make the overridden method get called when all it knows is that a compareAgainst(Object o) is being called on an instance of a CompareAgainst object? Here is where the compiler does a bit of magic. The compiler understands that even though we have specialized CompareAgainst with Main2, and thus conceptually created a compareAgainst(Main2) in the base class, in reality all that will be left in the base class when it gets compiled is compareAgainst(Object o). So if the compiler wants to enforce that the overridden method is called every time, what it actually has to do is override not compareAgainst(Main2) from the base class, but compareAgainst(Object o)!

If we use "javap -c" to examine the source code for Main2, we see that that is exactly what has happened:


public int compareAgainst(java.lang.Object);
Code:
 0:   aload_0
 1:   aload_1
 2:   checkcast       #4; //class Main2
 5:   invokevirtual   #9; //Method compareAgainst:(LMain2;)I
 8:   ireturn

public int compareAgainst(Main2);
Code:
 0:   iconst_m1
 1:   ireturn

Here, we see that the compiler has inserted a compareAgainst(Object o) automatically into our Main2 class. This causes the compareAgainst(Object o) from the superclass to be overridden. The compiler implements compareAgainst(Object o) so that it simply casts the argument to a Main2, and then calls our compareAgainst(Main2) method.

Now if we go back and check the code again, we can understand how our new compareAgainst() gets called even though the compiler only thinks it has a handle to a CompareAgainst class, and even though the parameter to compareAgainst() looks like an Object. During runtime, the JVM actually does invoke compareAgainst(Object), but there is a compareAgainst(Object) defined in Main2, namely the one the compiler defined for us. Thus that version gets called, and that version in turn calls our compareAgainst(Main2) method explicitly. Note that this mechanism will work across multiple levels of inheritance. If you were to create a class called Main3 that extended Main2, and then defined your own compareAgainst(Main2) in that Main3 class, the compiler would again insert the overridded compareAgainst(Object) into the Main3 instance to force the Main3 definition of the method to be called instead of any of the superclass versions.

This is the exact mechanism that enables Comparable to work in things like Arrays.sort(). Within the implementation of Arrays.sort(Object[]), the method casts each object to an instance of Comparable, and then calls compareTo(Object) on it. But because the compiler has inserted a compareTo(Object) into the definition of every class that implements Comparable, the correct compareTo(T) is called and the sorting algorithm works properly.

One last note: This mechanism works because the compiler is attempting to ensure that method overriding works correctly. This does not have anything to do with method overloading. If we define a compareAgainst(Integer) in any of our classes, then we are overloading compareAgainst(), not overriding it. As with any overloaded method, the compiler will only call that method if the types match up at compile-time. So unless we explicitly cast something to Integer and pass it into compareAgainst(Integer), that overloaded method will not be called.

Maybe Java Generics Really Are Useless

2006-08-08T20:41:00.000-07:00

So in March I thought I had found a pretty cool way to get around type erasure in Java 5 generics. If you overload methods with generic types, the resolution of which overloaded method to call happens at compile-time, so the compiler will hard-code the correct method to call. Or so I thought. This led me to think that I could do something cool like write a base class to handle all the correct operator equals() operations without requiring a runtime type-check. I describe this in detail in my blog posting titled "Overcoming Java 5 Generics Type Erasure with Method Overloading".

In that posting I basically posited that given:

class MyClass
{
public boolean equals( T o ) {}
public boolean equals( Object o) {}
}

MyClass myString = new ...
myString.equals( "someString" );
myString.equals( new Object() );

I thought that I could now assume that the proper equals() would always get called, because the compiler would at compile-time figure out the appropriate equals() and hard-code that into the bytecode, thus avoiding type erasure. In other words, I assumed that the "T" would get replaced with String in the definition of MyClass, and thus when I called equals() the compiler would hard-wire a call the correct overloaded version at compile-time.

Turns out I was mostly wrong, because the compiler only knows about the correct type in the scope in which the correct type is specified. In other words, if you say "MyClass<> myclass", then within the immediate scope of 'myclass' the compiler knows what type 'myclass' is and will call the appropriate overloaded type. That's why I got tricked into thinking this would always work.

This led me to think that I could send in MyClass to a generically-typed object, and then when the object called "equals()" the appropriate overloaded version of MyClass's equals() would get called. So if I had:

class GenericClass
{
public void doSomething( T a ) {
a.equals( "someString" );
}
}

// Code snippet
GenericClass o = new ....;
MyClass myclass = new ....;
o.doSomething( myclass );

I thought that within doSomething() the proper equals would get called in this situation, since the compiler would know that myclass is of generic type "String". Unfortunately this is not the case. Once the scope passed into the doSomething method, the compiler "forgot" about the generic type of myclass, and instead just defaulted to "Object". So instead of calling equals(T) the compiler coded in equals(Object).

This makes sense in a way, because the compiler can only see the generic type at the boundary level where it is specified. Once it enters a block of code that treats the code generically it can't keep track of the type anymore, not if it wants to allow the block of code to be pre-compiled and distributed in binary form.

Put into another way, the decision of which overloaded method to call is always made at compile-time. So if one block of code can be compiled separately from another block of code, then it's not possible for the compiler to figure out retroactively which overloaded method to call given the constraints of Java 5 generics, so it has to default to the lowest-common-denominator, which is usually Object.

Man, Java 5 generics really aren't all that useful...

Reflection IS Slow - Eating Major Crow!!

2006-08-02T19:45:00.000-07:00

So I'm an idiot, please disregard the last post about reflection being fast. Turns out I switched the names of two critical variables in my benchmark test, and the times got swapped. It's CGLIB that's 4x - 10x faster than reflection, not the other way around!! Don't use reflection unless performance really doesn't matter!!!

Munch, munch ... crow tastes like crap! ... Munch, munch

Java Reflection no longer that slow!

2006-08-01T19:04:00.000-07:00

[WARNING: Turns out I was wrong about reflection, see the blog post above for my addendum! I'm only leaving the contents here for "historical" reasons! REFLECTION IS SLOW!!!!]

I was in the process of evaluating validation frameworks for Java applications and came across iScreen. Looking at that framework, I liked the fact that it performed a mapping from any given object to a more specific object that the validation required. This eliminated the need to have every object implement some sort of a "IValidatable" interface that would enable some generic validation object to validate it.

For example, if you want to write a generic method that validates all objects that have a start date and an end date, you'd need for every object to have a "getStartDate()" and "getEndDate()" method so that this generic method could perform the validation. So you'd create an "IDateValidatable" interface that has a "getStartDate()" and "getEndDate()" method and force any object that wanted to be validated to implement these methods. This is rather clunky because it means that your data access objects might need to implement an interface that it really doesn't care about at all (most validation happens at the business layer, not the data access layer). Normally, one object needs to be validated in multiple ways, so this same data access object might also need to implement other methods in order to be validatable by other generic methods. This means the one object has to have knowledge of any and all validations that anyone might need to perform on it. Considering that two completely separate groups might be implementing the data access layer and the business layer, coordinating all the validations becomes a mess rather quickly.

iScreen has a different approach. Instead of trying to have interfaces that any and all objects must implement in order to be validated, iScreen allows each generic validator to define an object that contains the methods it needs in order to validate the data, and then iScreen will guarantee that an instance of that object is passed into the validator for validation. For example, a DateValidator might define an object that contains "getStartDate()" and "getEndDate()" methods, and then this object would be what iScreen passes into the DateValidator. iScreen is able to guarantee that this object is passed in because it allows the developer to define a set of rules to map the data from any arbitrary object into an instance of this object. So, if there was an Employee object that had a "getHireDate()" and "getTerminationDate()", iScreen allows the developer to map the "getHireDate()" method to the "getStartDate()" method, and the "getTerminationDate()" method to the "getEndDate()" method. Once that mapping is performed, anytime an Employee is sent in for validation, iScreen automatically creates a new instance of the object that the validator needs and populates it with the data from the Employee object.

It's quite a neat way around the fundamental problem of how to enable objects to be used in multiple generic contexts without requiring them to implement any and all interfaces that might be required across all of the layers of the application. Consider for example the need to display objects in a table. There might be a generic table component that always displays an ID, a name, and a description. Without this mapping functionality, it would be necessary for an interface to be created that had "getID()", "getName()", and "getDescription()" methods, and a data object that comes all the way from the back-end data access layer would be required to implement this interface. This would be extremely cumbersome, not to mention a terrible break in the encapsulation of the different layers of the application, since front-end UI concerns would be infecting the objects in the back-end. Instead, if the data object could be automatically mapped to some other object that the front-end UI component required, this would enable both the creation of the reuseable UI component without requiring the data object to be aware of it. It's really the best of both worlds.

iScreen achieves this mapping capability by using a technology called OGNL. Looking under the hood of OGNL, I was dismayed to see that it appeared to use reflection to perform the mapping. My understanding had always been that reflection was quite slow, so I decided not to use iScreen and OGNL, but instead to write my own mapping tool based on cglib, since it was my understanding that cglib performed byte-code manipulations to achieve faster reflection-like operations. Specifically, I used cglib's FastClass, FastMethod, and Enhancer APIs. (One very frustrating thing about cglib was its lack of documentation. It was obvious to me that there were other APIs that I could be making in order to potentially achieve more speed, but there is almost no documentation for how to use them, so all I could do was use FastClass and FastMethod).

When I was done coding the first part of the data mapping tool and began unit testing it, I decided to do some performance benchmarks to see how much of a speed gain I had achieved using cglib instead of reflection. After all, if the performance difference was minimal, I might as well stop my custom coding and just use OGNL and iScreen. I set up a synthetic microbenchmark that simply called a FastMethod versus a regular reflection Method over and over again and timed the operations. I was quite shocked when I saw the results.

# Iterations	control	reflection	cglib
10,000x	0ms	5ms	21ms
100,000x	4ms	17ms	105ms
1,000,000x	47ms	97ms	971ms

The control was simply calling a regular method without reflection over and over again. While the control was the fastest of all, the most surprising result was that cglib was much much slower than reflection! Over 10,000 iterations it was about 4x slower, but at 1,000,000 iterations it was almost 10x slower. Now granted, synthetic microbenchmarks are always a little bit of hooey, but these differences aren't small at all. I tried the benchmark both on a WindowsXP Pentium4 class machine and a FreeBSD laptop Pentium M, and in both cases the differences between the two were comparable. So there was basically no question about it, reflection is faster than cglib, at least in the way that I was using cglib, and it didn't make sense to write my own data mapping tool using cglib.

The only question is whether or not it's worth it to take the performance hit of using reflection versus regular methods. Personally, when I see that at 100,000 iterations each reflection method call only took 0.00017ms, or below the nanosecond threshold, and those numbers were even better at 1,000,000 iterations, I'd say the coding problem that's solved by using reflection is worth the minor cost in performance.

Overcoming Type Erasure with Method Overloading in Java 5 Generics

2006-03-08T15:55:00.000-08:00

I wrote a set of classes intended to represent a DatabaseID, and I wanted to make use of generics so that the appropriate overloaded equals() would be called. What I ended up doing was perhaps figuring out a way around type erasure. I've stripped everything down to make it relevant for this post:

// Base class that does most of the work, abstract.
public abstract class DatabaseID <>
{
protected
int
id;

protected
DatabaseID ( int newID )
{
id = newID;
}

// Overloaded equals(), specific to the parameterized type
public
boolean
equals ( ConcreteID otherID )
{
System.out.println( "Overloaded equals() called." );
return id == otherID.id;
}

// Overridden equals() that simply compares Objects
public
boolean
equals ( Object o )
{
System.out.println( "Object equals() called." );
return false;
}
}

// Concrete subclass of DatabaseID representing an OrderID
public class OrderID extends DatabaseID<>
{
// Contains nothing but a constructor to initialize its value
public
OrderID ( int newID )
{
super( newID );
}
}

// Concrete subclass of DatabaseID representing a UserID
public class UserID extends DatabaseID<>
{
// Contains nothing but a constructor to initialize its value
public
UserID ( int newID )
{
super( newID );
}
}

// The test class
public class Test
{
public static
void
main ( String[] argv )
{
OrderID o = new OrderID( 1 );
UserID u1 = new UserID( 1 );
UserID u2 = new UserID( 1 );

u1.equals( u2 ); // Calls overloaded equals()
o.equals( u2 ); // Calls equals( Object )
}
}

What surprised me was that I had expected type erasure to cause both of these calls to resolve to the overloaded equals(). Since I had defined the overloaded equals to take , the overloaded equals will ultimately erase to equals( DatabaseID ). Since both ID's are DatabaseID objects, I had expected that that version of equals() is the one that would have been called.

But if you run the code, you'll see that in the first case, where two UserID objects are being compared, the overloaded equals() is being called. But in the second case, where an OrderID object is being compared to a UserID object, the "normal" equals() that compares regular Objects is called.

At first I wondered what was going on, and then I realized that it's the compiler that chooses which overloaded method is called. The compiler is the one that looks at the types of the objects and determines which version of a method should get executed. Because this is done by the compiler, the compiler is still able to see the un-erased types, and thus in the second case it knows that two different types of objects are being compared, and chooses the equals( Object ) version. This can be confirmed by looking at the byte code of the Test class and seeing that in the first case, the compiler has inserted a call to the overloaded equals() method, but in the second case it has inserted a call to the regular equals() method.

So, method overloading is one case where you can use generics and type erasure won't come into play.

JSF and JSP: A Mismatched Combination

2006-02-14T11:11:00.000-08:00

Our company has recently been evaluating a whole series of technologies to use on Java-based web applications. So far we've liked what Hibernate and Spring bring to the table so we'll be using those on our next project. Now we've turned our minds to evaluating the front-end frameworks that will help build the UI, and our first stop was Java Server Faces (JSF), since it's now the "official" framework, courtesy of Sun.

While reading the JSF spec, I grew increasingly uncomfortable with what I was reading. It seemed to me that JSF was trying too hard to turn the web into a traditional client/server application. It gave me that save bad feeling that I got when I first read the EJB spec back in 1998. There was something fundamentally wrong with the idea, and I struggled to figure out what it was. After thinking about it for a while, I realized that the real problem was with attempting to use JSP as the view technology within JSF, not any fundamental flaw in JSF itself. I'm certainly not the first person to suggest that JSPs shouldn't be used with JSF, but it still seems like everyone who has used JSF on a real-world project has used JSP as the view. Here's one more potential reason not to use JSPs.

The Component Tree And The View in JSF
First, let's take a closer look at the inner workings of JSF. Central to the operation of JSF is the component tree, a collection of objects that represent the UI components that are on a page. This component tree is used not only to render the display to the requester, but also to hold the form values that the requester submits to the application. Also, type transformations and input validations are performed on the values in the component tree prior to updating the model, so the component tree is really the core of the JSF framework. The JSF lifecycle basically defines how the component tree is created and used.

The definition and construction of the component tree is delegated to the view itself. The view is responsible for indicating what components are within itself, instantiating those components, and then putting those components into a component tree for the rest of the JSF framework to use. This mimics the way that traditional client/server applications are built: typically, a Canvas or Panel UI component will directly instantiate any other UI components that are attached to it, which is then manipulated by the rest of the application.

JSF’s lifecycle dictates that the view will be called upon to build the component tree at the beginning of the lifecycle (during the “Restore View” phase), and then will be called upon to render the output back to the client at the end of the lifecycle (during the “Render Response” phase). This happens via a “ViewHandler” class that acts as the interface to the specific view technology. Again, this mimics the operation of traditional client/server applications: a Canvas/Panel/Window is instantiated, which causes all of its components to be built and initialized, and then that Canvas/Panel/Window is told to “paint” itself, which causes the components to be rendered and displayed.

The Role of JSPs in the JSF framework
Sun’s specification for Java Server Faces states that the framework is intended to be open with respect to the view technology that is used to build and display the component tree; The last chapter of Bergsten’s Java Server Faces book[1] describes several view options for use in JSF. However, the vast majority of his book, as well as the specification itself, describes how to use JSP as the view technology. The only view technology that vendors are required to integrate into any JSF implementation, and the only view technology that is integrated into Sun’s own reference JSF implementation, is JSP. For all intents and purposes, JSP is the default view technology for JSF.

This implies that the default “ViewHandler” implementation of JSF will use the JSP engine in order to have the component tree created during the “Restore View” phase, and will also use the JSP engine in order to have the component tree rendered for display during the “Render Response” phase. The JSP engine, in turn, will read in a JSP file to determine what UI components are on that page, and how to display those components to the user. The JSF spec provides custom JSP tags to facilitate this: the tags are internally mapped to a specific type of JSF UI component, and when the JSP engine executes a tag that is embedded in the JSP file, the tag manipulates its corresponding UI component within the component tree.

Practical Problems with JSPs as the View
The JSF 1.1 spec, when used in conjunction with JSP 2.0, has several well-documented problems related to how the JSP engine interacts with the JSF framework. One problem is that HTML text can get outputted out of sequence. For example, if HTML is interleaved with JSF tags, the JSP engine will have to render one component before the other, since JSF components ignore the interleaved text. Another problem is that certain components that don’t exist when a page is first loaded can magically appear if a page is refreshed, again due to the interaction between how the JSP engine parses JSP pages and the JSF tags on that page.

As of the time of this writing, Sun has completed revision 1.2 of the JSF spec, and also updated the JSP spec to revision 2.1, in an attempt to fix these types of issues. Their official release is being delayed due to the fact that these specs are tied to the overall update of J2EE, from version 1.4, to version 5. Once these changes are released, vendors will need to update their implementations and tools accordingly, and only then will these problems (hopefully) be solved.

The Fundamental Problem
While these practical problems are said to be fixed in the new versions of the JSF and JSP specs (1.2 and 2.1, respectively), the fact that changes to the JSP specification were required at all in order to interoperate properly with JSF brings up a more fundamental problem: JSP technology was never designed to fit into the JSF framework and specifically does not fit into the JSF lifecycle properly.

As previous stated, the JSF framework is based on the traditional client/server UI framework. In that framework, the view has separate instantiation and rendering phases in its lifecycle: The constructor of a Panel/Canvas/Window is responsible for instantiating the component and all subcomponents that it has, and the “paint” method (or equivalent method) is responsible for causing the component to render itself to the display. Correspondingly, in the JSF lifecycle there are separate instantiation and rendering steps (the “Restore View” and “Render Response” phases, respectively). In terms of the actual API, these two separate phases are embodied in the “ViewHandler” abstract class, which has separate “createView” versus “renderView” methods that map to each phase. The component tree is built by the “createView” method and returned to the JSF framework during the “Restore View” phase; that component tree is then passed in by the framework to the “renderView” method for use during the “Render Response” phase. It is this component tree that glues the two phases together by virtue of it being the output of one phase and the input to the other phase.

However, JSP technology was meant to output HTML to a browser, not to worry about building up or rendering component trees as part of some UI framework. When used as part of JSF, the JSP engine doesn’t even realize that it is building a component tree when it processes a JSP page, it is simply telling each custom tag it encounters to render itself. All it expects is for each custom tag to render HTML output. But when JSF custom tags are embedded into a JSP page, they will actually build up the component tree based on the parameters that are sent into them when they are executed by the JSP engine. The JSP engine is unaware of these activities, as far as it is concerned it is just calling a series of tags; the component tree is built as a byproduct of what the JSP engine does, not as its core functionality.

As a matter of fact, a typical JSP view handler implementation doesn’t even do anything during the “Restore View” phase of JSF. The implementation of the “createView” method in version 1.1.1 of MyFaces by Apache doesn't call the JSP engine at all and instead simply creates an empty component tree during the “Restore View” phase. Not until the implementation of the “renderView” method, during the “Render Response” phase of the lifecycle, is the JSP engine called to parse the JSP page, and the component tree actually built. The reason for this is that the JSP engine cannot separate the initialization of a component from the rendering of that component. The JSP “lifecycle” consists of only one externally accessible phase, which is the rendering of all the contents of a JSP file to the browser. Because of this, the first time a JSP page is parsed the component tree is empty, as none of the tags have been processed yet. As the JSP engine renders the JSP page by executing the tags embedded in the page, the component tree is built on the fly by each JSF custom tag. Thus, the UI components are initialized and rendered all in one step, during the “Render Response” phase of JSF. The “renderView” method is effectively performing both the “Restore View” and “Render Response” phases in one step. But the JSF lifecycle clearly expects these to be two separate steps, not one single step.

So JSP technology essentially violates the fundamental design of the JSF framework. Two of JSF’s phases have been compressed into one. It’s by clever engineering that the technology has been made to work with JSF. However, it’s clearly not an ideal fit, as JSP technology was never meant to perform the tasks that JSF requires of it as a view technology.

Possible Alternatives
It seems pretty clear to me that JSP technology, as it is currently defined, is not well-suited to work as the view technology for JSF. A view technology that features explicit instantiation versus rendering of its UI components is necessary in order for proper integration with JSF. I believe that eventually Sun will add such a view technology to the JSF specification in order to address this issue.

A simple approach might be for Sun to change the JSP engine so that it has an explicit “instantiate components” versus “render components” phase, and then use separate files for each phase. One file would define the components that should be on the page, and the other would specify the layout of those components. This would fix almost all of the core issues with using JSP technology as the view for JSF. However, it would be difficult to maintain backwards compatibility with existing JSP pages, and Sun has not indicated any desire to do this.

Outside of Sun, several attempts at building such an alternative view technology are already underway. Most are based on mimicking the functionality of Tapestry, which breaks apart the view into two separate files, one that defines the components on a page, and another that servers as the HTML template to enable those components to be laid out. One such alternative view is Facelets, which is being developed by one of the members of the Sun JSF implementation team. As such it has a good chance of eventually becoming a part of the spec. It breaks out the view into two separate files, a UI composition file where the components that make up the component tree are defined and initialized, and an HTML template file where the actual components are laid out for rendering. Another credible alternative is found in Apache’s Struts Shale project, which contains a technology called Clay that is meant to provide Tapestry-like HTML template capabilities as the view for their JSF implementation. Unfortunately it appears that the creator of Tapestry himself is not open to making Tapestry work with JSF.

Others have attempted to build a series of Swing-like objects for use as the view technology, effectively turning web application development into traditional client/server UI development. However there does not appear to be broad support for this approach, mostly due to the difficulty in having Java classes output HTML.

A final approach might be to utilize XML/XSLT to serve as the view technology. During the “Restore View” phase, an XML file could be read in that would then be converted into a component tree. Then during the “Render Response” phase the component tree could be converted back to XML and an XSL transformation performed on it to create the HTML output. This might be the simplest view technology to implement, but again, there have not been any suggestions to build such a view technology.

So the good news is that there are several attempts underway to replace JSP as the view technology for JSF. If JSF is the approach that you want to take for building web-based applications, I would encourage you to take a look at some of these alternatives rather than just automatically choosing to go with JSP.

[1] Java Server Faces, Bergsten, H., O’Reilly 2004

Regarding Inversion of Control

2006-01-03T21:20:00.000-08:00

Seems like Inversion of Control is one of the new rages (I say one of the new rages because it looks like the programming community, and Java in particular, is splintering into many different factions, each more rabid than the next). In the Java world, the Spring framework is one of the most well-known frameworks that makes Inversion of Control (IoC) easy to program to. The Spring framework website has a quick overview of IoC if you're not familiar with it.

I agree that IoC can be a very useful programming style because my own experiences in trying to build truly reuseable components has led me to construct my own code in a similar fashion. The idea is very simple: reuseable components often rely on certain objects that they should not be in charge of instantiating, because the actual object to be used should be determined further down the line by other components. So instead of instantiating these objects in your reuseable components, simply pass instances of those objects into the reuseable components, and it's a win-win situation.

For example, let's pretend you are writing a data access layer for your enterprise application (yes I know, most of the time an O/R mapping tool should be used instead). Specifically, you want to need to write a data access object that will persist User objects, where User objects have the usual username and userid fields, among other fields. Your first pass at
this might contain something like this:


public class UserDAO {
 private Connection conn;
 public UserDAO () {
   conn = DriverManager.getConnection( "jdbc:some:driver", "username", "password" );
 }
 public User loadUser ( int userID ) {
    // use conn to access the database
 }
}

Of course, as many people have found out when they write code like this, you have now locked this code into one particular use. You have locked in the JDBC url, which means you have forced any applicaton that wants to use this DAO to connect to a particular database. What happens if you want the DAO to connect to the QA database instead of the DEV database? What about the production database??

To fix this, many people then wrote this version, as Sun tells us to in their "official" J2EE guides:


public class UserDAO {
 private Connection conn;
 public UserDAO () {
   Context ctx = new InitialContext( );
   DataSource ds = (DataSource)ctx.lookup( "jdbc/MyDataSource" );
   conn = ds.getConnection( );
 }
 public User loadUser ( int userID ) {
    // use conn to access the database
 }
}

This appears to be better, because now the JDBC url and usename/password are configured by the app server and no longer hard-coded. Theoretically, now when you want to re-use this component, all you have to do is make sure a DataSource is defined in JNDI with the given name. But that's the problem: the code is now restricted to environments that are JNDI-enabled. If you want to use this DAO in a stand-alone Java application, you will have to construct an entire JNDI environment and expose a DataSource in that environment, just so that this DAO will run properly. That's an awful lot to do just to reuse one component. You're likely to give up and rewrite UserDAO rather than figure out how to do all of that, especially if all you're trying to do is write a simple Java app to fetch a report from the database.

Even worse, this JNDI requirement does not appear anywhere in the declaration of the class. While you could note this requirement in the javadocs for this class, there is no way a compiler can enforce that this requirement is met. You won't know until runtime, after you've written reams of code that relies on this component, before you realize that it won't work in your application. This is part of a greater trend that has grown over time and been a part of Java's philosophy from the beginning. Java is always willing to sacrifice compile-time ("static") safety for runtime ("dynamic") capabilities. It is difficult to write code that provides strong compile-time safety, while the most interesting technologies that have come about surrounding Java have usually harnessed the runtime capabilities of the VM.

A better way to write this code to optimize reuseability is to adopt the approach that if a component relies on another object, that object should be passed in by the caller:


public class UserDAO {
 private Connection conn;
 public UserDAO ( Connection cn ) {
   conn = cn;
 }
 public User loadUser ( int userID ) {
    // use conn to access the database
 }
}

Now, the UserDAO class makes no assumptions about how to obtain the database connection. A web-based application can obtain the Connection via JNDI while a standalone application can use the DriverManager to directly obtain a Connection. The UserDAO class is now more reuseable than it was before. At this point you effectively have used Inversion of Control. There is only one more step to go.

As a side note, the DAO should really take a DataSource and not a Connection, but I'll leave my opinions on that for a separate post. As a second note, some practitioners of IoC would insist that setters be used to set the Connection object onto UserDAO instead of using the constructor, but I disagree. Again, a topic for a separate post.

This style of coding points to a more general coding paradigm. You could treat the methods in the UserDAO class as implementing algorithms which operate on generic interfaces that are passed in, without needing to know exactly how those interfaces are implemented. This is essentially polymorphism taken to the level of generic programming. You can focus on writing reuseable algorithms by having the algorithm manipulate generic interfaces, and then somebody else is responsible for providing the actual implementations of those generic interfaces.

But when writing code in this manner, a central concern is trying to decide how to instantiate the concrete objects. If everyone is just dealing with interfaces, not caring what the concrete objects are, who actually creates the concrete object? This is why the GoF authors spend the entire first third of the book discussing patterns to instantiate an object. As an architect, if you want to write reuseable components that require instances of objects to be passed in, you will probably have to determine at what layer in the application those objects will be passed in, and you will probably have to use numerous patterns to enable new instances of those objects to be created when the reuseable components don't know the actual objects they have references to.

This is where the final step in Inversion of Control comes in, where the IoC containers come in handy. The IoC containers provide factories where you can configure what specific objects need to be passed into your reuseable components via XML files. This enables you to push the actual instantiations of the objects all the way into configurable files. Theoretically this should enable your reuseable components to be wired up with other components very quickly and easily, with none of them needing to know the actual objects they are using.

There's one big rub to using IoC containers though, and that's the fact that you can't avoid the laws of object instantiation. At some point somewhere in your code, you will have to hard-code how your objects are instantiated. In a web application, at some point (usually in the servlet) some piece of code is going to instantiate a Spring FactoryBean or ApplicationContext, and use that to instantiate other objects. For a standalone web application, you might define this in main() or in some Factory object somewhere. There's no other way for objects to just magically appear in your code, even with IoC.

If you want to limit how much of your application relies on the IoC container to instantiate objects (again, if you've made all this effort to hide how objects are instantiated so far, you're likely to want to do this), you need to carefully construct all your application layers so that only a few select layers (ideally the top-most layers) use the FactoryBean to instantiate objects. Everything else should be handled in the FactoryBean.

For example, let's say that the UserDAO is itself used by a business object UserBO that implements some business logic surrounding the manipulation of Users. If you want the UserBO object itself to be reuseable across multiple applications, then UserBO shouldn't instantiate a UserDAO object directly, because if it did that then it would have to know what database Connection to use, and we'd have the same set of problems we had with re-using UserDAO. You might be tempted to have the UserBO grab a handle to a Spring FactoryBean and instantiate the UserDAO that way, but if you do that then you've tied UserBO to Spring, and that forces any application that wants to re-use UserBO to use Spring, which might not always be feasible. For maximum reuseability of UserBO, it should either be passed a generic interface to some Factory class that will return objects, or it should be passed and instance to the actual UserDAO to use (in which case UserDAO should become an interface). There are numerous issues with attempting to pass in a generic Factory which are beyond the scope of this blog entry, so instead let's assume that you pass in a IUserDAO interface that represents the actual UserDAO for the UserBO to use.

At this point we still need to figure out who will actually instantiate the IUserDAO object to pass into the UserBO, so logically we go up one more layer. If this is a three-tier application, then the next layer up is the presentation layer. So, what if we have the presentation layer use Spring's FactoryBean to instantiate the actual UserDAO, and then send it into the UserBO? That works ... except that conceptually it's pretty ugly. Why does the presentation layer know exactly what DAO the business layer is supposed to use? That doesn't make much sense.

The solution in Spring is to have the UserBO configured via IoC to have the appropriate DAO passed in, and then have the presentation layer use Spring's FactoryBean to instantiate the UserBO. That way the presentation layer only knows about the UserBO, and the rest is configured in Spring. Specifically, this means that: 1.) the UserBO is configured in XML so that the appropriate UserDAO is passed in, 2.) the UserDAO is configured in XML so that the appropriate DataSource is passed in, and 3.) all of them get automatically instantiated by Spring when somebody uses the FactoryBean to instantiate a UserBO. This sort of chaining of configured objects is common in the Spring framework apparently, where all the objects are setup like dominoes, so that one layer instantiates one object which triggers a whole sequence of objects to be also instantiated.

If this sounds like it might be a little bit complicated to understand and properly configure, I don't think you're alone. Anytime configuring things requires that much coordination, it's usually a warning sign that long-term maintainability is going to be difficult. But only time will tell whether this is true or not, I personally have not had to maintain a long-term project that used Spring in this manner so I can't speak to whether this is maintainable or not. In any case, some could argue that this is the same as hard-coding all of these instantiations into the code itself, and thus no harder to maintain, and potentially easier to maintain because the XML files expose the interrelationships more clearly than code does.

In the meantime, one big question is whether all of this work is worth it. Is it truly that important that a DAO not hard-code a reference to a JNDI DataSource? Will your DAO ever need to be used outside of an application server container? On the projects I've been on, I've found that the answer is usually no. And if that's the case on your project, then you should carefully consider whether or not IoC makes sense.

The determining factor usually appears to be whether you expect your data access objects and/or business objects to be re-used across multiple application environments (web-based and client/server, or lightweight appserver versus full J2EE server, etc). One current design pattern that has gained a lot of traction in the past few years is for companies to expose their APIs via web services. Instead of exposing a business function via a class library that is used in several apps, that business function is exposed via a web service, and applications invoke the web service to obtain the functionality they need. In this way the reuseable code is always running in just one place, and much of the reuseability problem goes away. As always, the proper solution to your problem will be specific to the details of your project and your environment.

What's been your experiences with IoC containers? Where have you found that line to be where the complexity becomes worth it?