In the Spotlight - Howard Lewis Ship

According to Neal Gafter, the story for closures it still wide open.

As someone who writes a lot of code using closure-like mechanisms ... in the form of lots of inline inner classes ... I have a few idea of what I want in a solution. I think I'm writing some powerful and elegant code today, but that elegance in function is undermined by some severe awkwardness in its expression as Java code.

It really comes down to conciseness. I can accomplish pretty much everything I need using inner classes and holder objects, such as AtomicInteger and friends. But it ends up being more code than I'd like.

What I want (to borrow Stu's term) is to emphasize the essence of my logic, and strip away the ceremony: the naming of the interface (it should be known from context), the types of parameters (just the names, please), the list of thrown exceptions, etc.

First of all, let's constrain the problem. Closures would be defined in terms of an interface, an interface that contains a single method. Attempting to use the concise syntax with an interface that contains multiple methods would simply be a compiler error.

Second, the closure block should have free read/write access to parameters and local variables in the enclosing method. This can easily be accomplished with syntactic sugar: the variables can be converted into references to holder objects, such as AtomicInteger, that are stored on the heap. Today, shared fields must be final, to indicate that it is safe to share references to the object between the main method and any inner classes.

The hard part about sharing local variables in this way isn't implementing the syntactic sugar, it's about the updating of the information provided to the debugger so that the debugger can undo the syntactic sugar changes, and make variables in the enclosing method, or in the closure block, appear to be local.

Lastly, syntax. I think Groovy has the right syntax here. The important part is for the compiler to actually help out rather than for it to complain from the side-lines. Today's Java compiler has all the type information, but just uses that to build fancy error messages about what you should have typed. It should be using that type information to avoid the necessity of all the extra typing (that is, keyboard entry, not the need for types in the Ruby/Groovy sense of the word).

When a closure is passed to another method, the parameter type determines all the compiler should need to know. Likewise, when a closure is assigned to a variable, the variable defines the closure interface. Thus:

List<Widget> widgets = ... ;
Collections.sort(widgets,
  { w1, w2 -> return w1.getWidgetId().compareTo(w2.getWidgetId()); });

The generic type of the list, Widget, informs the generic type of the Comparator. Thus w1 and w2 are fully typed, as instances of Widget. The return type of the closure is pegged as int (also from the Comparator interface).

Side note: I'd also like to see a lot of other streamlining of Java, such as a Groovy-style implicit return.

In other words, the above could be expanded by the compiler to the following in terms of compilation:

List<Widget> widgets = ... ;
Collections.sort(widgets,
  new Comparator<Widget>()
  {
    public int compare(Widget w1, Widget w2)
    {
       return w1.getWidgetId().compareTo(w2.getWidgetId());
    }
  });

See, the essence is still there, the comparison of the widget Ids. But its now occluded by all that ceremony about interface names, method names, return values, generic types, and so forth.

A caveat: there are edge cases where we'll need to identify the closure interface type. This occurs when a method to be passed a closure is overridden. The compiler should be able to reduce the candidates based on parameter count, checked exceptions thrown inside the closure, and other factors ... but it may be necessary to implement an alternative syntax. For example:

ExecutorService service = ...;
Future<?> = service.submit(Runnable { performExpensiveOperation(); });

submit() is overridden to accept a Callable as well as a Runnable. Callable can return a value. Again, minimal syntax here: the interface name followed by the implementation of the closure method. This compares to the Groovy as keyword.

Now, it's easy to get carried away and put in stuff like Python style co-routines (the yield keyword), or want a syntax to allow the closure to force a return value from the enclosing block or method. <sarcasm> Yep, let's add a few more alien concepts to the language, people love that. </sarcasm>.

These are also not function objects, so you can't easily do magic things such as currying (currying is a way of pre-supplying some of the parameters to a function, such that a new function is created that takes fewer parameters). An "interface" that's curried is a whole new interface and that's OK by me.

I'm more more modest. I don't particularly want to add new features to the Java language, just new syntax for the compiler, to let it do the ugly plumbing. That's kind of my theory for Tapestry's relationship to Java and the Servlet API as well, and it works.

A few side notes:

Annotations could be used to help the compiler out. For example, if a common interface should be a closure, but has multiple methods, an annotation could be used to tell the compiler which method of the interface is applicable to use as a closure; any other methods (in the inner class) would be no-ops or (perhaps guided by additional annotations) failures.

Annotations on fields and parameters (or perhaps on the method) could help the compiler decide how to share visible variables: do we need to use a thread-safe approach (such as something based on AtomicReference) or simple, non-synchronized holder objects? The compiler could do some escape analysis as well to chose the best solution, but in many cases, it's on the coder's shoulders to make the right decision.

From what I can tell, my ideas are closest to the Concise Instance Creation Expressions proposal by Bob Lee, Doug Lea, and Josh Bloch. However, in my opinion, the compiler can do much more in terms of providing type information. I think CICE stumbles in that it allows for creation of classes, not just interfaces, and gets bogged down in defining closure types and parameter types ... again, things that (with the mandate of single-method interfaces), the compiler can do autonomously.

Likewise, CICE requires that variables visible to the inner block be marked "public". This to me is something that the compiler can analyze; it can identify any assignments to variables and promote such variables to stored-on-the-heap status. I don't see the need for final; the compiler has plenty of ability to determine if a parameter or local variable is ever updated within the body of a method (and the body of any inner classes or closures of that method).

My basic concept is: Less is More. Support fewer cases but do so more cleanly, more concisely, and more understandably. Simplify. Let the compiler do more work. Reduce the density and complexity of the Java code. Expose the essence.

Portland is a town where, while walking to the gym, you can come across someone sitting at a sidewalk table of a local cafe, leafing through a hard-copy of Programming in Scala. We chatted for a moment (even though I was not quite awake yet). One more thing for me to do ... check out Lift.

And, yes, he actually had a print shop print and bind the PDF (as the book is only available electronically right now).

It's looking very likely I'll be in London, UK next month for a bit of on-site training. I'm also speaking at JavaZone 2008. I'd love to optimize the number of flights I make to Europe, so if anyone is on the Continent is hankering for a little Tapestry training, now's a good time to get in touch!

After London, Suzy and I are then going to do a little trip around Europe, taking a couple of days in Paris and Amsterdam.

Huzzah! One of the problems I've had using FireBug is that it didn't record a component event request long enough for me to see it; all I'd see was the follow-up page render request. But the latest FireBug is showing me the component event and the 302 response, and then the page render request, which gives a much more accurate idea of what is going on and how long it takes.

My current task has put me in an odd mood. A client of Jesse's (Kuhnert) contracted him to make some changes to the Tapestry 3 code base, with me auditting those changes. Jesse has backported some ideas from Tapestry 4 and Tapestry 5: mostly the use of a backported version of java.util.concurrent to improve Tapestry 3 throughput by replacing single-thread synchronized access with concurrent access.

Many of these files haven't really been touched since 2004. It's brought back a lot of memories, such as working on TheServerSide's application.

To my jaundiced eye, the code looks almost primitive; no IoC container, no dependency management, heavy reliance on base classes, a lot of rather naive multithreading.

Overall, I'd say that Tapestry 5 is as far advanced over Tapestry 3 as Tapestry 3 was advanced over Struts. Probably more so. Looking back over this code has given me some perspective.