JavaOne Days 3-4: Mashups and garbage collection

If you are surprised that I am talking about mashups and garbage collection in the same post, well, so was I. But if one can talk about implementing servers in JavaScript, I most definitely can talk about mashups and garbage collection in the same breath.

Day 1 was primarily scripting, day 2 hardcore java. Days 3 and 4 saw sessions on the entire gamut of Java technologies - from garbage collection to mashups. I discuss some of them here.

Blueprints for Mashups

This was arguably the most informative session for me by far at this year's JavaOne. Kudos to Greg, Mark and Sean for putting together a to-the-point, practical and readily-usable session together. Personally, this session validated the REST and JSON concepts I had gathered over the past few months. I'll recommend this session to anybody interested in building mashups, REST services, AJAX and a whole lot more. (No, they haven't paid me to say this.)

Anybody building REST services should design their system with JavaScript in mind. In today's mashup world the browser (and hence JavaScript) is your first class client. XML and JSON are the typical content types returned by REST services. To get over the browser's cross-domain restrictions, there are 2 possible solutions:

• Server-side proxy: Clients always send requests to the server that is hosting the page. The server in turn acts as a proxy, sends your request to the (remote) mashup service and returns the data it gets from the mashup service to you.
• Dynamic script tags: Browsers allow script tags to communicate with cross domain servers. This opens up the opportunity for you to issue requests to any mashup service by generating dynamic script tags.

Consider the Atom format if you are serving XML. This opens up your service to the large number of feed readers and other Atom clients out there.

JSON has of course gained immense popularity of late. Although JSON is technically a serialized JavaScript object, the JSON format is highly portable and language independent. In addition to returning JSON, you can also support wrapping the JSON object in a JavaScript callback method (they called it jsonp - JSON with padding). This enables clients to specify a callback method which can readily work with the JSON that the server returns.

Various options are available for securing your REST services:

• User tokens
• Session based hash
• URL based API key
• Authentication headers

JavaScript best practices:

• Use namespaces
• Use CSS for applying styles
• Don't add to the prototype of common JavaScript objects
• Setting the innerHTML property is easier / better than DOM manipulation

Components of a ("good") Mashup API / library:

• A server-side service
• A client-side JavaScript API / library
• A client-side CSS for applying styles
• Document the API
• Create simple examples enabling a simple cut-and-paste approach to learning your API

As you can see, they have, true to the name, provided blueprints for mashups, best practices, possible hurdles and workarounds, problems and solutions. Once again - highly recommended for anybody who has / will have / may have anything to do with mashups.

Phobos - A Server-side JavaScripting framework

While JS clients are ubiquitous and obviously here to stay, for the life of me, I have not yet come to terms to implementing my server in JS as well. The 2 reasons they cited - impedance mismatch and all-you-need-is-an-F5 to redeploy - didn't quite do it for me. May be in another life I'll grow up to JavaScript servers, but not for now. And if at all that day were to come, there should be a JavaScriptOne and no JavaOne.

Garbage collection

If there was one objective of this session, it was to clear the GC myths out there. Some of the finest Java minds were refuting the urban legends out there and when they talk you listen:

• Object allocation is cheap. Reclaiming young objects is also cheap.
• Small, short-lived immutable objects are good. Large, long-lived mutable objects are bad.
• Nulling references rarely helps - except when it comes to arrays.
• Avoid finalizers - in most cases there are better alternatives possible.
• Avoid System.gc() - except between well-defined application phases and when the load on your system is low.
• Object pooling is not required in today's VMs. It is a legacy of older VMs. Exceptions are objects that are expensive to allocate / initialize or objects that represent scarce resources.
• Consider using reference-objects for limited interactivity with the garbage collector.

Certain memory-leak pitfalls:

• Objects in wrong scope
• Lapsed listeners
• Metadata mismanagement

They strongly advocated FindBugs for finding such pitfalls as well as other potential bugs.

JavaOne Day 2: Hardcore Java and Java EE 5

While Day 1 saw an overdose of scripting, sanity made a return on Day 2. Although JavaScript technology (whoever came up with that) sessions continued, there was a good supply of deep-dive Java and Java EE sessions to keep me busy. And there was also some WADLing.

Generics and Collections

Generics have been around for a while and most Java programmers have some understanding of it. There have been many criticisms against the erasure based implementation of generics - since parameter types are not reified, constructs that require type information at runtime don't work well. However erasures allow 2 most important benefits - migration compatibility and piecewise generification of existing code. Just these benefits make erasures a necessary bane.

Huge additions have been made to the Collections framework in Java 5 and 6. So much so that many recommend programmers to never use arrays - Collections all the way!

Concurrency Practices

The concurrency classes introduced in Java 5 are a great new toolset for the Java programmer. New concurrency related annotations such as @ThreadSafe, @GuardedBy, etc. are being considered. One important aspect to keep in mind - imposing locking requirements on external code is asking for trouble. Ensure that you make your code threadsafe yourself. Performance penalties incurred by synchronized constructs are overblown.

Immutable objects are your friends - final is the new private! They are automatically threadsafe. Object creation is cheap. Aim for less and less mutable state.

Performance is now a function of parallelization - write code such that it can use more cores to get the job done faster. So to improve scalability find serialization in your code and break it up:

• Hold locks for less time
• Use AtomicInteger for counters
• Use more than one lock
• Consider using ConcurrentMap
• Consider ThreadLocal for heavyweight mutable objects that don't need to be shared

Effective Java

Builder pattern allows you to construct objects cleanly in a valid state with both required and optional parameters. The basic premise can be expressed in code as such:

MyObject myobj = new MyObject.Builder(requiredParam1,requiredParam2).optionalParam1(opt1).optionalParam2(opt2).build();

Generics - avoid raw types in new code. Don't ignore compiler warnings. Annotate local variables with @SuppressWarnings rather than the entire method / class. Use bounded wildcards to increase applicability of APIs. 3 take home points for paramterized methods:

• Parameterize using <? extends T> for reading from a Collection
• Parameterize using <? super T> for writing to a Collection
• Parameterize using <T> for methods that both read and write

If a type variable appears only once in a method signature then consider using a wildcard. Avoid using bounded wildcards in return types. Generics and arrays don't mix well - consider using generics wherever possible.

Java EE 5 Blueprints

Filter.doFilter() may run in a different thread than the Servlet.service() method. The Servlet API does not use NIO, however it is possible to implement it yourself. New annotations may make web.xml optional.

The JSP and JSF EL have been unified. # is now reserved in JSP 2.1. The javax.faces.ViewState hidden field (for client side state) will be standardized. Include this field in your AJAX postback. Application-wide configuration of JSF resource bundles will be possible in faces-config.xml. The <f:verbatim> tag is no longer needed to interleave HTML and JSF content. @PostConstruct and @PreDestroy annotations will be supported for JSF managed beans.

WADL

WADL - Web Application Description Language. As a colleague of mine put it - it's WSDL for REST! And IMO that's almost what it is. It's all in good intent to introduce WADL - a formal definition of your REST resources, allows you to automatically stub out code in your favorite language, aids testing. However WADL has many rough edges and I don't see myself using it anytime soon until those have been addressed - overloaded / loosely typed query parameters, security, non-standard content negotiation, et al.

JavaOne Day 1: Scripting Everywhere

If you were new to JavaOne, you might be forgiven for thinking that the message was - Strongly typed languages are so-90s, scripting is in! And why wouldn't you - every other word you heard was - JRuby, Groovy, Server-side JavaScripting. And if that wasn't enough, Sun introduced a half-baked, half-after-thought, all put-together-in-a-last-minute-sprint JavaFX. Ok so I don't want to disregard the hard work put together by folks to get JavaFX ready for JavaOne but it all seemed so unconvincing. The demos were clunky, some didn't work and those that did were hardly "Wow!".

Nothing to "Wow!"

If you like showtime, there was none. No big announcements, no new features, no gennext projects. But IMO that was a good thing. Java has seen many evolutions, many innovations. It's time to make it just a little better, a little more robust, a little more performant. Java is a mature technology now and any radical change might actually be shaky news for the massive Java community.

Oh so Groovy

As I mentioned - scripting languages were the talk of the day. And Groovy is one of them. It has been around for a while and the rich set of features it offers are testimony to that - Java-like syntax, pre-compiled or runtime compilation, "duck typing", annotation support and a whole pageful of other features.

I wonder what has lead to this sudden wave of dynamic languages? Wasn't it only yesterday when everything-should-be-strongly-typed was the hallowed rule of programming. Maybe it's the advent of mashups. Maybe the uptake in AJAX. Or maybe we just need something new to keep us interested!

Web as a platform

"Integrated rich clients" - or what we know as mashups - are the killer apps of the day. Things (technologies, techniques, hacks) that allow for mashups to be assembled with minimal code will be on the victory parade. More code is executed on the client but most of that code is still sent to the client by the server. No wonder scripting is big today and getting bigger. But what I do wonder is where does that leave EJBs / ESBs?

JSR 311 - Java REST API

I had questioned the need for frameworks to build RESTful services earlier. And after having sat thru a session explaining the new Java REST API spec I am no more close to getting an answer to that. Ok so they showed a few annotations which got rid of a few lines of code. But does the same simplicity work if I have even a slightly complicated URI pattern? Or will it work if I don't use the "Accepts" header for conent negiotiation and use query params or path extensions? And from what I saw it seemed they were generating XML and JSON from Java code. Aren't JSPs (or any template technology) a whole lot easier to do that?

I got Closure

Neal Gafter gave a presentation on Closures in Java. Very well done, no gimmicks, to the point. A great presentation for a potentially very powerful feature in the Java language. If Neal needs more voices to move this up the JCP and into Sun's plans for Java SE 7, count me at +1.

Take home point

Like 'em or hate 'em - scripting is in town.

Do we need frameworks for REST?

Many REST frameworks such as Restlets and simple web are gaining popularity of late. There's also a new JSR to give Java developers a new API to build RESTful web services. It's quite natural that as more people "get" REST they look for ways to simplify building their next REST web application / service.

And while I admittedly have to yet fully "get" REST, whatever I have got so far is certainly not by using frameworks that let me implement REST by writing POJOs but rather by extensive use of the POW (Plain Old Web) and POGS (Plain Old Google Search). And applying the principles I learned by using POS (Plain Old Servlets).

This is not a criticism of these frameworks. May be I still haven't understood the role of these frameworks. And may be once I do understand their goodness I myself will start using them.

But my question is this - why are servlets not good enough? Sure they have their limitations. But rather than have yet another framework or a brand new API, why not have a JSR to fix the servlets and JSPs themselves? (A good start would be to not enable sessions for JSPs by default.) After all isn't it the convenience of using frameworks galore that has kept the larger community from understanding the goodness of HTTP? Isn't it the same convenience that has made it a common practice to use (bloated) sessions?

You have to wet your feet to tread the waters. You have to get your hands dirty in HTTP to implement REST.

SOAP over DCOM in the ArcGIS ADF explained

The ArcGIS Server Java ADF supports accessing the ArcGIS Server over the internet (http) as well as locally (dcom). Internet access uses SOAP. Local access can work with the server objects directly over DCOM or you can also issue SOAP calls over DCOM.

For local access, the ADF gives preference to SOAP over DCOM while it accesses the ArcObjects directly over DCOM only when the functionality is not available thru SOAP. There are 2 primary reasons why SOAP / DCOM is preferred to ArcObjects / DCOM:

• Performance
• Code reuse

Performance:

When you work with ArcObjects / DCOM the server gives you access to remote ArcObjects proxies (such as IMapDescription, ILayerDescription, etc.). Every method call on these proxies is a remote method call. So methods such IMapDescription.getName() and ILayerDescription.getID() are both remote method calls.

On the other hand, when you work with SOAP / DCOM, only the methods defined in the WSDLs are remote calls. Once you have made those remote calls, you get access to objects which are local value objects (such as MapDescription, LayerDescription, etc.). So methods such as MapDescription.getName() and ILayerDescription.getLayerID() are both local method calls.

As you can infer, ArcObjects / DCOM elicits more "chattiness" with the server than SOAP / DCOM. And the reduced number of remote calls in case of SOAP / DCOM obviously translates to better performance over the lifetime of your application.

Code reuse:

If you look at various functionalities supported by the ADF such as AGSMapFunctionality, AGSGeocodeFunctionality, etc., the same functionality classes are used for both internet and local access. This was possible because these functionalities were implemented by using the SOAP interface to the server. The transport is HTTP in case of internet access and DCOM in case of local access but the code remains the same allowing us to reuse the same functionality implementation in both cases.

Capabilities such as the EditingTask which are not available with SOAP have obviously been implemented by using ArcObjects / DCOM.

Bottom line:

In summary, if your functionality can be implemented by using the SOAP interface you should use it. The richness of ArcObjects / DCOM is of course always available to you in cases where SOAP does not suffice.

LinkedHashMap to implement LRU caches

I must be living under a rock to have not noticed this before. The LinkedHashMap has a 3 argument constructor. The last argument, if true, orders the map according to the access order. It also defines a protected removeEldestEntry() method which returns false by default. One can override this method to return true in certain situations, in which case the map will remove its eldest entry.

The implementation of an LRUMap will look something like this:

public class LRUMap<K,V> extends LinkedHashMap<K,V> {

 int maxLimit;

 public LRUMap(int maxLimit) {
   super(maxLimit * 10/7 + 1, 0.7f, true);
   this.maxLimit = maxLimit;
 }

 @Override
 protected boolean removeEldestEntry(Map.Entry<K,V> eldest) {
   return size() > maxLimit;
 }

}

The removeEldestEntry() method is invoked by the put*() methods. With the above implementation, if the size of the map exceeds the max limit, the eldest entry is removed.

Short and sweet. Albeit some 2 years too late for my unenlightened self.

ArcGIS Server coding practices

The ESRI dev summit was a great experience. Meeting fellow developers is always good - you talk the same language, exchange ideas, receive feedback, discuss how things can be made better,... I could go on and on.

Ok, so I was asked quite a few server related questions at the summit: What should I keep in mind while coding against the server? Any do's and dont's? Any particular classes / methods I should read more about? etc... So here I'll talk about certain points you should keep in mind while working with the ArcGIS Server, particularly the MapServer:

• Use the description objects: Sure the MapServer gives you access to the IMap but you don't have to go there. A lot can be accomplished by using the IMapDescription, ILayerDescription, et al.


• Release the ServerContext: In a pooled environment you are sharing the server object with other users. So it's your responsibility to release the context once you have performed your set of operations. For web applications, this translates to releasing the context after every request. Of course, if you are using the ADF, the ADF does that for you so you need not worry. But even then it's good to keep this in mind.


• Do not reference server objects after the context has been released: Do you continue to work with the Statement object once the JDBC Connection has been closed? NO. Why? Because a Statement can be executed only while the Connection is live. Similarly, you should not reference ANY server objects once you have released the IServerContext. If you want to persist the state of any server object, you should use the saveObject() method to get a serialized string representation of the object before releasing the context. You can once again rehydrate the object by using the loadObject() method once you have regained access to the context.


• In case you had forgotten - use the description objects: Did you know you could add serializable custom graphics to the IMapDescription object?


• Use methods exposed by the MapServer: Look at the javadoc for the MapServer and for all the interfaces that it implements. It can do a lot more than you might think - you can export maps, layouts, do queries, identifies, get feature info, handle SOAP requests, and a whole lot more.


• Create server objects on the server: You don't create RMI objects on the client. You don't create EJBs on the client. Similarly, you don't create ArcGIS server objects on the client. ALWAYS use the IServerContext.createObject() method to create all and any ArcObject in a server application within your server context at the server.


• BTW, use the description objects: Did you know you could change layer visibilities, select features, even set definition expressions with the ILayerDescription object?

This by no means is an exhaustive list but something which might help you when working with the server. Oh yeah, in parting, in case you had missed: use the description objects!

Exception handling in web frameworks

There are many good articles out there pertaining to good exception handling practices. Here I'll list some that I've learned over the course of working on web applications and as it pertains to web frameworks.

1. Chain Exceptions:
   Sure one should never lose sight of the destination, but with exceptions one should never lose sight of the source - the all important root cause, the crux of the problem. While you can catch exceptions and throw custom exceptions, never lose the exception that you caught. When creating custom exception classes, always include a constructor that takes the cause as an argument:
   

   public CustomException(String message, Throwable cause) {
     super(message, cause);
   }

   
   Or if you are using an exception which does not have such a constructor, you can always use the initCause() method:
   

   catch(AnException e) {
     IllegalStateException ise = new IllegalStateException("Something went wrong.");
     ise.initCause(e);
     throw ise;
   }

   
   


2. Throw framework specific exceptions:
   The root cause tells you what went wrong but you also want to know where it went wrong. This is especially critical in web frameworks where the framework makes callbacks, does dependency injection and other IoC stuff to call into app specific custom client code. When your client comes to you with a support call or posts the problem on the forums the first thing you want to know is where in your stack did it fail. Debugging and fixing the problem becomes a lot easier once you have narrowed it down to your courtyard. Framework exceptions are best implemented as a hierarchy of custom runtime exceptions. The root of this hierarchy could be a RuntimeException named MyFrameworkException and it could have any number of sub-classes such as MyFrameworkModelException, MyFrameworkControllerException, MyFrameworkViewException, etc. depending on how you have categorized different pieces of your framework.


3. Log, log, log:
   Where exception messages don’t tell much, debug messages logged from important places will. What comments are to source code, log messages are to runtime. The standard JDK logging API allows you to log messages at multiple levels including INFO, CONFIG, FINE, FINER, et al. Logging messages at various levels will not only provide answers to the hows, whens and wheres of the problem but will also go a long way in helping your customers understand the underpinnings of your framework. And more often than not that is all they need to fix the problem themselves!

We are constantly trying to improve the exception handling and logging in the ArcGIS ADF and you can be assured that we are employing these principles as well.

CONned by windows

Link: Naming a file

Do not use the following reserved device names for the name of a file: CON, PRN, AUX, NUL, COM1, COM2, COM3, COM4, COM5, COM6, COM7, COM8, COM9, LPT1, LPT2, LPT3, LPT4, LPT5, LPT6, LPT7, LPT8, and LPT9. Also avoid these names followed by an extension, for example, NUL.tx7.

I was writing an auto code generator which was generating many Java files and one of the files to be generated was Con.java. My generator generated every other file but when it came to Con.java it threw a FileNotFoundException. Which was very weird because I was trying to create a new file and so it not finding the file was in fact a good thing! To justify that it wasn't just a figment of my imagination I checked and confirmed that there was no such file. I rechecked and yet there was no such file. I cursed and ran the code generator again - this, my never failing trump card betrayed me as well.

Of course these days where everything else fails, Google doesn't. And sure enough it brought me to this page and it was clear that CON was one of the many reserved words and so I could not create a filed named so.

While I am ok with Windows not allowing me to create a file with a name which is a reserved word but not allowing files with a reserved word followed by an extension is all too limiting. And worse still the error messages when I try to create a file con.whatever vary from "access denied" to "file already exists" to "are you kidding me?" (no, not the last one but it came close).

Extend the ArcGIS ADF with POJOs - Part III

The first cardinal rule when working with pooled ArcGIS Server objects in a webapp is that you must release the server context after every web server request. This is because you are sharing the server object with many other users and you need to put it back into the pool so that other users can access it. The second cardinal rule is that you cannot reference any server object after you have released the server context. If you do, it would be like executing a JDBC Statement after closing the Connection. You always need a live connection or a context in a client-server environment to work with objects that are remotely hosted.

So the question then is: How do I persist with the current state of a server object after releasing the context? The answer lies in the saveObject() and loadObject() methods of the IServerContext. You can serialize objects to their string representations with saveObject() and you can deserialize them to their object forms with loadObject(). So calling saveObjects before releasing the context and calling loadObjects on reconnect sets you up well to persist with the current client state while working with pooled objects.

As always, the ADF assists you in making such experiences easy for you. Rather than you having to scratch your heads about when to calls loads and when to calls saves, where to call them, how to keep track of them, et al the ADF provides you a simple interface in WebLifecycle where in you can do all of these tasks and that for only those objects needed for that particular task. The ADF calls relevant methods of the WebLifecycle just prior to releasing the context as well as immediately after a reconnect.

In the CountFeatures class that we have been working on through Parts I and II we may want to persist with the SpatialFilter object. Generally you persist only those objects which have enough client state in them for you to justify the saves and loads. It's obvious that the SpatialFilter doesn't have much state in it to merit justification but the idea here is to showcase how to do it easily so that you can apply it to more pertinent objects such as graphic elements and symbols.

The WebLifecycle defines 3 methods - activate(), passivate() and destroy() - which are called at different stages of a request / session lifecycle by the ADF. The passivate() method is called after a request has been serviced giving you the opportunity to serialize server objects to strings. The activate() method is called before servicing the request where you can deserialize the strings so that they are available as live objects when you perform the business tasks. And finally the destroy() is called when the session is being terminated for you to perform the necessary cleanup.

Below is the code which extends the CountFeatures class to participate in this lifecycle:

public class CountFeatures implements WebContextInitialize, WebContextObserver, WebLifecycle {
    ...
    //serialized SpatialFilter - valid after passivate() and before activate()
    String serializedSpatialFilter;
    //the spatial filter object - only valid between activate() and passivate()
    SpatialFilter filter;

    public void init(WebContext webContext) {
      ...
      //create SpatialFilter and set spatial relationship
      filter = new SpatialFilter(agsctx.createServerObject(SpatialFilter.getClsid()));
      filter.setSpatialRel(esriSpatialRelEnum.esriSpatialRelContains);     
    }
    
    public void passivate() { //serialize to strings
      serializedSpatialFilter = agsctx.saveObject(filter);
    }
    
    public void activate() { //deserialize to objects
      filter = new SpatialFilter(agsctx.loadObject(serializedSpatialFilter));
    }
    
    public void destroy() { //cleanup
        filter = null;
        serializedSpatialFilter = null;
    }

    public String doCount() throws Exception {
        ...
        //spatial filter is already created - only need to set geometry to the current extent
        filter.setGeometryByRef(agsmap.getFocusMapExtent());
        ...
        return null;
    }
}

The complete source code can be downloaded from here.

It's important to note that the loads still create new instances of the objects on the server. So there are no performance benefits to saves and loads versus new instance creations. The benefit to gain is that you don't have to worry about persisting with the state of the object yourself. The server will do that work for you.

That does it for this trilogy (ok so that was blatant abuse of that word - but hey, who's to say... It's my world around here :)

JDK 5 concurrency API: group / batch thread pool

First up, let me say that the new concurrency API in JDK 5 is indeed a boon for the Java community especially for developers (including yours truly) who before this indulged in threads and concurrent programming only sparingly. Not because we didn’t know how to do it but because getting it right was quite an ordeal. The concurrency API should surely make concurrent programming, the bastion of only a selected few so far, more "mainstream".

So here’s my scenario: I need for a group / batch of tasks to execute concurrently and additionally, I need to wait until all of them have finished executing before moving forward.

Levaraging the new concurrency API; to implement this I can use the Executors factory to create a new thread pool. (A thread pool being an instance of ExecutorService.) To this pool I can submit the tasks of my batch which will be executed according to the policies of the pool. Then if I want to wait on all of them to complete, I need to call shutdown() followed by awaitTermination(). With this my code will indeed block until all tasks have been executed but the problem is that the thread pool no longer accepts any new tasks. So for my next batch of tasks I need to create a new thread pool all over again - which obviously is unneeded and expensive.

All said and done, I need an awaitExecution() method which like awaitTermination() blocks until all tasks have completed but unlike the shutdown() + awaitTermination() combo does not reject new tasks.

Below is a simple wrapper with the awaitExecution() method included. You can ofcourse use any of the extension patterns - decorator, adapter, etc. - for a more refined solution.

public class GroupThreadPool {
  protected ExecutorService pool;
  protected ArrayList<Future> futures = new ArrayList<Future>();

  public GroupThreadPool(int poolSize) {
    pool = Executors.newFixedThreadPool(poolSize);
  }

  public void submit(Runnable command) {
    futures.add(pool.submit(command));
  }

  public void awaitExecution() {
    try {
      for (Iterator<Future> iter = futures.iterator(); iter.hasNext(); ) {
        iter.next().get(); //blocking call
      }
    } catch (Exception ignore) {
    } finally {
      futures.clear();
    }
  }
}

The user creates this GroupThreadPool just once, calls submit() to submit various tasks in a batch and then calls awaitExecution() to block until all tasks have executed. He can continue to use the same GroupThreadPool object to execute subsequent batches.

The implementation adds the submitted tasks to a list of Futures. To block until all tasks have completed, it calls get() on all Futures which itself is a blocking operation. So awaitExecution() returns only after all tasks have been executed but before returning it clears the list of Futures to accept the next batch of tasks.

I would love suggestions / feedback on this implementation. Is there a better approach? Also, is this a common use case which merits inclusion of awaitExecution() in ExecutorService itself?

Extend the ArcGIS ADF with POJOs - Part II

In Part I we discussed how you could implement GIS functionalities in POJOs and plug them into the ADF. In this part we'll extend the POJO a little further.

In Part I, the CountFeatures object calculated and updated the feature count on a client interaction such as a button click. Suppose that we now need for this object to recalculate the count automatically whenever the current extent of the map changes or the map refreshes due to some other action.

The ADF provides a very simple way to do this. Objects can register themselves as observers of the WebContext and whenever the context is refreshed (by virtue of the user calling the refresh() method on the context), all observers will be intimated of this event and each observer can act upon it individually. This way we have loosely coupled objects reacting together to the context refresh.

With this background let's now extend our CountFeatures class to implement this behavior.

public class CountFeatures implements WebContextInitialize, WebContextObserver {

  public void init(WebContext context) {
    ...
    context.addObserver(this);
  }

  public void update(WebContext context, Object arg) {
    doCount(); //perform the business action on update
  }
  ...
}

First up, all observers of the WebContext need to implement the WebContextObserver interface. Next, they register themselves as observers of the context by calling the addObserver() method on the context. Finally, on every context refresh, the update() method of the WebContextObserver interface is called by the ADF and the object reacts (performs the business action) to the same. In this case we simply call the doCount() method which recalculates the feature count of the updated map. This will ensure that whenever the context refreshes (for example when the user zooms or pans), this object will recalculate and display the new count to the user.

As simple as that. Apart from a few modifications to the Java code, nothing else needs to change from the Part I source code. The JSP as well as the configuration files remain unchanged. You can download all the source code (including the unchanged JSP) for this part from here.

In Part III we'll extend CountFeatures further by implementing the WebLifecycle interface.

Extend the ArcGIS ADF with POJOs - Part I

This is the first of a 3 part series where we'll discuss how to add custom GIS functionality to the ADF as POJOs (Plain Old Java Objects). To accomplish this we'll be leveraging the IOC inherent in the ADF discussed earlier. We talked about 3 very important interfaces in the IOC discussion - WebContextIntialize, WebContextObserver and WebLifecycle. Putting these 3 interfaces in practice will be central to the 3 parts respectively. In this part we'll make use of the WebContextInitialize interface.

We'll keep the functionality to be implemented quite simple: Count the number of features of a given layer in the map's current extent.

To implement this scenario our POJO will need a few basic properties and methods - a read-only count property, a read/write layerId property representing the layer whose features are to be counted and a business method doCount() which implements the business task at hand. With this said, the skeleton of the class (we'll call it CountFeatures) will be as such:

public class CountFeatures {

  //properties
  int count;
  int layerId;
  public int getCount() { return count; }
  public int getLayerId() { return layerId; }
  public void setLayerId(int layerId) { this.layerId = layerId; }

  //business method
  public String doCount() {
    ...
    ...
    count = ...;
    return null;
  }
}

You might have noticed that the doCount() method returns a String. This is because the ADF is JSF based and when the user clicks on say a command button on a web page, it results in a call to doCount(). Based on the return value of this method the JSF framework decides which page to navigate to. Returning a null ensures that the webapp stays on the same page.

OK, so now we have the skeleton in place but we also need access to the ArcGIS Server and the underlying ArcObjects to perform the GIS task at hand. This is where the WebContextIntialize comes into the picture:

public class CountFeatures implements WebContextInitialize {

  //the context associated with this object
  AGSWebContext agsctx;
  public void init(WebContext context) {
    agsctx = (AGSWebContext)context;
  }
  ...
}

The ADF will call the init(WebContext) method of objects implementing WebContextInitialize immediately after the object is instantiated. This gives the object access to the WebContext. The AGSWebContext (which is the actual implementation of the WebContext that we work with) maintains references to the ArcGIS Server objects and ArcObjects (such as IMapServer, IMapDescription, etc.) as well as to other ADF objects (such as AGSWebMap). This implies that by virtue of gaining access to the AGSWebContext our custom object now has a hook into the whole of ArcObjects as well as the ADF - basically everything that you need to accomplish your GIS task at hand.

With access to everything that our class needs, the business logic can now be implemented in the doCount() method to perform the count operation and set the result to the count variable.

That's it - our Java code ends here. All that is left to do now is to register this object as a managed attribute of the WebContext so that the ADF can automatically instantiate the object on demand as well as call the init(WebContext) method immediately after instantiation. This is accomplished by adding the following lines of XML to managed_context_attributes.xml which you can find in the /WEB-INF/classes folder of your ADF webapp:

<managed-context-attribute>
  <name>countFeatures</name>
  <attribute-class>custom.CountFeatures</attribute-class>
  <description>counts features of a given layer in the current extent...</description>
</managed-context-attribute>

With this done, you can now access our custom object by name (countFeatures in this case).

You can download the full source here. In addition to the Java code, the ZIP file also contains a sample JSP. The JSP has a command button to trigger the business method, a dropdown to choose the layer and a text out to display the count.

In conclusion I'd like to mention that while admittedly the functionality that we have implemented here is trivial, you can essentially follow the same programming model to implement your own functionality as well: POJOs which implement WebContextInitialize

In Part II we'll extend this same object to be an observer of the context and in Part III we'll make this object participate in the ADF lifecycle.

Inversion of Control in the ArcGIS Java ADF

Martin Fowler in a recent blog gave a good short explanation of the inversion of control pattern... In ESRI's ArcGIS Java ADF we employ this approach at a few places.

1. The WebContextInitialize interface declares an init(WebContext) method. The ADF will call this method on objects which implement this interface and register themselves as attributes of the WebContext. This method will be called immediately after they are registered with the WebContext. Users interested in getting access to the associated WebContext object or want to do some initialization tasks should implement this interface.


2. The WebLifecycle interface declares methods which will be called by the ADF at various phases of the webapp's lifecycle. Users can implement activation, passivation and destroy logic in these methods. This interface is most relevant when using pooled objects since users may want to rehydrate and dehydrate the states of the server objects when the ADF reconnects and releases its connection to the ArcGIS server on every request.


3. The WebContextObserver interface declares an update(WebContext webContext, Object args) method. Objects implementing this interface can register themselves as observers of the WebContext by calling the addObserver(WebContextObserver) method. After users perform operations which change the state of the objects that they work with (for example zoom to a different extent, add a graphic element, etc.), they call the refresh() method on the WebContext. When this happens, the ADF iterates thru all the registered observers of the context and calls their update() methods. This ensures loose coupling among the various objects but at the same time gives these loosely coupled objects an opportunity to be in sync with the changed state of the app. This is a classic implementation of the observer pattern with the WebContext acting as the Observable object.

With the advent of JDK 5 annotations, it might be convenient for the users if #1 could be achieved by simply annotating a field or a setter method with an @Resource like annotation. The ADF on encountering this annotation on a WebContext field or setter method could inject the same into the interested object. Further, users can do the initialization tasks in any arbitrary method annotated with the @InjectionComplete annotation and the ADF will call this method immediately after injecting the WebContext. (Both these annotations are proposed by JSR 250 - Common Annotations).

#2 could also be achieved through annotations. Much like how EJB3 is proposing @PostActivate, @PrePassivate, et al; users can annotate the lifecycle methods with annotations such as @OnActivate, @OnPassivate and @OnDestroy. This would alleviate them of having to implement interfaces for lifecycle callbacks and further, they can choose to participate in only those phases of the ADF lifecycle which makes business sense for their objects.

Comments / feedbacks welcome.

Adding layers dynamically in the ArcGIS Java ADF

There have been many questions about adding layers dynamically in the ADF... And of course the requirement is that the added layer will reflect not only on the map but also on the TOC, layer drop downs, etc...

When working with non-pooled objects, there's a straightforward way of doing this. Look at the source code below:

AGSWebContext agsCtx = ...; //get hold of the AGSWebContext
AGSWebMap agsMap = (AGSWebMap)agsCtx.getWebMap();

//Step 1
agsCtx.applyDescriptions();

//Step 2
MapServer mapso = new MapServer(agsCtx.getServer());
IMap map = mapso.getMap(agsMap.getFocusMapName());
ILayer layer = ...; //create the layer
map.addLayer(layer);

//Step 3
agsCtx.reloadDescriptions();

Let's discuss the 3 steps now:

• Step 1: Before making stateful changes to the object graph (like adding a new layer in this case) you want to apply the current state of the MapDescriptions to the object graph. Calling the applyDescriptions() method on the AGSWebContext does exactly that.


• Step 2: Now that you have the object graph in the current state, you can make the modifications there. In this case we add a new layer to the map.


• Step 3: Once you have made changes to the object graph you want to reload the MapDescriptions to reflect the changed state and additionally, you also want the web controls to display this new state (in this case display the layer on the map control, on the TOC control as well as on any other components working with layers). A single call to the reloadDescriptions() method on the AGSWebContext will do all of this for you.

And that's about it! This sequence of steps holds true for any stateful changes that you want to make to non-pooled objects. The 3-word mantra is APPLY-CHANGE-RELOAD.

If you wanted to work with dynamic layers (or make any stateful changes) in the pooled context, there's indeed more work to do because you are now sharing the server object with others and you want to return the object back to the pool in the same state that you had received it. You need to get access to the server object, apply the current MapDescriptions to the object graph, make the changes to the object graph, reflect the changes in the MapDescriptions and the web controls, undo the changes to the graph and then return the object back to the pool. You can check out the dynamic layers sample on EDN to see this use case in action.

ArcGIS Server 9.2 (Java): Coming soon to a GIS near you

The ESRI User Conference is around the corner and it's time to talk about what the future holds for our users. Before I delve into the details let me say this: ArcGIS Java users, we have heard you! Here's what the future holds for you:

• A pure Java experience: No more query interfacing... no proxies to deal with... The Java programmer will get an API s/he lives by - pure Java... diagnostic exception handling... established design patterns... the whole shebang... you name it we got it!


• A rich SDK experience with Eclipse plug-ins to help you get easily started with using ArcGIS... one click deploy and run of samples... ready to use application templates... code snippets... an integrated and intuitive help system...


• The web tier is being reworked to leverage state of the art technologies (if you are thinking AJAX you are right, but there's MORE)... how about the ability to work with multiple data sources - ArcGIS, ArcIMS, ArcWeb Services... how about the ability to add your custom source... dynamic layer reordering and grouping... network analysis... the list goes on...


• Geoprocessing: You'll get a pure Java experience with GP too - tools, Java data types, geoprocessor, exceptions... You can also build your own custom tool / model, use a tool generator to generate a pure Java class for the same and just like that you are ready to incorporate new GP functionality into your Java GIS.


• OTB EJBs to integrate GIS with your J2EE infrastructure.

All this and more will be discussed at the UC in a couple of sessions:

• ArcGIS Road Ahead: What's Coming for GIS Developers at 9.2
  
  
  
  • Offering I - Wed 8:30 AM Room: 6A (SDCC)
  
  
  • Offering II - Thu 1:30 PM Room: 6A (SDCC)
  
  
  
  


• ArcGIS Road Ahead: What's Coming in ArcIMS and ArcGIS Server at 9.2
  
  
  
  • Offering I - Tue 1:30 PM Room: 6A (SDCC)
  
  
  • Offering II - Thu 8:30 AM Room: 6A (SDCC)
  
  
  
  

I'm sure you are rushing to add them to your agenda! Also, here's the UC Q and A for more info...

See ya!

Annotation use cases

My first blog here but will cut straight to the point.

The most talked about feature at J1 this year was annotations. It was as if every new API / framework had to have support for annotations or must have something pertaining to it on their radar to gain acceptance or even be considered a contender.

I have not yet being able to make my mind if this profileration of @YeahIHaveAnAnnotationToo is a good thing or not. For the time being I am trying to come up with use cases of where annotations make sense. Here are some that I have assimilated from various blogs, J1 sessions and my own brain dumps:

1. Dependency injection.

• The @Resource and the @EJB annotations introduced by EJB3 are obvious examples of dependency injections. Any custom framework will have some notion of a context or an environment or a manager and injecting such resources into users' classes would make a lot of sense.

2. Aspects (boiler plate stuff handled by the container / framework) and interceptors

• The @TransactionAttribute annotation associates a transaction aspect to a method. @Query and @Update annotations introduced by JDBC 4.0 perform tasks which go beyond just boiler plate - they actually populate and manipulate objects which directly affect business logic. I've put interceptors alongwith aspects coz I consider interceptors as an implementation technique for aspects (Correct me if I am wrong). I think one needs to be really careful when designing annotations in this category because taking this route could be a slippery slope - you design a bunch of annotations which perform certain core tasks and your co-developer has his/her own set of tasks which s/he religiously believes are "core" and your manager feels that it's critical to design a third set to appease that billion $ client. IMO, for this category of annotations just design those which strictly do boiler plate stuff and don't necessarily alter the business state for the client.

3. Callback methods

• The @PostActivate and @PrePassivate ones fall in this category. I am not sure if I am a fan of annotations in this category. Basically with this all you are trying to do is eliminate the need to implement certain interfaces. Which means all you gain is that you don't need to have empty implementations for methods that you don't need. But it becomes very difficult for somebody to understand what contracts your object fulfills or which methods are actually designated as callbacks. One might argue that objects are no longer POJOs if they have to adhere to some framework specific interfaces - if this argument is important for you then implement callback listeners (an alternative that EJB3 also provides) to achieve this objective.

4. Container contracts

• Tagging an EJB as @Stateless and @Stateful makes it known to the container how it should treat that bean. Moreover, it also sets a programming pattern for the user how to access and use these objects. Most marker annotations could be included in this category. So if your framework needs to handle certain objects in specific ways, you could think about such annotations.

5. Programmatic access to metadata for classes, fields and methods.

• This is an oft forgotten use case but one which I think makes sense for UI objects. While it's a given that it's important to publish information regarding a method or a field's behavior through detailed comments and Javadocs; for UI objects this information may also need to be presented to the user on desktop panels and HTML forms. So the next time you author a UserBean think about associating a runtime @Description annotation with the username and password fields so as to give consistent descriptions for these fields across different views of your app.

(I'll try to add more as I get my head around it more (or if you have any more points for me).)

While this push toward annotated POJOs for everything might be a good thing; the thing that I fear is that Java classes of the future would have less Java code and more of annotations. It may become increasingly difficult for the user to deterministically gauge what behavior any given method would exhibit because what the method does could change dramatically depending on which annotations were applied when the method was actually called.

Not trying to attach a pessimistic annotation to annotations (what can I say, I love meta-anything!) but just waving a flag of caution before we head heads first into the annotated unknown...