NioSax – Sax style xml parser for Java NIO

NioSax (pronounced ‘Neo-Sax’) provides a Java NIO friendly XML push parser similar in operation to SAX. Unlike SAX, with NioSax it is possible for the xml source to contain partial content (i.e. only part of the XML stream has been received over the network). When this occurs, instead of failing with an error, NioSax simply stops. As soon as your application receives more data you simply call the same instance of the parser again and it will resume parsing where it left off.

NioSax (pronounced ‘Neo-Sax’) provides a Java NIO friendly XML push parser similar in operation to SAX. Unlike SAX, with NioSax it is possible for the xml source to contain partial content (i.e. only part of the XML stream has been received over the network). When this occurs, instead of failing with an error, NioSax simply stops. As soon as your application receives more data you simply call the same instance of the parser again and it will resume parsing where it left off.

The public API consists of the classes within this package, although the bare minimum required for use are the NioSaxParser, NioSaxParserHandler and NioSaxSource classes.

To use NioSax you simply use NioSaxParserFactory to create a NioSaxParser, implement a SAX ContentHandler and finally create a NioSaxSource which references the content.

Then you can parse one or more ByteBuffer’s by updating the NioSaxSource with each buffer and pass it to the NioSaxParser.parse(NioSaxSource) method.

The only other two things you must to do with the parser is to ensure that you call NioSaxParser.startDocument() prior to any parsing, and call NioSaxParser.endDocument() once you are done with the parser so any resources used can be cleaned up.

Example

First in maven we need to add a dependency to NioSax. For details of the repository click on the ‘reteptools’ menu above. However you’ll need to add the following to your pom:

<dependency>
    <groupId>uk.org.retep</groupId>
    <artifactId>niosax</artifactId>
    <version>10.6</version>
</dependency>

Now we’ll create a parser:

import java.nio.ByteBuffer;
import uk.org.retep.niosax.NioSaxParser;
import uk.org.retep.niosax.NioSaxParserFactory;
import uk.org.retep.niosax.NioSaxParserHandler;
import uk.org.retep.niosax.NioSaxSource;

public class MyParser
{
    private NioSaxParser parser;
    private NioSaxParserHandler handler;
    private NioSaxSource source;

    public void start()
    {
        NioSaxParserFactory factory = NioSaxParserFactory.getInstance();

        parser = factory.newInstance();
        parser.setHandler( handler );
        source = new NioSaxSource();

        parser.startDocument();
    }
}

Next, when you receive data from some nio source and have the data in a ByteBuffer you need to pass it to the parser:

    public void parse( ByteBuffer buffer )
    {
        // flip the buffer so the parser starts at the beginning
        buffer.flip();

        // update the source (presuming the buffer has changed)
        source.setByteBuffer( buffer );

        // Parse the available content then compact
        parser.parse( source );
        source.compact();
    }

Finally we must call endDocument() to release any resources:

    public void close()
    {
        // releases any resources and notifies the handler the docment has completed
        parser.endDocument();
    }

Now all we need to is when we receive some data from an external source like a Socket, we pass the ByteBuffer to the parse method. This then passes it to the NioSax parser which in turn calls the ContentHandler as the parse progresses.

When it gets to the end of the available content, it compacts the buffer so that it can be reused.

Usually the buffer will now be empty, however if there was partial content (like only part of a Unicode character was present) then the parser would stop prior to that character and that character would remain in the buffer. The next packet received via nio would have the rest of that character and the parser would then continue where it left off.

This was originally posted early in 2009 but the post seemed to have vanished so this article is loosely based on the documentation for NioSax.

It’s been a busy couple of months

It’s been a couple of busy months with most of my time being taken up with my day job.

Most of my time has been spent with either tracking down issues with our live environment, or trying to finish off a couple major projects (both related to XMPP) interspersed with the usual major partner getting in the way.

Any how, over the last couple of weeks I’ve been finishing off some new features which cover most of my public projects and this post will hopefully cover some of the details.

Hopefully these will be released this weekend, time allowing.

The new features are:

RetepTools
* the builder api within retepTools has been updated
* the jaxb plugin library has been cleaned up with common generation code split out to enable reuse
* retepTools as a project is almost ready for deployment to maven central
* a new pligun has been added to jaxb which generates builders for jaxb objects

RetepMicroKernel
* spring has been updated to the latest version 3 (it was on 2.5)
* the core module has been broken up into individual & independent modules
* a new groovy module which enables groovy scripts to be run from the command line
* a major bug fix where exceptions thrown during application startup causes the process to hang has been fixed
* web applications can now be deployed as a war with either jetty or tomcat (they are both supported with their own modules)
* you can now embed Apache Derby within the environment

I’m leaving out the retepXMPP changes out of this list as they need their own article. Suffice it to say, I’ve got a lot waiting for release, just need the time.

Finally, this post is also a test of submitting a blog post from a BlackBerry using the WordPress app so the formatting may be off a tad – won’t know how it goes until I see it in a real browser.

retepTools Concurrency Support

retepTools Concurrency Support

The retepTools library provides additional concurrency support to that provided by the java.util.concurrency package of JDK 1.5 or later.

The core component of the concurrency support provided by retepTools is the locking. By utilising a set of annotations, it is possible to mark a method so that the entire body of that method is bound to the scope of that Lock.

For example, we have a bean with a property. The properties value can be read by any number of Threads, but it can be set by only one at a time. In the normal java.util.concurrent.lock way you would write something like this:

import java.util.concurrent.locks.ReadWriteLock;

import java.util.concurrent.locks.ReentrantReadWriteLock;

public class MyBean

{

  private final ReadWriteLock lock = new ReentrantReadWriteLock();

  private int value;

  public int getValue()

  {

    lock.readLock().lock();

    try

    {

      return value;

    }

    finally

    {

      lock.readLock().unlock();

    }

  }

  public void setValue( int newValue )

  {

    lock.writeLock().lock();

    try

    {

      value = newValue;

    }

    finally

    {

      lock.writeLock().unlock();

    }

  }

}

Now imaging doing that on an object with dozens of properties… a lot of boiler plate code which is prone to typing errors. retepTools removes this boiler plate code by using a set of thee annotations: @Lock, @ReadLock and @WriteLock.

All three annotations follow a simple contract:

    * your object must implement a corresponding method for each annotation.

    * the associated method is named as it’s annotation, i.e. @ReadLock expects readLock(), etc.

    * that method must be declared either private or “protected final”.

    * the Lock object returned by those methods must also be final – specifically it must be the same object returned for every invocation of that method on that instance.

For convenience the class uk.org.retep.util.collections.ConcurrencySupport implements this contract for objects using @ReadLock and @WriteLock.

So here’s the above code rewritten to use the annotations:

import java.util.concurrent.locks.Lock;

import java.util.concurrent.locks.ReadWriteLock;

import java.util.concurrent.locks.ReentrantReadWriteLock;

import net.jcip.annotations.ThreadSafe;

import uk.org.retep.annotations.ReadLock;

import uk.org.retep.annotations.WriteLock;

@ThreadSafe

public class MyBean

{

  private final ReadWriteLock lock = new ReentrantReadWriteLock();

  private int value;

  protected final Lock readLock()

  {

    return lock.readLock();

  }

  protected final Lock writeLock()

  {

    return lock.writeLock();

  }

  @ReadLock

  public int getValue()

  {

      return value;

  }

  @WriteLock

  public void setValue( int newValue )

  {

      value = newValue;

  }

}

Now thats alot cleaner, less error prone and yet easier to see what the business logic is rather than having it obscured by the locking mechanism.

retepTools also provides several concrete base classes to make this even easier. The main one is uk.org.retep.util.concurrent.ReadWriteConcurrencySupport which provides concrete implementations of readLock() and writeLock(). So the above example can be made even simpler by extending ReadWriteConcurrencySupport:

import java.util.concurrent.locks.ReadWriteLock;

import java.util.concurrent.locks.ReentrantReadWriteLock;

import net.jcip.annotations.ThreadSafe;

import uk.org.retep.annotations.ReadLock;

import uk.org.retep.annotations.WriteLock;

import uk.org.retep.util.concurrent.ReadWriteConcurrencySupport;

@ThreadSafe

public class MyBean extends ConcurrencySupport

{

  private int value;

  @ReadLock

  public int getValue()

  {

      return value;

  }

  @WriteLock

  public void setValue( int newValue )

  {

      value = newValue;

  }

}

Now thats alot cleaner, less error prone and yet easier to see what the business logic is rather than having it obscured by the locking mechanism.

Footnotes

  1. I originally started to write this article back in February however since retepTools 9.2 a new annotation processor was added which performs additional sanity checks.
  2. Project Coin is currently taking proposals for small changes to the Java Language. One of those proposals was Automatic Resource Management and during the debate about that proposal Lock’s were brought up. Now locks are out of scope for that proposal and there is an additional one which now looks like it’s not going to be accepted. I’ll be writing another article about that one later.

The Generic Messaging API

A common use case is the routing of messages from one component within a system to another. The standard API for this type of messaging within Java is the Java Messaging Service or JMS – but that can be a bit overkill for small embedded systems or if you are implementing another existing messaging protocol like Jabber. Originating from the retepXMPP project, retepTools 9.2 introduces a core messaging API which can form the basis of many simple messaging systems.

Introduction

The Messaging API is relatively simple comprising of four main classes in the uk.org.retep.util.messaging package:

  • MessagingService – a class that forms the entry point into a messaging system for messages
  • Message – an interface defining the message that can be handled by the service
  • Component – an interface defining a component that can accept messages
  • Router – an interface defining a component that can route messages to another Router or component.

The API is totally generic so the above classes have no concept of the inplementation of an address (where messages are sent to) or the route required to send them between Routers – this is left entirely to the implementing code. The only requirement is that the implementing classes obey the contract between hashCode and equals.

The API also provides some concrete implementations of the interfaces. These implementations are themselves generic, but they are also fully concurrent allowing the implemented messaging system to operate in a fully multi-threaded environment.

The rest of this article will show how to implement a simple messaging system where a group of interacting components can pass messages comprising of simple POJO’s between themselves.

Concepts

Addressing

The first concept within the Messaging API is addressing. An address is a POJO that defines the Component that the message should be sent to. It also defines where the message was sent from if a response is required. This POJO can be any Java object. For this example we will be using java.lang.String but for more complex systems you would implement a class specifically for that purpose.

For example, in Jabber an address is known as a JID (Jabber ID) and has the form node@domain/resource (where node and resource are optional). In retepXMPP the address is implemented by a JID class which holds those three components individually allowing the messages to be routed by domain first and then by node and finally by resource.

Component

The second concept is a Component, represented by the uk.org.retep.util.messaging.Component interface. It represents a POJO that can accept a Message for some form of processing.

Route

The third concept is a Route. A route is a POJO used by a Router to determine where to send messages. Usually a Route implementation would hold the part of an address relevant to the specific Router using it. You could think of the global Domain Name System (DNS). For DNS you could have an address of “retep.org”. Then sending a messsage to that address would invove the core router using the Route “org” to send the message to the router handling “org”. That router wouild then send the message to the end Component using the Route “retep”.

Router

The final concept is a Router. A Router, represented by the uk.org.retep.util.messaging.Router interface is a specialised Component that can contain other Components. It deals with forwarding Messages to one or more Components by using Route’s.

A Simple example

To show how simple it is to setup a MessagingService, we’ll create a simple system comprising of three Components that will interact with each other when an event occurs. We’ll call these components A, B and C (yes imaginative naming here).

Address

For this example our address is a java.lang.String.

Message

The first step is to create the message. This will hold an int describing the event.

package example;

import uk.org.retep.util.messaging.message.AbstractMessage;

public class Event

        extends AbstractMessage<String>

{

    private int event;

    public int getEvent()

    {

        return event;

    }

    public void setEvent( int event )

    {

        this.event = event;

    }

}

AbstractMessage is a concrete implementation of the Message interface implementing all of it’s methods. The <String> generic defines the type of the address, in this case java.lang.String.

Components

Now for our three components. Here component A will simply send a message to B when it’s setValue() method is called. B will log that event and forward the message to C. C will then log the event itself and reply back to the originator (in this case A) with the negative value. A will then log the returned value.

package example;

import net.jcip.annotations.ThreadSafe;

import uk.org.retep.util.messaging.MessageException;

import uk.org.retep.util.messaging.component.AbstractComponent;

@ThreadSafe

public class A

        extends AbstractComponent<String, Event>

{

    public void setValue( int value )

    {

        System.out.printf( “A.setValue( %d )\n”, value );

        

        Event event = new Event();

        event.setFrom( “A” );

        event.setTo( “B” );

        event.setEvent( value );

        

        try

        {

            send( event );

        }

        catch( MessageException ex )

        {

            ex.printStackTrace();

        }    }

    @Override

    public void consume( Event message )

            throws MessageException

    {

        System.out.printf( “A received event from %s: %d\n”,

                           message.getFrom(),

                           message.getEvent() );

    }

}

You’ll notice that the setValue method creates a new Event, sets the from and to addresses, the value of the event and then sends the message via the send() method. Once the Component is attached to the MessagingSystem then that message will be sent on to its destination.

@ThreadSafe

public class B

        extends AbstractComponent<String, Event>

{

    @Override

    public void consume( Event event )

            throws MessageException

    {

        System.out.printf( “B received event from %s: %s\n”,

                           event.getFrom(),

                           event.getEvent() );

        Event newEvent = new Event();

        newEvent.setFrom( event.getFrom() );

        newEvent.setTo( “C” );

        newEvent.setEvent( event.getEvent() );

        send( newEvent );

    }

}

@ThreadSafe

public class C

        extends AbstractComponent<String, Event>

{

    @Override

    public void consume( Event event )

            throws MessageException

    {

        System.out.printf( “C received event from %s: %s\n”,

                           event.getFrom(),

                           event.getEvent() );

        Event newEvent = new Event();

        newEvent.setFrom( “C” );

        newEvent.setTo( event.getFrom() );

        newEvent.setEvent( -event.getEvent() );

        send( newEvent );

    }

}

Router and Route

Now every messaging service has a core router which accepts every message as it first enters the system. For this example we’ll use the address as the route as it’s a single keyword:

package example;

import net.jcip.annotations.ThreadSafe;

import uk.org.retep.util.messaging.router.AbstractRouter;

@ThreadSafe

public class SimpleRouter

        extends AbstractRouter<String, String>

{

    @Override

    protected String getRoute( String to )

    {

        return to;

    }

}

The Generics here define the types for the Address and the Route respectively. Now as we are using String’s then both are set to String. Also our router implements the getRoute() method which should form the Route for this instance from the to address. In this case it’s just the address, but usually you would have some other logic here.

Deploy and Run

Now all there’s left is to deploy the system and send a message:

package example;

import uk.org.retep.util.messaging.MessagingService;

import uk.org.retep.util.messaging.Router;

public class Main

{

    private Main()

    {

    }

    public static void main( String… args )

            throws Exception

    {

        Router<String, String> router = new SimpleRouter();

        MessagingService<String, String> messagingService = new MessagingService<String, String>(

                router );

        A a = new A();

        messagingService.addRoute( “A”, a );

        messagingService.addRoute( “B”, new B() );

        messagingService.addRoute( “C”, new C() );

        messagingService.startService();

        a.setValue( 42 );

    }

}

Here we create the core Router and then the MessagingService using that router. We then create the A component and add it to the service. We then do the same for B and C. Finally we start the service, and then set the value of A to 42. When run we should then see the following as the event propagates through the system:

A.setValue( 42 )

B received event from A: 42

C received event from A: 42

A received event from C: -42

Note: C says it’s received the event from A and not B because the event’s from address is A and not B – this is from the statement newEvent.setTo( event.getFrom() ); in B.

Now with this example it’s pretty simple. We could have started the service before adding the components if we wanted to – components can be added or removed at will.

Also this example is bad in the sense that we are adding all components directly to the MessagingService. This would be correct for components routed by the core route, but say we had a router called D containing a component called D/A. The component D/A would be attached to router D and D would be connected to the messaging service.

Validating source with Annotation Processors

The release of retepTools 9.2 this week introduces a new annotation processor which validates the use of some of the annotations provided by the library generating either compilation warnings or errors dependent upon the errors found. This article describes how that processor works and the reasons behind it’s creation.

In the previous article I wrote about problems in reading values from Annotations in super types. That issue had cropped up due to some of the rules required for this processor.

Now this processor is still in it’s early stages of development but it works and is therefore useable. The rules that follow are those it will enforce and those will not change – although I will almost certainly add more over time.

The rules

Each rule defined below follows this template:

package.Annotation

Action when violated: Compile time error/warning

Mutually excludes: package.AnotherAnnotation…

Related: package.RelatedAnnotation

The Action when violated indicates if an error or warning wouild be produced if that rule is violated.

If the Mutually excludes line is present, then a compile time error will be generated if the annotation is present on an element with any of the listed annotations.

If the Related line is present, it lists annotations that are related and can affect the outcome of this rule.

1 Concurrency RULES

The main set of rules are for concurrency. In previous versions there’s a set of annotations which inject code into the methods they annotate ensuring those methods run within a specific lock – either a standard java.util.concurrent.locks.ReentrantLock or a java.util.concurrent.locks.RentrantReadWriteLock. The problem was that without compile time checking it would be possible to mark a method in one class as being a ReadLock, but then override that method in a subclass and annotate it with WriteLock. Now this would cause an immediate deadlock.

So here are the rules enforced for concurrency:

net.jcip.annotations.NotThreadSafe

Action when violated: Compile time error

Mutually excludes: net.jcip.annotations.ThreadSafe

This annotation is still optional, however it should be declared on a class that may be used in a concurrent context to indicate that it does not use any concurrency. For example an implementation of List should be marked @NotThreadSafe, but a ConcurrentList implementation with @ThreadSafe.

When it is present, this rule will generate a compile time error if any of the concurrency annotations are used in that class. Also subclasses cannot be ThreadSafe due to this class being NotThreadSafe.

For obvious reasons this annotation is mutually exclusive with NotThreadSafe

net.jcip.annotations.ThreadSafe

Action when violated: Compile time error

Mutually excludes: net.jcip.annotations.NotThreadSafe

For all classes that use any of the concurrency annotations must now be marked as ThreadSafe. The reason behind this is to ensure that documentation indicates that the class is ThreadSafe, but also to ensure that subclasses are also ThreadSafe as they can affect the concurrency of the super class. Because of this, a subclass of a class annotated with ThreadSafe must also be marked as ThreadSafe.

For obvious reasons this annotation is mutually exclusive with NotThreadSafe

uk.org.retep.annotations.Lock

Action when violated: Compile time error or warning

Mutually excludes: uk.org.retep.annotations.ReadLock, uk.org.retep.annotations.WriteLock

Related: uk.org.retep.annotations.Contract

This annotation indicates that the method runs within a shared lock. As defined by the annotations javadoc, it injects code into the method to gain the lock, run the method body and then release the lock.

This rule ensures that:

  • The method cannot be annotated with a ReadLock or WriteLock as those annotations are mutually exclusive with this one – generates an error.
  • If the method is overridden then a warning is issued that the overridden method code will be running outside of the lock.
  • As there’s a contract defined in the javadoc for a method called lock() to be defined then that method should be annotated with @Contract( Lock.class ) to document that it’s bound by that Contract. If it is not then a warning is generated.
  • The Contract for the support lock() method that is enforced is that the method has one of the following signatures. If it does not have these signatures then an error is generated: private java.util.concurrent.locks.Lock lock(); or protected final java.util.concurrent.locks.Lock lock();
  • The class is annotated with @ThreadSafe.

uk.org.retep.annotations.ReadLock

Action when violated: Compile time error or warning

Mutually excludes: uk.org.retep.annotations.Lock, uk.org.retep.annotations.WriteLock

Related: uk.org.retep.annotations.Contract

This annotation indicates that the method runs within a shared read lock. As defined by the annotations javadoc, it injects code into the method to gain the lock, run the method body and then release the lock.

This rule ensures that:

  • The method cannot be annotated with a Lock or WriteLock as those annotations are mutually exclusive with this one – generates an error.
  • If the method is overridden then a warning is issued that the overridden method code will be running outside of the lock.
  • As there’s a contract defined in the javadoc for a method called readLock() to be defined then that method should be annotated with @Contract( ReadLock.class ) to document that it’s bound by that Contract. If it is not then a warning is generated.
  • The Contract for the support readLock() method that is enforced is that the method has one of the following signatures. If it does not have these signatures then an error is generated: private java.util.concurrent.locks.Lock readLock(); or protected final java.util.concurrent.locks.Lock readLock();
  • The class is annotated with @ThreadSafe.

uk.org.retep.annotations.WriteLock

Action when violated: Compile time error or warning

Mutually excludes: uk.org.retep.annotations.Lock, uk.org.retep.annotations.ReadLock

Related: uk.org.retep.annotations.Contract

This annotation indicates that the method runs within a shared lock. As defined by the annotations javadoc, it injects code into the method to gain the lock, run the method body and then release the lock.

This rule ensures that:

  • The method cannot be annotated with a Lock or ReadLock as those annotations are mutually exclusive with this one – generates an error.
  • If the method is overridden then a warning is issued that the overridden method code will be running outside of the lock.
  • As there’s a contract defined in the javadoc for a method called writeLock() to be defined then that method should be annotated with @Contract( WriteLock.class ) to document that it’s bound by that Contract. If it is not then a warning is generated.
  • The Contract for the support writeLock() method that is enforced is that the method has one of the following signatures. If it does not have these signatures then an error is generated: private java.util.concurrent.locks.Lock writeLock(); or protected final java.util.concurrent.locks.Lock writeLock();
  • The class is annotated with @ThreadSafe.

2 Singletons

There are two annotations provided for supporting singletons:

uk.org.retep.annotations.NoInstance

Action when violated: Compile time error

Mutually excludes: uk.org.retep.annotations.Singleton

A class marked with NoInstance implies that there can be no instance of this class – i.e. a class with just static fields or methods.

This rule will generate an error if the class violates any of the following:

  • The class is not declared final as it cannot have subclasses
  • The class has a non private default constructor
  • The class has a non default constructor
  • The class has any instance methods or fields
  • The class has a field referencing itself – i.e. private static Class instance;
  • The class has a method who’s return type is that of the class
  • The class is annotated with Singleton as its mutually exclusive with NoInstance

uk.org.retep.annotations.Singleton

Action when violated: Compile time error

Mutually excludes: uk.org.retep.annotations.NoInstance

A class marked with Singleton implies that there is only a single instance of this class .

This rule will generate an error if the class violates any of the following:

  • The class is not declared final as it cannot have subclasses
  • The class has a non private default constructor
  • The class has a non default constructor
  • The class has no instance methods or fields
  • The class does not have a private static field referencing itself – i.e. private static Class instance;
  • The class does not have a public static method who’s return type is that of the class
  • The class is annotated with NoInstance as its mutually exclusive with Singleton

3 Miscellaneous

This set of rules are not associated with any annotations but enforce certain optional rules.

3.1 hashCode and Equals

When enabled ensures that if a class overrides either of the hashCode() or equals() methods then a compiler error is issued if the class does not override both of them. This is because there is a contract between those two methods where equals() can return true only if the hashCode of both object are also equal (read the docs for java.lang.Object if you disagree).

This rule came about because recently I had a class that was misbehaving in a map and it was because it had not defined both methods.

This rule is enabled by default and can be turned off by passing a configuration parameter to javac.

3.2 Missing javadoc comments

When enabled this rule will generate either a compiler warning or error if a non-private method has no documentation. This option is disabled by default but can be enabled by passing a configuration parameter to javac. The type of action when the rule is violated is itself configurable for this rule.

4 Configuration

The processor has some level of configurability, enabling certain rules to be enabled or disabled depending on user requirements.

4.1 Javac

When using the javac command you can pass any of the following options on the command line, prefixing them with -A. They then take a single value, either true or false to turn that option on or off. For example to turn on the missing javadoc rule, then you would pass -AwarnMissingJavadocs=true to javac.

4.2 Maven

When using the maven compiler plugin you are supposed to be able to add the javac options to the pom by using the compilerArgument attribute:

<compilerArgument>-AfailHashCodeEquals=true -AwarnMissingJavadocs=true</compilerArgument>

The problem is that although this is shown in the plugins documentation, it doesn’t work as it gets passed to javac as a single argument and not as a set of arguments.

To get around this the processor also supports a special option called mavenOpts who’s value is a comma separated list of the required features – if the feature is in the string then it is enabled. To disable a feature then simply prefix the feature with either ! or ^ – there’s two options to negate as some shells use ! so it’s not always possible to use on the command line:

<compilerArgument>-AmavenOpts=failHashCodeEquals,^warnMissingJavadocs</compilerArgument>

4.3 Currently supported options

failHashCodeEquals

Should an error occur if one of hashCode or equals is overridden but not the other? Enabled by default

warnMissingJavadocs

If enabled a compiler warning is generated for each non private method or field that has no documentation.

failMissingJavaDocs

This overrides warnMissingJavadocs if enabled. This will cause a compiler error to be issued if a non private method or field has no documentation.

Getting Class values from Annotations in an AnnotationProcessor

In annotation processors one common use case is to read the values contained within an Annotation. In most cases this works fine, however there are some caveats when it comes to accessing values within Annotations that are either Class or a Class array. It’s even worse if you are trying to access an annotation on a method within a super type of the class you are processing. Here I’ll go through how to access those Class values and how to get those values when they are in a super type – specifically when a method has been overridden.

First why?

Well inside retepTools we have an AnnotationProcessor that checks to ensure that certain annotations are used correctly, specifically the @ReadLock and @WriteLock annotations. It’s invalid for those annotations to be used together on the same method. It’s also invalid for a method to be annotated with one and then overridden in another class but annotated with the other, so the processor checks the overridden method and generates an error if the rules have been violated.

Now this caused be a big headache in reading the overridden annotations, because although the annotations were present, the values (and specific the Class ones) were null. Although @ReadLock and @WriteLock do not use values, another related annotation @Contract does, so here’s how I finally solved the problem.

The usual way of reading annotation values

In most cases, when you have an javax.lang.model.element.Element you can use the getAnnotation() method to return your annotation and then you have access to the values contained within it. For example say we have an annotation called Action which holds a single String value:

@Documented
@Retention( RetentionPolicy.RUNTIME )

@Target(ElementType.METHOD)

public @interface Action

{

    String value();

}

We annotate a class with that annotation against the run method:

public class A {

    @Action( “do something” )

    public void run() {

    }

}

Now within your AnnotationProcessor, you can obviously get the value easily by calling getAnnotation( Action.class ) and if it returns an instance call the value() method:

@SupportedAnnotationTypes( “*” )

@SupportedSourceVersion( SourceVersion.RELEASE_6 )

@ProhibitAnnotationProcessing

public class Processor

        extends AbstractProcessor

{

    @Override

    public boolean process( final Set<? extends TypeElement> annotations,

                            final RoundEnvironment env )

    {

        if( !env.processingOver() )

        {

            for( Element e : env.getRootElements() )

            {

                TypeElement te = findEnclosingTypeElement( e );

                System.out.printf( “\nScanning Type %s\n\n”,

                                   te.getQualifiedName() );

                for( ExecutableElement ee : ElementFilter.methodsIn(

                        te.getEnclosedElements() ) )

                {

                    Action action = ee.getAnnotation( Action.class );

                    

                    System.out.printf(

                            “%s Action value = %s\n”,

                            ee.getSimpleName(),

                            action == null ? null : action.value() );

                }

            }

        }

        return false;

    }

    public static TypeElement findEnclosingTypeElement( Element e )

    {

        while( e != null && !(e instanceof TypeElement) )

        {

            e = e.getEnclosingElement();

        }

        return TypeElement.class.cast( e );

    }

}

This generates the following output when run:

————————————————————————

Building scratch

   task-segment: [clean, install]

————————————————————————

[clean:clean]

Deleting directory /Users/peter/dev/retep/scratch/target

[compiler:compile {execution: compileAnnotations}]

Compiling 2 source files to /Users/peter/dev/retep/scratch/target/classes

[resources:resources]

Using default encoding to copy filtered resources.

[compiler:compile]

Compiling 2 source files to /Users/peter/dev/retep/scratch/target/classes

Scanning Type scratch.A

run Action value = do something

Reading annotation values from an overridden method

Now this is fine, but what happens if you are looking at an annotation thats in an overridden class?

Say we have class B which extends A and overrides run():

public class B extends A {

    @Override

    public void run() {

    }

}

Now when we run the processor, for each ExecutableElement we’ll first look for an annotation and then if not found we’ll look for an overridden method and check there. 

    @Override

    public boolean process( final Set<? extends TypeElement> annotations,

                            final RoundEnvironment env )

    {

        if( !env.processingOver() )

        {

            for( Element e : env.getRootElements() )

            {

                TypeElement te = findEnclosingTypeElement( e );

                System.out.printf( “\nScanning Type %s\n\n”,

                                   te.getQualifiedName() );

                for( ExecutableElement ee : ElementFilter.methodsIn(

                        te.getEnclosedElements() ) )

                {

                    Action action = ee.getAnnotation( Action.class );

                    if( action == null )

                    {

                        // Look for the overridden method

                        ExecutableElement oe = getExecutableElement( te,

                                                                     ee.getSimpleName() );

                        if( oe != null )

                        {

                            action = oe.getAnnotation( Action.class );

                        }

                    }

                    System.out.printf(

                            “%s Action value = %s\n”,

                            ee.getSimpleName(),

                            action == null ? null : action.value() );

                }

            }

        }

        return false;

    }

    public ExecutableElement getExecutableElement( final TypeElement typeElement,

                                                   final Name name )

    {

        TypeElement te = typeElement;

        do

        {

            te = (TypeElement) processingEnv.getTypeUtils().asElement(

                    te.getSuperclass() );

            if( te != null )

            {

                for( ExecutableElement ee : ElementFilter.methodsIn(

                        te.getEnclosedElements() ) )

                {

                    if( name.equals( ee.getSimpleName() ) && ee.getParameters().isEmpty() )

                    {

                        return ee;

                    }

                }

            }

        } while( te != null );

        return null;

    }

Now when we run we get the annotated value on A.run() when we are processing B.run():

[compiler:compile]

Compiling 2 source files to /Users/peter/dev/retep/scratch/target/classes

Scanning Type scratch.A

run Action value = do something

Scanning Type scratch.B

run Action value = do something

The problem with Class

Now this is fine, but what happens if the annotation’s value is Class instead of String? Well the problem here is that Javac does not load classes in the normal manner. In fact it doesn’t at all for classes that are in the source – it’s all contained within a model.

Now say our Action annotation had value defined as Class instead of String. In that case when we call action.value() it would fail:

[compiler:compile]

Compiling 2 source files to /Users/peter/dev/retep/scratch/target/classes

Scanning Type scratch.A

javax.lang.model.type.MirroredTypeException: Attempt to access Class object for TypeMirror java.lang.Runnable

So we have to find another way to get the value, and there are two available to us. The first is not to use getAnnotation() but getAnnotationMirrors(), and the second is a neat trick with MirroredTypeException.

Solution 1 use getAnnotationMirrors()

This solution is a little long winded but is the most reliable. When getAnnotationMirrors() is used, it returns a set of AnnotationMirror instances, one for each annotation on that Element, so the first step is to locate the correct AnnotationMirror for the annotation you require. The next step is to extract the AnnotationValue’s from that mirror which represents the values stored in the annotation. This is held in a map keyed by an ExecutableElement.

Why ExecutableElement? Well it’s because the annotation values are actually defined as methods – hence why in our Action value is defined as value().

So in the next example we run through the AnnotationMirrors on a method until we find the one for our annotation then run through until we find the required value.

Once we have the AnnotationValue we simply print it to System.out but normally you would use the getValue() method which returns the value as an Object. If the value is an array it returns a java.util.List containing the values. Oh and if the values are of type Class then it returns a TypeMirror or a List of TypeMirrors.

Element actionElement = processingEnv.getElementUtils().getTypeElement(

        Action.class.getName() );

TypeMirror actionType = actionElement.asType();

for( ExecutableElement ee : ElementFilter.methodsIn(

        te.getEnclosedElements() ) )

{

    ExecutableElement oe = ee;

    AnnotationValue action = null;

    while( action == null && oe != null )

    {

        for( AnnotationMirror am : oe.getAnnotationMirrors() )

        {

            if( am.getAnnotationType().equals( actionType ) )

            {

                for( Map.Entry<? extends ExecutableElement, ? extends AnnotationValue> entry : am.getElementValues().entrySet() )

                {

                    if( “value”.equals(

                            entry.getKey().getSimpleName().toString() ) )

                    {

                        action = entry.getValue();

                        break;

                    }

                }

            }

        }

        // Look for the overridden method

        oe = getExecutableElement(

                findEnclosingTypeElement( oe ),

                ee.getSimpleName() );

    }

    System.out.printf(

            “%s Action value = %s\n”,

            ee.getSimpleName(),

            action == null ? null : action );

}

Now at first that appears to work, and in most use cases it does – if we have a method thats overridden then we get the annotation values from the overridden method.

However, although its not obvious, if the super type is not part of the same CompilationUnit – i.e. it’s in a third party jar or from a previous call to javac then it will not find anything outside of that CompilationUnit.

The trouble with TypeMirrors

Now the cause on why the solution above fails isn’t obvious. The problem here is actually down to the TypeMirror’s. In the above example we get a TypeMirror for the annotation called actionType then search the AnnotationMirror set of each element using that TypeMirror.

Now TypeMirror acts in a similar way to how Class works at runtime. At run time Class is equal if it’s in the same ClassLoader, so here TypeMirror is equal if it’s in the same CompilationUnit. So the example above fails because they are different instances.


So the solution here is not to use TypeMirror.equals() but to convert the TypeMirror into a String representing the fully qualified class name and use equals() on that String. Now, no matter what  source the super type comes from, it will always match.

Here’s the new version:

final String actionName = Contract.class.getName();

for( ExecutableElement ee : ElementFilter.methodsIn(

        te.getEnclosedElements() ) )

{

    ExecutableElement oe = ee;

    AnnotationValue action = null;

    while( action == null && oe != null )

    {

        for( AnnotationMirror am : oe.getAnnotationMirrors() )

        {

            if( actionName.equals(

                    am.getAnnotationType().toString() ) )

            {

                for( Map.Entry<? extends ExecutableElement, ? extends AnnotationValue> entry : am.getElementValues().entrySet() )

                {

                    if( “value”.equals(

                            entry.getKey().getSimpleName().toString() ) )

                    {

                        action = entry.getValue();

                        break;

                    }

                }

            }

        }

        // Look for the overridden method

        oe = getExecutableElement(

                findEnclosingTypeElement( oe ),

                ee.getSimpleName() );

    }

    System.out.printf(

            “%s Action value = %s\n”,

            ee.getSimpleName(),

            action == null ? null : action );

}

Now that one works. The lesson here is to use the String version of TypeElement when searching as two TypeElement’s representing the same class are not always equal.

Solution 2 – Single Class values

Now if your value contains just one Class (i.e. not Class[] ) then there’s a much simpler solution. This one isn’t that obvious, but I found that someone had a similar problem in the sun forums[1]. There the trick is to actually use getAnnotation() and catch the MirroredTypeException. Surprisingly the exception then provides the TypeMirror of the required class:

for( ExecutableElement ee : ElementFilter.methodsIn(

        te.getEnclosedElements() ) )

{

    Action action = ee.getAnnotation( Action.class );

    if( action == null )

    {

        // Look for the overridden method

        ExecutableElement oe = getExecutableElement( te,

                                                     ee.getSimpleName() );

        if( oe != null )

        {

            action = oe.getAnnotation( Action.class );

        }

    }

    TypeMirror value = null;

    if( action != null )

    {

        try

        {

            action.value();

        }

        catch( MirroredTypeException mte )

        {

            value = mte.getTypeMirror();

        }

    }

    System.out.printf(

            “%s Action value = %s\n”,

            ee.getSimpleName(),

            value );

}

Notice getTypeMirror() method call? Here’s the output of the above loop:

[compiler:compile]

Compiling 2 source files to /Users/peter/dev/retep/scratch/target/classes

Scanning Type scratch.A

run Action value = java.lang.Runnable

Scanning Type scratch.B

run Action value = java.lang.Runnable

This trick works fine for individual classes, but sadly it does not work for Class[] arrays. According to the javadocs it should work for Class[] as it should then throw the MirroredTypesException instead and that exception contains an alternate method that returns a List of TypeMirrors.

However it doesn’t – it simply throws MirroredTypeException for the first element in the array. I think it’s where it’s running through the array to populate it and the first element is then throwing MirroredTypeException before it gets chance to throw MirroredTypesException.

Conclusion

Well, hopefully this article will save someone hours of trouble when they hit the same problem. I’ve spent about 20 hours of time searching the net and dabbling for these solutions – and at least these solutions use just the documented public APIs.

References

  1. http://forums.sun.com/thread.jspa?threadID=791053