Implementing Builders with JAXB generated objects

While JAXB is a good method of mapping XML into Java objects, if your schema’s are large or complex, generating the required Object graph prior to marshaling into XML can be both tedious and error prone due to the large amount of ‘boiler plate’ code of creating an object, calling accessor methods to set the values etc.

A common pattern for easing this sort of thing is known as the Builder pattern where you either have a common object holding configuration and then it builds the objects for you, or a set of objects each configured with the information and when required then build the final Object graph.

Previously however, this involved writing these builder classes manually once you have generated the model from a schema, however retepTools provides a JAXB plugin which, with a little configuration within the bindings will automatically generate these builders for you.

First an example of a simple pair of objects to be generated by JAXB – this is an abridged version of the jabber:client namespace in XMPP:

<?xml version='1.0' encoding='UTF-8'?>

<xs:schema
    xmlns:xs='http://www.w3.org/2001/XMLSchema'
    targetNamespace='jabber:client'
    xmlns='jabber:client'
    elementFormDefault='qualified'>

  <xs:element name='message'>
     <xs:complexType>
        <xs:sequence>
          <xs:choice minOccurs='0' maxOccurs='unbounded'>
            <xs:element ref='subject'/>
            <xs:element ref='body'/>
          </xs:choice>
        </xs:sequence>
        <xs:attribute name='from' type='xs:string' use='optional'/>
        <xs:attribute name='from' type='xs:string' use='optional'/>
     </xs:complexType>
  </xs:element>

  <xs:element name='body'>
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base='nonEmptyString'>
          <xs:attribute ref='xml:lang' use='optional'/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>

  <xs:element name='subject'>
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base='nonEmptyString'>
          <xs:attribute ref='xml:lang' use='optional'/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>

  <xs:simpleType name='nonEmptyString'>
    <xs:restriction base='xs:string'>
      <xs:minLength value='1'/>
    </xs:restriction>
  </xs:simpleType>

</xs:schema>

Now if run through JAXB this would generate three classes: Message, Body and Subject. With this default model you would have to do something like the following to create a valid Message:

Message message = new Message();
message.setFrom( "juliet@example.com/balcony" );
message.setTo( "romeo@example.net" );

Body body = new Body();
body.getBodyOrSubject().add( "Wherefore art thou, Romeo?" );
message.setBody( body );

Now with this simple example thats fine, but imagine something where your model is far more complex, or worse can contain arbitary namespaces as children such as in XMPP where Message can contain objects within any namespace?

This is where Builders become useful. Imagine the above being converted into this:

Message message = new MessageBuilder().
	setFrom( "juliet@example.com/balcony" ).
	setTo( "romeo@example.net" ).
	addBody( new BodyBuilder().
		 setValue( "Wherefore art thou, Romeo?") ).
	build();

Although at first this may look similar to the standard way, the builders break the code into more manageable chunks. The builders themselves can be reused, so if you are sending multiple messages then you could change the above code to:

MessageBuilder builder = new MessageBuilder().
	setFrom( "juliet@example.com/balcony" ).
	addBody( new BodyBuilder().
		 setValue( "Wherefore art thou, Romeo?") );

Message message1 = builder.setTo( "romeo@example.net" ).build();

Message message2 = builder.setTo( "tybalt@example.com/cousin" ).build();

As you can see, the use of builders becomes apparent, we created two Message objects with a single change between them in a lot less code.

On the next page we’ll configure maven to build these builders.

It’s been a busy couple of months

It’s been a couple of busy months with most of my time being taken up with my day job.

Most of my time has been spent with either tracking down issues with our live environment, or trying to finish off a couple major projects (both related to XMPP) interspersed with the usual major partner getting in the way.

Any how, over the last couple of weeks I’ve been finishing off some new features which cover most of my public projects and this post will hopefully cover some of the details.

Hopefully these will be released this weekend, time allowing.

The new features are:

RetepTools
* the builder api within retepTools has been updated
* the jaxb plugin library has been cleaned up with common generation code split out to enable reuse
* retepTools as a project is almost ready for deployment to maven central
* a new pligun has been added to jaxb which generates builders for jaxb objects

RetepMicroKernel
* spring has been updated to the latest version 3 (it was on 2.5)
* the core module has been broken up into individual & independent modules
* a new groovy module which enables groovy scripts to be run from the command line
* a major bug fix where exceptions thrown during application startup causes the process to hang has been fixed
* web applications can now be deployed as a war with either jetty or tomcat (they are both supported with their own modules)
* you can now embed Apache Derby within the environment

I’m leaving out the retepXMPP changes out of this list as they need their own article. Suffice it to say, I’ve got a lot waiting for release, just need the time.

Finally, this post is also a test of submitting a blog post from a BlackBerry using the WordPress app so the formatting may be off a tad – won’t know how it goes until I see it in a real browser.

Using JAXB to marshall XMPP

Back in late 2008, one of the original design goals for the rewrite of my retepXMPP project was to use JAXB for handling the marshalling of XMPP Stanzas into POJO’s. The main ideas behind this was to standardise against the XMPP schemas available online and to make the addition of further protocols easier – mainly not to break existing code.

Now this was fine until I started testing against a couple of client libraries, certain messages were being ignored. It turned out that the XML generated by JAXB does not follow the rules defined in RFC 3920bis. The problem here was that JAXB was placing all of the namespaces together in the root.

For example here’s one of the examples from RFC3921bis as generated by JAXB:

  <iq from='juliet@example.com/balcony'
       id='rg1'
       type='get'  xmlns='jabber:client'  xmlns:ns1='jabber:iq:roster'>
    <ns1:query/>
  </iq>

Here’s what it should look like:

  <iq from='juliet@example.com/balcony'
       id='rg1'
       type='get'>
    <query xmlns='jabber:iq:roster'/>
  </iq>

There’s two differences here, but the main issue is where ‘jabber:iq:roster’ is declared in the root iq element and not against the query element it’s supposed to be – the other is the declaration of the ‘jabber:client’ namespace but that one is not a problem here.

Although both are strictly correct XML wise, it isn’t for XMPP and most parsers expect the stream to follow the rules.

So why is JAXB doing this? Well apparently its by design and, if you use the JAXB-RI like I do, there’s not much we can do to change this.

In JAXB 1 it actually declared the namespaces at the appropriate places so we would get the ‘jabber:iq:roster’ namespace against the query element as expected and the XMPP stanza would then conform correctly.

However in JAXB 2.0 they changed this to the current behaviour for performance reasons. Apparently with large documents looking ahead for what namespaces are present uses a lot of resources, so they declare them first.

Now I can see their point but for one small issue – this presumes that, when a document is being parsed then unmarshalled by JAXB it was originally produced by JAXB – what if it wasn’t generated by JAXB? In that case JAXB would still do the lookup anyhow.

Unfortunately when I took a look at the source for JAXB 2.2 this declaring of namespaces is done pretty close to the start of the marshalling process, so unless they add an option to disable it we have to live with this.

So how to fix this so we can still use JAXB but generate conforming XML?

Well I’ve got a solution but it’s not pretty. The solution simply involves breaking the marshalling/unmarshalling process up into separate units of work, one per namespace. We marshal the first object, then – if it has children, marshal them.

Now this involves additional work during the marshalling/unmarshalling process but it works. The downside is that we have to modify the schemas to do this.

Fortunately the XSF’s schemas are not concrete. As Peter Saint-Andre’s put it a few weeks ago on the muc mailing list, those schemas are ‘descriptive, not normative’. This means they are representative of what the XEP’s define, but they can be changed. In fact for some (like XEP-0045/MUC) the schemas don’t define the extension points used by other XEP’s. This means we can make some simple modifications to them to get this ‘hack’ to work.

So now I have this working pretty well – marshalling to xml runs smoothly. Unmarshalling still has a few problems but I’m being hit by a nice issue with Java’s generics but that’s going to be a later post.

Binding custom objects to attributes in JAXB

When generating Java sources from an XML Schema with JAXB, the type of field used to represent an attribute is determined by it’s type in the schema.

For example, a snippet from jabber-client.xsd:

<xs:element name="message">
  <xs:complextype>
    <xs:attribute name="from" type="xs:string" use="optional">
      <xs:attribute name="to" type="xs:string" use="optional">
    </xs:complextype>
  </xs:sequence>
</xs:element>

By default, JAXB would create the object (Message in this case) with two String properties. To change this to another object (say JID) we would add a simple binding changing these to use JID and define two static methods who’s job it is to convert between a String and a JID.

So the bindings would be something like:

<bindings schemalocation="jabber-client.xsd">
  <bindings node="/xsd:schema/xsd:element[@name='message']/xsd:complexType">
    <bindings node="xsd:attribute[@name='from']">
      <property>
        <baseType>
          <javaType name="uk.org.retep.xmpp.JID"
            parseMethod="uk.org.retep.xmpp.jaxb.adaptor.XMPPDatatypeConverter.parseJID"
            printMethod="uk.org.retep.xmpp.jaxb.adaptor.XMPPDatatypeConverter.printJID"
          />
        </baseType>
      </property>
    </bindings>
  </bindings>
</bindings>

Now for mose usecases this is fine and works pretty well. The problem is when you have a large number of custom bindings.

What JAXB does for each binding is that it generates an anonymous Adapter class. In that adaper class are two methods which call the two methods declared in the binding – in the case of binding to a Double you would get the following:

public class Adapter1
    extends XmlAdapter<String, Double>
{

    public Double unmarshal(String value) {
        return (javax.xml.bind.DatatypeConverter.parseDouble(value));
    }

    public String marshal(Double value) {
        if (value == null) {
            return null;
        }
        return (javax.xml.bind.DatatypeConverter.printDouble(value));
    }

}

The problem here is that JAXB generates one of these for every instance, so you can end up having multiple Adapter classes present, all doing the same thing. In one instance I had a single schema contain 16 duplicates of the above code. In retepXMPP I had almost 60 spread over multiple schemas just for handling JID. This is a lot of wasted, duplicated code and if you use a large number of schemas then this is going to cause both your jar sizes and permgen use to increase for no real reason.

So it would be nice to reduce this down to a single adapter.

Fortunately there is a way when using the JAXB RI. There is an alternate javaType binding which takes the full class name of an adapter class instead – using this causes JAXB to use that single class instead of generating these duplicate classes. The main difference is replacing the parseMethod and printMethod attributes with a single adapter attribute, and qualifying the javaType with the http://java.sun.com/xml/ns/jaxb/xjc namespace.

So the bindings would be something like:

<bindings
  xmlns="http://java.sun.com/xml/ns/jaxb"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc"
  xmlns:retep="http://retep.org/xml/ns/retepTools"
  version="2.0">
  <bindings schemalocation="jabber-client.xsd">
    <bindings node="/xsd:schema/xsd:element[@name='message']/xsd:complexType">
      <bindings node="xsd:attribute[@name='from']">
        <property>
          <baseType>
            <xjc:javaType adapter="uk.org.retep.xmpp.jaxb.adaptor.JIDAdapter"
                          name="uk.org.retep.xmpp.JID" />
          </baseType>
        </property>
      </bindings>
    </bindings>
  </bindings>
</bindings>

Then just create a single adapter implementation. in retepXMPP thats some 60 classes reduced to one, if you generate any javadocs based on the generated sources there’s no more “Adapter” spam – large numbers of apparently duplicated classes in the javadocs etc.

Tip: Just make sure you keep an eye on that namespace – miss it out and XJC will spurt out some unusual error messages.

Update: I’ve disabled comments on this article simply because spammers are the only ones posting comments to it. Peter 27 Feb 2010

Implementing custom types within JAXB

With JAXB it does a pretty good job in mapping between XML and Java Objects. In normal use this is fine but what happens when you need to use a custom data type for an attribute? Well JAXB supports this in binding so this article will describe this.

The originating issue that’s caused this article to be written cropped up at work earlier today where we had an issue with currency values. Over time the various programmers had naturally used a Java double to represent them but problems started to appear when some testers started to use amounts equivalent to €1.2 Billion and things started to fall apart.

Now obviously it’s bad practice to use double for monetary values (we’ve all done it at some point) due to the fact that specific values cannot be represented accurately (eg 39.995 is represented as 39.9949989[1]), it gets worse when dealing with large values. With large amounts you start to loose precision at the milli and penny levels, so adding 1¢ to the amount will fail because the double cannot handle it because of rounding errors.

So the solution is to use BigDecimal as it has arbitrary precision. With JAXB this is not a problem as it supports BigDecimal when defining an attribute with type xsd:decimal – as long as you own the xsd then it’s simply a case of changing it from xsd:double. The problem is when you don’t own the xsd, i.e. it’s an external specification. In this case you cannot change it as you would break other implementations. This is exactly where JAXB bindings[2] come into play.

Changing the object used in an attribute

In this first example taken from my xmpp project we have a series of xsd files and a single xjb file containing the bindings.

From client.xsd which defines the “jabber:client” namespace[3]:

<xs:element name=’message’>

     <xs:complexType>

        <xs:sequence>

          <xs:choice minOccurs=’0′ maxOccurs=’unbounded’>

            <xs:element ref=’subject’/>

            <xs:element ref=’body’/>

            <xs:element ref=’thread’/>

          </xs:choice>

          <xs:any namespace=’##other’ minOccurs=’0′ maxOccurs=’unbounded’/>

          <xs:element ref=’error’ minOccurs=’0’/>

        </xs:sequence>

        <xs:attribute name=’from’ type=’xs:string’ use=’optional’/>

        <xs:attribute name=’id’ type=’xs:NMTOKEN’ use=’optional’/>

        <xs:attribute name=’to’ type=’xs:string’ use=’optional’/>

       <xs:attribute ref=’xml:lang’ use=’optional’/>

     </xs:complexType>

  </xs:element>

Now here the from and to attributes represent not just a string but a Jabber IDentifier JID which within the project is represented by the class uk.org.retep.xmpp.JID. This class ensures it’s both syntactically correct and provides access to the various components within the JID.

For JAXB to use this JID class instead of String we can either add the bindings into the original xsd or into a separate xjb file. I prefer the latter as it keeps the originals untouched so that if someone updates them then the bindings are not lost:

So to add the bindings for JID into the above message element we have the following bindings.xjb file:

<?xml version=’1.0′ encoding=’UTF-8′?>

<bindings

    xmlns=”http://java.sun.com/xml/ns/jaxb”

    xmlns:xsd=”http://www.w3.org/2001/XMLSchema”

    xmlns:retep=”http://retep.org/xml/ns/retepTools”

    version=”2.0”>

   <bindings schemaLocation=”client.xsd” node=”/xsd:schema/xsd:element[@name=’message’]/xsd:complexType/xsd:attribute[@name=’from’]”>

        <property>

            <baseType>

                <javaType name=”uk.org.retep.xmpp.JID” parseMethod=”uk.org.retep.xmpp.JID.parse” printMethod=”uk.org.retep.xmpp.JID.print” />

            </baseType>

        </property>

    </bindings>

    <bindings schemaLocation=”client.xsd” node=”/xsd:schema/xsd:element[@name=’message’]/xsd:complexType/xsd:attribute[@name=’to’]”>

        <property>

            <baseType>

                <javaType name=”uk.org.retep.xmpp.JID” parseMethod=”uk.org.retep.xmpp.JID.parse” printMethod=”uk.org.retep.xmpp.JID.print” />

            </baseType>

        </property>

    </bindings>

</bindings>

Here we define the attributes we want to change using an XPath expression and then the object we want to bind to that attribute. We also provide two methods to parse the attribute from a String to the object and to print it (from the object to a String). In this case those two methods are provided by the JID class itself but they don’t have to be.

Implementing the parse and print methods

The main things to watch out with the parse and print methods is that they have to be public static and they must handle null values. If the attribute is not present or should be a default value then they will be passed nulls:

From JID.java (abridged):

    public static JID parse( final String jid )

    {

        return jid == null ? null : new JID( jid );

    }

    public static String print( final JID jid )

    {

        return jid == null ? null : jid.toString();

    }

Back to the currency issue

Now with the currency issue which initially started off this rambling, there’s one more complication. In the specific DB the currency values are not actually stored as floats but in a custom SQL Type which I’ll call Money to protect the innocent. On the Java side there’s a similar object which uses a BigDecimal for the storage but also handles things like null’s etc. Anyhow in this case all that is required is to add a parse and print method into that class and add the equivalent bindings to the xjb file.

References

  1. Round and Round on the Daily WTF 
  2. Customising JAXB Bindings on sun.com
  3. jabber-client.xsd on xmpp.org