Broadcasting Ehcache CacheEvent’s across a Terracotta cluster

I’ve been trying to get CacheEvent’s working with ehcache – specifically across a terracotta cluster so that if an element is added, removed or updated then every node in the cluster is notified.

Now going by the api, this should be a simple case of creating a CacheEventListener and attaching it to the individual Ehcache. It’s not that simple. Doing that only means that events occuring on the local instance get the events – the other nodes don’t get them.

Now it’s not easy to find but the solution is to actually add a specific CacheEventListenerFactory to the cache.

I.e.

    <cache  name="myCache"
            maxElementsInMemory="100000"
            overflowToDisk="false"
            eternal="false"
            timeToIdleSeconds="1800"
            timeToLiveSeconds="3600"
            memoryStoreEvictionPolicy="LFU">
        <searchable>
            <searchAttribute name="online" expression="value.isOnline()" />
            <searchAttribute name="name" expression="value.getName()" />
        </searchable>
        <terracotta/>
        <cacheEventListenerFactory class="net.sf.ehcache.event.TerracottaCacheEventReplicationFactory"/>
    </cache>

Now in code simply register your listener with:

myCache.getCacheEventNotificationService().registerListener( myListener );

and you’ll find it will work!

Why is ehcache on Terracotta using Hibernate when it’s not meant to be?

I have this weird and interesting problem with hibernate interfering with ehcache on Terracotta.

I have 2 clients, one jboss the other a standalone vm. Both share a common ehcache instance on Terracotta.

Now this works, depending on which client starts first after terracotta.

If the standalone vm starts first then everything is fine.

If jboss starts first then when the standalone vm tries to use the cache it fails because the cache has configured itself to use HibernateElementSerializationStrategy for the keys. As this has a classpath dependency on org.hibernate.cache.CacheKey and that is not in the standalone vm the cache blows up.

So, is there a way to configure ehcache to use a specific strategy and not hibernate? For me hibernate is legacy so ehcache should have no need to refer to it.

Performing remote Maven 3 builds with NetBeans 7.0

This article covers how to get NetBeans to use a Maven installation on a remote server. This has been tested with Maven 3.0.2 but should work on any other version. It may also work for other IDE’s like Eclipse if you can get them to use an alternate maven installation other than the internal one.

Sometimes you may want to run maven on a remote machine from inside the NetBeans IDE. For me this is down to available memory on the local machine is limited enough on large projects that both NetBeans and Maven compete slowing the local machine down as it swaps the IDE in and out of main memory.

Another instance may be to perform builds on a standard OS (Linux in this case) rather than the OS running the IDE (OSX here).

Now this is pretty easy to implement. Here I have the IDE running on OSX and the builds on Linux (Ubuntu 10.10) but if you can get the prerequisites running in theory any OS should work.

Prerequisites

  • ssh access to both machines with public key authentication working
  • sshfs configured for mounting drives (you can use nfs or smb if you wish)
  • maven 3.0.2 installed on both machines – both machines need the same version

Setting up the environment

In this example:

  • sabrina – The local machine running NetBeans. It’s running OS X
  • kira – The remote machine which will run the builds. It’s running Ubuntu 10.10

Shared filesystems

The remote needs to have access to the local project directories for maven to work. As I have all development under a dev directory under home, on the remote machine I simply create a dev directory and then mount the local host from it using sshfs

<br />
peter@kira:~$ mkdir dev<br />
peter@kira:~$ sshfs peter@sabrina:dev dev<br />

If you have a custom settings.xml file, make sure it’s also present on the remote’s .m2 directory, i.e.

<br />
sabrina:~ peter$ scp .m2/settings.xml peter@kira:.m2<br />

Finally NetBeans needs access to the built artifacts, so to do this we mount the remotes .m2/repository locally. I use .m2/remoterepo for this

<br />
sabrina:~ peter$ mkdir .m2/remoterepo<br />
sabrina:~ peter$ sshfs peter@kira:.m2/repository .m2/remoterepo<br />

Maven

Next install your maven installation on both machines. I keep mine in the same place, but thats optional. For the purposes of this article I’m going to use /usr/local/apache/apache-maven-3.0.2

Next on the machine running NetBeans you need to create a duplicate installation /usr/local/apache/apache-maven-3.0.2-remote We need to have a copy for NetBeans to recognise it as a valid environment later on.

Now in this new installation we need to replace the /usr/local/apache/apache-maven-3.0.2-remote/bin/mvn script with the following:

<br />
#!/bin/bash<br />
#<br />
# Enable NetBeans 7.0 beta to run maven on a remote host</p>
<p># Pass the maven args as is<br />
MVN_ARGS=&quot;$@&quot;</p>
<p># The current directory, NetBeans does a cd so this will be the project<br />
# directory.<br />
#<br />
# If you need to translate the path between local and remote machines then<br />
# pass the output of pwd through sed first<br />
REMOTE_PROJECT_DIR=$(pwd)</p>
<p># Execute maven on remote host<br />
REMOTE_USER=peter<br />
REMOTE_HOST=kira</p>
<p># Location of remote maven installation<br />
REMOTE_MAVEN_HOME=/usr/local/apache/apache-maven-3.0.2</p>
<p># JAVA_HOME on remote machine<br />
REMOTE_JAVA_HOME=/usr/lib/jvm/java-6-openjdk/</p>
<p># ======================================<br />
# DO NOT EDIT ANYTHING BEYOND THIS POINT<br />
# ======================================</p>
<p># Remote mvn command<br />
REMOTE_MVN=$REMOTE_MAVEN_HOME/bin/mvn</p>
<p># Build the commands to run remotely<br />
CMD=&quot;cd ${REMOTE_PROJECT_DIR};&quot;<br />
CMD=&quot;${CMD}export JAVA_HOME=${REMOTE_JAVA_HOME};&quot;<br />
CMD=&quot;${CMD}$REMOTE_MVN ${MVN_ARGS}&quot;</p>
<p># Now run the remote build<br />
echo<br />
echo ------------------------------------------------------------------------<br />
echo Executing remote build on $REMOTE_HOST as $REMOTE_USER<br />
echo Remote command: $CMD<br />
echo ------------------------------------------------------------------------<br />
echo<br />
exec ssh ${REMOTE_USER}@${REMOTE_HOST} &quot;${CMD}&quot;<br />

In this script you need to ensure that the settings are correct:

  • REMOTE_USER and REMOTE_HOST point to your remote host,
  • REMOTE_MAVEN_HOME is the location of where maven is installed on the remote host,
  • REMOTE_JAVA_HOME is the location of the JDK on the remote host.

Now you may need to alter the line defining REMOTE_PROJECT_DIR. Shown above it gets set to the directory containing the maven project on the local host. If this differs on the remote then you need to either change to translate this to the location on the remote.

For example I have my NetBeans projects under a directory called dev in my home directory. As this is OS X that becomes /Users/peter/dev however on Linux the path is /home/peter/dev

I get around this by on the remote simply creating a symlink for /Users pointing to /home but you could modify the script to do that for you.

Configuring NetBeans

The last step is getting NetBeans to use the new remote.

Start NetBeans and enter preferences Then the Miscellaneous tab and finally Maven:

  • Under Maven Home select Browse and select the dummy remote installation, in my case /usr/local/apache/apache-maven-3.0.2-remote
  • Under Local Repository select Browse and point it at .m2/remoterepo
  • Select OK

Now if you try to run any build you’ll now see it run on the remote machine and as far as NetBeans is concerned it’s running locally. For example in the NetBeans output window you should see something like the following:

<br />
cd /Users/peter/dev/misc/example; JAVA_HOME=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home /usr/local/apache/apache-maven-3.0.2-remote/bin/mvn --batch-mode --lax-checksums --fail-fast --no-plugin-updates --batch-mode clean</p>
<p>------------------------------------------------------------------------<br />
Executing remote build on 192.168.30.108 as peter<br />
Remote command: cd /Users/peter/dev/misc/example;export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/;/usr/local/apache/apache-maven-3.0.2/bin/mvn --batch-mode --lax-checksums --fail-fast --no-plugin-updates --batch-mode clean<br />
------------------------------------------------------------------------</p>
<p>Command line option -npu is deprecated and will be removed in future Maven versions.<br />
Scanning for projects...<br />
------------------------------------------------------------------------<br />
Reactor Build Order:<br />

Future invocations

Once setup, whenever you restart either machines you simply need to mount each others drives before starting NetBeans as all the rest should then be setup.

<br />
sabrina:~ peter$ sshfs peter@kira:.m2/repository .m2/remoterepo<br />
peter@kira:~$ sshfs peter@sabrina:dev dev<br />

Compiling Java 7.0 on OS X

Since the announcement by Apple last week about deprecating Java on OS X, there’s been a few people wanting to know how to compile OpenJDK on the Mac.

Although I’ve not done this for JDK 6, this article cover’s how to compile and use the current development version of JDK 7.0 on OS X.

First a few notes:

  1. This only enables Java 7 within an X environment, native UI’s are not supported – one of the main parts of the Apple JVM
  2. When I tested this by running Netbeans 7.0 M2 within X the menus were a bit screwey – try it you’ll see what I mean
  3. These instructions are for 10.5.8 but should work for 10.6.x
  4. This is for Intel processors only

So as a word of warning: Don’t expect this to either work, or work well – and don’t use this in production – JDK7 isn’t due out for another 12 months or so…

Ok, first we need a bootstrap JDK6 environment. This is needed to do some of the initial java compilation during the build. For this the apple JVM can’t be used so we need to download and install the i386 Soylate binaries JDK – don’t get the amd64 version, get the i386 one…

Once you have it downloaded, copy it to /usr/local/soylatte16-i386-1.0.3 and test it:

sabrina:~ peter$ /usr/local/soylatte16-i386-1.0.3/bin/java -version
java version "1.6.0_03-p3"
Java(TM) SE Runtime Environment (build 1.6.0_03-p3-landonf_19_aug_2008_14_55-b00)
Java HotSpot(TM) Server VM (build 1.6.0_03-p3-landonf_19_aug_2008_14_55-b00, mixed mode)

Next create a blank directory under which we will build everything. I’m using ~/dev/ojdk but you could use any directory. Under this we need to create a couple of directories and some symlinks:

sabrina:~ peter$ mkdir -p dev/ojdk
sabrina:~ peter$ cd dev/ojdk
sabrina:~ peter$ mkdir -p bin ALT_COMPILER_PATH
sabrina:~ peter$ cd ALT_COMPILER_PATH
sabrina:~ peter$ ln -s /usr/bin .SOURCE
sabrina:~ peter$ ln -s .SOURCE/g++-4.0 g++
sabrina:~ peter$ ln -s .SOURCE/gcc-4.0 gcc
sabrina:~ peter$ cd ../bin

Now in the bin directory you need to create two scripts. Fortunately these are readily available from http://gist.github.com/617451 – specifically update.sh and update-usr-local.sh. Copy these two files into the bin directory and ensure they are executable.

Now open update.sh in your favourite editor and find the line with ALT_COMPILER_PATH in it. Change it to hold the full path to the ALT_COMPILER_PATH directory defined above. In my case this looks like:

ALT_COMPILER_PATH=/Users/peter/dev/ojdk/ALT_COMPILER_PATH/ \

Next we need to checkout a copy of the source:

sabrina:~ peter$ cd ~/dev/ojdk
sabrina:~ peter$ hg fclone http://hg.openjdk.java.net/bsd-port/bsd-port bsd

We should now be setup. The last step is to run a build. This can be done at any time. It will check for any updates, clear down and then run a full build:

sabrina:~ peter$ cd ~/dev/ojdk/bsd
sabrina:~ peter$ source ../bin/update.sh

If all goes well, after about 20 minutes you should see something like the following at the end of the build:

testing build: ./build/bsd-amd64/j2sdk-image/bin/java -version

openjdk version "1.7.0-internal"
OpenJDK Runtime Environment (build 1.7.0-internal-peter_2010_10_25_11_19-b00)
OpenJDK 64-Bit Server VM (build 19.0-b05, mixed mode)

If the build succedes the last step is to install it under /usr/local/java-1.7.0:

sabrina:~ peter$ cd ~/dev/ojdk/bsd
sabrina:~ peter$ source ../bin/update-usr-local.sh

To use Java 7, you need to simply point the app to the installed application, usually by setting JAVA_HOME=/usr/local/java-1.7.0 and running $JAVA_HOME/bin/java…

sabrina:bin peter$ /usr/local/java-1.7.0/bin/java -version
openjdk version "1.7.0-internal"
OpenJDK Runtime Environment (build 1.7.0-internal-peter_2010_10_25_11_19-b00)
OpenJDK 64-Bit Server VM (build 19.0-b05, mixed mode)

NioSax – Sax style xml parser for Java NIO

NioSax (pronounced ‘Neo-Sax’) provides a Java NIO friendly XML push parser similar in operation to SAX. Unlike SAX, with NioSax it is possible for the xml source to contain partial content (i.e. only part of the XML stream has been received over the network). When this occurs, instead of failing with an error, NioSax simply stops. As soon as your application receives more data you simply call the same instance of the parser again and it will resume parsing where it left off.

NioSax (pronounced ‘Neo-Sax’) provides a Java NIO friendly XML push parser similar in operation to SAX. Unlike SAX, with NioSax it is possible for the xml source to contain partial content (i.e. only part of the XML stream has been received over the network). When this occurs, instead of failing with an error, NioSax simply stops. As soon as your application receives more data you simply call the same instance of the parser again and it will resume parsing where it left off.

The public API consists of the classes within this package, although the bare minimum required for use are the NioSaxParser, NioSaxParserHandler and NioSaxSource classes.

To use NioSax you simply use NioSaxParserFactory to create a NioSaxParser, implement a SAX ContentHandler and finally create a NioSaxSource which references the content.

Then you can parse one or more ByteBuffer’s by updating the NioSaxSource with each buffer and pass it to the NioSaxParser.parse(NioSaxSource) method.

The only other two things you must to do with the parser is to ensure that you call NioSaxParser.startDocument() prior to any parsing, and call NioSaxParser.endDocument() once you are done with the parser so any resources used can be cleaned up.

Example

First in maven we need to add a dependency to NioSax. For details of the repository click on the ‘reteptools’ menu above. However you’ll need to add the following to your pom:

<dependency>
    <groupId>uk.org.retep</groupId>
    <artifactId>niosax</artifactId>
    <version>10.6</version>
</dependency>

Now we’ll create a parser:

import java.nio.ByteBuffer;
import uk.org.retep.niosax.NioSaxParser;
import uk.org.retep.niosax.NioSaxParserFactory;
import uk.org.retep.niosax.NioSaxParserHandler;
import uk.org.retep.niosax.NioSaxSource;

public class MyParser
{
    private NioSaxParser parser;
    private NioSaxParserHandler handler;
    private NioSaxSource source;

    public void start()
    {
        NioSaxParserFactory factory = NioSaxParserFactory.getInstance();

        parser = factory.newInstance();
        parser.setHandler( handler );
        source = new NioSaxSource();

        parser.startDocument();
    }
}

Next, when you receive data from some nio source and have the data in a ByteBuffer you need to pass it to the parser:

    public void parse( ByteBuffer buffer )
    {
        // flip the buffer so the parser starts at the beginning
        buffer.flip();

        // update the source (presuming the buffer has changed)
        source.setByteBuffer( buffer );

        // Parse the available content then compact
        parser.parse( source );
        source.compact();
    }

Finally we must call endDocument() to release any resources:

    public void close()
    {
        // releases any resources and notifies the handler the docment has completed
        parser.endDocument();
    }

Now all we need to is when we receive some data from an external source like a Socket, we pass the ByteBuffer to the parse method. This then passes it to the NioSax parser which in turn calls the ContentHandler as the parse progresses.

When it gets to the end of the available content, it compacts the buffer so that it can be reused.

Usually the buffer will now be empty, however if there was partial content (like only part of a Unicode character was present) then the parser would stop prior to that character and that character would remain in the buffer. The next packet received via nio would have the rest of that character and the parser would then continue where it left off.

This was originally posted early in 2009 but the post seemed to have vanished so this article is loosely based on the documentation for NioSax.

Getting NetBeans 6.9 RC1 to insert the correct copyright text in maven projects

Netbeans has supported for quite some time the ability to set the license for a project so that when a new file is created, the template automatically inserts the correct one. The problem is that in the past this has not been that well supported when it comes to maven based projects.

Now in some of the recent releases this now works, with a little bit of configuration – I’ve tested this only with 6.9 RC1, but this should (in theory) work with 6.8 and possibly 6.5.

First you need to choose your license. Open the Tools menu and select Templates. This should then open the Template Manager. Once open, expand the Licenses entry to show the available licenses.

Netbeans 6.9 RC1 Template Manager

Now the ones that can be used in maven are the ones named: license-*.txt. If the one you want is there, then make a note if it’s name.

Creating a new License Template

If the license you want is not there then you’ll need to create a template for it. For example the above screenshot shows bsd and gpl3cp which are the two I use for my Open Source projects but they are not provided by Netbeans so I had to create them.

For the BSD template, create the following file in the root directory of your project (so others can reuse the same license), in this case license-bsd.txt:

<#if licenseFirst??>
${licenseFirst}
</#if>
${licensePrefix} Copyright (c) 1998-${date?date?string("yyyy")}, Peter T Mount
${licensePrefix} All rights reserved.
${licensePrefix}
${licensePrefix} Redistribution and use in source and binary forms, with or without
${licensePrefix} modification, are permitted provided that the following conditions
${licensePrefix} are met:
${licensePrefix}
${licensePrefix} * Redistributions of source code must retain the above copyright
${licensePrefix}   notice, this list of conditions and the following disclaimer.
${licensePrefix}
${licensePrefix} * Redistributions in binary form must reproduce the above copyright
${licensePrefix}   notice, this list of conditions and the following disclaimer in the
${licensePrefix}   documentation and/or other materials provided with the distribution.
${licensePrefix}
${licensePrefix} * Neither the name of the retep.org.uk nor the names of its contributors
${licensePrefix}   may be used to endorse or promote products derived from this software
${licensePrefix}   without specific prior written permission.
${licensePrefix}
${licensePrefix} THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
${licensePrefix} "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
${licensePrefix} LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
${licensePrefix} A PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER
${licensePrefix} OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
${licensePrefix} EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
${licensePrefix} PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
${licensePrefix} PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
${licensePrefix} LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
${licensePrefix} NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
${licensePrefix} SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
<#if licenseLast??>
${licenseLast}
</#if>

Next select the Licenses heading and press Add. Select your new file then add. The template’s now been added. Later if you need to tweek it you can now select it and press Open in editor.

Enabling the template for your Maven project

The next step is to tell Netbeans to use this template. To do this we need to add another file, profiles.xml, which goes into the same directory as your project’s root pom.

<profilesXml>
    <profiles>
        <profile>
            <id>netbeans-private</id>
            <activation>
                <property>
                    <name>netbeans.execution</name>
                    <value>true</value>
                </property>
            </activation>
            <properties>
                <netbeans.hint.license>bsd</netbeans.hint.license>
            </properties>
        </profile>
    </profiles>
</profilesXml>

You’ll notice the netbeans.hint.license property. It’s value is the middle part of your template’s name.

Once set, reload the project within netbeans and every time you create a new file who’s template includes the license it will include the correct one for the project.

Netbeans Template Tips

When creating your own templates, you can include the license at any time using the following snippet:

<#assign licenseFirst = "/*">
<#assign licensePrefix = " * ">
<#assign licenseLast = " */">
<#include "../Licenses/license-${project.license}.txt">

Here you’ll notice we have to assign three values then include the license – the above example is for .java files but you can change it to any values for any type of file.

The other useful tip is the following line from the license template above:

${licensePrefix} Copyright (c) 1998-${date?date?string("yyyy")}, Peter T Mount

Here the template will always insert the current year into the file, rather than having to remember to change the templates every year – although you’ll still have to amend existing files.

Implementing Builders with JAXB generated objects

While JAXB is a good method of mapping XML into Java objects, if your schema’s are large or complex, generating the required Object graph prior to marshaling into XML can be both tedious and error prone due to the large amount of ‘boiler plate’ code of creating an object, calling accessor methods to set the values etc.

A common pattern for easing this sort of thing is known as the Builder pattern where you either have a common object holding configuration and then it builds the objects for you, or a set of objects each configured with the information and when required then build the final Object graph.

Previously however, this involved writing these builder classes manually once you have generated the model from a schema, however retepTools provides a JAXB plugin which, with a little configuration within the bindings will automatically generate these builders for you.

First an example of a simple pair of objects to be generated by JAXB – this is an abridged version of the jabber:client namespace in XMPP:

<?xml version='1.0' encoding='UTF-8'?>

<xs:schema
    xmlns:xs='http://www.w3.org/2001/XMLSchema'
    targetNamespace='jabber:client'
    xmlns='jabber:client'
    elementFormDefault='qualified'>

  <xs:element name='message'>
     <xs:complexType>
        <xs:sequence>
          <xs:choice minOccurs='0' maxOccurs='unbounded'>
            <xs:element ref='subject'/>
            <xs:element ref='body'/>
          </xs:choice>
        </xs:sequence>
        <xs:attribute name='from' type='xs:string' use='optional'/>
        <xs:attribute name='from' type='xs:string' use='optional'/>
     </xs:complexType>
  </xs:element>

  <xs:element name='body'>
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base='nonEmptyString'>
          <xs:attribute ref='xml:lang' use='optional'/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>

  <xs:element name='subject'>
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base='nonEmptyString'>
          <xs:attribute ref='xml:lang' use='optional'/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>

  <xs:simpleType name='nonEmptyString'>
    <xs:restriction base='xs:string'>
      <xs:minLength value='1'/>
    </xs:restriction>
  </xs:simpleType>

</xs:schema>

Now if run through JAXB this would generate three classes: Message, Body and Subject. With this default model you would have to do something like the following to create a valid Message:

Message message = new Message();
message.setFrom( "juliet@example.com/balcony" );
message.setTo( "romeo@example.net" );

Body body = new Body();
body.getBodyOrSubject().add( "Wherefore art thou, Romeo?" );
message.setBody( body );

Now with this simple example thats fine, but imagine something where your model is far more complex, or worse can contain arbitary namespaces as children such as in XMPP where Message can contain objects within any namespace?

This is where Builders become useful. Imagine the above being converted into this:

Message message = new MessageBuilder().
	setFrom( "juliet@example.com/balcony" ).
	setTo( "romeo@example.net" ).
	addBody( new BodyBuilder().
		 setValue( "Wherefore art thou, Romeo?") ).
	build();

Although at first this may look similar to the standard way, the builders break the code into more manageable chunks. The builders themselves can be reused, so if you are sending multiple messages then you could change the above code to:

MessageBuilder builder = new MessageBuilder().
	setFrom( "juliet@example.com/balcony" ).
	addBody( new BodyBuilder().
		 setValue( "Wherefore art thou, Romeo?") );

Message message1 = builder.setTo( "romeo@example.net" ).build();

Message message2 = builder.setTo( "tybalt@example.com/cousin" ).build();

As you can see, the use of builders becomes apparent, we created two Message objects with a single change between them in a lot less code.

On the next page we’ll configure maven to build these builders.