Downloading URL or copying Files using Categories in Groovy

Over the weekend I needed to write a script in groovy which first downloaded a file from a remote webserver then, if that file was retrieved & different copy it to a final location.

Now as Groovy is based on Java you could have done this in the usual manner but I wanted to find a groovy way & found this method which is way more elegant.

The way I found to do this is by using a category to extend the << operator. To do this we create a new class at the end of the script and add a method implementing the operator with the appropriate types.

First we want to implement File << Url which will write the content of a url to a file:

    def static leftShift(File file, URL url) {
       url.withInputStream { is->
            file.withOutputStream { os->
                def bs = new BufferedOutputStream( os )
                bs << is
            }
        }
    }

Now we have that we can simply download the remote file with a few lines. Here we set a URL of a remote rss feed, the file we want to download it to then, using our new category we do the actual copy.

def source = new URL( 'http://trainwatch.co.uk/forums/feed.php?mode=topics' )
def target = new File( 'trainwatch.rss' )

use( FileBinaryCategory ) {
  target << source} 

Extending the category to copy local files is just as simple, just overload leftShift with a File as the source:

    def static leftShift(File dst, File src) {
         src.withInputStream {
           is -> dst.withOutputStream {
          os -> def bs = new BufferedOutputStream( os )
          bs << is
        }
      }
    }

Now we can extend our example above to copy the file elsewhere:

def source = new URL( 'http://trainwatch.co.uk/forums/feed.php?mode=topics' )
def temp = new File( '/tmp/trainwatch.rss' )
def target = new File( 'trainwatch.rss' )

use( FileBinaryCategory ) {
  // Download the rss from the remote site to the temp file
  temp << source

  // Imagine we do some tests here & then copy the file
  target << temp
  } 

Here’s the full category class, all I do in standalone scripts is put this at the end:

class FileBinaryCategory {
  def static leftShift(File file, URL url) {
    url.withInputStream { is->
      file.withOutputStream { os->
        def bs = new BufferedOutputStream( os )
        bs << is
      }
    }
  }
  def static leftShift(File dst, File src) {
    src.withInputStream {
      is -> dst.withOutputStream {
        os -> def bs = new BufferedOutputStream( os )
        bs << is
      }
    }
  }
 }

Now this doesn’t just have to be used for file io either. One usecase I had was to take a list of command line arguments to pass on to a command. Simple except if the list contained another list then I had to flatten the two lists before passing them to the command. Now usually you’d flatten them but what if you had a null or even an empty string? In this case this was possible but I had to strip them out first.

So how to do it? well here’s the category:

class ArgumentsCategory {
 def static leftShift( List list, String arg ) {
   if( arg!=null && !arg.empty ) {
     list.add(arg)
   }
 }

 def static leftShift( List list, List args ) {
   args.each{
     leftShift( list, it )
   }
 }

 def static leftShift( List list, File file ) {
    list.add( file.canonicalPath )
 }
}

Now here we’ve defined three extensions to the << operator. For all three the left and side is the list were appending to while the right is either a String, List or File. For a string it simply appends it. For the file it’s the canonical Path thats appended otherwise we run through each element of the list as if it’s the same command.

Now we can just use this in a similar manner as before:

def src = new File( 'a.tiff' )
def dst = new File( 'a.png' )
def parms = [ '-rotate', '-90', '-resize', '200x200' ]

def args = []
use( ArgumentsCategory ) {
  args << src
  args << params
  args << dst
}

The above code would generate a single list of command arguments for the ImageMagick convert command.

Broadcasting Ehcache CacheEvent’s across a Terracotta cluster

I’ve been trying to get CacheEvent’s working with ehcache – specifically across a terracotta cluster so that if an element is added, removed or updated then every node in the cluster is notified.

Now going by the api, this should be a simple case of creating a CacheEventListener and attaching it to the individual Ehcache. It’s not that simple. Doing that only means that events occuring on the local instance get the events – the other nodes don’t get them.

Now it’s not easy to find but the solution is to actually add a specific CacheEventListenerFactory to the cache.

I.e.

    <cache  name="myCache"
            maxElementsInMemory="100000"
            overflowToDisk="false"
            eternal="false"
            timeToIdleSeconds="1800"
            timeToLiveSeconds="3600"
            memoryStoreEvictionPolicy="LFU">
        <searchable>
            <searchAttribute name="online" expression="value.isOnline()" />
            <searchAttribute name="name" expression="value.getName()" />
        </searchable>
        <terracotta/>
        <cacheEventListenerFactory class="net.sf.ehcache.event.TerracottaCacheEventReplicationFactory"/>
    </cache>

Now in code simply register your listener with:

myCache.getCacheEventNotificationService().registerListener( myListener );

and you’ll find it will work!

Why is ehcache on Terracotta using Hibernate when it’s not meant to be?

I have this weird and interesting problem with hibernate interfering with ehcache on Terracotta.

I have 2 clients, one jboss the other a standalone vm. Both share a common ehcache instance on Terracotta.

Now this works, depending on which client starts first after terracotta.

If the standalone vm starts first then everything is fine.

If jboss starts first then when the standalone vm tries to use the cache it fails because the cache has configured itself to use HibernateElementSerializationStrategy for the keys. As this has a classpath dependency on org.hibernate.cache.CacheKey and that is not in the standalone vm the cache blows up.

So, is there a way to configure ehcache to use a specific strategy and not hibernate? For me hibernate is legacy so ehcache should have no need to refer to it.

%d bloggers like this: