Java 8 running out of file handles on Linux

When I checked one of my websites this morning I discovered it had stopped responding. All it did was sit there then time out with a standard Tomcat error page. So I logged in to check the logs and found that tomcat was complaining about “Too many open files”. Now a few days earlier I had released a new version of the site which moved the serving of static content from Apache to Tomcat so that was my initial though of where the problem was.

This turns out to be true. Where I was reading a text file I was using the new Java 8 Files.line() method which returns a Stream consisting of each line in a file. Now this was nice as instead of writing a block of code to read a file into a string we could reduce it down to:

String page = Files.line( f.getPath() ).
    collect( Collectors.joining( "\n" ) ) );

The problem here is that although nice and concise, it never closes the file. So after a while the Operating System complains that too many files are open and tomcat grinds to a halt.

Now hidden away in the javadocs you’ll find tha Stream actually implements AutoClosable & the reason why is that you can close the stream once you have done with it. Now in 99% of all Streams you don’t need to do anything but if a Stream is operating against some external resource like a File then you are supposed to be closing the stream afterwards.

Now this is where I went wrong, and I suspect a lot of others would also fall foul of this as every example of using Java 8 streams do not show closing them – so who would know that they need to?

So, the correct way of reading a file using streams is this:

try( Stream lines = Files.line( path ) ) {
    // do something with the stream
}

For example, in my case it ended up looking something like this:

String page;
try( Stream lines = Files.line( f.getPath() ) ) {
    page = lines.collect( Collectors.joining( "\n" ) ) );
}

So the lesson here is make certain when using an external resource within a Stream then ensure you close it afterwards.

Installing Java 7 on Debian Squeeze

For all of my servers I use Debian, however that distribution has a few problems, mainly the packages can be a bit behind the cutting edge.

Now this is usually a good thing if you are looking for stability – cutting edge software can have issues, especially from new features etc, so for a live environment you want something thats stable.

However, there does come a time when this can bite back. You either need a feature thats not in the standard repositories or in this case the version is now unsupported.

In Debian Squeeze it has Java 6 – but that was EOL’d a couple of months ago so is no longer supported by Oracle. The current version is Java 7 update 17.

So how do we get Java 7 installed?

Well it’s pretty easy to do, we just need to add another repository into apt and install it.

First the repository:

sudo su -
echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list
echo "deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886
apt-get update
exit

What that does is to install the ubuntu ppa repository into apt, setup the public keys and then load the package lists.

Next we need to install it:

sudo apt-get install oracle-java7-installer

This will now download Oracle Java 7 and present you with a couple of screens about licensing. Just ok and accept it and it will now install.

That’s it. You now have Java 7 installed – but it’s not the default JDK (if you already had Java 6 installed). If you want it to be the default then there’s just one more thing to do:

sudo apt-get install oracle-java7-set-default

That’s a dummy package but it will make Java 7 the default on that machine. If you want to check then you can check:

peter@titan ~ $ java -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)

Writing Servlets the J2EE 6 / Servlet 3.0 way

In the past whenever you wrote a Servlet you had a lot of work to do. First you wrote your servlet, then you had to add configuration for that servlet into web.xml so that your application would use it.

For simple applications this was fine but the more servlets you wrote the harder it became as web.xml started to become large and unwieldy.

Now with J2EE 6 you got the ability to add annotations to your servlets. No more do you need to add anything to web.xml as the container scans the classpath for any servlet that’s annotated with @WebServlet getting the required configuration from there.

This also has the benefit that you can write libraries of servlets. Those servlets get deployed automatically and you don’t have to worry about duplicating the config.

Both Tomcat 7 and TomEE 1.5.1 support the J2EE 6 Web Profile as standard. I’m not certain of other containers as I’ve only tested this on those two.

Continue reading “Writing Servlets the J2EE 6 / Servlet 3.0 way”

The problem of mixing library versions

Usually you wouldn’t mix versions. Build environments like Maven should handle this for you – although even maven can get things wrong.

Yesterday I was playing with Apache TomEE looking at migrating some webapp’s away from Apache Tomcat 7. The impetus for this was to get Rest services working and one of the TomEE profiles supports JAX-RS rest services. Tomcat only supports JAX-WS as thats in Java EE6 Web profile.

Anyhow, with the theory that the webapp’s should just run with minor config changes to the container (declaring datasources, that sort of thing) I installed TomEE and got Netbeans to deploy to it.

Bang!

None of the apps would deploy. They all complained about slf4j missing a method:

java.lang.NoSuchMethodError: org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
    at org.apache.commons.logging.impl.SLF4JLocationAwareLog.debug(SLF4JLocationAwareLog.java:133)
    at org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager$1.getConnection(ThreadSafeClientConnManager.java:221)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:401)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)

Now if you ever see this, it’s down to having mixed versions of slf4j in your classpath – and unlike Tomcat, TomEE uses the most recent version of slf4j.

The solution

Many hours later I find this article where someone had a similar problem last year: SLF4J Dependencies and LocationAwareLogger.log Method Not Found Exception.

Now my webapps also use the latest standalone tiles library for layout so after getting maven to show me the dependency tree (mvn dependency:tree) on the webapp it showed that I was pulling in an old slf4j version into the webapp.

So, with a minor change to the pom telling maven to exclude slf4j it finally worked.

Here’s the changes I ended up making to the pom:

<dependency>
  <groupId>org.apache.tiles</groupId>
  <artifactId>tiles-extras</artifactId>
  <!-- needed to exclude slf4j which causes incompatibilities -->
  <exclusions>
    <exclusion>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-nop</artifactId>
    </exclusion>
    <exclusion>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
    </exclusion>
    <exclusion>
      <groupId>org.slf4j</groupId>
      <artifactId>jcl-over-slf4j</artifactId>
    </exclusion>
  </exclusions>
</dependency>

<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-api</artifactId>
  <scope>provided</scope>
</dependency>

The first dependency is for tiles. The <version/> element is missing as I declare that in the project’s root pom and ensures all webapps use the same version.

The important bit is the <exclusions/> element, here we tell maven to exclude those artifacts from the war. There’s one or more <exclusion/> elements which contain just the <groupId/> and <artifactId/> elements – no <version/> here.

The second dependency is for slf4j. It’s here to allow any code provided by the war to have access to it. Again <version/> is missing as its in the root pom but we provide a <scope/> of provided. This tells maven it can use it for compiling but does not include it in the war.

Finally in the project’s other modules I just make sure that any pom with a reference to slf4j is also declared as provided so they don’t cause maven to include it either.

That’s about it. It only took me about 4 hours of hunting around to find the underlying cause. In the Windows world this is known as DLL Hell – well it can happen to any OS/Environment & it is hell when it strikes.

Notes:

  • If you use another logging library within your webapp then that shouldn’t be affected. log4j should just work, this problem appears just to be for slf4j.
  • If you use apache commons logging then you can declare that as provided if you wanted. This is because TomEE provides it at the container level as well as slf4j.

How to fix OpenJDK-7 certificates on Ubuntu 11.10 running on Amazon EC2

After a second crash of my EC2 instance which was running Amazon‘s own Linux distribution I had to rebuild so this time I decided to put the latest official Ubuntu AMI on it. Everything ran fine until I fired up an application which takes a feed from Twitter using their stream api.

When I fired that up I got the following stack trace:

17 Feb 2012 21:13:00,790 ERROR [Twitter Stream consumer-1[Waiting for 500 milliseconds]] [in.setra.twitter.TwitterModule] Exception during processing
java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-emptyRelevant discussions can be on the Internet at:
 http://www.google.co.jp/search?q=b5e7486f or
 http://www.google.co.jp/search?q=24943238
TwitterException{exceptionCode=[b5e7486f-24943238 b5e7486f-2494320e b5e7486f-2494320e b5e7486f-2494320e], statusCode=-1, retryAfter=-1, rateLimitStatus=null, featureSpecificRateLimitStatus=null, version=2.2.5-SNAPSHOT}
 at twitter4j.internal.http.HttpClientImpl.request(HttpClientImpl.java:200)
 at twitter4j.internal.http.HttpClientWrapper.request(HttpClientWrapper.java:65)
 at twitter4j.internal.http.HttpClientWrapper.post(HttpClientWrapper.java:102)
 at twitter4j.TwitterStreamImpl.getFilterStream(TwitterStreamImpl.java:290)
 at twitter4j.TwitterStreamImpl$7.getStream(TwitterStreamImpl.java:279)
 at twitter4j.TwitterStreamImpl$7.getStream(TwitterStreamImpl.java:277)
 at twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java:427)
Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
 at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
 at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1697)
 at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1660)
 at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1643)
:
Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
 at java.security.cert.PKIXParameters.setTrustAnchors(PKIXParameters.java:200)
 at java.security.cert.PKIXParameters.<init>(PKIXParameters.java:120)
 at java.security.cert.PKIXBuilderParameters.<init>(PKIXBuilderParameters.java:104)
 at sun.security.validator.PKIXValidator.<init>(PKIXValidator.java:73)
 ... 23 more

After a brief search I found that for some reason when you install OpenJDK-7-jre-headless you don’t get the certificates installed & most people just switched back to the Sun/Oracle jre.

Now this worked for me – the install was a virgin setup so I hadn’t installed the sun JDK before but I found the Java 6 cacerts installed, so the following two lines fixed the problem:

cd /usr/lib/jvm/java-6-openjdk/jre/lib/security
sudo ln -s /usr/lib/jvm/java-7-openjdk-i386/jre/lib/security/cacerts cacerts

This may work elsewhere, it may not – in this instance it worked & I’m now getting a realtime stream in from Twitter.

Manually downloading an artefact in Maven

Usually maven will download an artefact on it’s own however there are times when you need to do this manually – in this instance the local Nexus installation is down for maintenance so I had no choice but do it the hard way.

In this instance I needed the exec-maven-plugin which my local repository didn’t have but fortunately the maven-dependency-plugin allows you to download them:

 mvn org.apache.maven.plugins:maven-dependency-plugin:2.1:get \
     -Dartifact=org.codehaus.mojo:exec-maven-plugin:1.2 \
     -DrepoUrl=http://repo1.maven.org/maven2

All you need is to set artifact= to the required artifact & repoUrl to the remote repository – in this case Maven Central

Downloading URL or copying Files using Categories in Groovy

Over the weekend I needed to write a script in groovy which first downloaded a file from a remote webserver then, if that file was retrieved & different copy it to a final location.

Now as Groovy is based on Java you could have done this in the usual manner but I wanted to find a groovy way & found this method which is way more elegant.

The way I found to do this is by using a category to extend the << operator. To do this we create a new class at the end of the script and add a method implementing the operator with the appropriate types.

First we want to implement File << Url which will write the content of a url to a file:

    def static leftShift(File file, URL url) {
       url.withInputStream { is->
            file.withOutputStream { os->
                def bs = new BufferedOutputStream( os )
                bs << is
            }
        }
    }

Now we have that we can simply download the remote file with a few lines. Here we set a URL of a remote rss feed, the file we want to download it to then, using our new category we do the actual copy.

def source = new URL( 'http://trainwatch.co.uk/forums/feed.php?mode=topics' )
def target = new File( 'trainwatch.rss' )

use( FileBinaryCategory ) {
  target << source} 

Extending the category to copy local files is just as simple, just overload leftShift with a File as the source:

    def static leftShift(File dst, File src) {
         src.withInputStream {
           is -> dst.withOutputStream {
          os -> def bs = new BufferedOutputStream( os )
          bs << is
        }
      }
    }

Now we can extend our example above to copy the file elsewhere:

def source = new URL( 'http://trainwatch.co.uk/forums/feed.php?mode=topics' )
def temp = new File( '/tmp/trainwatch.rss' )
def target = new File( 'trainwatch.rss' )

use( FileBinaryCategory ) {
  // Download the rss from the remote site to the temp file
  temp << source

  // Imagine we do some tests here & then copy the file
  target << temp
  } 

Here’s the full category class, all I do in standalone scripts is put this at the end:

class FileBinaryCategory {
  def static leftShift(File file, URL url) {
    url.withInputStream { is->
      file.withOutputStream { os->
        def bs = new BufferedOutputStream( os )
        bs << is
      }
    }
  }
  def static leftShift(File dst, File src) {
    src.withInputStream {
      is -> dst.withOutputStream {
        os -> def bs = new BufferedOutputStream( os )
        bs << is
      }
    }
  }
 }

Now this doesn’t just have to be used for file io either. One usecase I had was to take a list of command line arguments to pass on to a command. Simple except if the list contained another list then I had to flatten the two lists before passing them to the command. Now usually you’d flatten them but what if you had a null or even an empty string? In this case this was possible but I had to strip them out first.

So how to do it? well here’s the category:

class ArgumentsCategory {
 def static leftShift( List list, String arg ) {
   if( arg!=null && !arg.empty ) {
     list.add(arg)
   }
 }

 def static leftShift( List list, List args ) {
   args.each{
     leftShift( list, it )
   }
 }

 def static leftShift( List list, File file ) {
    list.add( file.canonicalPath )
 }
}

Now here we’ve defined three extensions to the << operator. For all three the left and side is the list were appending to while the right is either a String, List or File. For a string it simply appends it. For the file it’s the canonical Path thats appended otherwise we run through each element of the list as if it’s the same command.

Now we can just use this in a similar manner as before:

def src = new File( 'a.tiff' )
def dst = new File( 'a.png' )
def parms = [ '-rotate', '-90', '-resize', '200x200' ]

def args = []
use( ArgumentsCategory ) {
  args << src
  args << params
  args << dst
}

The above code would generate a single list of command arguments for the ImageMagick convert command.