Fixing HTTPS mixed content issues in Chrome

I’ve been beta testing the new Lets Encrypt Certificate Authority so have been adding HTTPS to various services however I hit on a problem where Chrome wasn’t showing the green padlock for a specific application. Instead it was showing the broken padlock because it was saying there was mixed content.

In https, mixed content is usually where you have a resource (css, image etc) thats being served in http instead.

So first I went in to developer tools to find that resource. Now it can be difficult to find, however one first tip is to go to the network tab and right click on any of the columns (e.g. Name) and ensure that scheme is ticked. This now adds a new column which should show https for everything. The one that shows http is the culprit.

Except in this case it didn’t. Everything was https. So what’s happening?

Well in Chrome it caches the https state with each resource, and in this instance I had https originally working with a test certificate. Now I have a real one, so chrome was being confused.

The solution was to restart chrome by opening a new tab and entering the url: chrome://restart

Once chrome restarts (it will restart every tab/window you have open) the problem was fixed.

Java 8 running out of file handles on Linux

When I checked one of my websites this morning I discovered it had stopped responding. All it did was sit there then time out with a standard Tomcat error page. So I logged in to check the logs and found that tomcat was complaining about “Too many open files”. Now a few days earlier I had released a new version of the site which moved the serving of static content from Apache to Tomcat so that was my initial though of where the problem was.

This turns out to be true. Where I was reading a text file I was using the new Java 8 Files.line() method which returns a Stream consisting of each line in a file. Now this was nice as instead of writing a block of code to read a file into a string we could reduce it down to:

String page = Files.line( f.getPath() ).
    collect( Collectors.joining( "\n" ) ) );

The problem here is that although nice and concise, it never closes the file. So after a while the Operating System complains that too many files are open and tomcat grinds to a halt.

Now hidden away in the javadocs you’ll find tha Stream actually implements AutoClosable & the reason why is that you can close the stream once you have done with it. Now in 99% of all Streams you don’t need to do anything but if a Stream is operating against some external resource like a File then you are supposed to be closing the stream afterwards.

Now this is where I went wrong, and I suspect a lot of others would also fall foul of this as every example of using Java 8 streams do not show closing them – so who would know that they need to?

So, the correct way of reading a file using streams is this:

try( Stream lines = Files.line( path ) ) {
    // do something with the stream
}

For example, in my case it ended up looking something like this:

String page;
try( Stream lines = Files.line( f.getPath() ) ) {
    page = lines.collect( Collectors.joining( "\n" ) ) );
}

So the lesson here is make certain when using an external resource within a Stream then ensure you close it afterwards.

Connecting an Arduino to a Raspberry PI using I2C

Some time ago I created a weather station using a Raspberry PI and an off the shelf weather station, connecting the two via USB.

However, for some time not I’ve been meaning to create a weather station from scratch – i.e. one or more Raspberry PI’s which connect to the network (via Ethernet or WiFi) and directly monitor the sensors directly.

Now the problem here is that some sensors are analog – for example the leaf, soil and UV sensors I have generate an analog signal so we need an ADC (Analogue to Digital Converter) which the Raspberry PI doesn’t have.

So we have two possible solutions:

  1. Add a Raspberry PI compatible ADC
  2. Use an Arduino

With the parts I have available, the Arduino won, not just on available ADC channels but also with the additional digital ports available.

Now how to connect it to the PI? Well the easiest way is to use USB, however the PI only has two USB ports (one for the Model A) and as I’m intending to use Model A’s for the final station I need that for WiFi (there won’t be room or power for hubs) so USB is out.

There’s RS232 which both support, however the PI runs on 3v3 whilst the Arduino (UNO) is 5v so I need to add a level converter between the two. It also limits me to just one arduino and I might need to use more than one so another solution is needed.

Continue reading “Connecting an Arduino to a Raspberry PI using I2C”

Getting IPv6 working on the UK Mobile Network

One problem with mobile data here in the UK is that everything is NATed to death. Most consumer mobile data connections do not support static IP‘s and those that do are expensive. Also, the UK is lagging behind most of the world in providing access to the IPv6 internet. It’s not that IPv6 is new either, it’s been around for 10 years but no, here in the UK they want to try not fork out the cash to replace older kit that can only handle IPv4 & it’s not exactly rocket science either.

Ok, my home network is IPv6 enabled. I have a /64 subnet routed to home out of my /48 allocation at my ISP (they support IPv6 natively). If you don’t know what this is don’t worry – l this means is that on the IPv6 internet I actually have more static IP’s on my home network than the old legacy internet put together. In fact the legacy has about 4 billion & I have 1.8*1019. Now you see the main selling point of IPv6 – there’s room for every single device that’s networkable to have just one IP address and be accessible from anywhere – obviously behind a firewall.

Anyhow, the problem I had to solve was this: I’ve got a fair few machines on my network and at times I need to be able to ssh into them remotely. Currently I can do this by either ssh into my firewall by it’s IPv4 address and then onto the internal machine or I could setup a VPN – but why should I when I’m fully IPv6 enabled?

Setting up a Tunnel Broker

Well there is a way. When native IPv6 isn’t available, one option is to use a tunnel broker. A broker sets up a tunnel between your device and an endpoint at the broker. IPv6 is then encapsulated in an IPv4 packet, sent through the tunnel to the broker, then sent out from there as IPv6. Now there are plenty out there like Hurricane Electric and SixXS but as I need to use this on a 3G device they won’t work as they require a static IPv4 address and we don’t have that – we’re behind a NAT, so the only available option is Gogo6 (which owns Freenet6). They are also a broker but they support NAT traversal which is what we need.

So, on my Linux Mint laptop it’s a simple case of installing the Gogo6 client:

sudo apt-get install gogoc
sudo /etc/init.d/gogoc stop
sudo /etc/rc5.d/S20gogoc

Now you might wonder why we stop then delete a file after installation. This is because when it installs it starts the service and we don’t want it running just yet. Also the rc5.d file means it starts on boot which we don’t want – we want to use this on 3G remember.

As it stands that’s all there is to do – by default it’s configured to use an anonymous account so the next time you’re on 3G you simply:

sudo /etc/init.d/gogoc start

and you’ll find you are now on the IPv6 internet. When you go offline just stop gogoc:

sudo /etc/init.d/gogoc stop

Getting a more permanent static IPv6 address

With an anonymous connection you’ll get an IP address out of a pool but if you want a static address you’ll need to register an account and edit /etc/gogoc/gogoc.conf

In that file:

  1. edit the lines with userid= and passwd= with you’re account’s username and password.
  2. The line server= needs to be their endpoint. Here you register against a specific one, so as I used amsterdam set this to amsterdam.freenet6.net
  3. Finally change the auth_method= line from anonymous to one of the other methods listed just above that line.

Now that last step might take some work to get working. any should always work but it risks sending your password in the clear but you might want to play with that later.

That’s it. When you start gogoc you’ll get a new IP address which will be permanent. You’ll also get a dns entry setup as well, username.broker.freenet6.net so now you can get into your laptop.

Tunneling an entire network

There is a final option available but out of scope here, and thats connecting an entire network to the tunnel. Thats simply a case of changing the host_type= line from host to router. Then you’re local network will get an IPv6 address with your laptop as the router.

How well does it work

Well I’ve tested it on T-Mobile UK and it works pretty well. I can access my home servers directly and as I use the non-anonymous option I can actually ssh from home to the laptop via it’s 3G connection.

I’ve even tried setting up a proxy on an Apache server which is accessible from the legacy IPv4 internet and it connects to the laptop’s Apache server fine – although sluggish but remember this is over 3G.

At some point I’ll try it on other operators (I also use GiffGaff & 3).

Next I need to figure out how to get this working on Android so that I can get my Nexus 4 & Nexus 7 3G online – both supports IPv6 when on the WiFi at home, just would be nice when out and about.

 

Getting an Icom IC-PCR1000 working with Linux Mint

I’ve just got a second hand Icom IC-PCR1000 communications receiver and needed to get it working with Linux Mint.

The Icom IC-PCR1000
The Icom IC-PCR1000

What’s unique to this receiver is that its a standalone unit which connects to a computer via RS232C (it dates from 1999) and covers the entire 10kHz to 1300MHz range (there is a US version which has US mobile frequencies blocked).

Now obviously most modern computers don’t have native serial ports these days so I’m using a USB-Serial adapter to connect it.

Installation

So to get it working with Linux. First ignore the floppies – they only contain software for Windows and even if you wanted to use it, the contents are available online anyhow.

Next plug it in to your Linux box. You’ll need a standard mini-plug audio cable which you’ll plug in to the Ext-SP socket on the radio and the other end into either Line-In or Microphone (some systems they are the same one).

Next you need is a recent version of a utility called pcrd. There’s actually two versions out there but the one I’m using is from https://www.crc.id.au/pcrd-pcr1000-on-linux/

From that page download the source for v0.12 and extract it. This should give you a pcrd-0.12 dircectory. Next you need to compile it:

peter@titan ~ $ cd pcrd-0.12
peter@titan ~/pcrd-0.12 $ make linux

Trying it out

peter@titan ~/pcrd-0.12 $ ./pcrd -d /dev/ttyUSB0 -v 40 89.100 wfm 230

Here

  • /dev/ttyUSB0 is the USB-Serial port.
  • -v is the volume in hexadecimal, so 00 is off, ff is full on.
  •  89.100 is BBC Radio 2 here in the South East of the UK.
  • wfm is the mode, so for broadcast FM radio you need to use Wide FM
  • 230 is the filter to use. If you leave this out it will default to 15 kHz

That’s effectively all that’s needed, sort of.

Here’s a few other stations to try:

BBC Radio Kent

peter@titan ~/pcrd-0.12 $ ./pcrd -d /dev/ttyUSB0 -v 40 96.7 wfm 230

BBC Radio 4 FM

peter@titan ~/pcrd-0.12 $ ./pcrd -d /dev/ttyUSB0 -v 40 93.6 wfm 230

BBC Radio 4 Long Wave – essential for Test Match Special, as long as you have the right antenna 😉

peter@titan ~/pcrd-0.12 $ ./pcrd -d /dev/ttyUSB0 -v 40 0.198

No audio?

You might have a problem with getting the audio being fed to your speakers. This seems to be a common problem with Linux mint, but there’s a solution:

First make sure you have pulseaudio-utils installed:

peter@titan ~ $ sudo apt-get install pulseaudio-utils

Next you need to add the loopback module:

peter@titan ~ $ pactl load-module module-loopback latency_msec=10

You should see a microphone icon appear next to the clock and, hopefully, the audio being fed through to the speakers.

Converting OS Grid coordinates into Lat/Long using PostGIS

I had a case of needing to convert Ordnance Survey grid coordinates like 548420,247240 that were in a table from NAPTAN into latitude & longitude so that I could display them in a map.

Most references online appear to do this by running the data through some application then importing it back into the database but as I’m using PostGIS I wanted to do it directly on the database.

So here’s how I did it.

First here’s the table I want to process:

gis=> select crscode,stationname,easting,northing from naptan limit 5;
 crscode |           stationname           | easting | northing
---------+---------------------------------+---------+----------
 ABA     | Aberdare Rail Station           |  300400 |   202800
 AUR     | Aberdour Rail Station           |  319100 |   685400
 AVY     | Aberdovey Rail Station          |  260600 |   296000
 ABE     | Aber Rail Station               |  314870 |   186950
 AGL     | Abergele & Pensarn Rail Station |  294612 |   378681
(5 rows)

Now you see we have the rail station’s coordinates as an easting & northing pair.

First I added a geometry column to the table naptan:

gis=> select AddGeometryColumn( 'public', 'naptan', 'the_geom', 27700, 'POINT', 2);
                  addgeometrycolumn
------------------------------------------------------
 public.naptan.the_geom SRID:27700 TYPE:POINT DIMS:2
(1 row)

Here 27700 is the ESRI code for the OS grid system.

Next I updated the table to create geometries for each entry:

gis=> update naptan set the_geom=GeomFromText('POINT('||easting||' '||northing||')',27700);
UPDATE 2603

This updates each row using the easting and northing columns.

Finally I needed to add two new columns for latitude & longitude and populate them with the final values:

gis=> alter table naptan add column lat real;
ALTER TABLE
gis=> alter table naptan add column long real;
ALTER TABLE
gis=> update naptan set long=st_x(st_transform(the_geom,4326)), lat=st_y(st_transform(the_geom,4326));
UPDATE 2603

That’s it, we now have the table populated with lat/long coordinates:

gis=> select crscode,stationname,lat,long from naptan limit 5;
 crscode |           stationname           |   lat   |   long
---------+---------------------------------+---------+----------
 ABA     | Aberdare Rail Station           | 51.7151 | -3.44308
 AUR     | Aberdour Rail Station           | 56.0546 | -3.30056
 AVY     | Aberdovey Rail Station          |  52.544 | -4.05707
 ABE     | Aber Rail Station               |  51.575 | -3.22983
 AGL     | Abergele & Pensarn Rail Station | 53.2946 | -3.58262

Hopefully this is correct – if it isn’t please let me know but the final data looks ok to me.

Installing Java 7 on Debian Squeeze

For all of my servers I use Debian, however that distribution has a few problems, mainly the packages can be a bit behind the cutting edge.

Now this is usually a good thing if you are looking for stability – cutting edge software can have issues, especially from new features etc, so for a live environment you want something thats stable.

However, there does come a time when this can bite back. You either need a feature thats not in the standard repositories or in this case the version is now unsupported.

In Debian Squeeze it has Java 6 – but that was EOL’d a couple of months ago so is no longer supported by Oracle. The current version is Java 7 update 17.

So how do we get Java 7 installed?

Well it’s pretty easy to do, we just need to add another repository into apt and install it.

First the repository:

sudo su -
echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list
echo "deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886
apt-get update
exit

What that does is to install the ubuntu ppa repository into apt, setup the public keys and then load the package lists.

Next we need to install it:

sudo apt-get install oracle-java7-installer

This will now download Oracle Java 7 and present you with a couple of screens about licensing. Just ok and accept it and it will now install.

That’s it. You now have Java 7 installed – but it’s not the default JDK (if you already had Java 6 installed). If you want it to be the default then there’s just one more thing to do:

sudo apt-get install oracle-java7-set-default

That’s a dummy package but it will make Java 7 the default on that machine. If you want to check then you can check:

peter@titan ~ $ java -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)