Category: Linux

Connecting 2 Fibre Channel switches together

I’ve been running a Fibre Channel fabric at home for storage for a while now. The fabric contains a 24 cartridge Tape Library, an HP DL380e acting as a san for disk storage and most of my full sized servers are also connected to it.

The rear of my core Fibre Channel switch, an EMC DS-5100B 8Gb FC switch with 24 licensed ports.
The fibre going off to the right is the one going to the second switch upstairs in the office.

However, as I’ve just setup a proper home office away from the rack (it’s a lot quieter as the rack is noisy) I needed fibre channel as well as Ethernet as my main workstation’s connected to both.

Extending Ethernet is easy, you can just connect another switch to the network (although I’m changing that, more of that in a future article), but initially I had the workstation setup with it’s own fibre from the fibre switch but that limited me to just the workstation, so if I needed more then it was either lay more fibres or get a second switch to serve the office.

So I got my hands on an IBM 2498-B24/24E 8Gb FC switch with 8 ports licensed. Except it wasn’t, turns out it was fully licensed so has 24 active ports, which is a bonus.

IBM 2498-B24/24E 8Gb FC switch in the office network cabinet.
Yes it’s the wrong way round with the hot end at the front but this cabinet doesn’t have rear access so there’s no access to the ports otherwise. I will have to keep an eye on how the temperatures go but so far it’s acceptible.

So the two are linked together with a single fibre but at first they didn’t want to talk. This turned out that both switches had the same domain id.

Fabric OS which both switches are running defaults the domain id to 1, so I had to change that on the new switch which is now domain 2:

  • Disable the switch using the switchdisable command
  • Run the configure command
  • When prompted Fabric parameters enter Y
  • When prompted Domain enter the new domainId, 2 in my case
  • Just press enter for the remaining options.
  • Finally once configure has completed run switchenable to re-enable the switch.

Once that was done I did a reboot of the new switch then switchshow and I could see that port 0 was now disabled, showing Disabled (Implicit Platform Service Enable operation blocked) against the port linking the two:

admin> switchshow
 switchName:     VHC_Switch_D
 switchType:     71.2
 switchState:    Online   
 switchMode:     Native
 switchRole:     Principal
 switchDomain:   2
 switchId:       fffc02
 switchWwn:      10:00:00:05:33:4a:66:53
 zoning:         OFF
 switchBeacon:   OFF
 Index Port Address Media Speed State     Proto
 0   0   020000   id    N8   In_Sync     FC  Disabled (Implicit Platform Service Enable operation blocked)
 1   1   020100   id    N8   Online      FC  F-Port  10:00:00:90:fa:ae:0d:a5

It turns out this was the original switch with the MS Platform Management Service being enabled. As the service was running the new switch would refuse to connect as it couldn’t configure itself.

So onn the new switch I ran msplatshow and it told me it was disabled:

admin> msplatshow
 *MS Platform Management Service is NOT enabled.

On the original switch it showed it was running & empty:

admin> msplatshow
 *MS Platform Management Service DB is empty.

So the fix was simple, just to deactivate the service on the original switch with msplmgmtdeactivate and saying yes to confirm I wanted it to be deactivated. Once done running the command a second time told me it was now disabled.

admin> msplmgmtdeactivate
 MS Platform Service is currently enabled.
 This will erase MS Platform Service configuration 
 information as well as database in the entire fabric.
 Would you like to continue this operation? (yes, y, no, n): [no] yes
 Request to deactivate MS Platform Service in progress……
 *Completed deactivating MS Platform Service in the fabric!
 Switch:admin> msplmgmtdeactivate
 MS Platform Service is already disabled!

Next I rebooted the new switch. Once it came back up I ran fabricshow on the core switch and it appeared.

admin> fabricshow
 Switch ID   Worldwide Name          Enet IP Addr    FC IP Addr      Name
 1: fffc01 10:00:00:05:33:49:da:92 192.168.2.62    0.0.0.0        >"Switch"
 2: fffc02 10:00:00:05:33:4a:66:53 192.168.3.124   0.0.0.0         "VHC_Switch_D"
 The Fabric has 2 switches

So next to check that it’s working, so as root on my workstation I triggered the adapter to rescan for new devices:

# echo 1 > /sys/class/fc_host/host10/issue_lip

then ran lsscsi to see what it found (this doesn’t need root):

$ lsscsi
 [2:0:0:0]    cd/dvd  hp       DVD A  DH16AFSH  DHH4  /dev/sr0
 [4:0:0:0]    disk    ATA      HGST HUS724040AL AC50  /dev/sda
 [4:0:1:0]    disk    ATA      CT500MX500SSD1   023   /dev/sdb
 [11:0:0:0]   disk    Linux    File-Stor Gadget 0409  /dev/sdc
 [12:0:0:0]   disk    Generic  STORAGE DEVICE   1404  /dev/sdd
 [13:0:0:0]   tape    HP       Ultrium 5-SCSI   I6RW  /dev/st0
 [13:0:0:1]   mediumx HP       MSL G3 Series    7.10  /dev/sch0

Brilliant, it’s found the tape library. It’s the last 2 lines, one for the tape and the mediumx is the robotics for changing the tape.

Running mtx it returns the list of tapes in the library, confirming the link is now working:

$ mtx -f /dev/tape/by-id/scsi-DEC07115EW status
   Storage Changer /dev/tape/by-id/scsi-DEC07115EW:1 Drives, 24 Slots ( 1 Import/Export )
 Data Transfer Element 0:Empty
       Storage Element 1:Full :VolumeTag=ARC010L5                       
       Storage Element 2:Full :VolumeTag=ARC011L5                       
       Storage Element 3:Full :VolumeTag=ARC012L5                       
       Storage Element 4:Full :VolumeTag=ARC013L5                       
       Storage Element 5:Full :VolumeTag=ARC014L5                       
       Storage Element 6:Full :VolumeTag=ARC008L5                       
       Storage Element 7:Full :VolumeTag=ARC009L5                       
       Storage Element 8:Empty
       Storage Element 9:Empty
       Storage Element 10:Empty
       Storage Element 11:Empty
       Storage Element 12:Full :VolumeTag=ARC001L5                       
       Storage Element 13:Full :VolumeTag=ARC002L5                       
       Storage Element 14:Full :VolumeTag=ARC003L5                       
       Storage Element 15:Full :VolumeTag=ARC004L5                       
       Storage Element 16:Full :VolumeTag=ARC005L5                       
       Storage Element 17:Full :VolumeTag=ARC006L5                       
       Storage Element 18:Full :VolumeTag=ARC007L5                       
       Storage Element 19:Empty
       Storage Element 20:Empty
       Storage Element 21:Empty
       Storage Element 22:Empty
       Storage Element 23:Full :VolumeTag=CLN001CU                       
       Storage Element 24 IMPORT/EXPORT:Empty

There’s a lot more you can do with these switches, for example creating zones to partition the network like you can with VLAN’s in Ethernet but for my use-case this is is perfect, and the single 8Gb connection between the two should be enough bandwidth – most of my storage on SAS drives doesn’t run at 8Gb, but if it did become an issue I can always try to increase the link by bonding multiple fibres – if that’s even possible, would have thought it would be.

Centralizing Certificate Management of LetsEncrypt with a Raspberry PI

Lets Encrypt is a new Certificate Authority (CA), run for the public’s benefit by the Internet Security Research Group (ISRG). At the time of writing it’s currently in Beta and is due to go public in December 2015.

Update: Lets Encrypt went into public -beta on December 3 2015. I have updated this article with the minor change needed to work with the live servers.

Now in the default mode, the standard Lets Encrypt client (it’s not the only one) can manage this automatically – however it’s not ideal if you have more than one server.

What I describe here is how to centralize managing certificate registration (& later renewal) on a central machine. When a certificate is then registered or renewed we can then copy the certs to the remote servers.

Continue reading “Centralizing Certificate Management of LetsEncrypt with a Raspberry PI”

Fixing HTTPS mixed content issues in Chrome

I’ve been beta testing the new Lets Encrypt Certificate Authority so have been adding HTTPS to various services however I hit on a problem where Chrome wasn’t showing the green padlock for a specific application. Instead it was showing the broken padlock because it was saying there was mixed content.

In https, mixed content is usually where you have a resource (css, image etc) thats being served in http instead.

So first I went in to developer tools to find that resource. Now it can be difficult to find, however one first tip is to go to the network tab and right click on any of the columns (e.g. Name) and ensure that scheme is ticked. This now adds a new column which should show https for everything. The one that shows http is the culprit.

Except in this case it didn’t. Everything was https. So what’s happening?

Well in Chrome it caches the https state with each resource, and in this instance I had https originally working with a test certificate. Now I have a real one, so chrome was being confused.

The solution was to restart chrome by opening a new tab and entering the url: chrome://restart

Once chrome restarts (it will restart every tab/window you have open) the problem was fixed.