IPSEC - Transport Mode - Debian

Do you need to secure the traffic between two nodes on a network you don´t trust? There are many VPN solutions as well as SSL and SSH. They all require settings for either the application or for the link itself to work.

IPSEC encrypts on Level 3 (in the IP stack) because of this the application does not need to know about the encryption.

There are two types of IPSEC modes, Tunnel and Transport. I use Tunnel mode in many cases for standard VPN. Transport mode is not used as often since it has more limited usecases.

I needed to secure traffic between two servers. The servers were initaly on a trusted network but needed to be moved to a network that I do not have full controll of. In my case the data sent over the link is occasionaly sensitive, so I needed a way to make it harder to listen to the link.

The nodes have direct IP access to each other. This example is with internal IPs but it works the same way with public IPs on both machines.

This guide will show you how to setup a static link, wich has a limited amount of packets that can bet sent. The maximum number of packets that can be sent over the link without a reset is 2^32. But if the kernels support 64 bit index the max amount of packets is 2^64.

This is how i did it

Install ipsec-tools

I started by installing ipsec-tools on the two machines.

apt-get install ipsec-tools

Flush IPSEC databases

I changed ipsec-tools.conf to flush the ipsec databases when running the setup:

# Flush the SAD and SPD
flush;
spdflush;

Setup

I created a conf file: /etc/ipsec-tools.d/transport-link1.conf on the host with ip 192.168.0.2. Allso make sure that the data in this file is secret since it contains the crypto-keys.

#!/usr/sbin/setkey
# for host 192.168.0.2.

# Put the ESP SAs in the SAD, 192 bit long keys
add 192.168.0.3 192.168.0.2 esp 31031 -E rijndael-cbc
                0xf86f440c0247a935c75be785486e535a90a636f0b189d588;
add 192.168.0.2 192.168.0.3 esp 43241 -E rijndael-cbc
                0xe9749d02219238454dd0dba8eae55155131c86c05a2f7d4e;

# Put the IPs in the Security Policy Database 
# to require the traffic to be encrypted
spdadd 192.168.0.2 192.168.0.3 any -P out ipsec
       esp/transport//require;

spdadd 192.168.0.3 192.168.0.2 any -P in ipsec
       esp/transport//require;

I created a conf file: /etc/ipsec-tools.d/transport-link1.conf on the host with ip 192.168.0.3 as well.

Note that the only difference between the files is that the IPs in the "spdadd" lines are swapped

#!/usr/sbin/setkey
# for host 192.168.0.3

# Put the ESP SAs in the SAD, 192 bit long keys
add 192.168.0.3 192.168.0.2 esp 31031 -E rijndael-cbc
                0xf86f440c0247a935c75be785486e535a90a636f0b189d588;
add 192.168.0.2 192.168.0.3 esp 43241 -E rijndael-cbc
                0xe9749d02219238454dd0dba8eae55155131c86c05a2f7d4e;

# Put the IPs in the Security Policy Database 
# to require the traffic to be encrypted
spdadd 192.168.0.3 192.168.0.2 any -P out ipsec
       esp/transport//require;

spdadd 192.168.0.2 192.168.0.3 any -P in ipsec
       esp/transport//require;

To generate a 192 bit key I used this command:

echo "0x$(hexdump -e '1/1 "%02x"' '/dev/random' -n 24)"

Enable the link

I started the link by the setkey init script on both machines:

/etc/init.d/setkey start

Make sure that the link is working

I tested the link by pinging between the machines and using tcpdump to see the traffic, when the IPSEC is working you get ESP packets when it is not you get ICMP packets in tcp-dump.

Faultfindning

When things go wrong you will notice by the fact that no normal traffic reaches the other host.

I use the tcpdump to listen for esp traffic:

tcpdump -i eth0 esp

To listen to ah traffic, not in the first conf example:

tcpdump -i eth0 ah

There are som basic issues that I have encountered.

  1. The data is sent but not encrypted. When the data is sent unencrypted it will not be accepted by the other host since esp/transport//require requires the traffic to be esp encrypted. The add to the SPD might be the same on the two hosts.

  2. The data is sent encrypted but the receiver does not decrypt it. The receiver might not be running IPSEC, or does miss the SPD record for incoming traffic. The receiver/sender have the wrong keys, the keys might be swapped in se SAD. The SAD records should be exactly the same on the two machines. The parameters 31031 & 43241 is called SPI and it is sent in the IPSEC header to tell the the receiver what key to use to decrypt the message, it the SPI is not the same the kernel will not be able find the correct key.

  3. Staring setkey gives incorrect key length error If you forget the ; at the end of the add statement the setkey gives the error incorrect key lenght. If you forget the 0x before the hex the key is interpreted as ASCI and will therefore also have the wrong lenght.

SAD Security Association Database

The SAD is storing the keys and methods to use for a certain combiantion of IPs. And a SPI in this case 31031, is the ID of the combination of encryptionmethod and key to use.

add 192.168.0.3 192.168.0.2 esp 31031 -E rijndael-cbc 0xf86f44...;

The SAD is direction aware this means it understands what direction a packet is traveling, and thanks to this the add statement needs to be exactly the same both nodes.

SPD Security Policy Database

The SPD is storing when the kernel should encrypt the traffic. The SPD is not direction aware this means it has to be adopted to the node it is on.

That means that one node needs to say:

spdadd 192.168.0.2 192.168.0.3 any -P in ipsec esp/transport//require;

And the other node needs to say:

spdadd 192.168.0.3 192.168.0.2 any -P in ipsec esp/transport//require;

If it is mixed the data on one node will not be encrypted.

Manpage

I found the setkey manpage very helpfull and easy to follow!

Encryption

AES

For the ESP I use the rijndael-cbc algorithm also know as AES.

SHA1

When I want AH as well I use the hmac-sha256 algorithm.

Authentication Header

I did not use Authentication Header(AH), AH does not encrypt the content of the message but it makes sure that the sender is known and that the IP-header is untouched.

Encapsulating Security Payloads (ESP) used in my example, is used for encrypting the data in the packet.

This is a example of AH and ESP in combination:

#!/usr/sbin/setkey

# for host 192.168.0.3
# AH SAs using 160 bit long keys
add 192.168.0.3 192.168.0.2 ah 33362 -A hmac-sha256
            0x3308cef208a75c1048182e0a5b11e25a3308ce7a59a1a638ba3245c264538f4a;
add 192.168.0.2 192.168.0.3 ah 30759 -A hmac-sha256
            0x3fe6f3d57a58dcedd930de5818ed86be3fe6f33249a2c678bc324f12645f8f4b;

# Put the ESP SAs in the DB, 192 bit long keys
add 192.168.0.3 192.168.0.2 esp 31031 -E rijndael-cbc
                0xf86f440c0247a935c75be785486e535a90a636f0b189d588;
add 192.168.0.2 192.168.0.3 esp 43241 -E rijndael-cbc
                0xe9749d02219238454dd0dba8eae55155131c86c05a2f7d4e;

# Put the IPs in the Security Policy Database 
# to require the traffic to be encrypted
# with esp and ah
spdadd 192.168.0.3 192.168.0.2 any -P out ipsec
       esp/transport//require
       ah/transport//require;

spdadd 192.168.0.2 192.168.0.3 any -P in ipsec
       esp/transport//require
       ah/transport//require;                       

This is a example of AH and ESP in combination for the other host:

#!/usr/sbin/setkey

# for host 192.168.0.2
# AH SAs using 160 bit long keys
add 192.168.0.3 192.168.0.2 ah 33362 -A hmac-sha256
            0x3308cef208a75c1048182e0a5b11e25a3308ce7a59a1a638ba3245c264538f4a;
add 192.168.0.2 192.168.0.3 ah 30759 -A hmac-sha256
            0x3fe6f3d57a58dcedd930de5818ed86be3fe6f33249a2c678bc324f12645f8f4b;

# Put the ESP SAs in the DB, 192 bit long keys
add 192.168.0.3 192.168.0.2 esp 31031 -E rijndael-cbc
                0xf86f440c0247a935c75be785486e535a90a636f0b189d588;
add 192.168.0.2 192.168.0.3 esp 43241 -E rijndael-cbc
                0xe9749d02219238454dd0dba8eae55155131c86c05a2f7d4e;

# Put the IPs in the Security Policy Database 
# to require the traffic to be encrypted
# with esp and ah
spdadd 192.168.0.2 192.168.0.3 any -P out ipsec
       esp/transport//require
       ah/transport//require;

spdadd 192.168.0.3 192.168.0.2 any -P in ipsec
       esp/transport//require
       ah/transport//require;

References

Written 2016-04-06 by Martin Harari Thuresson