Do you use let's encrypt?

1767 votes ~ 11 comments


Joining disparate hosts into a VPN with gvpe

Posted by Steve on Wed 20 Nov 2013 at 22:04

Tags: ,

The GNU Virtual Private Ethernet software allows you to join multiple distinct hosts into a small private network, via a static set of public/private keys. It is ideally suited to joining a small number of hosts in a secure fashion.

I use gvpe to allow all the virtual machines I run to have a secure link between each other. Imagine you have a typical cluster:

  • 2-3 webservers.
  • 1-2 database servers.

If you assign each host a private address and configure them all to be in a VPN then traffic between them will be encrypted. That means even if your hosting company were to sniff your traffic they wouldn't ever see your database queries over the wire. (Of course if you don't trust your hosting company you've got bigger problems; they could clone your disks, subvert your hardware, or worse.)

I chose gvpe because it is very simple to setup, has a low overhead, and can be configured easily for a small and static list of hosts.

Installation of the software is very simple:

apt-get install gvpe

Once that is done we'll need to configure it. That involves three simple steps:

  • Writing a configuration file, containing a list of all the nodes.
  • Generating and configuring the keys used.
  • Writing a simple shell-script to be invoked when the VPN is established.

The first is simple. In our case we're going to configure a VPN between the two hosts and So our configuration file /etc/gvpe/gvpe/gvpe.conf will look like this:

enable-udp = yes
udp-port   = 407
ifname     = vpn0

node     = one
hostname =

node     = two
hostname =

The configuration has some boiler-plate at the top to configure how the VPN will be setup, in our case using UDP traffic over port 407, and the name of the networking devices on each node (vpn0).

Now that we've configured the list of nodes we need to generate the keys:

# gvpectrl --config /etc/gvpe/ --generate

This will poluate the two directores /etc/gvpe/hostkeys, and /etc/gvpe/pubkey with some static keys, which are used for the encryption, and session negotiation. You need to generate the keys on only one node, then copy them to any other nodes - Every node must have identical keys.

In addition to creating keys for all nodes in the mesh we need to ensure that the host key for the local host is present. So we'd run:

# On the first node
root@one:~# ln -s /etc/gvpe/hostkeys/one /etc/gvpe/hostkey

# On the second node.
root@two:~# ln -s /etc/gvpe/hostkeys/two /etc/gvpe/hostkey

(i.e. The keys are named after the node-name, but we need to ensure the key for the current host is located at /etc/gvpe/hostkey.)

The final step is to write a simple shell-script which will be launched when the link(s) are established. Create the file /etc/gvpe/if-up, with contents like so:


# Set the MTU + MAC address
ip link set $IFNAME address $MAC mtu $MTU up

# Add an IP to the 'vpn0' device.
ip add add dev $IFNAME

# Route the range via this device.
ip route add dev $IFNAME

NOTE: The script here will give the IP address to the node it runs on. To be more dynamic, but still allow /etc/gvpe to be identical on all nodes you could rework the script like so:


# Set the MTU + MAC address
ip link set $IFNAME address $MAC mtu $MTU up

# Add an IP to the 'vpn0' device - give a different IP for each node
case $NODENAME in
     ip add add dev $IFNAME
     ip add add dev $IFNAME

# Route the range via this device.
ip route add dev $IFNAME

This script works because the nodename of the current host will be passed along with the other variables you see referenced such as $IFNAME, and $MAC.

By the time you've reached this step you'll have the following structure:

├── gvpe.conf  - The configuration file.
├── hostkey    - The key for this host, linked to from ./hostkeys
├── hostkeys
│   ├── one    - Private keys for the nodes.
│   └── two
├── if-up      - The script to launc the devices and configure the routing.
└── pubkey
    ├── one    - Public keys for the nodes.
    └── two

2 directories, 7 files

The only remaining step is to launch the link:

# On one side.
root@one:~# gvpe --config /etc/gvpe/ -D one

# On another side.
root@two:~# gvpe --config /etc/gvpe/ -D two

The launch command includes the name of the current node, which is why we prefer to name nodes after hostnames. It makes it nice and clear. If all goes well you'll see something like this:

root@one ~ # gvpe --config /etc/gvpe/ -D one
gvpe daemon 2.24 (Nov 15 2011 20:40:14) starting up.
two(udp/x.x.x.x:407): connection established (direct), protocol version 0.1.

Repeat the matching command on the other side and you'll have a two-way link, which you can verify via:

root@one ~ # ip -4 addr list dev vpn0
19: vpn0:  mtu 1422 ..
    inet scope global vpn0

root@one ~ # ip -4 route list dev vpn0  scope link

Finally to allow the daemon to be launched on bootup you can update /etc/default/gvpe to read:

DAEMON_ARGS="--config /etc/gvpe/ one"

Here I'm using "one" as the node-name to be nice and clear. In my case I name nodes after the local hostnames, so I can instead write ...$(hostname --short)"

At the end of this article you should be able to quickly setup a simple VPN containing 2+ hosts. There are many more options you can experiment with, which are documented via "man gvpe.conf", "man gvpe", and "man gvpectrl".



Re: Joining disparate hosts into a VPN with gvpe
Posted by Anonymous (149.171.xx.xx) on Thu 21 Nov 2013 at 03:47

I've done a similar thing with tinc and it works really well too. The steps are mostly the same except:

  • you have a different key for each host -- you copy the public part of machine the key around to any host you want to be able to connect
  • the networking setup is less work -- it's just set the ip address of the node in an ifup script (you can do lots more funky stuff and have it route and gateway etc if you want)
  • tinc is a mesh network -- any node can make a direct connnection to any other node if you want it to and you can transparently use multiple hops to get indirect connections if want or need to

I have tinc set up on all my machines and I can then reach any other machine as needs be. I work in a few NATed environments so I try to have a tinc inside each of these NATs that's permanently on and connected to others and then laptops that move from one NATed environment to another just connect to whatever local tinc they can see. I can always get into the machine inside these NATs from outside when I need to over the VPN too, using an indirect connection through tinc. Given that the mesh could be quite complicated, I have tinc write out a .dot file for me whenever the graph changes... very useful when I was first learning about it, but mostly I look at it now only for amusement purposes.

I have no idea if tinc is any better or worse than gvpe... I'll be interested to hear how things go in the long run for you!

[ Parent | Reply to this comment ]

Re: Joining disparate hosts into a VPN with gvpe
Posted by Steve (94.3.xx.xx) on Fri 22 Nov 2013 at 07:56
[ View Steve's Scratchpad | View Weblogs ]

Personally I think it is an advantage you get to write a shell script for gvpe, as you say for tinc it allows you to do a lot - you can add/remove routes, restart services, etc. My setup makes sure that the VPN address uses th last octet of the main IP because that makes it simple to remember:

# Add an IP to the 'vpn0' device.
# We take the last octet of eth0 to use
octet=$(grep address /etc/network/interfaces  | awk -F. '{print $4}')
ip add add 10.0.0.$octet dev $IFNAME

(For example the host will get the VPN address

There are also hooks to call scripts when nodes join/leave the VPN too, which are documented in the "gvpe.conf" manpage.

But yes, if I were to compare gvpe to anything it would be tinc, and both are good simple to configure and use.


[ Parent | Reply to this comment ]

Re: Joining disparate hosts into a VPN with gvpe
Posted by Anonymous (88.97.xx.xx) on Sun 24 Nov 2013 at 18:30
You can also use $NODEID to simplify your script

ip link set $IFNAME address $MAC mtu $MTU up
ip addr add 192.168.13.$NODEID/24 dev $IFNAME

[ Parent | Reply to this comment ]

Re: Joining disparate hosts into a VPN with gvpe
Posted by Anonymous (149.171.xx.xx) on Mon 25 Nov 2013 at 23:22

You set up the IP layer in the same way with tinc:

$ cat /etc/tinc/*/tinc-up 

ifconfig $INTERFACE netmask

(yeah sure, I should use ip(8) and could do more funky things with addresses and netmasks and... meh)

There's also a tinc-down script although I've not written any.

[ Parent | Reply to this comment ]

Re: Joining disparate hosts into a VPN with gvpe
Posted by ajt (92.25.xx.xx) on Fri 22 Nov 2013 at 21:06
[ View Weblogs ]

Brilliant timing, I was looking at GVPE earlier this week and wanted an example to think about...

"It's Not Magic, It's Work"

[ Parent | Reply to this comment ]

Filtering MDNS
Posted by Anonymous (88.97.xx.xx) on Sun 8 Dec 2013 at 09:46
When you run gvpe on a desktop machine you will see that the connections go up immediately even if you set connect=ondemand. I found this is because of service advertisements on MDNS. There are two ways to stop these advertisements:
  1. create an iptables OUTPUT rule to block the specific port numbers/addresses
    iptables --append OUTPUT ... --jump DROP
  2. disable multicast completely before adding the address:
    ip link set $IFNAME address $MAC mtu $MTU up
    ip link set dev $IFNAME multicast off
    ip addr add 10.0.0.$NODEID/8 dev $IFNAME   # or whatever IP range

[ Parent | Reply to this comment ]

Re: Joining disparate hosts into a VPN with gvpe
Posted by Anonymous (150.164.xx.xx) on Mon 22 Sep 2014 at 20:09

When starting gvpe I was getting

no protocols enabled. unrecoverable error while setting up network, exiting. terminating with exit code 1

It fixed adding on gvpe.conf

enable-rawip = true

[ Parent | Reply to this comment ]