Eating My Own (Socket) Dog Food
Both QEMU and UML come with plenty of helper scripts and Web Lore to connect them to network via tap. There is precious little (if any) information on how to use alternative transports. As a result of this people continue to use tap despite the fact that it sucks asteroids sidewise through a microbore.
I am going to try to address it in this article. After all, what is the point in writing high performance socket drivers if nobody uses them.
NOTE: some of this is for pre-release drivers which were published mid-July 2017. I am trying to get them included, but I cannot guarantee anything as far as when they will end up upstream. You can get the most recent versions out of the qemu-devel and user-mode-linux-devel archives.
The key hurdle on which most people fail and give up is configuring the host side of the tunnel. So we will start with the instructions for that first.
GRETAP is relatively simple. It is set up via the /sbin/ip utility on Linux and is fairly well supported.
ip link add gt1 type gretap local 192.168.128.1 remote 192.168.129.1
This sets up a gretap tunnel from 192.168.128.1 to 192.168.129.1. The IP addresses must be different. If GRE is being used to connect a VM to the host it is running on the best approach is to set up a number of aliases (or additional ip addresses for the lo interface).
The tunnel above is the most basic one - it uses GRE in its pristine form without any extra bells and whistles like sequencing, checksums or keys.
In most cases that is more than sufficient when connecting a VM to its host. If, however, the VM is being connected to a remote system not under our control, we may need to add these.
ip link add gt0 type gretap local 192.168.128.1 remote 192.168.129.1 okey 0xdeadbeef ikey 0xbeefdead
sets up a link from 192.168.128.1 to 192.168.129.1 with the host using output key 0xdeadbeef and expecting an incoming key of 0xbeefdead.
Adding seq and csum to the options turns sequencing and checksumming respectively.
These magic incantations can be added to /etc/network/interfaces on Debian/Ubuntu or the corresponding RHEL/Centos/Fedora interface definitions as "pre-up" statements - they are needed to set up the tunnel.
: The tunnels so far are just link layers - they create Ethernet-like virtual devices. We still need to set the ip addresses on them, turn on DHCP, etc.
Here is an example of a complete GRE interface definition from my test rig:
iface gt0 inet static
pre-up ip link add gt0 type gretap local 192.168.128.1 remote 192.168.129.1 || true
down ip link del gt0 || true
I have used || true because this is a test rig (so that the commands always complete). Normally, this should not be necessary.
Compared to GRE L2TPv3
looks distinctly arcane. Most implementations have my favourite "bug" - more options than GNU ls. It is also not set up in one go - there can be multiple sessions in a single tunnel and each of them forms a distinct interface on Linux. For our purposes we will use it in a very static fashion where we cannot use this feature and there will be only one session per tunnel.
First we have to set up the tunnel. The tunnel definition includes the source, the destination and the protocol. In the case of UDP the source and destination specifications also include source and destination ports.
ip l2tp add tunnel remote 127.0.0.1 local 127.0.0.1 encap udp tunnel_id 2 peer_tunnel_id 2 udp_sport 1706 udp_dport 1707
Once we have set up the tunnel we can set up one or more sessions. You can use only one session for a static L2TPv3
tunnel to QEMU/kvm (from version 2.1 onwards) or UML (the old Cisco patches or the new version which I will release in June),
ip l2tp add session name l2tp1 tunnel_id 2 session_id 0xffffffff peer_session_id 0xffffffff
This example sets up an interface called l2tp1 which we can now use on the Linux side. I have omitted the more complex options like cookies, etc for sake of simplicity. They are not really needed to connect a VM under our control to a host we control (localhost in this case).
The equivalent /etc/network/interfaces stanza will be:
iface l2tp1 inet static
pre-up ip l2tp add tunnel remote 127.0.0.1 local 127.0.0.1 encap udp tunnel_id 2 peer_tunnel_id 2 udp_sport 1706 udp_dport 1707 && ip l2tp add session name l2tp1 tunnel_id 2 session_id 0xffffffff peer_session_id 0xffffffff
down ip l2tp del session tunnel_id 2 session_id 0xffffffff && ip l2tp del tunnel tunnel_id 2
With the host networking set-up correctly we can now attach the pseudowire directly to a VM,
- 01 Oct 2017