Categories
Linux and L2TP -- a peek under the hood (part 3)
This post forms the third and final episode in our three-part series on programming to the Linux kernel L2TP API. The first post can be found here, and described the architecture of an L2TP application on Linux. The second post went on to look at the details of writing to the kernel L2TP API.
In this post we're going to cover debugging and unmanaged tunnels.
Debugging the Linux L2TP API
When working on L2TP applications it's very useful to be able to examine the kernel state.
There are three ways to do so:
- using iproute2 to list tunnels and sessions
- using the L2TP subsystem's debugfs support
- using the L2TP subsystem's printk log output
Using iproute2 to examine state
The standard Linux iproute2 tool can send and receive netlink messages to examine the kernel's tunnel and session populations:
# List all tunnels
[bob@slinky ~]$ sudo ip l2tp show tunnel
Tunnel 32893, encap UDP
From 192.168.1.72 to 127.0.0.1
Peer tunnel 47244
UDP source / dest ports: 9201/5555
UDP checksum: enabled
# List all sessions
[bob@slinky ~]$ sudo ip l2tp show session
Session 7419 in tunnel 32893
Peer session 46176, tunnel 47244
interface name: l2tpeth0
offset 0, peer offset 0
The iproute2 tool has a good interactive help system you can use to explore how to use it, e.g.
[bob@slinky ~]$ ip l2tp help
For a less terse description the iproute2 ip-l2tp describes the command set available in more detail.
Using L2TP's debugfs interface
If the l2tp_debugfs module is loaded, the kernel debugfs system can be used to examine the L2TP subsystem:
[bob@slinky ~]$ sudo cat /sys/kernel/debug/l2tp/tunnels
TUNNEL ID, peer ID from IP to IP
L2TPv2/L2TPv3, UDP/IP
sessions session-count, refcnt refcnt/sk->refcnt
debug tx-pkts/bytes/errs rx-pkts/bytes/errs
SESSION ID, peer ID, PWTYPE
refcnt cnt
offset OFFSET l2specific TYPE/LEN
[ cookie ]
[ peer cookie ]
config mtu/mru/rcvseq/sendseq/dataseq/lns debug reorderto
nr/ns tx-pkts/bytes/errs rx-pkts/bytes/errs
TUNNEL 32893 peer 47244 from 192.168.1.72 to 127.0.0.1
source port 9201, dest port 5555
L2TPv3, UDP
1 sessions, refcnt 3/3
0000000f rx 0/0/0 rx 0/0/0
SESSION 7419, peer 46176, ETH
refcnt 2
config 1300/0/-/-/-/LAC 0000000f 0
offset 0 l2specific 0/0
0/0 tx 0/0/0 rx 0/0/0
interface l2tpeth0
In our example the debugfs is mounted in the conventional location of /sys/kernel/debug, but this does depend on your Linux distribution. You can easily check where your distribution mounts yours:
[bob@slinky katalix.com]$ mount | grep debugfs
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
Using the L2TP subsystem's printk logging
When creating tunnels and sessions you can use the L2TP_ATTR_DEBUG attribute to enable kernel debug logging for that instance.
The attribute is a bitmask made up of the flags called out in enum l2tp_debug_flags (defined in the Linux L2TP API header).
Logging is emitted using the kernel's printk mechanism, which can be examined using dmesg(1).
Unmanaged or static tunnels
In this series so far we've assumed that user space is going to deal with the L2TP control protocol, and will be using a tunnel socket to do so.
Tunnels created in this manner are known as "managed" tunnels.
Linux also supports another mode of operation for "unmanaged" (sometimes called "static") tunnels.
Managed v.s. unmanaged
When creating a managed tunnel, the user space tunnel socket is then passed to the kernel in the L2TP_ATTR_FD attribute of the L2TP_CMD_TUNNEL_CREATE netlink message.
By contrast, when creating an unmanaged tunnel, the L2TP_ATTR_FD attribute is omitted from the L2TP_CMD_TUNNEL_CREATE netlink message. The kernel creates its own socket to use for the data path only.
The benefit of unmanaged tunnels is that they can be very easily established, with no need for a user space daemon to manage the connection.
The obvious drawback is that unmanaged tunnels are very inflexible:
- both peers must agree on all tunnel and session parameters ahead of time
- both peers must run commands to bring up "their end" of the tunnel and each session
- should there be an issue with the connection there's no daemon to handle recovery
For practical reasons, only L2TPv3 tunnels and Ethernet sessions can currently be created as unmanaged instances.
Despite the limitations of unmanaged tunnels, for some niche use-cases they may be all you need. They also represent a good way of experimenting with the L2TP data path without needing to write a lot of code to implement the control protocol!
Creating an unmanaged tunnel using iproute2
The standard Linux iproute2 tool supports creation of unmanaged tunnels and sessions. For example:
[bob@slinky ~]$ sudo ip l2tp add tunnel local 192.168.2.24 remote 192.168.2.109 \
udp_sport 5555 \
udp_dport 6666 \
tunnel_id 1982 \
peer_tunnel_id 2001
[bob@slinky ~]$ sudo ip l2tp add session tunnel_id 1982 \
session_id 1 \
peer_session_id 903
This sets up a tunnel from local address 192.168.2.24 to remote address 192.168.2.109; carrying a session with local session identifier 1 and peer session identifier 903.
The session will be represented by a network interface (for example, l2tpeth0). Frames hitting that interface will be encapsulated in L2TP headers and transmitted off to 192.168.2.109.
Of course, this only represents one side of the connection! The peer would have to run corresponding commands to set up the data path there.
References and useful links
One of the motivations for this series of posts is that it's hard to find an open source Linux L2TP project to recommend as a reference for someone starting out trying to write a Linux L2TP application.
Typically, most existing projects use L2TPv2 with UDP encapsulation and PPP pseudowires, but don't support L2TPv3. Or if they do support L2TPv3, they don't support IP encapsulation, or they don't support PPP pseudowires. It's hard to find a single project which does everything!
However, existing projects can still be a very useful reference, and we encourage interested readers to study the source code for existing applications to see how they work.
- iproute2 uses the netlink API to create unmanaged UDP encapsulated L2TPv3 tunnels and Ethernet pseudowires. Experimenting with iproute2 is a very good way to play about with L2TP tunnels and sessions without having to write any code.
- xl2tpd supports kernel-mode tunnels and sessions using the sockets API. It supports L2TPv2 only, hence only UDP encapsulation and PPP pseudowires.
- l2tp-ktest has unit tests for most of the kernel's L2TP features (L2TPv2 and v3, sockets and netlink API, IP and UDP encapsulation).
In conclusion
This post concludes our three part series on the Linux L2TP API.
In the first post we presented an architecture for L2TP daemons on Linux. In the second post we looked at coding to the L2TP netlink API. Finally in this post we've examined debugging techniques and unmanaged tunnels.
We hope this series has helped illustrate how the Linux L2TP API can be used to write client and server L2TP applications.
Before you go, one last mention of our ProL2TP suite of applications.
ProL2TP is a production-ready suite of daemons supporting L2TPv2 and L2TPv3, PPP, and PPPoE. It is scalable from small network devices up to big servers, and we've a range of licensing options, including source code licensing, to suit your deployment. Take a look at ProL2TP's feature set to see whether it could be a good fit for you.
Thanks for reading!