Categories
Linux and L2TP -- a peek under the hood (part 2)
This post forms the second instalment of a three-part series on programming to the Linux kernel L2TP API. The first part can be found here, and described the architecture of an L2TP application on Linux.
In this post we're going to move on to look at the details of writing to the kernel L2TP API.
In order to simplify the code presented here, we're using blocking sockets, and we're mostly ignoring error conditions. Of course a real application would be more careful!
Driving the L2TP API
As we discussed in the first post in this series, L2TP applications on Linux are split such that a user space process (daemon) handles the L2TP control protocol, while the kernel L2TP subsystem handles bulk data transmission.
The process of creating a new L2TP tunnel starts with the tunnel socket. The user space daemon uses this socket to exchange L2TP control messages with the peer in order to establish the tunnel.
Once the tunnel is established with the peer, the user space daemon must send commands to the kernel to create a tunnel context in the kernel's L2TP subsystem. These commands take the form of netlink messages.
Make sure the L2TP kernel modules are loaded
Before trying to talk to the Linux L2TP API, you need to ensure that the L2TP subsystem is loaded and running in your kernel.
Most modern Linux distributions package the L2TP subsystem as optional modules which can be loaded using modprobe:
[bob@slinky ~]$ sudo modprobe l2tp_eth l2tp_ip l2tp_ip6 l2tp_netlink l2tp_ppp
If you don't have the L2TP modules available, it's likely that they're packaged separately. For example, for Ubuntu the L2TP modules are a part of the linux-modules-extra package, which can be installed using apt-get:
[bob@slinky ~]$ sudo apt-get install linux-modules-extra-$(uname -r)
To check that the modules are loaded, you can use lsmod:
[bob@slinky ~]$ lsmod | grep l2tp
l2tp_eth 16384 0
l2tp_netlink 24576 1 l2tp_eth
l2tp_ip 20480 0
l2tp_debugfs 16384 0
l2tp_core 32768 4 l2tp_debugfs,l2tp_eth,l2tp_ip,l2tp_netlink
ip6_udp_tunnel 16384 1 l2tp_core
udp_tunnel 16384 1 l2tp_core
In more recent kernels, the L2TP modules may be automatically loaded when an application tries to use them.
Open the tunnel socket
To create a tunnel socket, use the standard POSIX socket API.
For example, for a tunnel using IPv4 addresses and UDP encapsulation:
int sockfd = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
Or for a tunnel using IPv6 addresses and IP encapsulation:
int sockfd = socket(AF_INET6, SOCK_DGRAM, IPPROTO_L2TP);
The IPPROTO_L2TP protocol
Eagle-eyed readers will have noticed that there are two socket protocols in use in the code snippets above: IPPROTO_UDP is used when creating a UDP-encapsulated tunnel; while IPPROTO_L2TP is used when creating an IP-encapsulated tunnel.
IPPROTO_L2TP is defined in the Linux L2TP API header.
The use of IPPROTO_L2TP goes hand in hand with some special sockaddr structures, namely struct sockaddr_l2tpip for AF_INET sockets, and struct sockaddr_l2tpip6 for AF_INET6 sockets.
This difference has to do with the technical details of how the UDP and IP encapsulation data paths are implemented in the kernel.
The practical implications of this for user space is that we must pay attention to the different sockaddr structures when using IP-encapsulated tunnels.
Binding and connecting the tunnel socket
Before using the tunnel socket, it is necessary to bind and connect it in order to set the socket local and peer address.
For UDP encapsulation this is a relatively straight-forward affair. A struct sockaddr must be built, and then passed to the bind(2) system call (to set the local address); or to the connect(2) system call (to set the peer address).
We recommend using the getaddrinfo(3) system call to obtain the sockaddr structure since this allows for DNS lookups, and also correctly populates the IPv6 address structure with flow and scope information.
For IP encapsulation things are slightly more complicated because user space must create an L2TPIP address structure (either struct sockaddr_l2tpip for IPv4/AF_INET or struct sockaddr_l2tpip6 for IPv6/AF_INET6) when making the bind system call. The structure returned by getaddrinfo must be augmented by the L2TP tunnel ID.
These example code snippets demonstrate address lookup and the bind call for two different scenarios. Note that the address length passed to the bind call depends on the type of address structure which is being passed!
/* Look up IPv4 address for UDP encap */
{
struct addrinfo hints = {
.ai_family = AF_INET,
.ai_protocol = IPPROTO_UDP,
.ai_socktype = SOCK_DGRAM,
};
const char *node = "172.24.1.55";
const char *service = "5555";
struct addrinfo *res, *addrs;
struct sockaddr_storage out = {};
socklen_t out_len = 0;
if (0 == getaddrinfo(node, service, &hints, &addrs)) {
for (res = addrs, res; res = res->ai_next) {
if (res->ai_family == AF_INET) {
memcpy(&out, res->ai_addr, res->ai_addrlen);
out_len = res->ai_addrlen;
break;
}
}
/* bind the tunnel socket to the address */
if (out_len) {
bind(sock_fd, (struct sockaddr *)&out, out_len);
}
freeaddrinfo(addrs);
}
}
/* Look up IPv6 address for IP encap */
{
struct addrinfo hints = {
.ai_family = AF_INET6,
.ai_protocol = IPPROTO_UDP,
.ai_socktype = SOCK_DGRAM,
};
const char *node = "2001:db8:fe12::";
const char *service = "5555";
struct addrinfo *res, *addrs;
struct sockaddr_storage out = {};
socklen_t out_len = 0;
if (0 == getaddrinfo(node, service, &hints, &addrs)) {
for (res = addrs, res; res = res->ai_next) {
if (res->ai_family == AF_INET6) {
memcpy(&out, res->ai_addr, res->ai_addrlen);
/* Augment address with tunnel ID */
{
struct sockaddr_l2tpip6 *l2tpip6 = (void*)&out;
/* We're using 42 as an example value: you'd need to use the ID of your tunnel */
l2tpip6->l2tp_conn_id = 42;
out_len = sizeof(struct sockaddr_l2tpip6);
}
break;
}
}
/* bind the tunnel socket to the address */
if (out_len) {
bind(sock_fd, (struct sockaddr *)&out, out_len);
}
freeaddrinfo(addrs);
}
}
We won't detail code for making the connect call since it's fundamentally the same procedure. Look up the address using getaddrinfo, augment with the L2TP connection ID for an IP-encapsulated tunnel, and then make the connect system call.
Making a netlink connection
Once the tunnel socket has been created and L2TP control messages exchanged with the peer to establish the tunnel, we can start sending netlink commands to create tunnel and session instances in the kernel.
The first step in sending a netlink command to the kernel is to open a netlink socket.
Here's how to open a generic netlink socket using libnl, and resolve the L2TP protocol name to the numeric ID which we'll need later when sending messages:
struct nl_sock *nl_sock;
int l2tp_nl_family_id;
nl_sock = nl_socket_alloc();
genl_connect(nl_sock);
l2tp_nl_family_id = genl_ctrl_resolve(nl_sock, L2TP_GENL_NAME);
Note that the netlink socket is used only to manage and query L2TP state in the kernel. L2TP control packets use the tunnel UDP or L2TPIP socket. An application may use a single netlink socket to manage any number of L2TP tunnel and session instances.
Before sending any netlink messages, don't forget that you can check the API documentation and the kernel netlink handers to determine what attributes are required for each message type.
Creating a tunnel instance
Having opened the netlink socket, we are now able to send netlink messages in order to create tunnels and sessions.
The netlink message L2TP_CMD_TUNNEL_CREATE is used to create a tunnel instance in the kernel.
struct nl_msg *msg = nlmsg_alloc();
int ret;
genlmsg_put(msg, NL_AUTO_PID, NL_AUTO_SEQ, l2tp_nl_family_id, 0, NLM_F_REQUEST,
L2TP_CMD_TUNNEL_CREATE, L2TP_GENL_VERSION);
/* we would use 2 for L2TPv2, or 3 for L2TPv3 */
nla_put_u8(msg, L2TP_ATTR_PROTO_VERSION, 3);
/* the file descriptor of the tunnel socket */
nla_put_u32(msg, L2TP_ATTR_FD, sockfd);
/* our tunnel ID */
nla_put_u32(msg, L2TP_ATTR_CONN_ID, 42);
/* the peer's tunnel ID, which would be determined by the control protocol */
nla_put_u32(msg, L2TP_ATTR_PEER_CONN_ID, 2001);
/* for a UDP-encapsulated tunnel */
nla_put_u16(msg, L2TP_ATTR_ENCAP_TYPE, L2TP_ENCAPTYPE_UDP);
ret = nl_send_auto_complete(nl_sock, msg);
if (ret > 0) nl_wait_for_ack(nl_sock);
nlmsg_free(msg);
The sockfd we pass in L2TP_ATTR_FD is the file descriptor of the tunnel socket.
Creating a session instance
The netlink message L2TP_CMD_SESSION_CREATE is used to create a session instance in the kernel.
Note that you need to supply the parent tunnel's ID when creating the session.
struct nl_msg *msg = nlmsg_alloc();
int ret;
genlmsg_put(msg, NL_AUTO_PID, NL_AUTO_SEQ, l2tp_nl_family_id, 0, NLM_F_REQUEST,
L2TP_CMD_SESSION_CREATE, L2TP_GENL_VERSION);
/* local and peer tunnel ID for the parent tunnel */
nla_put_u32(msg, L2TP_ATTR_CONN_ID, 42);
nla_put_u32(msg, L2TP_ATTR_PEER_CONN_ID, 2001);
/* local session ID */
nla_put_u32(msg, L2TP_ATTR_SESSION_ID, 100);
/* the peer's session ID, which would be determined by the control protocol */
nla_put_u32(msg, L2TP_ATTR_PEER_SESSION_ID, 200);
/* the pseudowire type */
nla_put_u16(msg, L2TP_ATTR_PW_TYPE, L2TP_PWTYPE_ETH);
/* there are many other session options which can be set using netlink
attributes during session creation: review the kernel code to see the
full range! */
ret = nl_send_auto_complete(nl_sock, msg);
if (ret > 0) nl_wait_for_ack(nl_sock);
nlmsg_free(msg);
We're only using the bare minimum of the available session parameters here. You can refer to the kernel netlink handlers for the L2TP subsystem for reference on what other attributes can be set.
Other commands are available!
We've given you a whistle-stop tour of the API to show how to instantiate tunnel and session instances.
Of course, the API offers more commands than just these. You can also:
- delete tunnels and sessions using
L2TP_CMD_TUNNEL_DELETEandL2TP_CMD_SESSION_DELETE - modify attributes using
L2TP_CMD_TUNNEL_MODIFYandL2TP_CMD_SESSION_MODIFY - examine state using
L2TP_CMD_TUNNEL_GETandL2TP_CMD_SESSION_GET
While we'd love to explore all the commands more fully, hopefully what we've covered so far should give you enough information to continue to investigate the API under your own steam.
Next time...
In this post we've looked at how to create a tunnel socket for a user space daemon to use, and how to create tunnel and session contexts in the kernel.
Don't forget, if you're looking for an L2TP solution for a Linux system, our ProL2TP might fit the bill.
ProL2TP is a production-ready suite of daemons supporting L2TPv2 and L2TPv3, PPP, and PPPoE. It is scalable from small network devices up to big servers, and we've a range of licensing options, including source code licensing, to suit your deployment. Take a look at ProL2TP's feature set to see whether it could be a good fit for you.
In the third and final post of this series we'll look at how to debug the Linux L2TP API, and discuss unmanaged tunnels.