Ever on and on

In my first post in this series I described an auto-connecting setup for Cisco VPNs using OpenWRT and vpnc, and in my second post I showed the details of how I did it. To wrap it up, here are some thoughts about future improvements.

One thing that bugs me about this setup is that I hardcoded some IP addresses. This is a really bad idea, in fact half the reason DNS exists is to stop people from doing this. Someday the VPN administrators could decide to renumber their gateway, and then I would have to connect to each and every WRT and update the configuration, or all my customers would be cut off from the VPN.

I could stop hardcoding the VPN gateway pretty easily, by having it auto-update whenever the WRT boots. Look up the correct IP address and write it to /etc/hosts. (You need to be careful not to trigger the VPN autoconnect when you do the lookup, of course.)

The DNS servers are a little harder to fix. To look them up, we need to actually connect the VPN, and then pull out the nameservers that the VPN tells us to use. But this could be done automatically each time the VPN is connected, and it would make the whole setup more robust. Don’t forget to restart dnsmasq after you reconfigure it.

Another thing that bothers me is that there are a lot of gyrations before the VPN gets connected: a packet goes to a private IP address, matches on the vpn_trigger rule, which queues a message for the ulog daemon, which writes to a file, which causes the file to exist, and also causes tail to create some output, which finally causes the script to start vpnc. Whew!

One suggestion is to use inotify to watch the file, which is a good idea but still requires a file. (Also I don’t see inotify in the OpenWRT Kamikaze package list.) Another possibility would be to write a ulog plugin so that I can omit the file altogether. That might make VPN startup quicker, which would be a good thing.

Finally, if a customer wanted it, there are more sophisticated setups possible. It might be useful to connect to more than one VPN (either serially or at the same time), or to have the different LAN ports on the OpenWRT connect to different VPNs. All of these are possible if the scenario warrants it.

What are your ideas?

In my last post I outlined a design for auto-connecting Cisco VPNs using OpenWRT and the vpnc client. In this post I’ll share the code, and highlight a couple of details. Finally, in my next post, I’ll share some thoughts on improving these scripts.

This process requires some knowledge about your VPN setup. To keep my post from getting too long, I’m assuming that you know your VPN domain name and IP address range, that you can quickly figure out the IP addresses of hosts and DNS servers, and that you have a working vpnc config file.

The first step is installing additional packages on OpenWRT. You can install these from the web interface, or using opkg install at the OpenWRT shell:

1. vpnc, the Cisco VPN client
2. ulogd-mod-extra, which pulls down the ulog daemon
3. kmod-ipt-ulog, kernel modules for iptables and ulog
4. iptables-mod-ulog, part of the tool for adding rules to iptables

Next, we need to make sure you can always reach the VPN gateway host. So we configure it into /etc/hosts. Look up the IP address (using nslookup, dig, or a similar tool) and add a line to /etc/hosts like:

aaa.bbb.ccc.ddd vpn.example.org


Now that we know we can reach the VPN gateway, we will redirect dnsmasq to always use the internal servers for the VPN’s domain. Look up the domain nameservers using nslookup or dig from inside the VPN, or just look at the nameservers in /etc/resolv.conf when you’re connected from your PC. Then edit /etc/config/dhcp on OpenWRT and add lines like this:

# EXAMPLE.ORG private servers
list server '/example.org/aaa.bbb.ccc.ddd'
list server '/example.org/aaa.bbb.ccc.eee'


Now, we need the script that will manage the VPN connection. Cut and paste this code into /usr/bin/autostart-vpn.sh:

#!/bin/sh
#
# Autostart vpnc
#
# From http://www.kaufmanfamily.net/blog/2010/05/how-to-auto-connect-a-cisco-vpn-with-openwrt
# DHK 4/17/2010

MYPID=/var/run/autostart-vpnc.pid
LOGFILE=/var/log/ulogd.syslogemu
PIDFILE=/var/run/vpnc/pid

is_vpn_connected() {
        connected=0
        if [ -s $PIDFILE ]; then
                ps=`ps | awk -v pid=\`cat $PIDFILE\` '$1 == pid && $5 == "vpnc" { print $0 }'`
                if [ -n "$ps" ]; then
                        connected=1
                fi
        fi
}

# Fill in our PID file
echo $$ > $MYPID

# Loop, monitoring the VPN
while true; do
	is_vpn_connected

	if [ $connected -eq 0 ]; then
		# VPN is not connected. Wait for a request, then start it

		# Wait for a log message, denoting that someone is trying to connect
		while [ ! -e $LOGFILE ]; do
			sleep 10
		done
		pkt=`tail -0 -f $LOGFILE | head -1`
		logger "autoconnect[$$] connecting due to $pkt"

		# Start the vpn
		date >> /tmp/autoconnect-vpnc.log
		vpnc >> /tmp/autoconnect-vpnc.log

		# Let the VPN settle
		sleep 2
	else
		# VPN is connected. Wait for it to drop, then clean up

		# Wait for the VPN to disconnect
		while [ $connected -eq 1 ]; do
			is_vpn_connected

			# sleep 10 seconds, check again
			sleep 10
		done
		logger "autoconnect[$$] disconnected"

		# Clean up route table
		if [ -f /var/run/vpnc/defaultroute ]; then
			outsidegw=`awk '{print $3}' /var/run/vpnc/defaultroute`
			currentgw=`netstat -rn | awk '$1 == "0.0.0.0" && $4 == "UG" {print $2}'`
			if [ "X$outsidegw" != "X$currentgw" ]; then
				if [ "X$currentgw" != "X" ]; then
					route delete default gw $currentgw
				fi

				# Restore exactly what was saved, except vpnc syntax is slightly wrong
				route add `sed -e 's/via/gw/;' /var/run/vpnc/defaultroute`
			fi
		fi

		# Clean up resolv.conf
		resolvconf -d
	fi
done

We’re almost there. Cut and paste the following code into /etc/init.d/autostart-vpn; this is the startup script that creates the iptables rules and starts the last script at boot time. Make sure you edit the script to list the correct networks for your VPN, and check that the locations (hardcoded, unfortunately) for inserting vpn_trigger in the FORWARD and OUTPUT rulesets makes sense:

#!/bin/sh /etc/rc.common
#
# From http://www.kaufmanfamily.net/blog/2010/05/how-to-auto-connect-a-cisco-vpn-with-openwrt
# DHK 4/17/2010
#
# Start after dnsmasq
START=80

VPN_NETWORKS="10.0.0.0/8 192.152.0.0/16"
EXT_IF="eth0.1"

check_firewall_rules() {
	trigger=`iptables -L vpn_trigger 2>/dev/null`

	if [ "X$trigger" == "X" ]; then
		# Fill in vpn_trigger ruleset
		iptables -N vpn_trigger

		for net in $VPN_NETWORKS; do
			iptables -A vpn_trigger -o $EXT_IF --dest "$net" -j ULOG
		done

		# Hook vpn_trigger into OUTPUT and FORWARD rules
		# Ought to do something smarter than hardcoding the position
		iptables -I OUTPUT  4 -j vpn_trigger
		iptables -I FORWARD 4 -j vpn_trigger
	fi
}

start() {
	check_firewall_rules

	/usr/bin/autostart-vpnc.sh &
}

stop() {
	if [ -f /var/run/autostart-vpnc.pid ]; then
		kill `cat /var/run/autostart-vpnc.pid`
	fi
}

One detail I skipped last time is that vpnc, as packaged for OpenWRT Kamikaze, will stomp on your resolv.conf file. Its default configuration just doesn’t work on OpenWRT. (The issue is that OpenWRT puts the WAN resolv.conf details in a non-standard place.) There’s an easy fix for this, though. Cut and paste the following code into /sbin/resolvconf. vpnc will find resolvconf and use it to manage /etc/resolv.conf correctly.

#!/bin/sh
#
# Simple resolvconf manager to integrate vpnc better with WRT
#
# Update /tmp/resolv.conf.auto, NOT /etc/resolv.conf. This affects the
# DNS resolver operation, which is actually the right thing.
#
# Usage:
#
#   resolvconf -a [if] < new-resolvconf
#
#   resolvconf -d [if
#
# We ignore the [if] argument.
#
# From http://www.kaufmanfamily.net/blog/2010/05/how-to-auto-connect-a-cisco-vpn-with-openwrt
# DHK 4/13/2010

BACKUP=/tmp/resolv.conf.bak
RESOLV=/tmp/resolv.conf.auto

if [ $1 == "-a" ]; then
	# Change resolv.conf

	if [ ! -e $BACKUP ]; then
		cp $RESOLV $BACKUP
	fi

	cat > $RESOLV
fi

if [ $1 == "-d" ]; then
	# Restore original resolv.conf

	if [ -e $BACKUP ]; then
		mv $BACKUP $RESOLV
	fi
fi

Now is a good time to make sure you’ve installed your VPN configuration into /etc/vpnc/default.conf. It’s a good idea to test out your vpnc config on another machine before running it on OpenWRT.

OK, let’s enable the services we need. You can do this from the OpenWRT web interface, or the command line:

/etc/init.d/ulogd enable
/etc/init.d/autostart-vpnc enable


Reboot your OpenWRT to get all the services set up. You’ll want to watch the system message log, so in one ssh connection run the log reader:

logread -f


and in another ssh connection start pinging a host in the VPN:

ping somehost.example.org


You should see a message in the system log, and after a short delay you’ll start getting ping responses. Make sure to test the auto-connect from a host plugged in to your OpenWRT’s LAN port as well as from the shell: if auto-connect works directly from the OpenWRT shell, but not from the LAN, then your iptables OUTPUT rule is correct but your FORWARD rule isn’t. (If the problem is reversed, then the rules are reversed.)

If auto-connection doesn’t work, you can check the log at /tmp/autoconnect-vpnc.log and then debug the process step-by-step:

1. First, check that your vpnc configuration works:
   vpnc
2. Then, check that the vpn_trigger iptables rule is being called by looking at the packet counts:
   iptables -L vpn_trigger -v
3. If vpn_trigger is being called, make sure that ulogd is writing to the correct file:
   cat /var/log/ulogd.syslogemu
4. check that the autoconnect script is actually running with ps

The hardest thing to check is that you have your DNS setup correct. I usually do this by checking the vpn_trigger rules first, then use nslookup to query a behind-the-vpn host.

vpnc is an open source client for Cisco VPNs, which is pretty widely used. It’s intended as a user-driven interface: user decides to connect, user runs vpnc (or one of the many GUI wrappers around it) to connect, user uses remote resources, user disconnects. But for some users, manually connecting & disconnecting the VPN is a burden. I developed an auto-connecting VPNC setup for just such a customer scenario.

I started with OpenWRT (of course!) and vpnc. So far I’ve deployed this setup using OpenWRT Kamikaze, versions 8.09.1 and 8.09.2. I’ll describe how the process works first, and then show the details in my next post.

The first thing we need to do is intercept DNS queries for the secure network. (Most secure networks will have a private DNS server behind the firewall, with lots of information that’s not in the public version.) OpenWRT uses the dnsmasq name resolver, and happily dnsmasq provides for exactly this case. So, we configure dnsmasq to know that the servers for the VPN domain are on its private addresses, and not to use the public servers.

Some companies might have the name of the VPN gateway in the same domain as their private, through the VPN environment. This could be a problem: how do we find the IP address for the gateway, if we need the VPN to talk to the domain servers? The easy answer for now is to just hardcode it in /etc/hosts on the OpenWRT.

OK, so we dealt with DNS name resolution, but how do we get the IP packets to go through the VPN? Actually, once vpnc connects, it will set up routing on the OpenWRT for us. So the only tricky bit is starting up vpnc at just the right time.

One way to do this would be to start vpnc whenever the WRT boots. This works, pretty much, but might be considered hostile by the VPN administrators. Instead we want to start the vpn tunnel whenever a user tries to access a machine in the private environment. To do this, we need to capture IP packets being routed to private IP addresses. This sounds like a job for iptables!

iptables is part of the Linux kernel which lets you define rules for handling IP packets. It does all kinds of cool things – filtering, QOS, NAT – but all we need it to do is tell us when someone is trying to connect through the VPN, so that we can start up the VPN tunnel. To do this, I used the iptables “ULOG” target: ULOG writes a message to a userspace daemon called ulogd, which acts like a hub (kind of like syslogd) and routes the messages to other programs. I kept it simple and just wrote the message to a file.

To tie it all together, I wrote a script which monitors the ulog file. The script gets automatically started at boot time: if the VPN isn’t up yet, it just monitors the ulog file. When a packet trace appears in the ulog file, we know it’s time to start the VPN up. Once the VPN is connected, the script hangs out, waiting for it to get shut down — either manually or by an idle timer. If the VPN gets shut down, then the script goes back to watching the ulog file.

OK, so we have DNS requests going to the private servers, and IP traffic causes the VPN to autoconnect. We’re done, right? Well, almost. It turns out that sometimes vpnc shuts down messily. It can leave the OpenWRT without a default route, or it can leave the resolv.conf file pointing to behind-the-tunnel servers. Either one of these problems is enough to prevent reconnecting the VPN! So I added a cleanup step to my script, which makes sure that vpnc has cleaned up after itself.

Add some logging and a simple web interface, and we’re all set. Details in my next post.