Ever on and on

I’m running multi-homed while I exercise FiOS and make sure everything works. It took a few tricks to make everything work. My Linux box actually has three Ethernet NICs:

• eth0, the onboard 100M NIC, for FiOS
• eth1, a 10M NIC, for RCN
• eth2, a 1G NIC, for the internal LAN

The first thing to do is adjust your firewall. I run iptables, with very few dependencies on the external IP address. I did have to add some rules to the FORWARD and INPUT chains for the FiOS NIC - but only for network 192.168.0.0/16, because of the d-link firewalling. Things will get more interesting if/when the d-link comes out of the network, because I’ve seen VZ allocate IP addresses from both net 70 and net 71. I’ll probably change things around so the network-specific rules get changed dynamically, as the IP address changes.

The next thing you have to do is set up per-interface route tables. If you don’t do this then only one external IP will actually work, because a packet may be received on NIC A, but its reply be routed out NIC B. This especially doesn’t work in a NATted environment! A good reference to get you started is an article in Linux Journal.

The third thing is to make sure that your various services don’t rely on specific IP addresses. DNS (BIND) and Apache configuration are good places for this dependency to sneak in. I’ve been around this block before, so I didn’t have those problems. I did get bitten by my CUPS configuration, because it “knew” that eth1 was the LAN interface - when the LAN moved to eth2, CUPSd was broadcasting its notifications on the wrong network.

Email is another challenge. I’m running my outbound email server (postfix) bound to the RCN IP address for the time being. (The dynamic Verizon IP addresses are running into SPAM filters around the network.) I have worked out how to feed email from my domain into Verizon’s servers. You need to a) turn on SMTP AUTH, and b) pass your Verizon username & password in the SMTP AUTH transaction. In postfix this looks like:

1. /etc/postfix/main.cf

   smtp_sasl_auth_enable = yes
smtp_sasl_security_options =
smtp_sasl_password_maps = hash:/etc/postfix/saslpass

2. /etc/postfix/saslpass

   # remote user:password
outgoing.verizon.net user@verizon.net:password

3. create /etc/postfix/saslpass.db with:

   % sudo postmap /etc/postfix/saslpass

4. Last thing: I noted above that my outgoing email is all coming off the RCN IP address. This broke the connection to amavis, the Virus/Spam scanner. Previously amavis was set up to only accept connections from the loopback (127.0.0.1) IP address, now it needs to accept the RCN IP address as well:

   in /etc/amavisd.conf

   @inet_acl = qw(207.172.210.134 127.0.0.1 [::1]);

I already mentioned that I want access to my Linux server from the network. Verizon’s d-link - still in my network, but not forever - is set up to prevent this. Verizon assumes you have a standard PC, so they preconfigure the d-link to firewall any incoming connections.

That’s not my situation, I have a Linux server with extensive firewalling and more than a few services running on it. I want all the network traffic coming to MY firewall, please, not Verizon’s.

The solution - partial as usual - is the “DMZ” setting. In the d-link configuration you can set a “DMZ” host, which gets most of the traffic the d-link sees forwarded to it. Specifically it seems to forward TCP and UDP traffic, which is a big step. But it’s still intercepting ICMP - so any Ping or traceroute traffic goes to d-link, not Linux. Yet another reason to disable the d-link.

The embedded web server on the VZ FiOS d-link box sucks. That’s all there is too it. I’m polling it every 10 minutes to check the IP address, and it consistently locks up in less than an hour. Every. Single. Time.

The only way to reset the Web server, that I’ve found, is to power-cycle the box. This has a nice symmetry, because power-cycling the d-link gets you a new PPPoE tunnel which gets you a new IP address. But it’s unacceptable that VZ can change my public IP address when the d-link web server is locked up, and I won’t know it.

At least it’s only the web server that locks up, the d-link keeps forwarding packets.

So what are the choices?
1. Restart the d-link every hour - impractical
2. Rewrite script to use http://dynamic.zoneedit.com/checkip.html - works, but not very elegant
3. Turn off the d-link and bring the PPPoE tunnel all the way to my Linux box

Stay tuned.

VZ installed a d-link DI-604 “Broadband router” in between my home network and the ONT on the side of the house. The d-link’s job is setting up the PPPoE connection back to Verizon, NAT, and firewalling. Verizon uses dynamic IP addressing for public IP addresses, and rumour has it that they reassign IP addresses frequently - but the d-link hides these changes.

(Apparently there are some specialized diagnostic tests the d-link knows how to run as well. It appears to be running a custom firmware load, with a version number that doesn’t appear on d-link’s support site. And when I tried to upgrade its firmware, the d-link rejected the new version.)

I want to know my public IP address so I can access my home computer from work or other places. (I use zoneedit.com to maintain an Internet-visible domain name.) The d-link makes this hard! I have used ddclient for this job in the past - basically it connects to the embedded web server on the d-link (or similar devices), and scrapes the IP address out of the web pages. (We used to use this kludge to interface then-modern computers to IBM mainframes - I guess old kludges never die.)

Unfortunately, ddclient only understands “basic” HTML authentication, and the d-link wants “digest” authentication. (Basic is pretty insecure, and Digest uses MD5 hashes which are much better. But all of this is already hidden behind the d-link’s own firewall, so who cares?) In fact, d-link is very picky about its digest authentication - it would talk to firefox and curl, but not lynx, links, or wget, all of which support digest authentication.

So here’s the snippet that works - for a while:

curl -f -s --digest --connect-timeout 10 --interface eth0 -u admin:password http://192.168.0.1/st_devic.html |
egrep '^[0-9]?[0-9]?[0-9]\.[0-9]?[0-9]?[0-9]\.[0-9]?[0-9]?[0-9]\.[0-9]?[0-9]?[0-9]‘ |
head -n 3 | tail -n 1 | tr -d ‘[:space:]‘

But see the next entry.

The VZ tech who installed my FiOS had his own procedures for speed-testing and speed-tuning, according to him the sites that VZ runs for the purpose give terrible results so none of the service techs use them. (They’re still targeted at DSL, not FiOS speeds.)

He recommended Speakeasy.net’s speed test - Speakeasy has done a very nice job as usual, too bad I can’t get FiOS speeds (and prices) from them - and also http://miranda.ctd.anl.gov:7123/for more repeatable numbers. My Linux box showed near 15M downloads and 1.8M uploads right off the bat, not bad for a 15M/2M service.

The web site he used for tuning Windows downloaded an app that set a bunch of Registry settings. This worked fine (IBM Thinkpad T42, Windows XP) - and anyway if it blew up, my work IT guy would have to fix it! So I didn’t save this URL.

The Apple iBook was a little out of his league, however. He had a CD-R with a collection of tools, one of which was an ethernet driver update, which he thought would do the trick. Only problem is that my iBook already had a newer Ethernet driver! So I let him leave at this point, and did my own research. I’ll post the iBook settings later.