Ever on and on

In a comment on my earlier pivotroot post, nathane says:

I was hoping you might have a solution to the problem of keeping /etc/config/system and /[flash|mnt]/config/system in sync.

and mentions another howto that does a bind mount of /etc between the flash and USB filesystems, so they stay in sync.

There’s an obvious problem with a bind mount of /etc, which is that the contents of /etc/config depend on the software in the rest of the filesystem. Let’s say you’re booted to the USB root filesystem. If you install a new package, or a new version of a current package, then you might inadvertently create a config file that’s incompatible with the software on the flash root filesystem. Most likely you won’t discover the incompatibility until it causes a significant problem. (Imagine if the webif or ssh doesn’t come up when you do an emergency boot off the flash root filesystem!)

My perspective is that having a completely separate root filesystem in flash is a feature, not a bug. When I deliver OpenWRT routers to remote locations, I configure the flash filesystem with the minimum requirements for remote maintenance: generally, this is the firewall, dynamic dns, and ssh with a key. Any added-value software goes on the USB root filesystem – generally this software is bigger or has more complex configuration.

If something goes wrong in the remote location, especially if somebody misconfigures OpenWRT, then recovery is easy: pull out the USB drive and reboot. (If the USB drive dies, simply rebooting is enough.) There’s a separate, known good configuration which will be sufficient for me to have remote access.

In a past life I helped design embedded network devices that got delivered to remote (unstaffed) locations. We used dual flash partitions and a custom boot ROM, to ensure that the devices would always be able to boot. Even a cosmic ray flipping the wrong bit wouldn’t brick the device: it would simply boot from the alternate partition. Similarly, software upgrades that affected the flash image would only be written to one partition, so you could fallback by booting the alternate.

My strategy for OpenWRT borrows from this prior experience, and also from the hidden “recovery” partition on many Windows computers, where there’s a stripped-down partition to allow restoring the original software in case of disaster.

This entry was posted on Sunday, January 10th, 2010 at 2:12 pm and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.