More on pivotroot

January 10th, 2010

In a comment on my earlier pivotroot post, nathane says:

I was hoping you might have a solution to the problem of keeping /etc/config/system and /[flash|mnt]/config/system in sync.

and mentions another howto that does a bind mount of /etc between the flash and USB filesystems, so they stay in sync.

There’s an obvious problem with a bind mount of /etc, which is that the contents of /etc/config depend on the software in the rest of the filesystem. Let’s say you’re booted to the USB root filesystem. If you install a new package, or a new version of a current package, then you might inadvertently create a config file that’s incompatible with the software on the flash root filesystem. Most likely you won’t discover the incompatibility until it causes a significant problem. (Imagine if the webif or ssh doesn’t come up when you do an emergency boot off the flash root filesystem!)

My perspective is that having a completely separate root filesystem in flash is a feature, not a bug. When I deliver OpenWRT routers to remote locations, I configure the flash filesystem with the minimum requirements for remote maintenance: generally, this is the firewall, dynamic dns, and ssh with a key. Any added-value software goes on the USB root filesystem – generally this software is bigger or has more complex configuration.

If something goes wrong in the remote location, especially if somebody misconfigures OpenWRT, then recovery is easy: pull out the USB drive and reboot. (If the USB drive dies, simply rebooting is enough.) There’s a separate, known good configuration which will be sufficient for me to have remote access.

In a past life I helped design embedded network devices that got delivered to remote (unstaffed) locations. We used dual flash partitions and a custom boot ROM, to ensure that the devices would always be able to boot. Even a cosmic ray flipping the wrong bit wouldn’t brick the device: it would simply boot from the alternate partition. Similarly, software upgrades that affected the flash image would only be written to one partition, so you could fallback by booting the alternate.

My strategy for OpenWRT borrows from this prior experience, and also from the hidden “recovery” partition on many Windows computers, where there’s a stripped-down partition to allow restoring the original software in case of disaster.

Presidentials

September 23rd, 2009

What’s a Presidential peak? My niece asked me this question, and it’s an excellent question with a very confusing answer.

A Presidential peak is generally considered to be Mt. Washington, and one of the major peaks on the ridges connected to Washington. North of Washington, these are Madison, Adams, and Jefferson. Mt. Clay sits between Jefferson and Washington; it is not an official 4K peak (for the usual insufficient col reason), nor is it named after a President. There are 5 peaks in the Adams group; the tallest is named after John Adams, the next tallest is John Quincy Adams, then Sam Adams, then imagination failed and the call the smallest bumps Adams 4 and Adams 5. (I and many others think Adams 4 should be called Abigail.) John Quincy Adams, by the way, sits across the col from Mt. Madison and is in fact taller than Madison.

South of Washington the Presidential ridge is Monroe, Little Monroe, Franklin (an even smaller bump on the ridge than Clay), Eisenhower, Pierce/Clinton, Jackson, and Webster. Of these peaks, only half are named after Presidents: Monroe, Eisenhower, and Pierce. (But the Clinton is a former governor of NH, not the President). The Jackson was a state geologist.

There are several other ridges leading to Washington. The Montalban ridge is just east of the Southern Presidentials: its highest peaks are Isolation and Davis, as well as the very recognizable Giant Stairs. The Rocky Branch Ridge, to the east of Montalban, has no major peaks. Then the ridges to the east, which define Washington’s ravines: Boott Spur, Glen Boulder, and Chandler Ridge (where the auto road runs). The peaks on Chandler Ridge, all higher than the Southern Presidentials, are named Ball Crag and Nelson Crag. None of these peaks are considered Presidentials.

Why is this such a mess? Mountain naming is necessarily haphazard (don’t get me started about the peaks named after Native American chiefs who never set foot in New England), and necessarily represents a moment in time. The Presidentials (Adams through Monroe) were named by a group of “prominent citizens”, who climbed Washington (named after George Washington when he was a general and not yet President) and named the next 4 highest peaks after the next 4 Presidents. They got it wrong, by the way, because Monroe – named after the fifth president – is actually the fourth tallest, and Madison is only fifth tallest.

None of this accounts for Lincoln, which is named after President Lincoln, but is not in the Presidential range. (It’s on Franconia Ridge.)

I wrote above that Clay is not named after a President, but that is not entirely true. There has been a national drive to rename features after President Reagan, and Clay is the peak that was targeted in the Whites. The New Hampshire legislature has renamed Clay to Reagan, but the US Bureau of Geological Names, which has its own rules, does not recognize the renaming. So Clay has two names, depending on whether you answer to NH or the US.

Favorite Peaks

September 16th, 2009

These are some of my favorite peaks from the past 4 years.

  • Wheeler Mt. Absolutely my favorite hike of the past 4 years was climbing Wheeler, a short hike (about 2 miles round-trip) on a small mountain with dramatic cliffs, but it was the company that made the day. My son was with me – his 2nd or 3rd time on Wheeler – and my father – his first time hiking in years, and the first time hiking with my son when he was actually walking, and not on my back. My wife completed the team. Standing on the final outlook with them was a very special moment.
  • Bondcliff, very possibly my favorite NH 4K. Positioned in the middle of the Pemigewasset Wilderness, it is hidden from most of the overt signs of civilization. It has close-up dramatic views of South Twin and the other Bonds, nearby views of Franconia Ridge, and more-distant views of the Sandwich Range and the Presidentials. And it has a luxurious summit area, very flat and broad. It’s remote, so you know you’ve earned your solitude.
  • Guyot, not actually a 4K, but on the Twins/Bonds ridge. Another remote spot with awesome views. I had a full day planned and had to press on, but someday I plan to hike to Guyot by whatever route seems easiest, and have lunch.
  • South Kinsman is another broad summit area, with unique views over the wall of Franconia notch into the Pemigewasset. I was so enthralled by the views I nearly missed the storm sneaking up on me from Moosilauke.
  • Jefferson. It seems unfair to pick the Presidential I most recently climbed, but I think Jefferson probably is my favorite. The unique double-humped summit, with views into the headwall of the Great Gulf, and of course Adams and Washington up close, are simply awesome.
  • Somehow I’ve left off Franconia Ridge, and my favorites in the Sandwich Range. Some peaks get short shrift because I climbed them on busy days (South Twin, Liberty). Actually, there’s hardly a peak I’ve climbed that I wouldn’t go back to. But these are some of the ones that stand out.

Finishing the Separate LANs Configuration

September 14th, 2009

Last time I wrote about how OpenWRT configures networks at the port level. Once I split the LAN ports into two separate LANs, there was higher-level configuration that also needed to be taught about the LANs.

Once again, I’ll show the configuration from /etc/config files. The first thing we need to do is define a network for the new LAN (in fact, I renamed the old LAN as well). The type is “bridge” so we can bridge it with the wireless LAN, as we noted last time. The interface is eth0.2, which is the new VLAN. This looks something like:

config 'interface' 'homelan'
option 'type' 'bridge'
option 'ifname' 'eth0.2'

Finally I wrote some iptables rules to prevent forwarding between the two LANs, and to prevent access to the router itself.
Then we need to add a section to the dhcp configuration. In this case we don’t want to run DHCP on this interface, so I set “ignore” to 1.

config 'dhcp'
option 'interface' 'homelan'
option 'ignore' '1'

Finally, and this is key, we need to add some sections to the firewall config file. First we need to define a “zone”, which is an OpenWRT concept. This (typical) zone allows packets to be transmitted or received on the homelan interface, but not to be forwarded between interfaces:

config 'zone'
option 'name' 'homelan'
option 'input' 'ACCEPT'
option 'output' 'ACCEPT'
option 'forward' 'REJECT'

Then we configure forwarding from the homelan into the WAN:

config 'forwarding'
option 'src' 'homelan'
option 'dest' 'wan'
option 'mtu_fix' '1'

Open Source in Startups

September 11th, 2009

I’ve worked for several New England startups that used open source software to jump-start their product development. What are some of the tradeoffs? I’m interested in open source software integrated into a product; there are different issues when the company exists to extend an open source project, or when open source software is used as part of the company’s IT infrastructure.

The classic case of this usage is embedding the Linux kernel in the product. Linux is mature, actively-developed, and very modular – these are all attractive attributes. It’s also free, as in royalty-free, and the source code is similarly freely-available. The competition – embeddable kernels, often with a real-time flavor – is also mature, several options are actively-developed, but you pay for using the code, and presumably receive better support in your development. (The technical fit will depend on your application, I won’t address that here.)

Other classic examples include using gcc in your product, or building a web-based service around the Apache web server, or even using common tools such as Perl and PHP. Each one of these software packages is mature and actively-developed, and has competitive commercial products (depending on your application) that are also mature and actively-developed, but require payment.

One company I helped start made the decision (free or not free) several times during our lifetime, and changed the direction we went. Initially, we licensed a commercial embedded real-time operating system, bought a commercial suite of compilers, and used them to develop the first products. Later, in our 3rd-generation products, we switched to a Linux base. (We revised the physical designs at the same time, moving to a more off-the-shelf approach.) Our server application team made different decisions, building in more open source software from the beginning. All of these decisions had an impact on the business years later, and some of them led to last-minute rewriting of non-trivial parts of the product.

The problems we faced – and bigger problems that have been faced by much larger companies – evolved from that word “free”. When I described Linux and other open source projects as “free”, that was sloppy thinking, and in the case of this company it led to some sloppy management practices. (I was part of these practices, I share the responsibility.)

The Linux kernel is copyrighted, and that copyright is released under a license. In this way it is no different from the commercial products I called “competitors”. gcc is also copyrighted, and so are all other open source projects. (If there’s no copyright, we call the software public domain, and the rules are different.)

You wouldn’t found a startup – or start a new project – by saying “I’m going to steal this software and use it to build our next-gen best-seller.” No, you would say “we need to license this software package, and that will cost us X dollars up front with a per-unit royalty of $Y” (or whatever the vendor’s terms are). But, because the industry often calls open source projects “free”, we as software managers often forget to account for the licenses that control them.

Open source software is licensed. The choice to accept those licenses, and abide by their terms, is a business decision that needs to be made by the business managers. Just as you wouldn’t make an individual engineer responsible for counting the units-shipped of a commercial component, and paying the vendor’s royalty, you shouldn’t leave open source licensing decisions to engineers either. These are business decisions that need to be made by technically-savvy business managers.