(no subject)

[torkell]

Today's discovery is that if you're foolish enough to enable DHCP on an alternate network interface (eth3, in my case) on Linux, then the DHCP client will overwrite your hand-configured default route that actually works with the one it received from the DHCP server.

Which is not particularly helpful when eth3 is connected to a different LAN with a router that actually checks the IP addresses of packets it forwards. At which point it eats the SSH connection I was using (because that was to an IP address on eth0) and I had to wander down to the lab and dig out a keyboard and monitor.

I am becoming more and more convinced that the Linux network stack just Does Not Work as soon as you plug it into more than one network.

Comments

[tau-iota-mu-c.livejournal.com]

The 12 eth interfaces each coming out of our production boxes at the Bureau of Met disagree with you.

Which were you trying to do something bizarre with dhcp on eth3?

[torkell]

To begin with, eth0 was plugged into the main lab LAN (which has a router/firewall between it and my desktop). It's configured with a static address and that router as the gateway in /etc/sysconfig/network-scripts/ifcfg-eth0, and everything worked fine.

I then wanted to add a connection to an alternate LAN for some testing, so I plugged eth3 into it (well, configured the switch to present the right VLAN to that port). The alternate LAN runs a DHCP server and dynamic DNS so I just enabled DHCP in /etc/sysconfig/network-scripts/ifcfg-eth3. I then did a "service network restart", at which point my SSH connection disappeared.

My best guess, once I had a keyboard and monitor plugged in to the server, is that the DHCP-assigned default route overrode my previously configured gateway for eth0. Linux pays no attention to the source address when choosing which interface to use to transmit a packet (unless you add static routing rules that do), so anything it transmitted to my desktop it sent from eth3 as that was where the default route ended up. My desktop was still directing packets to the lab LAN and eth0 as that's the IP address I was using for my SSH connection. The router for the alternate LAN appears to run a stateful firewall, and so since it only saw the outbound traffic from the server it binned all the packets.