2011-06-08 – In which IPv6 day turns out to be unexpectedly exciting

As part of IPv6 Day we have enabled IPv6 on the important parts of our mail service: mx.cam (incoming mail), ppsw.cam (outgoing), and smtp/imap/pop/webmail.hermes. Most of the University is IPv4-only, apart from the Computing Service's staff network, the Institute of Astronomy, the CUSU/SRCF network, and a smattering elsewhere. Today has been mostly unexciting for us, which is cheerful. Mostly.

This afternoon I got an alarming email and phone call from a departmental sysadmin whose mail was all of a sudden being rejected by our outgoing relay. (This turned out to be one of the very few occasions where my special "accept all mail to postmaster regardless of ACLs" rule actually got used by someone to work around a problem!)

Looking at the logs I immediately saw that the rejected mail was arriving over IPv6 from a 6to4 address; our ACLs do not treat 6to4 addresses as being inside the University, so the mail was being rejected.

The quickest way to fix mail delivery was to add disable_ipv6 to the Exim configuration on the sender.

The breakage suddenly started happening at 15:24, which immediately made me suspect a bogus IPv6 router had appeared on the server's LAN at that time. Malc has a good description of how Windows Internet Connection Sharing breaks connectivity by issuing rogue router advertisements.

It is possible to identify the machine responsible for this kind of braindamage. The unwanted 6to4 address was of the form 2002:836f:... where 2002::/16 is the 6to4 prefix and the two following hexadectets are the IPv4 address of the tunnel endpoint. 836f:xxxx is hex for 131.111.ddd.ddd.

I trawled my logs for other 6to4 lossage and found a couple of colleges that had a few messages from their web servers rejected. One of them had at least four machines issuing rogue RAs.

It is going to be, um, interesting dealing with this in the future...

⇐ 2011-05-31 ⇐ The state of DNSSEC deployment ⇐ ⇒ IPv6 day stats ⇒ 2011-06-09 ⇒