The Saga of Converting Some Hosts and Services to Dual-Stack
In the Westin rack, we had a dual-stack network running quite happily for a long while. I wanted to enable IPv6 for the major public-facing hosts and services. It was less simple than one might like. This is the story as of the moment.
All the servers ran FreeBSD, most 8-CURRENT and psg.com was 7-BETA2. The servers, in the order I worked on them, were as follows:
The DNS
First, I wanted to have the DNS ready. Because I wanted ip6.arpa and many other things we expect from IPv4 life to work, I decided to use fixed address of the form <prefix>.<same-as-v4>.
The upstream that gave me the IPv6 transit had not delegated the reverse ip6.arpa zone to me. I looked at the SOA RR of the parent zone and wrote to the required email address in the RNAME field. The mail bounced. I then spent two weeks working with their most kind and diligent NOC to get through to the actual responsible folk in their organization.
Making the ip6.arpa zone file was easy but required editing, as I could not just steal the IPv4 in-addr.arpa zone and stick in an
$ORIGIN 0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.0.0.0.8.1.4.0.1.0.0.2.ip6.arpa.
because one of the great leaps forward with IPv6 is that it does reverse DNS with nibbles instead of bytes. So the zone had to be broken up into nibbles, as in
0.1.0 PTR psg1.psg.com. 0.0.1 PTR roam.psg.com. 0.2.1 PTR voip0.psg.com.
I then built a kernel with IPv6 enabled, which it is by default in FreeBSD's GENERIC kernel, and changed /etc/rc.conf to configure what I minimally wanted.
ipv6_enable=YES # Set to YES to set up for IPv6.
ipv6_ifconfig_bge0="2001:418:1::39/64" # Sample manual assign entry
ipv6_ifconfig_bge0_alias0="2001:418:1::40/64"
ipv6_defaultrouter="2001:418:1::1"
ipv6_ipv4mapping=NO # Set to NO to disable IPv4 mapped IPv6 addr
I rebooted and it actually worked.
Then I had to rebuild BIND with
./configure --enable-ipv6 --with-openssl
Note that the vanilla BIND in the build will have these enabled. So if you use the normal buildworld BIND, you're fine.
To enable IPv6 transport for BIND, one has to add
listen-on-v6 { any; };
to the options section of the configuration file. Of course, one can be more restrictive specifying which address BIND listens.
Finding a secondary who had dual-stack transport was not as easy as I expected. But ISC volunteerd, and I could even reuse my existing tsig key with them. You do use tsig for axfr, don't you?
I still do not know how to get AAAA RRs added to COM and root zones for glue records.
Email: Exim and Dovecot
Exim was a bit cute. As it used colon in the local_interfaces configuration line already, there is a <; hack one uses
local_interfaces = <; \
127.0.0.1 ; \
147.28.0.39 ; \
147.28.0.40 ; \
2001:418:1::39 ; \
2001:418:1:40
Dovecot was my imaps server, and I needed to change the configuration file to add
listen = *, [::]
Checking for Missed Services
Issuing the command
netstat -af inet | egrep -v '(http|smtp|domain|imap|ssh)'
gave me an idea of what other services needed configuring. There were none that were not already handled some other way. Unfortunately, this took 15 minutes to run, as the host had a few TCP connections.
Testing
So I rebooted everything and tested that they all worked, i.e.
% dig +short @2001:418:1::39 1.0.0.0.1.0.0.0.8.1.4.0.1.0.0.2.ip6.arpa. ptr
psg1.psg.com.
and
% telnet 2001:418:1::39 993
Trying 2001:418:1::39...
Connected to 2001:418:1::39.
Escape character is '^]'.
quit
^]
telnet> q
Connection closed.
But then I hit a suprise
% telnet 2001:418:1::39 25
Trying 2001:418:1::39...
telnet: connect to address 2001:418:1::39: Permission denied
telnet: Unable to connect to remote host
A bit of debugging revealed a line in the sending host's /etc/ipfw.conf that said
add pass tcp from me to any smtp
add deny tcp from any to any smtp
which I think should have been fine. But, when I replaced it with
add deny log tcp from any to me smtp
everything was fine. Clearly I need to do more research on IPv4/IPv6 differentiation in ipfw.
Turning it Up and Moving On
I then felt confident that I could add the AAAA record(s) to the forward zone file in relative safety.
I then went over to the second of the three main systems, psg.com itself, and did pretty much the same thing: BIND, dovecot, /etc/ipfw.conf, etc.
Apache
Apache on psg.com was a bit tougher. I was still running Apache 1.3 on this host, and 1.3 did not support IPv6. I could either apply the Kame patch, which probably would have been easy considering that I worked in a office with leading Kame developers, or upgrade to Apache 2.x. I decided to upgrade to 2.2 as that was forward progress and probably needed doing anyway. So, at 16:00, I bit the bullet and did
pkg_deinstall -Rv php4* apache* mod_*
I then installed php5 and apache22 using portupgrade. Then I had to edit the configuration files to get the 42 virtual servers, SSL, etc.
I also had to go over to my key/certificate generation system and tell it that psg.com was now Apache 2.2, and automatically generate and install the appropriate certificates and keys. A million thanks to Rob Austein for the automated X.509 generation system software.
Turning Up psg.com and an Exim DNSBL Problem
I then checked out the services and found them to be working! So I changed the zone file for psg.com to show the AAAA RRs.
WHAM! My Exim started rejecting mail over IPv6 with some sort of bad dnsbl lookup at mail-abuse.org. A bounce looked like
rcpt to: <foo@psg.com>
550-rejected because 2001:4d00:0:e:62:89:13:66 is in a black list at
550-blackholes.mail-abuse.org
550 Blackholed - see <URL:http://mail-abuse.org/cgi-bin/lookup?2.0.0.0>
Connection closed by foreign host.
Note that the lookup was for 2.0.0.0, there is probably some other bug lurking there. The relevant config was
drop log_message = REJECT mail from rbl
message = rejected because $sender_host_address is \
in a black list at \
$dnslist_domain\n$dnslist_text
dnslists = blackholes.mail-abuse.org
I immediately turned the AAAA records off until I cound find a cure. I googled but no luck. I could not even ask on the exim-users mailing list because its server used sender verify and my sender was rude to it. I had to fall back to gmail to ask for advice.
The problem turned out to be that DNSBLs do not handle IPv6. So, for IPv6 connections, you want to skip the DNSBL tests. I did this by adding the conditional shown in bold.
drop log_message = REJECT mail from rbl
message = rejected because $sender_host_address is \
in a black list at \
$dnslist_domain\n$dnslist_text
condition = ${if isip4{$sender_host_address}}
dnslists = dialups.mail-abuse.org
Next I went to work0.psg.com, the host which has net services such as xmpp/jabber, asterisk, etc. First I did the same old basics that were begining to feel familiar to me, kernel, rc.conf, named, /etc/ipfw.conf, and exim. It was already running Apache 2.2, so no problem there.
MRTG
When IPv6 was up, I did get start getting whines from the mrtg running on the host,
socket: Protocol not supported
every time it ran a cycle. I need to chase this down.
ejabberd
The instructions said that ejabberd was very simple, merely add "inet6" to all the listeners, as in
{ listen,
[ {5222, ejabberd_c2s, [ inet6,
{ access, c2s },
starttls,
{ certfile, "/usr/local/etc/key.pem" },
] },
It turns out that ejabberd will not support listening on the same port from IPv4 and IPv6, see bug report. The work-around is to have the IPv6 listener on a different port. This would be pretty hard for users. But this does not explain why , if IPv6 was enabled on the box at all, ejabberd would not run, with or without the configuration change. So I had to leave it IPv4 only for the moment.
---
2008.01.22