Tony Finch – blog

Originally posted at https://www.dns.cam.ac.uk/news/2020-01-15-rollover.html

Here are some notes on how to upgrade a zone’s DNSSEC algorithm using BIND. These are mainly written for IT staff in departments who run their own DNS in the University of Cambridge.

I’ll use botolph.cam.ac.uk as the example zone. I’ll assume the rollover is from algorithm 5 (RSASHA1) to algorithm 13 (ECDSA-P256-SHA-256).

• Generate new keys
• Wait for child TTL to expire
• Update parent with new DS record
  • Semi-automated DS updates
  • Manual DS updates
• Wait for parent TTL to expire
• Decommission old algorithm
• Done!
• Future

Generate new keys

First we add the new algorithm to the zone alongside the old one.

Run the following commands in your key directory:

    dnssec-keygen -L 24h -a 13 -f ksk botolph.cam.ac.uk
    dnssec-keygen -L 24h -a 13 botolph.cam.ac.uk

• Algorithm 13 is ECDSA-P256-SHA-256.

• Each algorithm needs two keys, one with the KSK flag, and a ZSK without a special flag.

• The -L 24h flag sets the TTL on the DNSKEY records. This is optional but it’s a good idea if the zone’s default TTL is shorter.

In my setup I use group access to make the keys readable by named:

    chgrp named Kbotolph.cam.ac.uk.*.private
    chmod g+r Kbotolph.cam.ac.uk.*.private

Then get named to reload the keys:

    rndc loadkeys botolph.cam.ac.uk

Or you can use a bigger hammer to reload everything:

    rndc reload

Look at the name server logs. You should see the keys being reloaded, followed by a lot of zone transfer activity caused by the zone being signed with the new keys.

If you look at the zone now, you should see four DNSKEY records (two algorithm 5 and two algorithm 13) and two sets of signatures like RRSIG DNSKEY 5 and RRSIG DNSKEY 13.

    dig +dnssec botolph.cam.ac.uk DNSKEY

When a zone is signed with a new algorithm, named keeps track of progress with TYPE65534 records. You can tidy them away after it has finished signing with:

    rndc signing -clear all botolph.cam.ac.uk

Wait for child TTL to expire

Now you need to wait for the longest TTL in your zone to pass. For the cam.ac.uk algorithm rollover this was 48h, because the zone has some subdomain delegations with long TTLs.

You need to ensure that all records that are only signed with the old algorithm have expired from caches. After the longest TTL has expired, everywhere should be seeing records signed with both algorithms, so they will be able to successfully validate records using the new algorithm instead of the old one.

Update parent with new DS record

Now we can swap the chain of trust from the old algorithm to the new one. There are two ways to do this, depending on how recent your software is and your parent zone works.

Semi-automated DS updates

For subdomains of cam.ac.uk and other centrally-managed University of Cambridge zones (reverse DNS, etc.) you can use CDS records. CDS records are instructions from the child zone to its parent zone saying what the DS records should be. They are configured by setting timing parameters on your keys.

Have a look at the public key files for your zone. This will show you the timing metadata as well as the DNSKEY records:

    grep ^ Kbotolph.cam.ac.uk.*.key

Look for the key files with a first line like:

    ; This is a key-signing key, keyid XXXXX, for botolph.cam.ac.uk.

Ensure there are no CDS records for the old algorithm, and add CDS records for the new algorithm, like this:

    dnssec-settime -Dsync now Kbotolph.cam.ac.uk.+005+XXXXX
    dnssec-settime -Psync now Kbotolph.cam.ac.uk.+013+YYYYY

• -Dsync means delete sync records

• -Psync means publish sync records

• “sync records” is the term BIND uses for CDS and CDNSKEY records.

Then (if necessary) fix the permissions, and get named to reload the keys:

    chgrp named Kbotolph.cam.ac.uk.*.private
    chmod g+r Kbotolph.cam.ac.uk.*.private
    rndc loadkeys botolph.cam.ac.uk

Now wait. The university’s central systems will automatically notice the requested change, using my dnssec-cds program. We will send you a confirmation email when the change has been put into effect. (This is not yet entirely automated.)

Manual DS updates

If you are running an older version of BIND, or for zones under most other parent domains, you will need to explicitly update the delegation with the new DS record. (Automatic updates with CDS records are supposed to work for RIPE reverse DNS, but it seems RIPE-NCC have not yet added support as expected.)

As before, look at the key files to identify the key-signing keys. Get the DS record for the new algorithm like this:

    dnssec-dsfromkey -2 Kbotolph.cam.ac.uk.+013+YYYYY

• The -2 option forces SHA-2 and avoids the deprecated SHA-1 hash.

Using whatever facilities the parent zone provides (e.g. email to ip-register@uis.cam.ac.uk), replace the old DS record(s) with the new one.

Wait for parent TTL to expire

After you have confirmed that the DS record has been changed, you need to wait for the parent’s DS TTL to pass. For subdomains of cam.ac.uk and other centrally-managed University of Cambridge zones this is 24 hours; in other cases (such as for 111.131.in-addr.arpa) it is often 48 hours.

This wait ensures that the old algorithm is no longer being relied on as part of the chain of trust, before it can be safely decommissioned.

Decommission old algorithm

Once again this is done by updating the key timing parameters:

    dnssec-settime -D now -I now Kbotolph.cam.ac.uk.+005+KKKKK
    dnssec-settime -D now -I now Kbotolph.cam.ac.uk.+005+ZZZZZ

• -D is when the DNSKEY record is deleted from the zone

• -I is when the key becomes inactive, no longer used for signing records

• You need to do this for both the key-signing key and the zone-signing key for the old algorithm.

Then (if necessary) fix the permissions, and tell named to put the change into effect:

    chgrp named Kbotolph.cam.ac.uk.*.private
    chmod g+r Kbotolph.cam.ac.uk.*.private
    rndc loadkeys botolph.cam.ac.uk

After that, named no longer needs the old keys, so they can be removed from the filesystem.

Done!

That’s it!

Future

There are some changes still in progress that will make this somewhat smoother with newer versions of BIND.

The first improvement will be to combine the steps before the DS change. At the moment there is a bug that stops you from scheduling new CDS records in the future. I have submitted a fix; when the fix is released, you will be able to do the key generation and schedule the CDS swap in one go:

    dnssec-keygen -L 24h -a 13 -Psync now+50h -f ksk botolph.cam.ac.uk
    dnssec-keygen -L 24h -a 13 botolph.cam.ac.uk
    dnssec-settime -Dsync now+50h Kbotolph.cam.ac.uk.+005+XXXXX
    rndc loadkeys botolph.cam.ac.uk

Another improvement will come from an updated dnssec-checkds that knows about CDS records. (It works but I have not properly submitted it upstream yet.). This will allow you to automatically and safely verify that a DS change has taken effect.

When a DS change has been confirmed, you can immediately schedule the decommissioning of the old algorithm, with commands like:

    dnssec-settime -D now+50h -I now+50h Kbotolph.cam.ac.uk.+005+KKKKK
    dnssec-settime -D now+50h -I now+50h Kbotolph.cam.ac.uk.+005+ZZZZZ

This works now, but you might want something to remind you to clear away unused key files.

Further in the future will be fully automated rollovers, but I will discuss that another time.