Marcel is one of the gentlemen who maintain the NixSPAM DNSBL, and he contacted me a few weeks ago. The project maintains very volatile DNS data on a BIND name server, and they are beginning to experience difficulties in keeping up with updates to the data. In other words, the amount of updates they want to perform per second is higher than what their current methods allow.
Even though I'm not privy to how the exactly perform the updates, the topic piqued my interest, so I thought about how they could go about speeding up dynamic DNS updates. (We've discussed updating the DNS before, including using SIG(0) so I won't go into that again.)
If we look at the manual page for the
nsupdate command, there are some examples
that clearly indicate that batches of updates are possible:
# nsupdate > update delete oldhost.example.com A > update add newhost.example.com 86400 A 172.16.1.1 > send
This is possible only if all updates are performed for the same zone, which is the case for the NixSPAM list.
Using a bit of dnspython, I fired off 40,000 TCP signed updates to a local DNS server (localhost), in different batch sizes, listed in the first column in the table. I always started afresh, resetting the zone to serial number 1 and restarting my BIND server. Here are the results:
Transaction Time Final Updates size seconds serial per sec. ----------------------------------------- 1 125 40001 320 2 67 20001 597 4 37 8001 1081 8 28 4446 1428 16 25 2354 1600 50 19 786 2105 80 20 495 2000 100 18 398 2222 250 16 161 2500 500 16 81 2500
What is a bit surprising is that there is hardly a difference in wall clock time between batches of 250 and 500. Update: see below for a very likely reason explained by Geoff. (The final serial number in the zone is proof that batched updates are gathered together into a single transaction.)
The maximum batch size of an update depends on the length of individual updates
A records only), but we clearly see it is advantageous to batch
updates. Furthermore, update batches lower the number of zone transfers (AXFR)
and incremental zone transfers (IXFR) slave servers will have to perform.