Marcel is one of the gentlemen who maintain the NixSPAM DNSBL, and he contacted me a few weeks ago. The project maintains very volatile DNS data on a BIND name server, and they are beginning to experience difficulties in keeping up with updates to the data. In other words, the amount of updates they want to perform per second is higher than what their current methods allow.

Even though I’m not privy to how the exactly perform the updates, the topic piqued my interest, so I thought about how they could go about speeding up dynamic DNS updates. (We’ve discussed updating the DNS before, including using SIG(0) so I won’t go into that again.)

If we look at the manual page for the nsupdate command, there are some examples that clearly indicate that batches of updates are possible:

# nsupdate
> update delete oldhost.example.com A
> update add newhost.example.com 86400 A 172.16.1.1
> send

This is possible only if all updates are performed for the same zone, which is the case for the NixSPAM list.

Using a bit of dnspython, I fired off 40,000 TCP signed updates to a local DNS server (localhost), in different batch sizes, listed in the first column in the table. I always started afresh, resetting the zone to serial number 1 and restarting my BIND server. Here are the results:

Transaction    Time     Final   Updates
   size       seconds   serial  per sec.
-----------------------------------------
    1          125      40001     320
    2           67      20001     597
    4           37       8001    1081
    8           28       4446    1428
   16           25       2354    1600
   50           19        786    2105
   80           20        495    2000
  100           18        398    2222
  250           16        161    2500
  500           16         81    2500

What is a bit surprising is that there is hardly a difference in wall clock time between batches of 250 and 500. Update: see below for a very likely reason explained by Geoff. (The final serial number in the zone is proof that batched updates are gathered together into a single transaction.)

The maximum batch size of an update depends on the length of individual updates (I used A records only), but we clearly see it is advantageous to batch updates. Furthermore, update batches lower the number of zone transfers (AXFR) and incremental zone transfers (IXFR) slave servers will have to perform.