Knot has become much more mature and solid since I last looked at it almost five years ago – time flies!

It sports a new configuration file format (YAML) which is optionally backed by an LMDB database, a key manager which takes care of ZSK/KSK key rollovers and timings, and lots of changes which make the server and its utilities appear much more solid and quite a bit snappier. It’s worth scrolling through Knot’s changelog to see how the program has evolved.

A project I’ll likely soon be asked to start working on has a requirement for Knot because a front-end bit of DNS software requires a DNSSEC signer and the implementation specifically calls for the Czech software.

Simultaneously, the same company will soon embark on a project to rejuvenate an OpenDNSSEC installation, and I’ve been asked to assist with that. My gut feeling is that the current signer software is seeing very little love, and since the requirement of using HSMs for storing private DNSSEC key material has been dropped (does anybody need a half a dozen high-end HSM?), I recommend using Knot as a signer for two reasons:

  1. it will be the same software used in the first department; this will simplify and hopefully ease knowledge transfer
  2. DS-PUSH, which I’ll explain in a moment.

The authoritative Knot DNSSEC signer optionally publishes CDS/CDNSKEY records in a signed zone. (We’ve previously discussed CDS and CDNSKEY records and what they’re used for here.)

   - id: knotmaster
     key: km_updater

   - id: ds_checker
     check-interval: 10s
     parent: knotmaster

   - id: rsa00
     algorithm: RSASHA256
     ksk-size: 2048
     zsk-size: 1024
     cds-cdnskey-publish: rollover
     ksk-submission: ds_checker

   - domain: f8.example
     dnssec-signing: on
     dnssec-policy: rsa00

These records can be used to signal to the parent zone that a DS record has changed in the child zone. During a KSK roll, Knot can attempt to consume (i.e. verify if a record exists in the parent zone) a DS record before actually activating a KSK to ensure that a key roll is performed correctly; this is configured by the (IMO confusingly-named submission stanza).

info: [f8.example.] DS check, outgoing, remote, KSK submission attempt: negative

We could use the log message to trigger a DS upload or do it manually using knsupdate; note the k:


cds=$(dig +short @ $z CDS)

knsupdate <<END
zone ${p}.
update delete ${z}. DS
update add ${z}. 60 DS ${cds}

As soon as the DS is seen in the parent zone, Knot logs the following messages:

info: [f8.example.] DS check, outgoing, remote, KSK submission attempt: positive
notice: [f8.example.] DNSSEC, KSK submission, confirmed

The operator ensures the DS is uploaded to the parent and then, using knotc zone-ksk-submitted $zone signals Knot that the roll is to be completed whereupon the CDS/CDNSKEY records in the child are removed.

automation: ds-push

Knot can help me eliminate a lot of lovingly hand-written code from the above-mentioned “legacy” project by automatically submitting DS records. That’s what ds-push accomplishes: automatic upload of DS records to parent zones. This environment consists of a completely separate DNSSEC-signed root zone, and all zones must have their DS records uploaded to the root zone and be subsequently signed. Seeing we control the whole infrastructure, that is relatively easy to manage.

   - id: rsa00
     cds-cdnskey-publish: rollover
     ksk-submission: ds_checker
     ds-push: knotmaster

Knot is capable of doing this by itself. In the above configuration, I add a ds-push setting to the signing policy; this instructs Knot to push DS records to the specified remote (a remote configures a particular server). In this specific case, as the parent zone is signed and served by the same server, the remote specifies the Knot master server proper.

When a zone is signed, Knot publishes CDS records and uploads the hashes as DS records to the parent’s server via a dynamic DNS update

info: [f7.example.] DS push, outgoing, remote, success
info: [f7.example.] DS check, outgoing, remote, KSK submission attempt: positive
notice: [f7.example.] DNSSEC, KSK submission, confirmed

I’ve had dozens of KSK rollovers done in the course of the last 24 hours (one every five minutes), and this is working very well, with the exception that Knot appends the new DS only where it should replace; I reported this as an issue and it was solved 20 hours later – long live Open Source maintainers!

With the exception of the root’s KSK which for which I’ll likely use CDS signalling as the trust anchor has to be copied to all manner of places, this automatic handling of DS uploads is very promising.

adding zones

Knot supports adding zones to a running server, using the knotc utility:


knotc <<EOF
conf-set zone[$z]
conf-set zone[$z].file $
conf-set zone[$z].acl jp01
conf-set zone[$z].dnssec-signing on
conf-set zone[$z].dnssec-policy rsa00
conf-set zone[$z].zonefile-sync -1
conf-set zone[$z].zonefile-load difference
conf-set zone[$z].journal-content changes
conf-set zone[$z].serial-policy increment

I still have to learn how to best manage the configuration: knot.conf (the YAML) can be backed by an LMDB database, and that’s where the above zone is added to. Using knotc I can read the runtime configuration and all sorts of other information from the running server, but I’ll want to persist that.


Knot has for some releases provided the possibility for logging via dnstap, and we discussed this here some time ago.

DNS :: 13 Nov 2019 :: e-mail

I had the pleasure of attending EuroBSDCon in Lillehammer, Norway this year. It was a first visit to Norway and my first time at EuroBSDCon. The conference felt familiar, as I saw and spoke to many of the people I’d first met in Ottawa during BSDCan 2019.

The first two days of the event were dedicated to the tutorials, and my Ansible tutorial on day 2 was, I believe, a success. We had a full house with, I’m told, 35 participants. There were lots of interesting questions, and I was dead tired at the end of the day, so it must have been good. :-)

Norway being Norway, everything is rather expensive, but we weren’t really concerned because the conference put up hectolitres of free homeopathic beer. WiFi at the Scandic hotel was excellent, and all-in-all the venue was comfortable, and we were served interesting and tasty snacks during the day.

The third and fourth days were conference days, and I opted to listen to these talks:

  • Patricia Aas gave the opening keynote, entitled Embedded Ethics, and she made me tear up. Literally. I recommend you listen to it if you can and as soon as the video is published.
  • Paul Vixie of DNS fame, spoke about blocking DNS over HTTPS. He gets very involved and as such he ran out of time unfortunately. I later chatted to him, and as I’d brought a copy of my “Alternative DNS Servers” book for him, I presented this signed copy to him, not that I believe he can learn anything from it!
  • Mischa Peters gave a talk entitled The OpenBSD hypervisor in the wild, a short story in which he explains the setup of which I knew of. I’ve also experimented with VMM/VMD open OpenBSD and learned some new things from Mischa’s talk.
  • Modernizing relayd and the road to HTTP/2, by Reyk Floeter, was a deep dive into the work he’s invested in modernizing relayd.
  • I then listened to Dan Langille speak about why he prefers thick jails over thin jails. I think there’s a lot of interesting stuff in his notes and await the publication of his slides for that.

I decided to take a break to give the old brain cells a rest and enjoyed some peace and quiet in my room until 17:30 at which time we assembled for the short walk to the social event at Maihaugen Open Air Museum. The weather was fine, the buildings nice to see, and I got told off badly by the school teacher in the 1866 school; very realistic!

Day two consisted of this program for me:

  • Philipp Buehler spoke about how he is adding OpenBSD VMM to If this interests you, he has a bit of a remix on his vagrant and packer talks as presented in Warsaw earlier this year, and his presentations are here.
  • Dan’s ZFS for Newbies was a bit light in content for me: just mentioning commands didn’t impress me. A diagram or two on what a zpool or vdev is would have helped, in particular because a newbie (which I am) is likely quickly overwhelmed by terms.
  • Eric Allman gave a wonderful talk about “Lessons learned from Sendmail”, full of bits of history and humor. A bit of a highlight was when Paul Vixie came to the microphone to ask a question: two Internet giants next to each other.
  • Colin Percival gave a talk which was way over the top of what I can handle: 23 years of software side channel attacks was much too mathematical for me.
  • Last but not least, Pablo Carboni spoke about how he implemented Unbound on FreeBSD and the road to success for the project he undertook.

I very much enjoyed the conference. The hotel was adequate and comfortable, food was much better than I’d imagined it would be up north, and we made sure that the bar wasn’t empty.

Thank you to the EuroBSDCon Foundation for inviting me to give the tutorial.

conference :: 24 Sep 2019 :: e-mail

I’m an old fart but relatively new to so-called “meetups” – organized meetings for people with a common interest. I think it was Ton who first dragged me to one, and I seem to recall going mainly because it was to be followed by a rijsttafel; I could not resist.

I didn’t regret going to the meetup – quite the contrary – and I’ve since been to several, but it’s dreadful how low the turnout typically is. I’ve verified my numbers with some of the organizers of prior meetups:

  • Nijmegen: 30% no-shows.
  • Hilversum: 50% attendance; masses of food had been ordered, and I helped chuck large amounts of it out.
  • (location): less than half of people who said they’d attend.
  • another meetup in city X recently had 175 RSVPs but only 75 in attendance.

Andreas, a student I once had, came up with the idea of an Ansible meetup in Hannover and asked me whether I’d do the honour of attending. I promised I would and that I’d prepare a presentation. He took upon himself the trouble of organizing a location, etc.

tweet announcing start of meetup

I spent a couple of hours thinking up some possibly controversial content to present, created and worked over slides, and dragged swag along which had been sent me for this purpose by Carol. Of fifteen (15) people who had signed up and said they would come, only six (6) turned up (spoiler: I didn’t show either!). One of the no-shows wrote me a message 50 minutes before the scheduled begin that he couldn’t due to “family obligations”.

There have been suggestions on how to get people to actually show when they’ve signed up, such as “charge EUR 10 and reimburse when participants show”, but that’s a bit of an administrative nightmare, even though there would certainly be charities who’d welcome the no-show tenners…

Many of the meetup organizers go to a lot of effort to reserve spots and maybe even organize and possibly finance drinks and food. No-shows mean the effort and expense are wasted, and the food is also wasted. Adding insult to injury, by reserving and not showing, you are denying somebody else a chance to attend.

Ton brought in a new suggestion which goes a bit like this:

To prevent this from happening we are working with the following “no-show” principle:

  • If you register and cancel on the day of the meetup or the day before, you get a “yellow card”.
  • If you end up with two “yellow cards” you will be denied access to the next two (2) meetups.
  • If you have a “yellow card” and register and show up, the yellow card is removed from your name, and you restart with a clean slate.

Carol’s already handed me my first yellow card for not showing in Hannover:

JP, sorry to hear you were not able to make the Hannover meetup after all. As a consolation, you get a yellow card! (j/k…)

She’s right: the reason for no-show doesn’t matter; I get the card. Carol was also saying that anecdotal data suggests that in general less than 50% of people who RSVP ‘yes’ actually show up.

Bas wrote:

note, that the value of meetups is beholden by the people that do show up, if only a handful it adds depth to the conversation. … yet the wasted money and spilled food and the extra time spent removing the goodies from the empty seats feels a bit lonely.

It’s impolite. It’s demotivating. It’s disgraceful.

Most of us don’t just not show up to a party we’ve been invited to; let’s think of Meetups as parties, which they often almost are. People invest personal time to present an idea, open a discussion, and maybe even teach something. Show some respect for that, please, by showing up when you’ve said you will.

my first yellow card

P.S.: I’ve been given my first yellow card for not appearing at the Hannover meeting. I alloted two hours to drive the 110 kms (which is more than ample for that stretch; I detest arriving late), but actually required over four hours for the first 60 kms. There was a massive Stau when the A2 was closed down for the night, and I was in it; luckily I escaped and could return home after seven hours. I am dreadfully sorry to have basically killed this meetup; I’ve apologized to Andreas, and I’ve volunteered to attempt this anew.


  • Paul writes: Those numbers tally with my experience, usually 30-50% dropout for free events.
  • Tobias says: I see the same at our Cloud Native Meetups. This observation sadly applies to Switzerland as well.
  • John writes: I’ve just checked the London numbers. That hovers around 50% (which is 80 people).
  • Florian reports: One guy with multiple large meetups simply said “its normal, a meetup isn’t that important and people have lives. Care about those who come” … Assume a 30–50% missing in action and calculate with that; worst case is a cramped room. ;)

meetings :: 14 Sep 2019 :: e-mail

I discovered the OnlyKey via this post on Mastodon, and the description of the device tickled a fancy, so I ordered one. While the ordering process went as smoothly as ordering processes sometimes go, I was a bit confused about having ordered something at “OnlyKey”, receiving a confirmation mail from a company called “CrytpTrust” and finding an envelope marked and posted by “Amazon”. I wrote to the people requesting an invoice (because I didn’t get one!) and their email address is “”. All those different names don’t specifically convey “trust” to me.


The red protective case I ordered was a €8 mistake: I thought I’d been led to believe the OnlyKey base product didn’t have one, but returning to the product page now, I clearly see indicated that it does come with the black one.

the key assembled

The documentation is adequate, but I cringe at the PDF document which is a link to an HTML to PDF conversion site…

The software works well: I used the OnlyKey desktop app to avoid the strange-sounding setup via the Chrome browser. The desktop app exists for macOS, Windows, and Linux (.deb). Due to the fishy-looking domains I mentioned above I actually checked the SHA checksums of the downloads; names are important. :-)

In order to setup the OnlyKey, I run the desktop app and follow instructions to set up a PIN for each of the two profiles the device offers. When I later insert the key into a computer, I unlock the device by entering the PIN, and the device indicates happiness via a bright colourful LED. You’ll note the PIN pad has six pads only, numbered 1 through 6, so that limits the combinations I can use, and I found it a bit confusing the first ten times because muscle memory dictates where zero and nine should be, but they’re not.

Each OnlyKey slot can be configured to send (it acts as a keyboard when I later tap on one of the six pads) a URL, delays, TAB characters, usernames, and passwords, and each slot can have a label. The idea is I take a card along in my wallet with the labels. Nice detail: if I touch pad 2 for 5+ seconds and let go, the device “types” the labels at me (use cat, notepad.exe, or whatever to grab them). My use-case is not to have the device paste URLs and whatnot, but just a single password or two.


So far I’ve experimented with simple passwords only, but it appears to support TOTP via Google Authenticator or Yubikey OTP as well, in addition to being OpenPGP compatible and a “plug and play encryption device”. These features are explained in the documentation. There’s also an OnlyKey SSH/GPG agent which looks as though it could work; unfortunately the documentation suggests using to generate keys which is a shame. Basically what one has to do is to copy/paste a private RSA key onto the OnlyKey.

When traveling to specific countries the International Travel Edition might come in very handy:

OnlyKey allows the use of a hidden profile (Primary standard profile) and a fake profile (Second profile set to plausible deniability) that essentially provides a cover story. If compelled to unlock an OnlyKey the fake profile can be activated by entering the second profile PIN code. The goal of this feature is that there is no proof that the first profile even exists.

The plan for this device, is for it to be programmed with a few passwords on it and to use it in case of emergency or death to unlock other password stores, and as far as I can tell after a couple of hours, it fits the bill perfectly for that, and I appreciate that the data on the OnlyKey can be backed up and later restored.

passwords, security, and hardware :: 26 Aug 2019 :: e-mail

I frequently present during conferences or when giving trainings, and during the course of several years I’ve been very satisfied with a laser pointer which fits in my hand comfortably and can page-forward and page-backwards. The only issue I had with the device was a press on the laser in Terminal produced ugly escape sequences, but I killed those with Karabiner-Elements years ago.

What more could I want?

Well, I’m being confronted more and more frequently with TV panels instead of projectors, and the laser pointer doesn’t reflect off the panels.

I think it was Gerrit who, last year, showed me an equivalent solution, but it was then Bernd who, during a small demonstration, convinced me to opt for a Logitech Spotlight, which I did. I was afraid the software would suck, but it’s ok other than consuming quite a bit of CPU (5%) even when the Spotlight isn’t plugged in.


The device is pleasant to hold and the buttons are fine. I’ve configured it for a “laser highlight” only because using multiple different software pointers was confusing. Viewers also like the new pointer, and in particular, somebody who has a red/green deficiency convinced me to change the pointer’s color to a blueish, and he was then very happy!

blue laser

There’s one thing which is naturally unnatural: as a speaker, when I’m about to point at something with a real laser, the hand goes to the right direction before pressing the switch; this means the light comes on pretty much exactly where I want it. Using the Spotlight, this doesn’t work because only the software knows where the marker was last positioned, so I have to first press the light on and then drag the marker to its intended destination, wobbling and all. It takes a bit of getting used to, but I’ll manage.

I decided to solve the dilemma easily: I pack both, and if I’m given a projector with a laser-compatible screen I’ll use the laser and for a TV panel I’ll use the Spotlight.

presentation :: 04 Aug 2019 :: e-mail

Other recent entries