I pretty much stumbled over sematicon and their N200 Series on my quest for Hardware Security Modules (HSM) of almost any shape and size, and after a very pleasant phone call with one of their managers, I was quickly given documentation to study and access to a dedicated device over a Wireguard VPN.

Every appliance features two or more independently configurable crypto processors, which work with a variety of keys. Therefore, test and production can be operated separately of each other on only one platform.

Although the N200 series isn’t specifically tailored to PKCS#11, it’s what I’m mainly interested in so what I focus on. I pretty much ignored the actual PKI capabilities of the device. It can be a certification authority (CA), produce X.509 certificates, perform symmetric encryption, and a load of other features. I toyed with creating a user and assigning access rules. The Web-based UI is neat and functional. There were some labels and captions which appeared confusing to me, but I got used to them.

PKCS#11 support is relatively new, and the documentation doesn’t yet mention about it, but I’m told that will change for the upcoming new version. A small interim PDF gave some tips, but I had a bit trouble actually using the interface because after initializing the device, I wasn’t aware that I had to create an access rule for a user who wants to use the PKCS#11 interface without which I kept getting an access denied. A brief meeting with the person who’s blog article initially led me to sematicon got me set up.

I copy the PKCS#11 sematicon-provided shared object library to the directory I want it in and configure the n200.conf file, adding a high log level to begin with, but it turns out the level is ignored: setting it to 0 doesn’t change much. Beware the log: it will contain the user and password in clear, but I’ve reported this.

% echo "10.195.10.11 seSAM-nserie-101" >> /etc/hosts    # avoid TLS verification errors on hostname
                                                        # the appliance has a cert with subject CN= this name

% install -m 555 pkcs11-sesam-x64-3.so /usr/local/lib

% export p=/usr/local/lib/pkcs11-sesam-x64-3.so

% cat $N200_CONF 
{
   "url": "https://10.195.10.11/n200/web/postv1",
   "tlscert": "/etc/tls/server63.crt",
   "user": "pkmaster",
   "pass": "<password>",
   "core": 1,
   "keygen_pin": "12345678",
   "log_level": 10,
   "log_output": "/tmp/seSAM_p11.log"
}

% pkcs11-tool --module $p --show-info
Cryptoki version 3.0
Manufacturer     sematiconAG
Library          se.SAM N200 pkcs11 (ver 1.0)
Using slot 0 with a present token (0xc13cc61)

The pkcs11-sesam.so speaks to the N200 via their REST API using curl, and judging by the number of CURL_GLOBAL_INIT occurrences, there is plenty of opportunity to improve performance of the shared library. (I will gloss over performance, because the network connection I had to the HSM was so flaky as to not be funny.)

Web UI

I’ve already mentioned the N200 has a REST API which I want to test. After learning how to do so in the Web UI, I use the API to generate a key-pair on the device:

% curl \
        --cacert server63.crt \
        -u $user:$pass \
        --no-progress-meter \
        -d command=genecckey \
        -d core=1 \
        -d curve=NIST \
        -d keysize=256 \
        -d acl=FFFF \
        -d pin=12345678 \
        -d nice_name=JP.example \
        https://seSAM-nserie-101/n200/web/postv1
{       
  "command": "genecckey",
  "core": 1,
  "clustersync": "n/a",
  "nice_name": "JP.example",
  "publickey": "FACD4CF599AD0CBD43A5C38B294AF3DDC06FBC0883D419EEBCBF8039C132FC61FE972236B38A1C236E5EF0F1E78B64B31C9632C0D5B9CFA49AECDB89144A2C3C",
  "result": "hsm010123F13C56131E1EEE0000000000000025"
}

% pkcs11-tool --module $p --token-label hsm010123F13C56131E1EEE00000025 --list-objects
Public Key Object; EC  EC_POINT 256 bits
  EC_POINT:   044104facd4cf599ad0cbd43a5c38b294af3ddc06fbc0883d419eebcbf8039c132fc61fe972236b38a1c236e5ef0f1e78b64b31c9632c0d5b9cfa49aecdb89144a2c3c
  EC_PARAMS:  06082a8648ce3d030107
  label:      JP.example
  ID:         aaab68606be2
  Usage:      encrypt, verify
  Access:     local
Private Key Object; EC
  label:      JP.example
  ID:         aaab68606be2
  Usage:      decrypt, sign, non-repudiation
  Access:     sensitive, always sensitive, never extractable, local
  Allowed mechanisms: ECDSA,ECDSA-SHA1,ECDSA-SHA224,ECDSA-SHA256,ECDSA-SHA384,ECDSA-SHA512

The result from the key generation has an identifier (also visible in the UI) which sadly has eight more zeroes in it than the token label in the slot; I don’t know if that’s on purpose. (A limit of 32 characters in a token slot name maybe?) I say sadly because it requires an extra step (pkcs11-tool --module $p -T) to find the correct token label.

Web details of the created ECC key

It’s time to sign our zone with our Combined Signing Key or Single Signing Key, both euphemisms for “I’m too poor to afford a second key pair”.

% cat $OPENSSL_CONF
openssl_conf = openssl_init

[openssl_init]
engines=engine_section

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = pkcs11
dynamic_path = /usr/lib64/engines-3/libpkcs11.so
MODULE_PATH = /usr/local/lib/pkcs11-sesam-x64-3.so
init = 1

After configuring OpenSSL’s engine to use our pkcs11-sesam.so module, I generate the pair of key files referencing the key material in the HSM. Use of the PIN value here depends on the setting in the HSM proper: the N200 can store the PIN on the device in order to not require it on the client, IMO six of one and half-a-dozen of the other in terms of security.

% cat jp.example 
$TTL 3600
@ SOA  ns  jpmens  1 3H 1H 1W 1H
  NS   @
  A    192.0.2.43

% dnssec-keyfromlabel -E pkcs11 -a13 -l "token=hsm010123F13C56131E1EEE00000025;object=JP.example;pin-value=12345678" -fKSK jp.example 
Kjp.example.+013+29467

% dnssec-signzone -E pkcs11 -t -z -S -o jp.example jp.example 
Fetching jp.example/ECDSAP256SHA256/29467 (KSK) from key repository.
Verifying the zone using the following algorithms:
- ECDSAP256SHA256
Zone fully signed:
Algorithm: ECDSAP256SHA256: KSKs: 1 active, 0 stand-by, 0 revoked
                            ZSKs: 0 active, 0 stand-by, 0 revoked
jp.example.signed
Signatures generated:                        5
Signatures retained:                         0
Signatures dropped:                          0
Signatures successfully verified:            0
Signatures unsuccessfully verified:          0
Signing time in seconds:                 5.302
Signatures per second:                   0.942
Runtime in seconds:                      5.984

I wrote earlier that I would gloss over performance, so I will. I’ve no idea where the device is (network-wise) so I’m taking these numbers with spoon-fulls of salt. (To be fair: I wouldn’t expect miracles if the device were close to me because of the seemingly large amount of HTTP requests I see in the log; recall that each DNS and NSEC{3} record needs to be signed and causes a HTTP request.)

I see in the Web UI that I can import a key to the device and try a PKCS#8-wrapped key, but that doesn’t work for me, but the API should work, and it does:

#!/usr/bin/env python3

import requests
import json

url = "https://seSAM-nserie-101/n200/web/postv1"

data = {
        "command" : "importecckey",
        "core" : 1,
        "curve" : "NIST",
        "acl" : "FFFF",
        "pin" : "12345678",
        "nice_name" : "example-dot-com-EC",
        "key_to_import" : "443A5A0F630559075B8ED453E69FB134E73BED96B7E7695D55AFBBFC085278CD"
}

r = requests.post(url, data, verify='server63.crt')

print("status: %s, %s" % (r.status_code,
        json.dumps(json.loads(r.text), indent=4)))

The RAW key data for key_to_import I obtained from the ECDSA key I want to import:

% dnssec-keygen -a13 -fksk example.com
Generating key pair.
Kexample.com.+013+37354

% awk '/PrivateKey:/ { print $2 }' Kexample.com.+013+37354.private | base64 -d | ./hexlify
443A5A0F630559075B8ED453E69FB134E73BED96B7E7695D55AFBBFC085278CD

% ./jpimport.py
status: 200, {
    "command": "importecckey",
    "core": 1,
    "result": "hsm010123F13C56131E1EEE0000000000000018"
}

There’s a so-called “TCP-RAW-API” which allows direct access to the crypto processor as well as an HTTP GET API which uses the URL to transport a request to the appliance; the response to this is JSON. Here are a few examples including symmetric encryption and decryption. (I’ve not yet tried the factoryreset…) In the following examples I omit specifying the certificate verification option in the interest of brevity.

% curl https://seSAM-nserie-101/n200/timestamp/  
{"command": "timestamp", "result": "1664657599"}

% curl https://seSAM-nserie-101/n200/getrandom/5/
{"command": "getrandom", "numbers": "5", "core": 1, "result": "EBFB604875"}

% curl https://seSAM-nserie-101/n200/getkeyattributes/k01/
{
  "command": "getkeyattributes",
  "result": {
    "hsm010123F13C56131E1EEE0000000000000026": {
      "key_id": "hsm010123F13C56131E1EEE0000000000000026",
      "key_type": "SYM",
      "key_len": 256,
      "key_acl": "FFFF",
      "key_owner": "pkmaster",
      "nice_name": "k01"
    }
  }
}

% curl https://seSAM-nserie-101/n200/encrypt/k01/12345678/CBC/hello-world/
{
  "command": "encrypt",
  "core": 1,
  "result": "A84663E043099BC74C11628DAFB69B6D8FAABCFBF4FC4AF33657B719C96617B772D229AB8A9A0E7B3903C6E404121980"
}

% curl https://seSAM-nserie-101/n200/decrypt/k01/12345678/CBC/A84663E043099BC74C11628DAFB69B6D8FAABCFBF4FC4AF33657B719C96617B772D229AB8A9A0E7B3903C6E404121980/
{
  "command": "decrypt",
  "core": 1,
  "result": "hello-world"
}

% curl https://seSAM-nserie-101/n200/listkeys/NIST/
{
  "command": "listkeys",
  "core": 0,
  "result": {
    ...
    "hsm010123F13C56131E1EEE0000000000000025": {
      "slot": "hsm010123F13C56131E1EEE00000025",
      "key_id": "hsm010123F13C56131E1EEE0000000000000025",
      "id": "AAAB68606BE2",
      "key_type": "NIST",
      "key_len": 256,
      "key_acl": "FFFF",
      "nice_name": "JP.example",
      "date_of_creation": "2022/10/01 20:15",
      "date_of_last_use": "2022/10/01 20:32",
      "date_planned_lifetime_until": null,
      "token_name": "hsm010123F13C56131E1EEE00000025",
      "key_owner": "pkmaster",
      "public_key": "FACD4CF599AD0CBD43A5C38B294AF3DDC06FBC0883D419EEBCBF8039C132FC61FE972236B38A1C236E5EF0F1E78B64B31C9632C0D5B9CFA49AECDB89144A2C3C"
    }
  }
}

The sematicon N200 is a network-attached HSM and is positioned as an IoT-specialized device, and I felt comfortable using it after a bit of study. The documentation is good, and the chapter on APIs describes each call in detail for the three access methods (RAW, GET, POST).

My specific use-case (signing DNS records via a PKCS#11 interface) is not a forte of the appliance, but I was told that from the outset. In spite of this, I was positively surprised that all the PKCS#11 functions required by the utilities I used were supported.

As an all-purpose HSM this appears to be a very interesting device at a price far below what the market “leaders” demand for a networked appliance.

dnssec and hsm :: 02 Oct 2022 :: e-mail

The level of complexity caused by KSK/ZSK split in DNSSEC isn’t necessary for most purposes; you can in fact just have one key for a zone and sign everything with it such as co.uk. does. I make frequent use of these Combined (or Single) Signing Keys as shown here.

However, the KSK/ZSK split has one advantage: it allows us to keep the KSK offline and only bring it out occasionally to sign the DNSKEY RRset. This key can be kept stable (i.e. it doesn’t need to be changed frequently or even at all) and thus the interaction with the parent (DS submission) be kept at a minimum.

Jaromir Talir held a presentation about offline KSK with Knot DNS, and from the terminology he used in 2019 I made this diagram a few years ago to help me better understand the flow of keys.

key flow in offline KSK signing

The idea is that the KSK (depicted on the left) is managed completely offline, e.g. on a laptop which is otherwise kept in a safe place. Only public key material is carried around – on floppy disks or USB memory keys, say, and as such there is an extremely low risk of the KSK being compromised.

The ZSK and the zone it signs are maintained on the right, online and/or on a hidden primary. Since the ZSK can easily be replaced there is little risk of compromise, and the zone data is public anyway.

Once created, the ZSK (or ZSKs – plural) are copied to the KSK machine for signing. The KSK signer uses the KSK to sign the DNSKEY RRset and the resulting RRset and its signatures (RRSIG) are transported back to the zone signer on the right. This data is now public: the DNSKEY RRset contains the public keys for the zone.

The zone is now signed (on the right) with the ZSK only, so this signs all of the RRsets in the zone, and as a last step we enrich the newly-signed zone with the public DNSKEY of the KSK and its signatures, and this zone (named example.com.final in the diagram) we then load and serve.

The diagram shows “key requests” being passed left and right. These could be signed (PGP?) containers carrying public key material, but for our purposes simple files containing the public DNSKEY records suffice.

In the example which follows, the offline KSK will be managed with tools from the ldns toolchain, and the zone signer with tools from BIND and optionally from ldns. Why? Just to introduce diversity.

creating the keys

We begin by preparing our environment. Everything in the directory left contains files which we will have on the offline KSK machine; conversely, the directory right contains files on our actual zone signer. Obviously these directories will be on distinct machines.

% mkdir left right
% touch left/KSK.data                       # this file will contain public data

% cd right/
right% dnssec-keygen -a13 example.com       # generate ZSK
Generating key pair.
Kexample.com.+013+54147

right% ln -s Kexample.com.+013+54147.key ZSK.key

We copy the public ZSK.key to the left. In the following script, I have to replace $INCLUDE.*$ by the actual public ZSK DNSKEY RR because ldns-read-zone and friends don’t understand $INCLUDE.

% cd left/
left% ldns-keygen -a13 -k example.com       # generate KSK
Kexample.com.+013+11238

left% cat example.com
$TTL 3600
@ SOA  unused.invalid. jp.invalid.  1 3H 1H 1W 1H
  NS   unused.invalid.

; $INCLUDE "../zone/ZSK.key"

; This is a ZONE-SIGNING key, keyid 54147, for example.com.
example.com. IN DNSKEY 256 3 13 ah+p9L1ydOMAWsHTW9sLokRDrA1by6TV6ZPLKPDhkowEMAN7o36Vs1Bt db+obk6h22D548XWTWV6cWEkbNYHPQ==

left% cat Kexample.com.+013+11238.ds
example.com.	IN	DS	11238 13 2 2d0d834fe6aea4c80ec3bcbf5af753653c9d9ea6dc997ea86e90c2615edef19f

Note the DS record which we will later submit to the parent. This is public data, so we can safely copy it to the right for further submission.

On the left, the content of the zone doesn’t matter, and I’ve used invalid to emphasize the fact. We will now sign that zone with the KSK, and I set the signature validity to a date I’ll recognize in the final zone. As there’s no portable method of getting date(1) to add an offset to epoch, I’ve created eitime for this. In future: -e $(eitime -z 60) should do the trick.

The only records from this signed zone we are interested in are the DNSKEY and RRSIG over DNSKEY.

left% ldns-signzone -e 20221224 -o example.com example.com Kexample.com.+013+11238

# now grab the KSK DNSKEY record and its RRSIG
left% ldns-read-zone example.com.signed |
	awk '($4 == "DNSKEY" && $5 == 257) || ($4 == "RRSIG" && $5 == "DNSKEY") { print }' | tee KSK.data
example.com.	3600	IN	DNSKEY	257 3 13 cMW3t5/jWcESqrzNgNjksnfUZdS4TtqO1gOae0cBRxgopoT/FNIa5GATNQAg64TJw3tgECVOq1beXpbJGSjG7Q== ;{id = 11238 (ksk), size = 256b}
example.com.	3600	IN	RRSIG	DNSKEY 13 2 3600 20221224000000 20220922073938 11238 example.com. 35J+toX0FyIyXRTNpamM6UVyivLFV1deUSWrJl+3DCVH2feolbDP45i/myLDnnywAS4NzQIZqjdkZQMTyVSxfg==

left% wc -l KSK.data
       2 KSK.data

The file KSK.data contains the public key and signature which we transfer (floppy disk, remember?) back to the right.

signing the zone

Back on the right we have the zone, the ZSK, and the public KSK in the file KSK.data, so we can now sign this:

right% ./bsig.sh
*** sign the zone; set SOA serial to epoch
example.com.signed
*** remove dsset file as it contains ZSK only
*** replace RRSIG over DNSKEY created by ZSK with that created by KSK
*** verify signed zone using ldns-verify-zone
Zone is verified and complete
*** verify signed zone using dnssec-verify
Loading zone 'example.com' from file 'example.com.signed'
Verifying the zone using the following algorithms: ECDSAP256SHA256.
Zone fully signed:
Algorithm: ECDSAP256SHA256: KSKs: 1 active, 0 stand-by, 0 revoked
                            ZSKs: 1 active, 0 stand-by, 0 revoked
*** verify signed zone using validns
records found:       17
skipped dups:        0
record sets found:   10
unique names found:  3
delegations found:   0
    nsec3 records:   0
not authoritative names, not counting delegation points:
                     0
validation errors:   0
signatures verified: 8
time taken:          0.003s

The b in bsig.sh is for signing with the BIND utilities, and there’s an lsig.sh which uses ldns for signing.

Both scripts work in a similar way: they first sign the zone using only our ZSK (BIND’s smart signing finds the key by itself, with ldns I have to specify it). Then they remove the RRSIG over DNSKEY which was created by the ZSK (we don’t need that; our DNSKEY RRset is signed by the KSK), and add the public DNSKEY of the KSK and its RRSIG, both obtained from left. Finally, we use three distinct tools to verify the validity of the zone.

The signed zone file is the one we (re-)load on our (hidden) primary server.

refreshing

Signing the zone and therewith refreshing signatures on the (hidden) primary shouldn’t be too much of a problem and can easily be scheduled; cron(8) comes to mind.

A bit more involved is refreshing the signatures over the DNSKEY RRset with the KSK on left. Depending on the chosen method of public DNSKEY and RRSIG transport (the file KSK.data) from left to right this can actually mean a manual task (“floppy disk”) which must be done before signatures expire.

finally

As the diagram above suggests, there’s no reason why you couldn’t add complexity security by adding a bunch of HSMs here and there.

And if you don’t like using floppy disks to transport the public keys from right to left and back, but want to make life difficult for intruders, a serial cable and the likes of UUCP or Kermit might fit the bill. :-)

Further reading

dnssec and dns :: 22 Sep 2022 :: e-mail

I had to abort an exercise I gave students last night during a BIND training, because after adding

allow-new-zones yes;

and reconfiguring a running BIND 9.16 server, the process complained it couldn’t open _default.nzd. (If compiled with LMDB support, BIND uses the memory-mapped database for storing new zone definitions instead of the “old” .nzf file.)

We run these training systems with SELinux enabled (I wouldn’t, but my colleague likes it :-), and that’s the reason I aborted the lab: I couldn’t tell students how to solve the cause other than by disabling SELinux entirely, but there wasn’t enough time for that.

After getting a good pointer on Twitter (thank you Howard, Stef, and Evgeni), I saw this:

#!!!! This avc can be allowed using the boolean 'domain_can_mmap_files'
allow named_t named_zone_t:file map;

The comment contained “boolean”, which I recognized, so I was able to fix the issue an hour after the training. The command

setsebool -P domain_can_mmap_files 1

has now found its way into the setup playbooks for these machines.

dns :: 13 Aug 2022 :: e-mail

The idea for an Ansible reference (or cheat) sheet was reborn last week at the Linuxhotel; a few students who knew they would be receiving a tmux mug as a gift when leaving asked why we don’t give out Ansible mugs.

One person approached me and said mugs are impractical: most people already have their favorite beverage mug, mugs have little space for information, and they typically find their way into the back of some kitchen cabinet. Why don’t we consider a nicely-printed DIN A3 sheet on good (plastified) paper? I thought that a good idea.

Ansible cheat sheet

And so began the work of gathering the information I wanted to present and packing it onto a double-sided A3 page. I’d have very much liked to add details for our advanced course, but there’s simply not enough space on these two pages. (If your eye sight is very good you might be able to read it when printed on A4.)

So, here’s the reference card, and please let me know if you have feedback.

ansible and documentation :: 24 Jul 2022 :: e-mail

My request was pretty easy, I thought, when I asked the person to give me a list of student names and their email addresses so I could send out invitations.

I got a spreadsheet:

spreadsheet

Five empty rows. Eight rows with an additional third column containing a first name which is already in column A. Two additional rows with some blurb and a given name, but as there’s no email address associated, I can’t use the entry.

Mind-boggling. Even more so when you know what this IT person’s title is.

people :: 19 Jul 2022 :: e-mail

Other recent entries