YubiHSM hardware test suite¶

Validating secsy-pki against a real device, not a software token.

Every other HSM test in this repository runs against SoftHSM. SoftHSM implements the PKCS#11 API faithfully and shares none of the constraints that make a real HSM a real HSM: it has no USB transport, no SCP03 secure channel, no 62-slot append-only audit log, no attestation certificates, and no device options that cannot be undone. Those are precisely the properties the enterprise claims in this codebase rest on, so they can only be validated on hardware.

server/internal/yubihsmtest is that validation — one suite, seven tiers, from the wire up to the product.

Running it¶

The suite is off unless you ask for it:

./scripts/yubihsm-test.sh              # everything
./scripts/yubihsm-test.sh --quick      # skip the slow RSA-4096 case
make test-yubihsm                      # same as the first form

or directly:

SECSY_YUBIHSM_TESTS=1 go test -tags sqlite -p 1 -count=1 ./internal/yubihsmtest/ -v

-p 1 is not a preference. The device admits one session at a time over direct USB, so two test binaries running at once fight over the interface and report device or resource busy instead of anything about the code.

The parts of tier 4c and all of tier 10 need the device through the PKCS#11 module and the native driver at once, which is how a running server uses it. They used to skip on a yhusb:// run, with an instruction to go and start Yubico's daemon; they now start the embedded connector instead — the same in-process relay a single-container deployment runs — so a plain run covers them:

SECSY_YUBIHSM_TESTS=1 ./scripts/yubihsm-test.sh

Pointing the suite at an external connector still works and still exercises that path:

yubihsm-connector -l 127.0.0.1:12345 &
SECSY_YUBIHSM_CONNECTOR=http://127.0.0.1:12345 ./scripts/yubihsm-test.sh

Tier 4c's first test is the measurement that explains why a relay is needed at all: it holds a PKCS#11 session open, deliberately without the embedded connector, and asserts that the drain fails as a recognised device-busy condition. See the transport must be shareable.

Why an environment variable and not a build tag¶

The suite compiles on every ordinary build and skips at runtime, so it is type-checked and linted continuously — which is where test rot is actually caught — while costing CI nothing but a compile. A build tag would hide it from both.

The per-package hardware tests that predate this suite (internal/hsm, internal/hsmattest, internal/hsmaudit, internal/pki, internal/yubihsm) stay behind the yubihsm build tag, because each declares a TestMain that would otherwise take over that package's SoftHSM tests. internal/yubihsm was untagged until it was found to hang: it skipped when no device answered, but a device that is enumerated and unresponsive — mid-reboot after a factory reset, or on a USB/IP link whose far side went away — answers neither, and the probe lands in a usbfs wait no context deadline can cancel. Run them alongside the suite with ./scripts/yubihsm-test.sh --legacy.

Configuration¶

Variable	Default	Meaning
`SECSY_YUBIHSM_TESTS`	(unset)	`1` enables the suite. Without it, every test skips.
`SECSY_YUBIHSM_DESTRUCTIVE`	(unset)	`1` additionally allows irreversible device changes (see below).
`SECSY_YUBIHSM_RESET`	(unset)	`1` additionally allows the genesis tier to factory-reset the device, erasing every key and the whole log. Requires `SECSY_YUBIHSM_DESTRUCTIVE=1` as well.
`SECSY_YUBIHSM_CONNECTOR`	`yhusb://`	Connector URL. Falls back to `YUBIHSM_CONNECTOR`.
`SECSY_YUBIHSM_PASSWORD`	`password`	Authentication key password. Falls back to `YUBIHSM_PASSWORD`.
`SECSY_YUBIHSM_AUTH_KEY_ID`	`1`	Authentication key object id.
`SECSY_YUBIHSM_PKCS11_MODULE`	(autodetected)	Path to `yubihsm_pkcs11.so`. Probed across the usual distribution paths.

With SECSY_YUBIHSM_TESTS=1 set and no device reachable, the suite fails rather than skipping. That is deliberate: you asked for hardware tests, so an unplugged or busy device is a result, not a non-event. A suite that reported PASS while touching no hardware would be worse than no suite.

The container tier¶

The suite above runs go test on your host, against whatever yubihsm_pkcs11.so that host happens to have. What operators run is the container, whose module is a different build — and, before the image compiled it from source, was a different version. Neither the suite nor a passing CI run can see that difference, so there is a second script for it:

./scripts/yubihsm-container-test.sh                       # the published main-yubihsm image
./scripts/yubihsm-container-test.sh --image secsy-pki:local-yubihsm
make test-yubihsm-container                               # same as the first form

Everything it checks runs inside the image, as the image's own non-root user, reaching the device the way container deployment tells operators to — --device plus --group-add, with the udev rule the image ships installed on the host. Nothing is compiled from the working tree; the binaries under test are the ones in the image.

It settles, in order: the module the image carries and its version; that the device answers over yhusb:// from inside as uid 65532; that secsy-ca import-key places RSA-2048, RSA-3072 and RSA-4096 keys and that each is verified by signing on the device; that the resulting object really is an asymmetric-key with the right algorithm, may sign, and is neither exportable under wrap nor unwrap-capable; that the same import works for every encoding openssl emits — PKCS#8, traditional PKCS#1, both of their encrypted forms, bare DER and PKCS#12, each generated on the host outside the image, each asserted to have been read as that format rather than merely accepted, and the encrypted ones driven through both ways a passphrase can reach a container (-e SECSY_KEY_PASSPHRASE and a mounted -pass-file); that secsy-ca inventory lists each at the expected key type and reports it non-extractable, which is the product's own view of that invariant; that ca import adopts a legacy RSA CA whose certificate was self-signed before the migration and issue then produces a leaf which openssl verify accepts under it; that init-root -key-type rsa-2048 generates in the device — the sibling of the import bug below, so importing alone proves half of it (2048 rather than 4096 for generation: the bug is not size-specific, and 4096 bits take a minute and a half on this hardware where 2048 take seconds); and finally where the device audit log is permanently written: that SECSY_YUBIHSM_AUDIT_LOG_FILE is the path hsm-audit status then reports and the one the server creates on a mounted volume, that the file it creates verifies offline with hsm-audit verify-file, that an uncommissioned device is refused with the reason instead of silently writing nothing, and that a destination the server cannot append to stops it starting rather than being ignored.

--legacy-module DIR repeats the import against a pre-2.7.2 module mounted over the image's, which is the shape of a deployment that supplies its own vendor stack. --help prints a recipe for building such a directory out of Debian's packages. The claim it checks is that the template adapts and the import still works, not that the old module is fine.

It owns object ids 0x7d00–0x7d1f and labels t201-c-* / t203-c-*, which do not overlap the ranges the Go suite uses, and deletes what it created — both object types, not just the one it meant to make, since the failure it is built to detect leaves the wrong one behind.

What it cannot settle is the drain itself, on a device nobody may commission for it: hsm-audit provision turns on irreversible forced audit logging and requires a factory-reset device, so a test may not perform it. On an uncommissioned device the script therefore asserts the refusal and its wording; on a commissioned one it asserts that the records land in the configured file. Which branch ran is in the transcript, along with the device's own force-audit option as the script found it.

Like the Go suite it drains the device audit log when the 62-entry ring runs short, for the same reason: a device with force-audit enabled stops accepting commands once the log fills, and this script performs enough operations to fill it. Draining destroys the device's only copy of those entries, so do not point it at a device whose log a deployment is collecting.

What it does to a device¶

Do not point this at a device a deployment is using. Specifically:

It creates and deletes scratch objects in the 0x7f00–0x7f1f handle range, and keys labelled t172-* at module-assigned handles. It touches nothing else, and sweeps both before and after every run. Tier 3b additionally uses the reserved device-challenge slot 0xfa00, because that range is fixed by the production code a deployment also runs; it clears the slot before and after itself.
It consumes device audit-log entries. With forced audit the log is 62 slots deep and the device refuses every audited operation once it is full. A full run produces well over 62 audited operations, so the suite drains the log as it goes and says so in the output. Drained entries are gone from the device — which is exactly what a deployment's audit collector needs to see.
It does not write device options unless SECSY_YUBIHSM_DESTRUCTIVE=1.

The destructive gate covers exactly one thing: provisioning forced audit on a fresh device. Setting the per-command audit levels to fixed survives every power cycle, and the only way back is a factory reset that destroys every key on the device. Running the hardware tests is not consent to that.

It is worth knowing what the gate costs, though, because the answer is a tier of coverage rather than a detail. Six tests in internal/hsmaudit -tags yubihsm need a commissioned device and skip on any other — forced-audit irreversibility, signature counting, the RFC 3161 freshness round trip, the per-key proof, the exportable-key refusal, and the device-signed serial commitment. They are the ones that exercise the audit argument end to end, so a device nobody will commission leaves the most interesting half of this suite unrun. The round trip is one command each way:

secsy-ca hsm-audit reset -confirm <serial>   # dry run without -confirm
secsy-ca hsm-audit provision                 # irreversible until the next reset

Over USB/IP a reset needs the client to re-import the device afterwards; see the audit-log page.

SECSY_YUBIHSM_RESET=1 is a second gate on top of it, for the genesis tier, which factory-resets the device several times. A reset erases more than a configuration — every key and every log entry go with it — and it runs mid-suite, so it would also undo the forced audit tier 4 had just established. Opting in separately keeps a destructive run predictable. After a genesis run the device is at factory defaults: authentication key id 1, password password, no forced audit.

internal/hsmaudit had a latent version of the same hazard, now fixed: a Service built over a scripted fake device provisioned through a zero hsm.Config, whose empty connector URL resolves to the real direct-USB default — so a plain go test ./internal/hsmaudit/ on a machine with a YubiHSM attached silently force-audited it. Provisioning is now a method on the Device interface, so a fake can only provision itself.

The tiers¶

Bottom-up on purpose: a failure in tier 1 explains failures in every tier above it, so the first failing tier names the layer at fault.

Tier	File	What it establishes
1	`driver_test.go`	The wire works: device identity is stable, SCP03 survives payloads across every AES-block and USB-packet boundary, the counter chains over 200 exchanges, concurrent callers are serialised without crossing responses, wrong password and wrong key id are both rejected, device refusals arrive as typed `DeviceError`s and do not poison the session, cancellation reaches the device, and reading the audit log does not consume it.
2	`keys_test.go`	Key lifecycle over the full algorithm matrix: generate / read public half / sign / verify with the standard library on P-256, P-384, P-521 and Ed25519; ECDSA nonces differ and Ed25519 is deterministic; imported keys sign as the key that was imported; capabilities are enforced; delete deletes; two keys from one template differ; and `GET PUBLIC KEY` agrees with the attestation certificate.
3	`attestation_test.go`	Attestation says something true: a device-generated non-exportable key verifies, an exportable one fails, an imported one fails, an attestation binds to a specific public key and rejects another, a flipped signature bit is caught, label resolution is exact, and chain anchoring is reported honestly (see below).
3b	`device_test.go`	The device the assertions come from is real: its factory certificate chains to Yubico's published attestation root and the serial it asserts is the one `GET DEVICE INFO` reports, it answers a fresh nonce with its attestation key, two challenges produce two different answers and neither satisfies the other, the throwaway challenge key is gone when the call returns, and an unexpected serial fails. See device attestation.
4	`audit_test.go`	The audit trail can be complete: the device's forced-audit configuration meets the baseline, a fixed setting cannot be downgraded, signatures are attributed to the handle that made them, collection stays continuous across drain seams spanning more than one 62-entry ring, and a full log stops the device rather than dropping records.
4b	`audit_sink_test.go`	Where the drained entries end up, since acknowledging the ring destroys the device's only copy: a spread of operations — generate, sign ECDSA, sign EdDSA, attest, delete, and the same through PKCS#11 — each reaches both the database and the append-only file, the file verifies from its raw device records alone, every command the driver sends prompts a drain while the drain's own three commands do not, and 100 operations across a 62-entry ring complete without the device ever refusing.
4c	`steady_state_test.go`	That the audit argument survives ordinary operation. Direct USB is measured to be exclusive — a drain while the PKCS#11 module holds a session fails as a recognised device-busy condition, which is the fact the server's startup refusal rests on. Then, through a connector (the embedded one on a direct-USB run, or an external one when configured): 70 leaves and a CRL issued by `ca.Manager` with the collector running after every operation, the device log never rising above a couple of entries, both durable copies agreeing, and the device log reconciling exactly against the signature ledger across two keys — no surplus, which would read as key abuse, and no deficit. The same for the secret layer's signing service, whose signatures reached neither half of the subsystem before Task 202. Needs `-tags sqlite`.
5	`genesis_test.go`	What a factory reset writes, and therefore what the chain anchor is worth: the device-init sentinel's hashed bytes are the constant `0001ffffffffffffffffffffffffffff` on every reset, while the digest the device reports for them differs on every reset and is reproducible from no publicly guessable seed — so the anchor cannot be recomputed and must be pinned out of band. Also that a reset restarts the log at entry 1 with no unlogged-operation counters. Gated on `SECSY_YUBIHSM_RESET=1`; see why the anchor cannot verify itself.
6	`pkcs11_test.go`	The layer the product signs through: generate/find/sign/verify for every offered key type through `keyprovider`, the readiness probe and hardware RNG, concurrent signing through the session pool over a device that cannot parallelise, the secret-envelope round trip, keys created non-exportable, and PKCS#11 labels and native handles resolving to the same object.
6b	`import_test.go`	Existing key material going the other way — onto the device. RSA-2048/3072/4096 imported through `keyprovider`, each proved to be the key that was sent by signing with both PKCS#1 v1.5 and PSS and verifying under the host key; a decrypt-only RSA KEK unwrapping what the host wrapped; a requested `CKA_ID` honoured so the handle a config names is the handle the key is at; the imported key attesting as imported and still non-exportable; a legacy RSA CA issuing a leaf that verifies under the root certificate published before the migration; and the sizes and exponents the device cannot hold being refused on the host with a sentence rather than `CKR_ATTRIBUTE_VALUE_INVALID` from the far side of USB.
7	`pki_test.go`	The product itself, on the device: a root CA, an intermediate signed by it, a leaf whose chain verifies as a TLS client would build it, revocation and a signature-checked CRL, and an SSH CA whose certificate `ssh.CertChecker` accepts for its principal and rejects for another. Needs `-tags sqlite`.
10	`embedded_test.go`	That one process is enough. The embedded connector's status document names the attached device and satisfies the field Yubico's module gates on; the native driver reaches the device by its plain `yhusb://` URL while the relay holds the only claim; Yubico's own module generates a key and signs over the relay, which no fake device can establish; and the device audit log is drained while that module's session pool is open — the combination `hsmaudit.DrainContention` refuses a deployment for — with the relay's operation hook shown to have signalled the module's work. Finally, that the runtime behaviour and the startup check agree: the same configuration is refused without the relay and accepted with it.

Findings this suite pinned down¶

Written down because each was a surprise, and a surprise that SoftHSM could not have produced.

The secret layer was broken on YubiHSM. The KEK template asked for CKA_WRAP/CKA_UNWRAP. Yubico's module maps a template requesting wrapping onto a device wrap-key object, and a wrap-key is not exposed as CKO_PRIVATE_KEY — so secret.ProvisionKEK generated a KEK successfully and the immediately following lookup failed with private key label not found. Every secret in such a deployment would have been unencryptable. The attributes were never used (the envelope layer unwraps with C_Decrypt, not C_UnwrapKey), so the fix was to drop them. SoftHSM draws no distinction between object types, which is why this survived until the suite ran.

A CKA_UNWRAP = FALSE in an RSA template made Yubico's module create a wrap-key. Before 2.7.2 the module tested the presence of the attribute rather than its value — if (template.unwrap), against an enum whose ATTRIBUTE_FALSE is 1 — so an assertion that a key must not unwrap produced a device wrap-key instead of an asymmetric key. This was on the import path (C_CreateObject) and both generate paths, inside the RSA arm, which is why it took RSA and left EC alone. Upstream fixed all three sites in 2.7.2; Debian bookworm-backports ships 2.6.0 and bookworm proper ships nothing, so the module an operator installs from their distribution is affected today. Two things follow, and both are in the tree: the -yubihsm image compiles 2.8.0 from Yubico's signed release rather than installing Debian's, and the template consults the module before asserting the attribute — omitted for the affected releases, where a YubiHSM derives capabilities from the template and so omitting it means exactly what the explicit FALSE was asking for, and kept everywhere else, because SoftHSM's default for a private key is CK_TRUE and dropping it outright would hand every software-token deployment an unwrap-capable CA key. See server/internal/pki/module_quirks.go.

A full audit log presents as CKR_DEVICE_MEMORY. The module maps the device's log-full refusal onto a PKCS#11 error that reads like exhausted storage. Worth knowing before diagnosing a phantom capacity problem.

Failed operations are logged. A signature attempt against a nonexistent handle appears in the audit log with a non-zero result. The suite relies on this to anchor a freshly drained log without leaving an object behind, and it strengthens the audit claim: a trail that recorded only successes would hide every probe of a key an attacker does not have.

Ed25519 works through this PKCS#11 module. Older Yubico releases exposed no EdDSA mechanism, which made "the YubiHSM supports Ed25519" true of the device and false of the only path the product can reach it through. It is now true of both, and tier 6's matrix is where a regression would show.

Attestation chains anchor to Yubico's published roots. The device certificate is issued by a Yubico YubiHSM <n> Sub-CA which Yubico publishes individually, named after its own subject key identifier; that sub-CA and the Yubico YubiHSM Root CA above it both ship embedded, so hsmattest.Policy.RequireAnchoredChain is on by default and tier 3 asserts the chain rather than merely reporting where it terminates. A device whose sub-CA postdates the binary fails with the URL that fixes it. See key attestation.

RSA is slow — to generate. On a YubiHSM 2, RSA-3072 key generation measured 25-45 seconds and RSA-4096 about 1m33s. Any timeout on an issuance or key-ceremony path has to accommodate that; ECDSA is about a second. Importing the same keys takes 0.9-2.3 seconds, because the device is storing primes rather than searching for them — so a migration is not subject to the ceremony timeouts a generation is.

The device takes three RSA sizes and one exponent, and will not say which is wrong. A YubiHSM 2 implements rsa2048, rsa3072 and rsa4096 with public exponent 65537. A 2560-bit key, or a 2048-bit key with e=3, is refused with CKR_ATTRIBUTE_VALUE_INVALID — the same code that covers an unsupported algorithm — after a round trip, with nothing to distinguish the causes. SoftHSM accepts all of them, so the import path looked healthy under test and failed opaquely on hardware. Both constraints are now checked on the host before the device is touched (see import), and tier 6b asserts on the substance of each message: an unhelpful rejection is the failure mode being guarded against, so "it errored" would not be the property worth testing.

Reference device¶

The suite was developed and run against:

serial 31650425, firmware 2.4.0, part 78CLUFX5000P
audit log 62 slots, force-audit = fixed, 24 commands force-audited

↩ Back to the HSM & key management index · documentation map