OpenSSL with libp11 for Signing, Verifying and Encrypting, Decrypting

OpenSSL can be used with pkcs11 engine provided by the libp11 library, and complemented by p11-kit that helps multiplexing between various tokens and PKCS#11 modules (for example, the system that the following was tested on supports: YubiHSM 2, YubiKey NEO, YubiKey 4, Generic PIV tokens and SoftHSM 2 software-emulated tokens).

Signing and Verifying

RSA-PKCS#1 v1.5

$ openssl dgst -engine pkcs11 -keyform engine -sign
   "pkcs11:token=YubiHSM;id=%04%01;type=private" -out t3200.pkcs1.sig
   -sha384 t3200.dat
engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:

$ openssl dgst -engine pkcs11 -keyform engine -verify
   "pkcs11:token=YubiHSM;id=%04%01;type=public" -signature t3200.pkcs1.sig
   -sha384 t3200.dat
engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
Verified OK
$

RSA-PSS

$ ~/openssl-1.1/bin/openssl dgst -engine pkcs11 -keyform engine -sign
    "pkcs11:token=YubiHSM;id=%04%01;type=private" -out t6400.txt.sigpss
    -sigopt rsa_padding_mode:pss -sha384 t6400.txt
engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:

$ ~/openssl-1.1/bin/openssl dgst -engine pkcs11 -keyform engine -verify
    "pkcs11:token=YubiHSM;id=%04%01;type=public" -signature t6400.txt.sigpss
    -sigopt rsa_padding_mode:pss -sha384 t6400.txt
engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
Verified OK
$

ECDSA

$ openssl dgst -engine pkcs11 -keyform engine -sign
   "pkcs11:token=YubiHSM;id=%02%03;type=private" -sha384 -out
   t3200.ecdsa.sig t3200.dat
 engine "pkcs11" set.
 Enter PKCS#11 token PIN for YubiHSM:

 $ openssl dgst -engine pkcs11 -keyform engine -verify
    "pkcs11:token=YubiHSM;id=%02%03;type=public" -sha384 -signature
     t3200.ecdsa.sig t3200.dat
 engine "pkcs11" set.
 Enter PKCS#11 token PIN for YubiHSM:
 Verified OK
 $

Encrypting and Decrypting

RSA-PKCS

$ cat t64.txt
4aa58c448f3264c777be1b5ad94cf3e0a68911ed3f18db9e568ff2179e263f76

$ ~/openssl-1.1/bin/openssl pkeyutl -engine pkcs11 -keyform engine
     -pubin -encrypt -inkey "pkcs11:token=YubiHSM;id=%04%02;type=public"
     -pkeyopt rsa_padding_mode:pkcs1 -in t64.txt -out t64.txt.pkcs1
engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:

$ ~/openssl-1.1/bin/openssl pkeyutl -engine pkcs11 -keyform engine
     -decrypt -inkey "pkcs11:token=YubiHSM;id=%04%02;type=private"
     -pkeyopt rsa_padding_mode:pkcs1 -in t64.txt.pkcs1
engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
4aa58c448f3264c777be1b5ad94cf3e0a68911ed3f18db9e568ff2179e263f76
$

RSA-OAEP

$ cat t64.txt
4aa58c448f3264c777be1b5ad94cf3e0a68911ed3f18db9e568ff2179e263f76

$ ~/openssl-1.1/bin/openssl pkeyutl -engine pkcs11 -keyform engine
     -pubin -encrypt -inkey "pkcs11:token=YubiHSM;id=%04%02;type=public"
     -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt
      rsa_mgf1_md:sha384 -in t64.txt -out t64.txt.oaep
engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:

$ ~/openssl-1.1/bin/openssl pkeyutl -engine pkcs11 -keyform engine
     -decrypt -inkey "pkcs11:token=YubiHSM;id=%04%02;type=private"
     -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt
      rsa_mgf1_md:sha384 -in t64.txt.oaep
engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
4aa58c448f3264c777be1b5ad94cf3e0a68911ed3f18db9e568ff2179e263f76
$

ECDH

$ openssl pkeyutl -engine pkcs11 -keyform engine -derive -inkey
    "pkcs11:token=YubiHSM;id=%02%04;type=private" -peerkey peer_key.der
engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
34a03079c38947a679a924f3e20657cd4f69dd36df395b7e759e727524da87dc