• HSM

HSM Hardware security Module

• Secure way of saving private/public keys, that get wiped/destroyed when attempts are made to retrieve the private key.
  • Standards: FIPS-120 Compliance.

Links

• Linux/Vault

• http://cloudhsm-safenet-docs.s3.amazonaws.com/007-011136-002_lunasa_5-1_webhelp_rev-a/startpage_Left.htm

• http://nicolas.corrarello.com/general/vault/security/2018/03/05/Understanding-the-encryption-overhead.html

• https://docs.aws.amazon.com/cloudhsm/latest/userguide/ssl-offload-overview.html

• https://www.openssl.org/docs/man1.1.1/man1/engine.html

AWS CloudHSM

• https://docs.aws.amazon.com/cloudhsm/latest/userguide/ssl-offload-overview.html

• https://docs.nginx.com/nginx/fips-compliance-nginx-plus/

• User creation - account layout in CloudHSM
  • CloudHSM SO - Security officer, manage other users, creates the CU(Crypto Users) and reset passwords
  • CloudHSM CU - Crypto User, the account that does the actual work, and owner of the private keys. (Lockout after 10tries)
    • Recommendation:
      • Use 1-CU to create keys, delegate to 2-CU for usage. If 2-CU gets locked you can create a 3-CU and delegate to it.
• Setup

  1. https://docs.aws.amazon.com/cloudhsm/latest/userguide/install-and-configure-client-linux.html

     • wget https://s3.amazonaws.com/cloudhsmv2-software/CloudHsmClient/Xenial/cloudhsm-client_latest_amd64.deb

  2. mv private CA to /opt/cloudhsm/etc/customerCA.crt
  3. run sudo /opt/cloudhsm/bin/configure -a 127.0.0.1

  4. ssh portforward to hsm, add line's to .ssh/config for ssh host connected to cloudHSM e.g. IP 10.0.0.55.

     LocalForward 2223 10.0.0.55:2223
     LocalForward 2225 10.0.0.55:2225

  5. start the cloudhsm_mgmt_util /opt/cloudhsm/bin/cloudhsm_mgmt_util /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg
     1. 1st Time initialize

        1. enable end-to-end encryption. aws-cloudhsm>enable_e2e ##Only once.

        2. aws-cloudhsm>listUsers

        3. aws-cloudhsm>loginHSM PRECO admin password

        4. aws-cloudhsm>changePswd PRECO admin <NewPassword> ##Do not lose

        5. Follow steps to get keys, create own CA, sign HSM csr, and upload

           • https://docs.aws.amazon.com/cloudhsm/latest/userguide/getting-started-ssl.html

     2. aws-cloudhsm>loginHSM CU <user01> <pwd01>

     3. aws-cloudhsm> quit

  6. Environment=n3fips_password=<usr01>:<pwd01>

     1. sudo service cloudhsm-client start
     2. Install the OpenSSL engine

        • wget https://s3.amazonaws.com/cloudhsmv2-software/CloudHsmClient/Xenial/cloudhsm-client-dyn_latest_amd64.deb

        • provides us with /opt/cloudhsm/lib/libcloudhsm_openssl.so
     3. /opt/cloudhsm/bin/key_mgmt_util

  7. Capabilities of the engine

     $ openssl engine cloudhsm -v -c
     (cloudhsm) CloudHSM hardware engine support
      [RSA, RAND, DES-EDE3-CBC, DES-CBC, AES-128-CBC, AES-192-CBC, AES-256-CBC, RC4, DES-EDE3, DES-ECB, AES-128-ECB, AES-192-ECB, AES-256-ECB, AES-128-CTR, AES-192-CTR, AES-256-CTR, SHA1, SHA256, SHA384, SHA512, rsaEncryption]

  8. Speed tests
     • $ openssl speed -engine cloudhsm rsa
     • e.g. with and without cloudhsm

       • Host	Crypto	CloudHSM	sign/s	verify/s
         Remote VM, ssh tunnel	rsa2048	No	1,465	49,348
         		Yes	3,900	9,700
         AWS local VM	rsa2048	No	1,196	27,642
         		Yes	7,100	30,060

         $ openssl speed rsa2048
         Doing 2048 bit private rsa's for 10s: 14658 2048 bit private RSA's in 10.00s
         Doing 2048 bit public rsa's for 10s: 493487 2048 bit public RSA's in 10.00s
         OpenSSL 1.0.2g  1 Mar 2016
         built on: reproducible build, date unspecified
         options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx) 
         compiler: cc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
                           sign    verify    sign/s verify/s
         rsa 2048 bits 0.000682s 0.000020s   1465.8  49348.7
         $ 
         
         $ openssl speed -engine cloudhsm 
         $ openssl speed -engine cloudhsm rsa2048
                 SDK Version: 2.03
         engine "cloudhsm" set.
         Doing 2048 bit private rsa's for 10s: 39 2048 bit private RSA's in 0.01s
         Doing 2048 bit public rsa's for 10s: 97 2048 bit public RSA's in 0.01s
         OpenSSL 1.0.2g  1 Mar 2016
         built on: reproducible build, date unspecified
         options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx) 
         compiler: cc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
                           sign    verify    sign/s verify/s
         rsa 2048 bits 0.000256s 0.000103s   3900.0   9700.0
         $
         
         
         
         $ openssl speed -engine cloudhsm rsa2048
                 SDK Version: 2.03
         engine "cloudhsm" set.
         Doing 2048 bit private rsa's for 10s: 71 2048 bit private RSA's in 0.01s
         Doing 2048 bit public rsa's for 10s: 1503 2048 bit public RSA's in 0.05s
         OpenSSL 1.0.2g  1 Mar 2016
         built on: reproducible build, date unspecified
         options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx) 
         compiler: cc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
                           sign    verify    sign/s verify/s
         rsa 2048 bits 0.000141s 0.000033s   7100.0  30060.0
         ubuntu@ip-10-0-0-233:~$ 
         
         
          openssl speed  rsa2048
         Doing 2048 bit private rsa's for 10s: 11362 2048 bit private RSA's in 9.50s
         Doing 2048 bit public rsa's for 10s: 263432 2048 bit public RSA's in 9.53s
         OpenSSL 1.0.2g  1 Mar 2016
         built on: reproducible build, date unspecified
         options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx) 
         compiler: cc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
                           sign    verify    sign/s verify/s
         rsa 2048 bits 0.000836s 0.000036s   1196.0  27642.4
         ubuntu@ip-10-0-0-233:~$ 

AWS CloudHSM Errors

• Added a new CU user, running nginx. 1stLine hsmclient, 2nd Nginx everytime there is a request

  <date> liquidSecurity ERR: e2e_send_error_to_app: Sending error 168 to app
  <date> [crit] 47#47: *21 SSL_do_handshake() failed (SSL: error:1409B006:SSL routines:ssl3_send_server_key_exchange:EVP lib) while SSL handshaking, client: 172.24.0.1, server: 0.0.0.0:443

  • Switch cloudhsm_client.cfg from "log_level": "DEBUG"
  • Problem was that the new CU user did not have access to the locked CU user's keys, this you have to delegate using the locked user.
  • Trying to reset the locked CU user using the SO failed as aws-cloudhsm kept insisting user is already logged in. Had to recreate hsm and restore.

• Used wrong password CU user locked out of hsm

   2019-11-28T05:59:32Z liquidSecurity ERR: login_hsm_reply_handler: Login failed with response code 0xa4
   Cfm2LoginHSM returned 164  HSM Error: User login fail count has reached configured maximum value, This user is locked. Get the password reset by CO 
   2019/11/28 05:59:32 [emerg] 53#53: ENGINE_set_default("cloudhsm", ENGINE_METHOD_ALL) failed (SSL: error:260B806D:engine routines:ENGINE_TABLE_REGISTER:init failed)
   nginx: [emerg] ENGINE_set_default("cloudhsm", ENGINE_METHOD_ALL) failed (SSL: error:260B806D:engine routines:ENGINE_TABLE_REGISTER:init failed)
           SDK Version: 2.03

...