The first secret, known as secret zero, for initial authentication to the Vault is typically the most difficult part to implement. Established approaches bootstrap the initial credentials either through a trusted orchestrator that injects those secrets along the way (e.g., HashiCorp Developer: Retrieving CI/CD secrets from Vault) or a trusted platform that issues credentials accepted by Vault (e.g., HashiCorp Developer: Kubernetes auth method).

As of Vault 1.14, the HashiCorp Vault PKI engine can issue certificates using the standard ACME protocol. The Automatic Certificate Management Environment (ACME) was made popular by Let’s Encrypt, which has been the default mechanism to request valid certificates from a public CA for over 10 years. Nowadays, these certificates can be issued by a corporate CA for company-wide usage. This enlarges the field of adoption significantly. Requesters no longer rely on a platform or on some entity that injects the initial secret along the way.

In this new setting, the root of trust is established through DNS or more specifically the name resolution of the requester. Corporate DNS is typically managed and tightly controlled by dedicated teams to ensure the reliable operation of critical services. Given this solid foundation and the flexibility of ACME (e.g., not bound to a specific type of certificate), we built upon the idea of using the ACME protocol to issue server certificates to also issue client certificates. These client certificates can be used as credentials to authenticate against Vault.

Client preparation

Initially, the target system has no built-in ACME capabilities. Therefore, an engineer first rolls out an ACME client (and Vault Agent, if needed) on the target system.

The target host needs to be whitelisted on the certificate authentication backend on the Vault server (see step 1 in the diagram “Host Preparation & Rollout” below). Afterward, the ACME client on the host can request a client certificate from Vault, e.g., using the HTTP-01 challenge for verification (2 and 3).

Vault configuration

The PKI engine needs to be configured with ACME protocol that allows issuing client certificates. Additionally, the certificate authentication method has to be enabled and a role mapping the certificates to Vault identities has to be configured. Custom permissions can then be added to those identities.

Accessing secrets

The target system hosts the code of the application (see step 4 in the diagram “Vault Authentication and Secrets Retrieval” below). Developers write code that uses the client certificates to authenticate to Vault and access secrets (5 and 6). Vault Agent can be used to simplify authentication and token lifecycle in the application or for templating secret values into (configuration) files.

At this stage, the developers can assume an authenticated connection to the Vault server with the client certificate. Authorization is assured by Vault policies attached through the mapping of the client certificate to a Vault internal identity, e.g., read a secret from a specific path or fetch dynamically generated secrets for a target such as a database.

Advantages and challenges

The outstanding advantage of this integration is the vast adoption range within a corporate network. Requesters are not bound to a platform or an entity that injects initial credentials. All that is required is an ACME client and a host or application able to authenticate against Vault and further interact with it. Furthermore, those initial secrets can be issued with a short TTL that can be renewed without any interaction with a third party.

The challenge is to ensure the corporation is in charge of DNS and to educate the personnel, considering that the ACME protocol is not always well known.