Security continues to be a hot topic in IT. We’ve seen the trend reflected, for example, in the development of a variety of security-related tools, such as Square’s Keywhiz or HashiCorp Vault. We’ve taken a closer look at Vault and thought about how we might implement Vault at Adfinis SyGroup.

What is Vault?

In its simplest form, Vault is a hierarchical key/value store for the storage of secrets. The idea is that you can connect Vault up to a persistent database (e.g. MySQL, PostgreSQL, Consul, …) of your choice or, alternatively, opt for the built-in file-backend. Interaction with Vault takes place through an HTTP REST API, for which there are client libraries available in a wide range of programming languages.

But before we get into the details of how Vault works, we need to specify which problems Vault is intended to solve. If you look at the lifecycle of a generic secret, e.g. a password for a database user, it usually looks like this:

1. The admin establishes a secret in accordance with the company password guidelines.
2. The password is registered in a password manager of their choice.
3. When the user is removed, the password list is revised accordingly.

Unfortunately, there is one important point which is often forgotten, namely the regular rotation of the secret. Assuming that, with enough time and resources, any encryption can be broken, it would be disastrous if an attacker were to come across a copy of the encrypted password database.

HashiCorp has come up with a vision for what they call “dynamic secrets”. Instead of the system administrator selecting the password while setting up a web application, the web application itself obtains the secret from Vault at regular intervals. Each secret comes with a TTL which limits the duration of the secret to a few hours or days. In such a set-up, if an attacker managed to access the encrypted passwords, “no harm” would be done, since after a day, they would all be rotated anyway.

Centralisation also has another side-effect – it allows for “break glass procedures”. If you discover a weak point in your own infrastructure, access to the affected subset of secrets can be blocked. Once you’ve found and eliminated the weak point, the secrets are rotated and unblocked once again.

At this time, very few applications support Vault natively. But if you use HashiCorp’s own Consul as a data backend, new configuration files can be generated automatically through key/value modification using the consul-template tool.

In order to be able to log changes in the data content, Vault provides audit backends (structured logs and syslog). This way, mutation can also be viewed later on. In addition, access to secrets can be limited using a sophisticated ACL system. ACLs can, in turn, be linked to different authentication backends. For example, for a given LDAP group, you could limit access to a path to “read-only”.

Vault is configured either through an HCL file or a JSON file. In the following example, we see a configuration with a Consul backend:

backend "consul" {
address = "127.0.0.1:8500"
path = "vault"
}

listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = 0
tls_cert_file = "/etc/ssl/host.crt"
tls_key_file = "/etc/ssl/host.key"
tls_min_version = "tls12"
}

> As for creating the TLS certificate, I’ll refer you at this point to our post on creating OpenSSL x509 Zertifikate certificates.

Demo

To give you a feel for how Vault is operated, here is a little demo.

Once the Vault server is running, it needs to be initialised.

$ vault init
Unseal Key 1: aRqypTLVENeZ9Tt1Kw2sy7XlnqHLT8XOUn3yqGTFMvsB
Unseal Key 2: o4Rd/1ZyzBwZaVEksIHsMpCJimv/pOt1+pSfQFiOEUkC
Unseal Key 3: S1x0hTZ5SrqiQlH1Boh8c/teKtfypu/PS6c5aHGj31cD
Unseal Key 4: TNdq4mdDDUMXAlxbtH8GCRRn3KU4HR3VSiefbWDUjtAE
Unseal Key 5: pA9DmAdIi+WsKVyKAnaWSH+wfBk1Hxlv+xQ5RUn5QM4F
Initial Root Token: 53368d66-8d50-ba5a-f953-7ef6b2dc03de
...

$ for KEY in ${UNSEAL_KEYS}; do vault unseal ${KEY}; done
...
Key (will be hidden):
Sealed: false
Key Shares: 5
Key Threshold: 3
Unseal Progress: 0
...

$ vault auth $ROOT_TOKEN
Successfully authenticated! You are now logged in.
token: 1bb7c7c6-6d4e-be92-cb5c-2838ccde3b5c
token_duration: 0
token_policies: [root]
...

A master key is generated and split up into several subkeys (see Shamir’s Secret Sharing). In the standard configuration, we need a quorum of 3 keys out of the subkeys generated in order to “unseal” the Vault instance. This procedure has to be done each time the Vault instance is started. The initial root token is then required for authentication to Vault.

From this point on, secrets can be written in the standard secret/ backend.

$ vault mounts
Path Type Default TTL Max TTL Description
cubbyhole/ cubbyhole n/a n/a per-token private secret storage
secret/ generic system system generic secret storage
sys/ system n/a n/a system endpoints used for control, policy and debugging

$ vault write secret/test secret_key=secret_value
Success! Data written to: secret/test

$ vault list secret/
Keys
----
test

$ vault read secret/test
Key Value
--- -----
refresh_interval 720h0m0s
secret_key secret_value

This is just a small sample of the features available in Vault. But here are a few more advanced use cases which would go beyond the limits of this blog post:

• Generating Amazon Web Services IAM Policies
• Managing dynamic database user/roles for MySQL, PostgreSQL, MongoDB…
• Complete PKI for creating and signing TLS certificates
• Securing SSH access to systems through OTP

I highly recommend taking a look at the Vault documentation. It’s excellent. Coming soon, we’ll also be posting a second part, telling you how we’ve adapted Vault.