Secrets

Cloud applications frequently need to store sensitive information like web API credentials or encryption keys in a medium that is not fully secure. For example, an application that interacts with GitHub needs to store its OAuth2 client secret and use it when obtaining end-user credentials. If this information was compromised, it could allow someone else to impersonate the application. In order to keep such information secret and secure, you can encrypt the data, but then you need to worry about rotating the encryption keys and distributing them securely to all of your application servers. Most cloud providers include a key management service to perform these tasks, usually with hardware-level security and audit logging.

The Go CDK provides access to key management providers in a portable way called “secret keepers”. These guides show how to work with secret keepers in the Go CDK.

Opening a SecretsKeeper🔗

The first step in working with your secrets is establishing your secret keeper provider. Every secret keeper provider is a little different, but the Go CDK lets you interact with all of them using the *secrets.Keeper type.

The easiest way to open a secrets keeper is using secrets.OpenKeeper and a URL pointing to the keeper, making sure you “blank import” the driver package to link it in. See Concepts: URLs for more details. If you need fine-grained control over the connection settings, you can call the constructor function in the driver package directly (like awskms.OpenKeeper).

See the guide below for usage of both forms for each supported service.

Using a SecretsKeeper🔗

Once you have opened a secrets keeper for the secrets provider you want, you can encrypt and decrypt small messages using the keeper.

Encrypting Data🔗

To encrypt data with a keeper, you call Encrypt with the byte slice you want to encrypt.

plainText := []byte("Secrets secrets...")
cipherText, err := keeper.Encrypt(ctx, plainText)
if err != nil {
	return err
}

Decrypting Data🔗

To decrypt data with a keeper, you call Decrypt with the byte slice you want to decrypt. This should be data that you obtained from a previous call to Encrypt with a keeper that uses the same secret material (e.g. two AWS KMS keepers created with the same customer master key ID). The Decrypt method will return an error if the input data is corrupted.

var cipherText []byte // obtained from elsewhere and random-looking
plainText, err := keeper.Decrypt(ctx, cipherText)
if err != nil {
	return err
}

Large Messages🔗

The secrets keeper API is designed to work with small messages (i.e. <10 KiB in length.) Cloud key management services are high latency; using them for encrypting or decrypting large amounts of data is prohibitively slow (and in some providers not permitted). If you need your application to encrypt or decrypt large amounts of data, you should:

  1. Generate a key for the encryption algorithm (16KiB chunks with secretbox is a reasonable approach).
  2. Encrypt the key with secret keeper.
  3. Store the encrypted key somewhere accessible to the application.

When your application needs to encrypt or decrypt a large message:

  1. Decrypt the key from storage using the secret keeper
  2. Use the decrypted key to encrypt or decrypt the message inside your application.

Keep Secrets in Configuration🔗

Once you have opened a secrets keeper for the secrets provider you want, you can use a secrets keeper to access sensitive configuration stored in an encrypted runtimevar.

First, you create a *runtimevar.Decoder configured to use your secrets keeper using runtimevar.DecryptDecode. In this example, we assume the data is a plain string, but the configuration could be a more structured type.

decodeFunc := runtimevar.DecryptDecode(keeper, runtimevar.StringDecode)
decoder := runtimevar.NewDecoder("", decodeFunc)

Then you can pass the decoder to the runtime configuration provider of your choice. See the Runtime Configuration How-To Guide for more on how to set up runtime configuration.

Supported Services🔗

Google Cloud Key Management Service🔗

The Go CDK can use keys from Google Cloud Platform’s Key Management Service (GCP KMS) to keep information secret. secrets.OpenKeeper will use Application Default Credentials. GCP KMS URLs are similar to key resource IDs:

import (
	"context"

	"gocloud.dev/secrets"
	_ "gocloud.dev/secrets/gcpkms"
)

keeper, err := secrets.OpenKeeper(ctx,
	"gcpkms://projects/MYPROJECT/"+
		"locations/MYLOCATION/"+
		"keyRings/MYKEYRING/"+
		"cryptoKeys/MYKEY")
if err != nil {
	return err
}
defer keeper.Close()

GCP Constructor🔗

The gcpkms.OpenKeeper constructor opens a GCP KMS key. You must first obtain GCP credentials and then create a gRPC connection to GCP KMS.

import (
	"context"

	"gocloud.dev/secrets/gcpkms"
)

// Get a client to use with the KMS API.
client, done, err := gcpkms.Dial(ctx, nil)
if err != nil {
	return err
}
// Close the connection when done.
defer done()

// You can also use gcpkms.KeyResourceID to construct this string.
const keyID = "projects/MYPROJECT/" +
	"locations/MYLOCATION/" +
	"keyRings/MYKEYRING/" +
	"cryptoKeys/MYKEY"

// Construct a *secrets.Keeper.
keeper := gcpkms.OpenKeeper(client, keyID, nil)
defer keeper.Close()

AWS Key Management Service🔗

The Go CDK can use customer master keys from Amazon Web Service’s Key Management Service (AWS KMS) to keep information secret. AWS KMS URLs can use the key’s ID, alias, or Amazon Resource Name (ARN) to identify the key. You can specify the region query parameter to ensure your application connects to the correct region, but otherwise secrets.OpenKeeper will use the region found in the environment variable AWS_REGION or your AWS CLI configuration.

import (
	"context"

	"gocloud.dev/secrets"
	_ "gocloud.dev/secrets/awskms"
)

// Use one of the following:

// 1. By ID.
keeperByID, err := secrets.OpenKeeper(ctx,
	"awskms://1234abcd-12ab-34cd-56ef-1234567890ab?region=us-east-1")
if err != nil {
	return err
}
defer keeperByID.Close()

// 2. By alias.
keeperByAlias, err := secrets.OpenKeeper(ctx,
	"awskms://alias/ExampleAlias?region=us-east-1")
if err != nil {
	return err
}
defer keeperByAlias.Close()

// 3. By ARN.
const arn = "arn:aws:kms:us-east-1:111122223333:key/" +
	"1234abcd-12ab-34bc-56ef-1234567890ab"
keeperByARN, err := secrets.OpenKeeper(ctx,
	"awskms://"+arn+"?region=us-east-1")
if err != nil {
	return err
}
defer keeperByARN.Close()

AWS Constructor🔗

The awskms.OpenKeeper constructor opens a customer master key. You must first create an AWS session with the same region as your key and then connect to KMS:

import (
	"github.com/aws/aws-sdk-go/aws/session"
	"gocloud.dev/secrets/awskms"
)

// Establish an AWS session.
// See https://docs.aws.amazon.com/sdk-for-go/api/aws/session/ for more info.
sess, err := session.NewSession(nil)
if err != nil {
	return err
}

// Get a client to use with the KMS API.
client, err := awskms.Dial(sess)
if err != nil {
	return err
}

// Construct a *secrets.Keeper.
keeper := awskms.OpenKeeper(client, "alias/test-secrets", nil)
defer keeper.Close()

Azure KeyVault🔗

The Go CDK can use keys from Azure KeyVault to keep information secret. secrets.OpenKeeper will use [default credentials from the environment]Azure Environment Auth, unless you set the environment variable AZURE_KEYVAULT_AUTH_VIA_CLI to true, in which case it will use credentials from the az command line.

Azure KeyVault URLs are based on the Azure Key object identifer:

import (
	"context"

	"gocloud.dev/secrets"
	_ "gocloud.dev/secrets/azurekeyvault"
)

// The "azurekeyvault" URL scheme is replaced with "https" to construct an Azure
// Key Vault keyID, as described in https://docs.microsoft.com/en-us/azure/key-vault/about-keys-secrets-and-certificates.
// You can add an optional "/{key-version}" to the path to use a specific
// version of the key; it defaults to the latest version.
keeper, err := secrets.OpenKeeper(ctx, "azurekeyvault://mykeyvaultname.vault.azure.net/keys/mykeyname")
if err != nil {
	return err
}
defer keeper.Close()

Azure Constructor🔗

The azurekeyvault.OpenKeeper constructor opens an Azure KeyVault key.

import "gocloud.dev/secrets/azurekeyvault"

// Get a client to use with the Azure KeyVault API, using default
// authorization from the environment.
//
// You can alternatively use DialUsingCLIAuth to use auth from the
// "az" CLI.
client, err := azurekeyvault.Dial()
if err != nil {
	return err
}

// Construct a *secrets.Keeper.
keeper, err := azurekeyvault.OpenKeeper(client, "https://mykeyvaultname.vault.azure.net/keys/mykeyname", nil)
if err != nil {
	return err
}
defer keeper.Close()

HashiCorp Vault🔗

The Go CDK can use the transit secrets engine in Vault to keep information secret. Vault URLs only specify the key ID. The Vault server endpoint and authentication token are specified using the environment variables VAULT_SERVER_URL and VAULT_SERVER_TOKEN, respectively.

import (
	"context"

	"gocloud.dev/secrets"
	_ "gocloud.dev/secrets/hashivault"
)

keeper, err := secrets.OpenKeeper(ctx, "hashivault://mykey")
if err != nil {
	return err
}
defer keeper.Close()

HashiCorp Vault Constructor🔗

The hashivault.OpenKeeper constructor opens a transit secrets engine key. You must first connect to your Vault instance.

import (
	"context"

	"github.com/hashicorp/vault/api"
	"gocloud.dev/secrets/hashivault"
)

// Get a client to use with the Vault API.
client, err := hashivault.Dial(ctx, &hashivault.Config{
	Token: "CLIENT_TOKEN",
	APIConfig: api.Config{
		Address: "http://127.0.0.1:8200",
	},
})
if err != nil {
	return err
}

// Construct a *secrets.Keeper.
keeper := hashivault.OpenKeeper(client, "my-key", nil)
defer keeper.Close()

Local Secrets🔗

The Go CDK can use local encryption for keeping secrets. Internally, it uses the NaCl secret box algorithm to perform encryption and authentication.

import (
	"context"

	"gocloud.dev/secrets"
	_ "gocloud.dev/secrets/localsecrets"
)

// Using "base64key://", a new random key will be generated.
randomKeyKeeper, err := secrets.OpenKeeper(ctx, "base64key://")
if err != nil {
	return err
}
defer randomKeyKeeper.Close()

// Otherwise, the URL hostname must be a base64-encoded key, of length 32 bytes when decoded.
savedKeyKeeper, err := secrets.OpenKeeper(ctx, "base64key://smGbjm71Nxd1Ig5FS0wj9SlbzAIrnolCz9bQQ6uAhl4=")
if err != nil {
	return err
}
defer savedKeyKeeper.Close()

Local Secrets Constructor🔗

The localsecrets.NewKeeper constructor takes in its secret material as a []byte.

import "gocloud.dev/secrets/localsecrets"

secretKey, err := localsecrets.NewRandomKey()
if err != nil {
	return err
}
keeper := localsecrets.NewKeeper(secretKey)
defer keeper.Close()