Skip to content
This repository was archived by the owner on Jul 26, 2022. It is now read-only.

Commit 20496ab

Browse files
feat(ibm): add spec option keyByName to support the use of a name, instead of id, as the key (#850)
* Look up secrets based on names when keyByName is true. * Allow unit tests for the backend running without mocking in development. * Add missing unit tests for generic secrets and IAM credentials. * Add keyByName property to CRD. * Document keyByName in README.
1 parent ca549f5 commit 20496ab

File tree

5 files changed

+227
-38
lines changed

5 files changed

+227
-38
lines changed

README.md

+29-7
Original file line numberDiff line numberDiff line change
@@ -847,20 +847,22 @@ The secrets will persist even if the helm installation is removed, although they
847847

848848
### IBM Cloud Secrets Manager
849849

850-
kubernetes-external-secrets supports fetching secrets from [IBM Cloud Secrets Manager](https://cloud.ibm.com/catalog/services/secrets-manager)
850+
kubernetes-external-secrets supports fetching secrets from [IBM Cloud Secrets Manager](https://cloud.ibm.com/catalog/services/secrets-manager).
851851

852-
create username_password secret by using the [ui, cli or API](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-user-credentials).
853-
The cli option is illustrated below:
852+
Create username_password secret by using the [UI, CLI or API](https://cloud.ibm.com/docs/secrets-manager?topic=secrets-manager-user-credentials).
853+
The CLI option is illustrated below:
854854

855855
```bash
856-
# you need to configure ibm cloud cli with a valid endpoint
856+
# You need to configure ibm cloud cli with a valid endpoint.
857857
# If you're using plug-in version 0.0.8 or later, export the following variable.
858858
export SECRETS_MANAGER_URL=https://{instanceid}.{region}.secrets-manager.appdomain.cloud
859+
859860
# If you're using plug-in version 0.0.6 or earlier, export the following variable.
860861
export IBM_CLOUD_SECRETS_MANAGER_API_URL=https://{instance_ID}.{region}.secrets-manager.appdomain.cloud
862+
861863
ibmcloud secrets-manager secret-create --secret-type username_password \
862-
--metadata '{"collection_type": "application/vnd.ibm.secrets-manager.secret+json", "collection_total": 1}' \
863-
--resources '[{"name": "example-username-password-secret","description": "Extended description for my secret.","username": "user123","password": "cloudy-rainy-coffee-book"}]'
864+
--metadata '{"collection_type": "application/vnd.ibm.secrets-manager.secret+json", "collection_total": 1}' \
865+
--resources '[{"name": "example-username-password-secret","description": "Extended description for my secret.","username": "user123","password": "cloudy-rainy-coffee-book"}]'
864866
```
865867

866868
You will need to set these env vars in the deployment of kubernetes-external-secrets:
@@ -880,7 +882,27 @@ spec:
880882
# The guid id of the secret
881883
- key: <guid>
882884
name: username
883-
property: username
885+
property: username
886+
secretType: username_password
887+
```
888+
889+
890+
Alternately, you can use `keyByName` on the spec to interpret keys as secret names, instead of IDs.
891+
Using names is slightly less efficient than using IDs, but it makes your ExternalSecrets more robust, as they are not tied to a particular instance of a secret in a particular instance of Secrets Manager:
892+
893+
```yml
894+
apiVersion: kubernetes-client.io/v1
895+
kind: ExternalSecret
896+
metadata:
897+
name: ibmcloud-secrets-manager-example
898+
spec:
899+
backendType: ibmcloudSecretsManager
900+
keyByName: true
901+
data:
902+
# The name of the secret
903+
- key: my-creds
904+
name: username
905+
property: username
884906
secretType: username_password
885907
```
886908

charts/kubernetes-external-secrets/crds/kubernetes-client.io_externalsecrets_crd.yaml

+5
Original file line numberDiff line numberDiff line change
@@ -138,6 +138,11 @@ spec:
138138
type: string
139139
description: >-
140140
Used by: gcpSecretsManager
141+
keyByName:
142+
type: boolean
143+
description: >-
144+
Whether to interpret the key as a secret name (if true) or ID (the default).
145+
Used by: ibmcloudSecretsManager
141146
oneOf:
142147
- properties:
143148
backendType:

examples/ibmcloud-secrets-manager.yaml

+4-2
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,10 @@ metadata:
44
name: ibmcloud-secrets-manager
55
spec:
66
backendType: ibmcloudSecretsManager
7+
# optional: true to key secrets by name instead of by ID
8+
keyByName: true
79
data:
8-
# The guid id of the secret
9-
- key: guid
10+
- key: my-creds
1011
name: username_password
12+
# Secret Manager secret type: username_password, arbitrary, or iam_credentials
1113
secretType: username_password

lib/backends/ibmcloud-secrets-manager-backend.js

+20-5
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ class IbmCloudSecretsManagerBackend extends KVBackend {
1919

2020
_secretsManagerClient () {
2121
let authenticator
22-
if (process.env.IBM_CLOUD_SECRETS_MANAGER_API_AUTH_TYPE && process.env.IBM_CLOUD_SECRETS_MANAGER_API_APIKEY) {
22+
if (process.env.IBM_CLOUD_SECRETS_MANAGER_API_AUTH_TYPE) {
2323
authenticator = getAuthenticatorFromEnvironment('IBM_CLOUD_SECRETS_MANAGER_API')
2424
} else {
2525
authenticator = new IamAuthenticator({
@@ -36,16 +36,31 @@ class IbmCloudSecretsManagerBackend extends KVBackend {
3636
/**
3737
* Get secret_data property value from IBM Cloud Secrets Manager
3838
* @param {string} key - Key used to store secret property value.
39-
* @param {object} specOptions - Options for this external secret, eg role
39+
* @param {object} specOptions.keyByName - Interpret key as secret names if true, as id otherwise
4040
* @param {string} keyOptions.secretType - Type of secret - one of username_password, iam_credentials or arbitrary
4141
* @returns {Promise} Promise object representing secret property value.
4242
*/
43-
async _get ({ key, keyOptions: { secretType } }) {
43+
async _get ({ key, specOptions: { keyByName }, keyOptions: { secretType } }) {
4444
const client = this._secretsManagerClient()
45-
this._logger.info(`fetching secret ${key} from IBM Cloud Secrets Manager ${this._credential.endpoint}`)
45+
let id = key
46+
keyByName = keyByName === true
47+
this._logger.info(`fetching ${secretType} secret ${id}${keyByName ? ' by name' : ''} from IBM Cloud Secrets Manager ${this._credential.endpoint}`)
48+
49+
if (keyByName) {
50+
const secrets = await client.listAllSecrets({ search: key })
51+
const filtered = secrets.result.resources.filter((s) => (s.name === key && s.secret_type === secretType))
52+
if (filtered.length === 1) {
53+
id = filtered[0].id
54+
} else if (filtered.length === 0) {
55+
throw new Error(`No ${secretType} secret named ${key}`)
56+
} else {
57+
throw new Error(`Multiple ${secretType} secrets named ${key}`)
58+
}
59+
}
60+
4661
const secret = await client.getSecret({
4762
secretType: secretType,
48-
id: key
63+
id
4964
})
5065
if (secretType === 'iam_credentials') {
5166
return JSON.stringify(secret.result.resources[0].api_key)
Original file line numberDiff line numberDiff line change
@@ -1,48 +1,193 @@
11
/* eslint-env mocha */
22
'use strict'
33

4-
process.env.IBM_CLOUD_SECRETS_MANAGER_API_AUTH_TYPE = 'noauth'
5-
process.env.IBM_CLOUD_SECRETS_MANAGER_API_APIKEY = 'iamkey'
6-
74
const { expect } = require('chai')
85
const sinon = require('sinon')
96

107
const IbmCloudSecretsManagerBackend = require('./ibmcloud-secrets-manager-backend')
118

9+
// In the unit test suite, these tests mock calls to IBM Secrets Manager, but mocking can be disabled during development to validate actual operation.
10+
// To diable mocking and enable real calls to an instance of Secrets Manager:
11+
//
12+
// 1. Set the three credential environment variables:
13+
// SECRETS_MANAGER_API_AUTH_TYPE=iam
14+
// SECRETS_MANAGER_API_ENDPOINT=https://{instance-id}.{region}.secrets-manager.appdomain.cloud
15+
// SECRETS_MANAGER_API_APIKEY={API key with Read+ReadSecrets access to the instance}
16+
//
17+
// 2. Add the three secrets described in the data object below to Secrets Manager.
18+
// When you add the IAM secret, be sure that "Reuse IAM credentials until lease expires" is checked.
19+
//
20+
// 3. Set the following three environment variables to the IDs of those secrets:
21+
// IBM_CLOUD_SECRETS_MANAGER_TEST_CREDS_ID
22+
// IBM_CLOUD_SECRETS_MANAGER_TEST_SECRET_ID
23+
// IBM_CLOUD_SECRETS_MANAGER_TEST_IAM_ID
24+
//
25+
// 4. Set the following environment variable to the API key generated as part of the IAM credential:
26+
// IBM_CLOUD_SECRETS_MANAGER_TEST_IAM_APIKEY
27+
//
28+
// Note: In the Secrets Manager UI, you can select "Show snippet" from the secret's overflow menu to show a curl command that will retrieve the value.
29+
// Or you can use the "ibmcloud sm secret" CLI command to handle authentication for you.
30+
//
31+
// You can switch back to mocking simply by unsetting SECRETS_MANAGER_API_AUTH_TYPE.
32+
// This makes it easy to switch back and forth between the two modes when writing new tests.
33+
34+
const endpoint = process.env.IBM_CLOUD_SECRETS_MANAGER_API_ENDPOINT || 'https://fake.secrets-manager.appdomain.cloud'
35+
36+
const data = {
37+
creds: {
38+
id: process.env.IBM_CLOUD_SECRETS_MANAGER_TEST_CREDS_ID || 'id1',
39+
name: 'test-creds',
40+
secretType: 'username_password',
41+
username: 'johndoe',
42+
password: 'p@ssw0rd'
43+
},
44+
secret: {
45+
id: process.env.IBM_CLOUD_SECRETS_MANAGER_TEST_SECRET_ID || 'id2',
46+
name: 'test-secret',
47+
secretType: 'arbitrary',
48+
payload: 's3cr3t'
49+
},
50+
iam: {
51+
id: process.env.IBM_CLOUD_SECRETS_MANAGER_TEST_IAM_ID || 'id3',
52+
name: 'test-iam',
53+
secretType: 'iam_credentials',
54+
apiKey: process.env.IBM_CLOUD_SECRETS_MANAGER_TEST_IAM_APIKEY || 'key'
55+
}
56+
}
57+
1258
describe('IbmCloudSecretsManagerBackend', () => {
59+
const mock = !process.env.IBM_CLOUD_SECRETS_MANAGER_API_AUTH_TYPE
1360
let loggerMock
14-
let clientMock
1561
let ibmCloudSecretsManagerBackend
1662

17-
const username = 'fakeUserName'
18-
const password = 'fakeSecretPropertyValue'
19-
const secret = { result: { resources: [{ secret_data: { password: password, username: username } }] } }
20-
const returnsecret = JSON.stringify({ password: password, username: username })
21-
const key = 'username_password'
22-
2363
beforeEach(() => {
24-
loggerMock = sinon.mock()
25-
loggerMock.info = sinon.stub()
26-
clientMock = sinon.mock()
27-
clientMock.getSecret = sinon.stub().returns(secret)
64+
if (mock) {
65+
process.env.IBM_CLOUD_SECRETS_MANAGER_API_AUTH_TYPE = 'noauth'
66+
}
67+
68+
loggerMock = {
69+
info: sinon.stub()
70+
}
2871

2972
ibmCloudSecretsManagerBackend = new IbmCloudSecretsManagerBackend({
30-
credential: { endpoint: 'https//sampleendpoint' },
73+
credential: { endpoint },
3174
logger: loggerMock
3275
})
33-
ibmCloudSecretsManagerBackend._secretsManagerClient = sinon.stub().returns(clientMock)
3476
})
3577

78+
afterEach(() => {
79+
if (mock) {
80+
delete process.env.IBM_CLOUD_SECRETS_MANAGER_API_AUTH_TYPE
81+
ibmCloudSecretsManagerBackend._secretsManagerClient.restore()
82+
}
83+
})
84+
85+
function mockClient ({ list = [], get = {} }) {
86+
if (mock) {
87+
const client = {
88+
listAllSecrets: sinon.stub().resolves({ result: { resources: list } }),
89+
getSecret: sinon.stub().resolves({ result: { resources: [get] } })
90+
}
91+
sinon.stub(ibmCloudSecretsManagerBackend, '_secretsManagerClient').returns(client)
92+
}
93+
}
94+
3695
describe('_get', () => {
37-
it('returns secret property value', async () => {
38-
const specOptions = {}
39-
const keyOptions = { secretType: 'password' }
40-
const secretPropertyValue = await ibmCloudSecretsManagerBackend._get({
41-
key: key,
42-
specOptions,
43-
keyOptions
96+
describe('with default spec options', () => {
97+
it('returns a username_password secret', async () => {
98+
const { id, secretType, username, password } = data.creds
99+
mockClient({ get: { secret_data: { password, username } } })
100+
101+
const secretPropertyValue = await ibmCloudSecretsManagerBackend._get({
102+
key: id,
103+
specOptions: {},
104+
keyOptions: { secretType }
105+
})
106+
expect(secretPropertyValue).equals('{"password":"p@ssw0rd","username":"johndoe"}')
107+
})
108+
109+
it('returns an arbitrary secret', async () => {
110+
const { id, secretType, payload } = data.secret
111+
mockClient({ get: { secret_data: { payload } } })
112+
113+
const secretPropertyValue = await ibmCloudSecretsManagerBackend._get({
114+
key: id,
115+
specOptions: {},
116+
keyOptions: { secretType }
117+
})
118+
expect(secretPropertyValue).equals('{"payload":"s3cr3t"}')
44119
})
45-
expect(secretPropertyValue).equals(returnsecret)
120+
121+
it('returns an API key from an iam_credentials secret', async () => {
122+
const { id, secretType, apiKey } = data.iam
123+
mockClient({ get: { api_key: apiKey } })
124+
125+
const secretPropertyValue = await ibmCloudSecretsManagerBackend._get({
126+
key: id,
127+
specOptions: {},
128+
keyOptions: { secretType }
129+
})
130+
expect(secretPropertyValue).equals(`"${apiKey}"`)
131+
})
132+
})
133+
134+
describe('with key by name enabled', () => {
135+
it('returns a secret that matches the given name and type', async () => {
136+
const { name, secretType, username, password } = data.creds
137+
const list = [
138+
{ name, secret_type: 'arbitrary' },
139+
{ name, secret_type: secretType },
140+
{ name: 'test-creds2', secret_type: secretType }
141+
]
142+
mockClient({ list, get: { secret_data: { password, username } } })
143+
144+
const secretPropertyValue = await ibmCloudSecretsManagerBackend._get({
145+
key: name,
146+
specOptions: { keyByName: true },
147+
keyOptions: { secretType }
148+
})
149+
expect(secretPropertyValue).equals('{"password":"p@ssw0rd","username":"johndoe"}')
150+
})
151+
152+
it('throws if there is no secret with the given name and type', async () => {
153+
mockClient({ list: [] })
154+
155+
try {
156+
await ibmCloudSecretsManagerBackend._get({
157+
key: 'test-missing',
158+
specOptions: { keyByName: true },
159+
keyOptions: { secretType: 'username_password' }
160+
})
161+
} catch (error) {
162+
expect(error).to.have.property('message').that.includes('No username_password secret')
163+
return
164+
}
165+
expect.fail('expected to throw an error')
166+
})
167+
168+
// Defensive test: this condition does not appear to be possible currently with a real Secrets Manager instance.
169+
if (mock) {
170+
it('throws if there are multiple secrets with the given name and type', async () => {
171+
const { name, secretType, username, password } = data.creds
172+
const list = [
173+
{ name, secret_type: secretType },
174+
{ name, secret_type: secretType }
175+
]
176+
mockClient({ list, get: { secret_data: { password, username } } })
177+
178+
try {
179+
await ibmCloudSecretsManagerBackend._get({
180+
key: name,
181+
specOptions: { keyByName: true },
182+
keyOptions: { secretType }
183+
})
184+
} catch (error) {
185+
expect(error).to.have.property('message').that.includes('Multiple username_password secrets')
186+
return
187+
}
188+
expect.fail('expected to throw an error')
189+
})
190+
}
46191
})
47192
})
48193
})

0 commit comments

Comments
 (0)