Skip to content

GODRIVER-2728: Implement automatic Azure token acquisition callback #1703

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 85 commits into from
Jul 22, 2024
Merged
Show file tree
Hide file tree
Changes from 80 commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
a47681f
GODRIVER-2911: Initial attempted to untie the Gordian not, this will …
pmeredit Jun 11, 2024
279635a
GODRIVER-2911: We're going to have to go this way and implement some …
pmeredit Jun 12, 2024
9170d50
GODRIVER-2911: Ok, not great, but this will work
pmeredit Jun 12, 2024
590662d
GODRIVER-2911: Renaming oidc sasl
pmeredit Jun 12, 2024
171204c
GODRIVER-2911: Implement Operation based private sasl conversation fo…
pmeredit Jun 12, 2024
dbc5699
GODRIVER-2911: Privitize all the oidc sasl api, move AuthConfig up so…
pmeredit Jun 12, 2024
ff73302
GODRIVER-2911: Move things as necessary for authentication registration
pmeredit Jun 12, 2024
0db7c3e
GODRIVER-2911: Let's use a bit better naming
pmeredit Jun 12, 2024
be99139
GODRIVER-2911: Add Reauth to Authenticators
pmeredit Jun 12, 2024
f400d18
GODRIVER-2911: Check point
pmeredit Jun 12, 2024
eed3dd5
GODRIVER-2911: Initial plumbing, the Client Authenticator is going to…
pmeredit Jun 13, 2024
2ee93cc
GODRIVER-2911: Set authenticator in topology
pmeredit Jun 13, 2024
f6def8d
GODRIVER-2911: Set authenticator from Command to Operation
pmeredit Jun 13, 2024
bd5c9f2
GODRIVER-2911: Remove authenticator so we can readd it programmatically
pmeredit Jun 13, 2024
36ba008
GODRIVER-2911: Remove authenticator so we can readd it programmatically
pmeredit Jun 13, 2024
a2a4029
GODRIVER-2911: Remove authenticator so we can readd it programatically
pmeredit Jun 13, 2024
d2c75f1
GODRIVER-2911: Add all that authenticator plumbing programmatically s…
pmeredit Jun 13, 2024
4070d06
GODRIVER-2911: Thread through Authenticator
pmeredit Jun 13, 2024
4a44090
GODRIVER-2911: Move OIDC back to auth package, yay
pmeredit Jun 14, 2024
4ea9b9c
GODRIVER-2911: Move Config = AuthConfig to top of the file
pmeredit Jun 14, 2024
2b5cde6
GODRIVER-2911: Update comment
pmeredit Jun 14, 2024
368cedd
GODRIVER-2911: Some implementation
pmeredit Jun 14, 2024
e00e057
GODRIVER-2911: Add OIDCTokenGenID to Connection interface
pmeredit Jun 14, 2024
1666c6c
GODRIVER-2911: Add OIDCTokenGenID to Connection interface for types i…
pmeredit Jun 14, 2024
d90ee3f
GODRIVER-2911: Actually add the oidc file, whoops
pmeredit Jun 14, 2024
19ed261
GODRIVER-2911: Fix nil pointer error
pmeredit Jun 14, 2024
4112208
GODRIVER-2911: Fix fmt
pmeredit Jun 14, 2024
03c4c08
GODRIVER-2911: Fix build failure
pmeredit Jun 14, 2024
dac0468
GODRIVER-2911: well, that was silly
pmeredit Jun 14, 2024
651af66
GODRIVER-2911: Add licenses and fix comment
pmeredit Jun 14, 2024
6b16e91
GODRIVER-2911: Fix receiver names and remove authenticator field from…
pmeredit Jun 14, 2024
26412ae
GODRIVER-2911: Fix many lints. Linter not running for me locally
pmeredit Jun 14, 2024
98e8cbe
GODRIVER-2911: Fix lints
pmeredit Jun 15, 2024
78fa217
GODRIVER-2911: Fix spelling error
pmeredit Jun 15, 2024
46fa6f3
GODRIVER-2911: Testing checkpoint
pmeredit Jun 18, 2024
c6d23de
GODRIVER-2911: Fix config, fix spec auth
pmeredit Jun 18, 2024
c137399
GODRIVER-2911: Checkpoint
pmeredit Jun 19, 2024
1be9498
GODRIVER-2911: OIDC working
pmeredit Jun 19, 2024
8542f76
GODRIVER-2911: add machine_1_2
pmeredit Jun 19, 2024
286525f
GODRIVER-2911: add machine_1_2, actually helps to call it
pmeredit Jun 19, 2024
4013ccb
GODRIVER-2911: Remove unneeded debugging
pmeredit Jun 19, 2024
83ffaa7
GODRIVER-2911: Add more tests
pmeredit Jun 20, 2024
f33dca7
GODRIVER-2911: Updates
pmeredit Jun 20, 2024
3c00307
GODRIVER-2911: Change to using errors
pmeredit Jun 20, 2024
590a3c8
GODRIVER-2911: Add more tests that do not require fail points
pmeredit Jun 20, 2024
e88ebe7
GODRIVER-2911: See if it fails with 10 tries
pmeredit Jun 20, 2024
58f0f42
GODRIVER-2911: Not sure how to get fail points working
pmeredit Jun 20, 2024
1be1e13
GODRIVER-2911: Appease linter
pmeredit Jun 20, 2024
6e1fd3a
GODRIVER-2911: Appease linter
pmeredit Jun 20, 2024
640907d
GODRIVER-2911: Change 3_3 to use fail on find, add 4_1
pmeredit Jun 21, 2024
4d30705
GODRIVER-2911: Manually create fail points
pmeredit Jun 23, 2024
9dd40c9
GODRIVER-2911: This is working except 3_3 seems to be hanging
pmeredit Jun 23, 2024
b343ebb
GODRIVER-2911: Tests all passing
pmeredit Jun 23, 2024
5240a91
GODRIVER-2911: Appease linter
pmeredit Jun 23, 2024
0cdd7a2
GODRIVER-2911: Remove test func that is unneeded
pmeredit Jun 23, 2024
4613c5f
Update x/mongo/driver/auth/oidc.go
pmeredit Jun 26, 2024
40998b6
SQL-1937: Remove spurious authenticators, move mutex
pmeredit Jun 26, 2024
2d09cc5
SQL-1937: Change Reauth interface
pmeredit Jun 26, 2024
d45c7e4
Update Makefile
pmeredit Jul 1, 2024
ae9c34f
GODRIVER-2911: Apply httpclient patch
pmeredit Jul 1, 2024
1d86914
GODRIVER-2911: Fix races
pmeredit Jul 1, 2024
30ed4c4
GODRIVER-2911: Back out changes to sasl, add comment, remove Println …
pmeredit Jul 2, 2024
519205c
GODRIVER-2911: Move public OIDC configuration types into public, non-…
pmeredit Jul 2, 2024
5f0c68d
GODRIVER-2911: Improve comment
pmeredit Jul 2, 2024
6410109
GODRIVER-2728: Add azurecallback support
pmeredit Jul 3, 2024
7dc89cb
GODRIVER-2728: Update test framework
pmeredit Jul 3, 2024
b84cc12
GODRIVER-2728: No idea what this incredibly unhelpful shellscript fai…
pmeredit Jul 3, 2024
6a3af5a
GODRIVER-2911: Update script comment
pmeredit Jul 3, 2024
386389d
GODRIVER-2728: cp
pmeredit Jul 4, 2024
7340ddd
GODRIVER-2911: Use conversion functions instead of type redeclarations
pmeredit Jul 4, 2024
902bb9b
Merge branch 'GODRIVER-2911' into GODRIVER-2728
pmeredit Jul 8, 2024
e86b80b
GODRIVER-2728: Appears to be working once I can figure out the json side
pmeredit Jul 8, 2024
9878c73
GODRIVER-2728: Should work now
pmeredit Jul 8, 2024
aa887de
GODRIVER-2728: Working, remove debugging
pmeredit Jul 8, 2024
39d045b
GODRIVER-2728: Cleanup
pmeredit Jul 8, 2024
c48ffec
Merge branch 'v1' into GODRIVER-2728
pmeredit Jul 15, 2024
0a4c641
GODRIVER-2728: Merge v1
pmeredit Jul 15, 2024
ac1782b
GODRIVER-2728: Merge v1
pmeredit Jul 15, 2024
95c22f3
GODRIVER-2728: Merge v1
pmeredit Jul 15, 2024
f081162
GODRIVER-2728: Check error
pmeredit Jul 15, 2024
1bcf07e
Update x/mongo/driver/auth/oidc.go
pmeredit Jul 15, 2024
d825e33
GODRIVER-2728: Fix spelling
pmeredit Jul 15, 2024
4450d5f
Update x/mongo/driver/auth/oidc.go
pmeredit Jul 22, 2024
8735082
GODRIVE-2728: Use auto deserialize conversion
pmeredit Jul 22, 2024
c9990a2
Merge branch 'v1' into GODRIVER-2728
pmeredit Jul 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 52 additions & 1 deletion .evergreen/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -365,7 +365,7 @@ functions:
script: |
${PREPARE_SHELL}
export OIDC="oidc"
bash ${PROJECT_DIRECTORY}/etc/run-oidc-test.sh
bash ${PROJECT_DIRECTORY}/etc/run-oidc-test.sh 'make -s evg-test-oidc-auth'

run-make:
- command: shell.exec
Expand Down Expand Up @@ -1975,6 +1975,31 @@ tasks:
commands:
- func: "run-oidc-auth-test-with-test-credentials"

- name: "oidc-auth-test-azure-latest"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test is compiled ahead of time and just the compiled exe is ran on the azure vm. This matters more for gcp vm which is far slower, but it's good to be consistent

commands:
- command: shell.exec
params:
working_dir: src/go.mongodb.org/mongo-driver
shell: bash
script: |-
set -o errexit
${PREPARE_SHELL}
export AZUREOIDC_DRIVERS_TAR_FILE=/tmp/mongo-go-driver.tar.gz
# we need to statically link libc to avoid the situation where the VM has a different
# version of libc
go build -tags osusergo,netgo -ldflags '-w -extldflags "-static -lgcc -lc"' -o test ./cmd/testoidcauth/main.go
rm "$AZUREOIDC_DRIVERS_TAR_FILE" || true
tar -cf $AZUREOIDC_DRIVERS_TAR_FILE ./test
tar -uf $AZUREOIDC_DRIVERS_TAR_FILE ./etc
rm "$AZUREOIDC_DRIVERS_TAR_FILE".gz || true
gzip $AZUREOIDC_DRIVERS_TAR_FILE
export AZUREOIDC_DRIVERS_TAR_FILE=/tmp/mongo-go-driver.tar.gz
# Define the command to run on the azure VM.
# Ensure that we source the environment file created for us, set up any other variables we need,
# and then run our test suite on the vm.
export AZUREOIDC_TEST_CMD="PROJECT_DIRECTORY='.' OIDC_ENV=azure OIDC=oidc ./etc/run-oidc-test.sh ./test"
bash $DRIVERS_TOOLS/.evergreen/auth_oidc/azure/run-driver-test.sh

- name: "test-search-index"
commands:
- func: "bootstrap-mongo-orchestration"
Expand Down Expand Up @@ -2293,6 +2318,30 @@ task_groups:
tasks:
- oidc-auth-test-latest

- name: testazureoidc_task_group
setup_group:
- func: fetch-source
- func: prepare-resources
- func: fix-absolute-paths
- func: make-files-executable
- command: subprocess.exec
params:
binary: bash
env:
AZUREOIDC_VMNAME_PREFIX: "GO_DRIVER"
args:
- ${DRIVERS_TOOLS}/.evergreen/auth_oidc/azure/create-and-setup-vm.sh
teardown_task:
- command: subprocess.exec
params:
binary: bash
args:
- ${DRIVERS_TOOLS}/.evergreen/auth_oidc/azure/delete-vm.sh
setup_group_can_fail_task: true
setup_group_timeout_secs: 1800
tasks:
- oidc-auth-test-azure-latest

- name: test-aws-lambda-task-group
setup_group:
- func: fetch-source
Expand Down Expand Up @@ -2642,3 +2691,5 @@ buildvariants:
tasks:
- name: testoidc_task_group
batchtime: 20160 # Use a batchtime of 14 days as suggested by the CSFLE test README
- name: testazureoidc_task_group
batchtime: 20160 # Use a batchtime of 14 days as suggested by the CSFLE test README
72 changes: 61 additions & 11 deletions cmd/testoidcauth/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -74,17 +74,26 @@ func main() {
fmt.Println("...Ok")
}
}
aux("machine_1_1_callbackIsCalled", machine11callbackIsCalled)
aux("machine_1_2_callbackIsCalledOnlyOneForMultipleConnections", machine12callbackIsCalledOnlyOneForMultipleConnections)
aux("machine_2_1_validCallbackInputs", machine21validCallbackInputs)
aux("machine_2_3_oidcCallbackReturnMissingData", machine23oidcCallbackReturnMissingData)
aux("machine_2_4_invalidClientConfigurationWithCallback", machine24invalidClientConfigurationWithCallback)
aux("machine_3_1_failureWithCachedTokensFetchANewTokenAndRetryAuth", machine31failureWithCachedTokensFetchANewTokenAndRetryAuth)
aux("machine_3_2_authFailuresWithoutCachedTokensReturnsAnError", machine32authFailuresWithoutCachedTokensReturnsAnError)
aux("machine_3_3_UnexpectedErrorCodeDoesNotClearTheCache", machine33UnexpectedErrorCodeDoesNotClearTheCache)
aux("machine_4_1_reauthenticationSucceeds", machine41ReauthenticationSucceeds)
aux("machine_4_2_readCommandsFailIfReauthenticationFails", machine42ReadCommandsFailIfReauthenticationFails)
aux("machine_4_3_writeCommandsFailIfReauthenticationFails", machine43WriteCommandsFailIfReauthenticationFails)
env := os.Getenv("OIDC_ENV")
switch env {
case "":
aux("machine_1_1_callbackIsCalled", machine11callbackIsCalled)
aux("machine_1_2_callbackIsCalledOnlyOneForMultipleConnections", machine12callbackIsCalledOnlyOneForMultipleConnections)
aux("machine_2_1_validCallbackInputs", machine21validCallbackInputs)
aux("machine_2_3_oidcCallbackReturnMissingData", machine23oidcCallbackReturnMissingData)
aux("machine_2_4_invalidClientConfigurationWithCallback", machine24invalidClientConfigurationWithCallback)
aux("machine_3_1_failureWithCachedTokensFetchANewTokenAndRetryAuth", machine31failureWithCachedTokensFetchANewTokenAndRetryAuth)
aux("machine_3_2_authFailuresWithoutCachedTokensReturnsAnError", machine32authFailuresWithoutCachedTokensReturnsAnError)
aux("machine_3_3_UnexpectedErrorCodeDoesNotClearTheCache", machine33UnexpectedErrorCodeDoesNotClearTheCache)
aux("machine_4_1_reauthenticationSucceeds", machine41ReauthenticationSucceeds)
aux("machine_4_2_readCommandsFailIfReauthenticationFails", machine42ReadCommandsFailIfReauthenticationFails)
aux("machine_4_3_writeCommandsFailIfReauthenticationFails", machine43WriteCommandsFailIfReauthenticationFails)
case "azure":
aux("machine_5_1_azureWithNoUsername", machine51azureWithNoUsername)
aux("machine_5_2_azureWithNoUsername", machine52azureWithBadUsername)
default:
log.Fatal("Unknown OIDC_ENV: ", env)
}
if hasError {
log.Fatal("One or more tests failed")
}
Expand Down Expand Up @@ -686,3 +695,44 @@ func machine43WriteCommandsFailIfReauthenticationFails() error {
}
return callbackFailed
}

func machine51azureWithNoUsername() error {
opts := options.Client().ApplyURI(uriSingle)
if opts == nil || opts.Auth == nil {
return fmt.Errorf("machine_5_1: failed parsing uri: %q", uriSingle)
}
client, err := mongo.Connect(context.Background(), opts)
if err != nil {
return fmt.Errorf("machine_5_1: failed connecting client: %v", err)
}
defer client.Disconnect(context.Background())

coll := client.Database("test").Collection("test")

_, err = coll.Find(context.Background(), bson.D{})
if err != nil {
return fmt.Errorf("machine_5_1: failed executing Find: %v", err)
}
return nil
}

func machine52azureWithBadUsername() error {
opts := options.Client().ApplyURI(uriSingle)
if opts == nil || opts.Auth == nil {
return fmt.Errorf("machine_5_2: failed parsing uri: %q", uriSingle)
}
opts.Auth.Username = "bad"
client, err := mongo.Connect(context.Background(), opts)
if err != nil {
return fmt.Errorf("machine_5_2: failed connecting client: %v", err)
}
defer client.Disconnect(context.Background())

coll := client.Database("test").Collection("test")

_, err = coll.Find(context.Background(), bson.D{})
if err == nil {
return fmt.Errorf("machine_5_2: Find succeeded when it should fail")
}
return nil
}
2 changes: 1 addition & 1 deletion etc/run-oidc-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,4 @@ export TEST_AUTH_OIDC=1
export COVERAGE=1
export AUTH="auth"

make -s evg-test-oidc-auth
$1
59 changes: 57 additions & 2 deletions x/mongo/driver/auth/oidc.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,11 @@ package auth

import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strconv"
"strings"
"sync"
"time"
Expand Down Expand Up @@ -166,10 +169,15 @@ func (oa *OIDCAuthenticator) providerCallback() (OIDCCallback, error) {
}

switch env {
// TODO GODRIVER-2728: Automatic token acquisition for Azure Identity Provider
case azureEnvironmentValue:
resource, ok := oa.AuthMechanismProperties[resourceProp]
if !ok {
return nil, newAuthError("resource must be specified for Azure OIDC", nil)
}
return getAzureOIDCCallback(oa.userName, resource, oa.httpClient), nil
// TODO GODRIVER-2806: Automatic token acquisition for GCP Identity Provider
// This is here just to pass the linter, it will be fixed in one of the above tickets.
case azureEnvironmentValue, gcpEnvironmentValue:
case gcpEnvironmentValue:
return func(ctx context.Context, args *OIDCArgs) (*OIDCCredential, error) {
return nil, fmt.Errorf("automatic token acquisition for %q not implemented yet", env)
}, fmt.Errorf("automatic token acquisition for %q not implemented yet", env)
Expand All @@ -178,6 +186,53 @@ func (oa *OIDCAuthenticator) providerCallback() (OIDCCallback, error) {
return nil, fmt.Errorf("%q %q not supported for MONGODB-OIDC", environmentProp, env)
}

// getAzureOIDCCallback returns the callback for the Azure Identity Provider.
func getAzureOIDCCallback(clientID string, resource string, httpClient *http.Client) OIDCCallback {
// return the callback parameterized by the clientID and resource, also passing in the user
// configured httpClient.
return func(ctx context.Context, args *OIDCArgs) (*OIDCCredential, error) {
resource = url.QueryEscape(resource)
var uri string
if clientID != "" {
uri = fmt.Sprintf("http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=%s&client_id=%s", resource, clientID)
} else {
uri = fmt.Sprintf("http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=%s", resource)
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, uri, nil)
if err != nil {
return nil, newAuthError("error creating http request to Azure Identity Provider", err)
}
req.Header.Add("Metadata", "true")
req.Header.Add("Accept", "application/json")
resp, err := httpClient.Do(req)
if err != nil {
return nil, newAuthError("error getting access token from Azure Identity Provider", err)
}
defer resp.Body.Close()
var azureResp struct {
AccessToken string `json:"access_token"`
ExpiresOn string `json:"expires_on"`
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: You can skip the separate integer parsing step by using the ,string struct tag option for ExpiresOn.

E.g.

var azureResp struct {
	AccessToken string `json:"access_token"`
	ExpiresOn   int64  `json:"expires_on,string"`
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, cool!


if resp.StatusCode != http.StatusOK {
return nil, newAuthError(fmt.Sprintf("failed to get a valid response from Azure Identity Provider, http code: %d", resp.StatusCode), nil)
}
err = json.NewDecoder(resp.Body).Decode(&azureResp)
if err != nil {
return nil, newAuthError("failed parsing result from Azure Identity Provider", err)
}
expiresOn, err := strconv.ParseInt(azureResp.ExpiresOn, 10, 64)
if err != nil {
return nil, newAuthError("failed converting experiation field from Azure Identity Provider to int64", err)
}
expiresAt := time.Unix(expiresOn, 0)
return &OIDCCredential{
AccessToken: azureResp.AccessToken,
ExpiresAt: &expiresAt,
}, nil
}
}

func (oa *OIDCAuthenticator) getAccessToken(
ctx context.Context,
conn driver.Connection,
Expand Down
Loading