Managing AWS profiles #67

chrisgilmerproj · 2020-06-30T17:42:50Z

In this change we want to be able to handle a couple cases:

Update existing profiles when running the tool where before the tool would error out if profiles existed
Allow adding new profiles as they are made available
Generate all profiles at once given a list of new profiles

Setup

New setup-new-aws-user setup usage:

setup-new-aws-user setup \
  --iam-user cgilmer-delete-me \
  --iam-role engineer \
  --no-mfa \
  --aws-profile-account delete-id:123456789012 \
  --aws-profile-account delete-ci:123456789013 \
  --aws-profile-account delete-sandbox:123456789014 \
  --aws-profile-account delete-prod:123456789015

You should see this output:

Enter Access Key ID: *****
Enter Secret Access Key: *****
Added credentials to profile "delete-id-base" in vault
Getting the existing MFA device...
Updating the AWS config file: /Users/cgilmer/.aws/config
Adding the profile "delete-id-base" to the AWS config file
Adding the profile "delete-id" to the AWS config file
Adding the profile "delete-ci" to the AWS config file
Adding the profile "delete-sandbox" to the AWS config file
Adding the profile "delete-prod" to the AWS config file
Removing aws-vault session
Rotating AWS access keys
A new unique MFA token is needed to rotate the AWS access keys
Third MFA token (5 attempts remaining): *****
Creating new access key
Added credentials to profile "delete-id-base" in vault
Deleting old access key
Victory!

Add Profiles

The new subcommand allows for adding a new profile based on an existing profile. It's important to note that the assumption here is that the BASE account and the MFA will remain the same. If this was not true then I'd expect the user to run the setup subcommand instead of the add-profile sub command.

New setup-new-aws-user add-profile usage:

setup-new-aws-user add-profile \
  --iam-user cgilmer-delete-me 
  --iam-role diff-role \
  --aws-profile delete-id \
  --aws-profile-account delete-test:123456789016

go run ./cmd/ add-profile

You should see this output:

Adding new profiles to the AWS config file: /Users/cgilmer/.aws/config
Adding the profile "delete-test" to the AWS config file
Victory!

Generates this in the ~/.aws/config file:

[profile delete-test]
source_profile=delete-id-base
mfa_serial=arn:aws:iam::REDACTED:mfa/cgilmer-delete-me
role_arn=arn:aws:iam::123456789016:role/diff-role
region=us-west-2
output=json

Testing

Try out the new tests with make test. To review test coverage you can run make test_coverage and view the covered code in the browser.

References

Pivotal tickets:

cmd/setup.go

…ic function

…ser to SetupConfig for clarity.

chrisgilmerproj · 2020-07-10T20:17:50Z

I believe this PR is ready. I'm struggling to think of ways to pull this into more, smaller and focused PRs. Let's give reviewing this a go and if its too hard I can try again.

eeeady · 2020-07-10T21:16:48Z

I haven't done the full... pull/test/read code cycles of this PR... just purely a thought/question around your example:

Add Profiles

The new subcommand allows for adding a new profile based on an existing profile. It's important to note that the assumption here is that the BASE account and the MFA will remain the same. If this was not true then I'd expect the user to run the setup subcommand instead of the add-profile sub command.

New setup-new-aws-user add-profile usage:
setup-new-aws-user add-profile \
  --iam-user cgilmer-delete-me 
  --iam-role diff-role \
  --aws-profile delete-id \
  --aws-profile-account delete-test:123456789016 
go run ./cmd/ add-profile

You should see this output:
Adding new profiles to the AWS config file: /Users/cgilmer/.aws/config
Adding the profile "delete-test" to the AWS config file
Victory!
Generates this in the ~/.aws/config file:
[profile delete-test]
source_profile=delete-id-base
mfa_serial=arn:aws:iam::REDACTED:mfa/cgilmer-delete-me
role_arn=arn:aws:iam::123456789016:role/diff-role
region=us-west-2
output=json

is --aws-profile flag the new profile delete-test will be inheriting? I guess the name of the flag throws me off.

eeeady · 2020-07-10T21:26:02Z

Actually reading the code makes me think the --aws-profile flag is actually the profile you're copying source profile etc from.

chrisgilmerproj · 2020-07-10T21:47:59Z

Actually reading the code makes me think the --aws-profile flag is actually the profile you're copying source profile etc from.

I landed on this for a couple reasons:

with spf13/pflags --aws-profile is actually using the same reserved env var as the AWS cli, $AWS_PROFILE.
If we were going to make add-profile subcommand do any future calls via the AWS SDK for validation it would have to be with this pre-existing profile
In all our examples in ~/.aws/config the *-id profile is the template, making it seem like the right one to copy from.

I had considered some other options but they really muddied the meaning or use of the tool. This seemed more elegant and reusuable for future cases.

eeeady

That took me a while to go through... Generally I'm happy with it... still confused about naming that variable... gotta reread your response.

eeeady · 2020-07-10T22:43:53Z

README.md

+    aws-vault exec $AWS_PROFILE -- aws sts get-session
+    ```
+
+#### How `add-profile` modifies your ~/.aws/config


Probably should add a note that it will override existing configuration if you supply a profile name that already exists.

I added a note at the bottom of this section.

eeeady · 2020-07-10T22:52:02Z

Actually reading the code makes me think the --aws-profile flag is actually the profile you're copying source profile etc from.

I landed on this for a couple reasons:

with spf13/pflags --aws-profile is actually using the same reserved env var as the AWS cli, $AWS_PROFILE.

If we were going to make add-profile subcommand do any future calls via the AWS SDK for validation it would have to be with this pre-existing profile

In all our examples in ~/.aws/config the *-id profile is the template, making it seem like the right one to copy from.

I had considered some other options but they really muddied the meaning or use of the tool. This seemed more elegant and reusuable for future cases.

TBH I have no idea what I'd call it... cuz it's not like you're explicitly defining the source profile here so you can't call it that...

I get that if you know your "base" account is in *-id you could move to the *-id folder and use $AWS_PROFILE as the argument for that flag and it would work... To me it's confusing because I'm not immediately sure if --aws-profile somehow the name of the profile i'd like to add or is the source... when really it is where we're copying some values from.

chrisgilmerproj · 2020-07-10T22:56:54Z

TBH I have no idea what I'd call it... cuz it's not like you're explicitly defining the source profile here so you can't call it that...

I get that if you know your "base" account is in *-id you could move to the *-id folder and use $AWS_PROFILE as the argument for that flag and it would work... To me it's confusing because I'm not immediately sure if --aws-profile somehow the name of the profile i'd like to add or is the source... when really it is where we're copying some values from.

Yeah, its a tough problem. I'd love to get opinions on this because honestly its easy to change if folks hate it. I think the logic of the PR is sound, its this user interface that needs some attention.

chrisgilmerproj · 2020-07-10T23:08:05Z

Bug: null pointer dereference when creating a new MFA device. Double check that this works.

chrisgilmerproj · 2020-07-10T23:43:44Z

Bug: null pointer dereference when creating a new MFA device. Double check that this works.

Fixed!

eeeady

chrisgilmerproj self-assigned this Jun 30, 2020

chrisgilmerproj commented Jun 30, 2020

View reviewed changes

cmd/setup.go Outdated Show resolved Hide resolved

chrisgilmerproj mentioned this pull request Jul 6, 2020

Add the logger to the user struct to pass around #71

Merged

Chris Gilmer added 21 commits July 10, 2020 10:52

Abstract out the update of aws profiles in aws config to a more gener…

5f21c23

…ic function

Allow exiting profiles to be updated with this tool

415bbf3

Quote the profile name

8ec7ee7

Add profile using existing profile

6673a8d

Add base profile flag

7ba7728

Add tests for cli

08a0f72

Rename test file for setup subcommand

e83ffcf

Upgrade aws-vault code and allow for multiple account setup. Rename U…

2f69367

…ser to SetupConfig for clarity.

No need to add mfa serial to base profile

274bc7c

Complete add-profile subcommand

a0fd9ab

Add test for add-profile cli inputs

fd74bbf

Update readme with new instructions on usage.

c63a75c

Start tests for add-profile

c108f5d

Move test data into test functions

470b1c9

Validate the output of the config file after it is created

5c8df8e

Start using testing suites to separate code

544bee4

Move shared code and tests for that code

696ae86

Convert tests for add-profile and setup to using test suite

87dc582

Use suite for common code

e582a1d

Use a struct to capture inherited code

2e47044

Make deleteSession part of the setupConfig methods

87e86db

chrisgilmerproj force-pushed the cg_update_profiles branch from fe2c825 to 87e86db Compare July 10, 2020 20:14

chrisgilmerproj marked this pull request as ready for review July 10, 2020 20:17

chrisgilmerproj requested a review from a team July 10, 2020 20:17

eeeady reviewed Jul 10, 2020

View reviewed changes

Add a note about overwriting configs

0390fca

Chris Gilmer added 3 commits July 10, 2020 16:32

Fix error where baseprofile is empty

fc4bc08

Fix error with null MFA serial

1f0e88e

Add logging for mfa creation, which helps in debugging

62a9dbf

chrisgilmerproj requested a review from mdawn July 14, 2020 22:49

eeeady approved these changes Jul 20, 2020

View reviewed changes

chrisgilmerproj merged commit e9206e0 into master Jul 20, 2020

chrisgilmerproj deleted the cg_update_profiles branch July 20, 2020 17:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Managing AWS profiles #67

Managing AWS profiles #67

chrisgilmerproj commented Jun 30, 2020 •

edited

Loading

chrisgilmerproj commented Jul 10, 2020

eeeady commented Jul 10, 2020

Add Profiles

eeeady commented Jul 10, 2020

chrisgilmerproj commented Jul 10, 2020

eeeady left a comment

eeeady Jul 10, 2020

chrisgilmerproj Jul 10, 2020

eeeady commented Jul 10, 2020

chrisgilmerproj commented Jul 10, 2020

chrisgilmerproj commented Jul 10, 2020

chrisgilmerproj commented Jul 10, 2020

eeeady left a comment

Managing AWS profiles #67

Managing AWS profiles #67

Conversation

chrisgilmerproj commented Jun 30, 2020 • edited Loading

Setup

Add Profiles

Testing

References

chrisgilmerproj commented Jul 10, 2020

eeeady commented Jul 10, 2020

Add Profiles

eeeady commented Jul 10, 2020

chrisgilmerproj commented Jul 10, 2020

eeeady left a comment

Choose a reason for hiding this comment

eeeady Jul 10, 2020

Choose a reason for hiding this comment

chrisgilmerproj Jul 10, 2020

Choose a reason for hiding this comment

eeeady commented Jul 10, 2020

chrisgilmerproj commented Jul 10, 2020

chrisgilmerproj commented Jul 10, 2020

chrisgilmerproj commented Jul 10, 2020

eeeady left a comment

Choose a reason for hiding this comment

chrisgilmerproj commented Jun 30, 2020 •

edited

Loading