(TES): Expired credentials when RequiresPrivilege set to true

[acorbel]

Describe the bug
When we set RequiresPrivilege to true for a component directly accessing AWS services (TES), we get expired credentials. However, it works fine if it set to false.

To Reproduce

• Create an S3 bucket with appropriate permissions given to the Greengrass role
• Create a component (we're using nodejs) and TES as dependency which uploads a static file to the bucket (PutObjectCommand with SDK V3 js)
• In the recipe, set RequiresPrivilege to true

Expected behavior
We should get valid credentials by TES.

Actual behavior
We get an The provided token has expired error.

Environment

• OS: Windows 10
• Nucleus version: 2.14.3

Additional context
It works as intended if RequiresPrivilege is set to false.

[yitingb]

Hi,
Could you share your component recipe please? We were trying to reproduce it but didn't encounter any issues to upload an object to s3.
May we also know how you're importing your AWS credentials please?

Thanks

[patrzhan]

TES credentials are accessible to the component as Greengrass sets the AWS_CONTAINER_CREDENTIALS_FULL_URI environment variable which is the 5th option in the credentials chain: https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials-chain.html. This means that if credentials are available to the component via one of the earlier items in the credentials chain, then your component will likely be using those credentials rather than the TES ones.

It's possible that when your component is running with RequiresPrivilege, some other credentials are being picked up before the SDK attempts to check the TES provided credentials. The expired credentials you're seeing are likely sourced from one of these alternative higher-priority options for the SDK to retrieve credentials--for example, if the standard AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables are set.

Can you check if any of these alternative credential retrieving paths are being used by your privileged user? It is unlikely that this is necessarily a bug with Greengrass or TES.

[acorbel]

Hello,

Thank you for your responses.
I've done several tested on different machines with different scenarios.

You are right.
We use SSM (hybrid mode) to manage patching as well on these machines.
When using RequiresPrivilege to true the SDK tries to assume the SSM role and not the Greengrass one.

Do you know a way to avoid this kind of side effect? To force the component to use Greengrass TES container under elevated privileges?

[patrzhan]

It seems like the credential provider used can be specified directly with the SDK. Since you mentioned you're using the SDK for JS v3 I found this: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-credential-providers/#fromcontainermetadata-and-frominstancemetadata. I believe the fromContainerMetadata() one is what you'll need here, and the fromHttp() provider documented below is a generic version that also uses the same environment variables in case the former doesn't work.