Urgent: InvalidIdentityTokenException with AWS SDK in Version 3.503.0 causes immediate failure (govcloud)

[jonathon-mcnabb]

Checkboxes for prior research

• I've gone through Developer Guide and API reference
• I've checked AWS Forums and StackOverflow.
• I've searched for previous similar issues and didn't find any solution.

Describe the bug

Urgent: Services cannot connect to AWS

InvalidIdentityTokenException with AWS SDK in Version 3.503.0

Environment:

AWS SDK Version: 3.503.0
Services we use that are affected: Amazon S3, Amazon DynamoDB (likely affects others)

Description:
After upgrading to @aws-sdk/client-s3 and @aws-sdk/client-dynamodb version 3.503.0, we are encountering an InvalidIdentityTokenException when attempting to access S3 and DynamoDB services. The same issue is not observed in previous versions of the package. No changes were made to the source code other than updating the package version.

Reproduction Steps:

Upgrade @aws-sdk/client-s3 to version 3.503.0.
Upgrade @aws-sdk/client-dynamodb to version 3.503.0.
Attempt to perform standard operations on S3 and DynamoDB.
Observe the InvalidIdentityTokenException.

Expected Behavior:
Operations on S3 and DynamoDB should succeed without identity token issues, as they did in previous versions of the SDK.

Actual Behavior:
Received an error with the following details:

Error Name: InvalidIdentityTokenException
Fault: client
HTTP Status Code: 400
Request ID: [REDACTED]
Error Type: Sender
Error Code: InvalidIdentityToken
Error Message: No OpenIDConnect provider found in your account for [REDACTED URL]

SDK version number

@aws-sdk/package-name@version, ...

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v16.20.2

Reproduction Steps

"dependencies": {
"@aws-sdk/client-s3": "^3.501.0",
"@aws-sdk/client-dynamodb": "^3.501.0",
}

Observed Behavior

When connecting to AWS services, the following error is seen.

Received an error with the following details:

Error Name: InvalidIdentityTokenException
Fault: client
HTTP Status Code: 400
Request ID: [REDACTED]
Error Type: Sender
Error Code: InvalidIdentityToken
Error Message: No OpenIDConnect provider found in your account for [REDACTED URL]

Expected Behavior

I expect to be able to utilize AWS SDKs to connect to S3 and DynamoDB.

Possible Solution

No response

Additional Information/Context

No response

[RanVaknin]

Hi @jonathon-mcnabb ,

Thanks for reaching out.
We just released a fix for some other breaking changes. Can you pull 3.503.1 and see if it fixes your issue?

Thanks,
Ran~

[jonathon-mcnabb]

Hey @RanVaknin,

Thanks for the quick response. It looks like with the new version this issue is still occurring.

For more context, we are running an EKS cluster with an OIDC provider.

[kuhe]

@jonathon-mcnabb could you provide an example of how to initialize the SDK client, including how you get credentials?

If you are relying on the default credential chain, please confirm that.

[jonathon-mcnabb]

Hey @kuhe, here's an example of a client init:

import { S3Client } from '@aws-sdk/client-s3';
// Create an Amazon S3 service client object.
// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment, @typescript-eslint/no-unsafe-call
const s3Client: S3Client = new S3Client({ region: process.env.AWS_REGION });
export { s3Client };

Out stack utilizes EKS with node microservices, using an ODIC provider. The following shows what env variables are defined on the node:

AWS_STS_REGIONAL_ENDPOINTS: regional
AWS_ROLE_ARN: arn:aws-us-gov:iam::[REDACTED_ACCOUNT]:role/[REDACTED_ROLE]
AWS_WEB_IDENTITY_TOKEN_FILE: /var/run/secrets/eks.amazonaws.com/serviceaccount/token

I have confirmed that a token is present at the path.

I am not sure if I am relying on the default credential change, though this may be occurring under the hood.