Failed to set Endpoint via EndpointProdiver or endpointUrl in S3Client

[hantsy]

Describe the bug

I tried to update my codes to use the latest AWS SDK Kotlin 0.20.1-beta, found the endpointResolver is removed.

I have tried to used the following two approaches to set endpoint, but failed.

S3Client {
       // other config.
        endpointProvider = properties.endpoint?.let { url ->
            EndpointProvider { params ->
                Endpoint(url)                }
        }
}

or set it via endpointUrl.

S3Client {
       // other config.
        endpointUrl = properties.endpoint?.let { Url.parse(it) }
}

The properties.endpoint is http://localhost:4566/ in local testing env.

Expected behavior

It should work as EndpointResolver. The old config worked well in a previous version.

 endpointResolver = properties.endpoint?.let {
            StaticEndpointResolver(
                AwsEndpoint(it)
            )
       }

Current behavior

I got the exception like this when calling the S3Client in my codes.

	error=exception while calling s3 with unknown operation: Unable to find operation for request to service s3: PUT /

Steps to Reproduce

I used a localstack to mock all AWS locally, so I need to setup the endpoint to http://localhost:4566

Possible Solution

No response

Context

No response

AWS Kotlin SDK version used

0.20.1-beta

Platform (JVM/JS/Native)

Java 17

Operating System and version

Windows 10

[lauzadis]

Hi, I was able to reproduce this. Our recent Endpoints 2.0 update had to change some things regarding bucket names in the request URL. We strip the bucket names from URLs due to how S3 parses them internally. That is the reason why you are getting the PUT / error, the bucket name is being stripped from the request.

To fix this you have to use endpointUrl and configure forcePathStyle to true

S3Client {
       // other config.
        endpointUrl = properties.endpoint?.let { Url.parse(it) }
        forcePathStyle = true
}

[hantsy]

Yes, my endpoint does not included a bucket name, the bucket name is configured by another property.

[hantsy]

Adding forcePathStyle = true make my tests work again on local machine.

But in my original config, endpoint is optional, only for the localstack for testing purpose.

Not sure this property(forcePathStyle = true) will affect the production config(only use AWS credential and bucket name, no endpoint there).

[hantsy]

If put the bucket name on the url, that means there is no need to set bucketname in the s3 operations?

I have checked the examples in this repo and the official doc https://docs.aws.amazon.com/sdk-for-kotlin/latest/developer-guide/kotlin_s3_code_examples.html, there is no examples demos how to build a S3 Client using the new Endpoint.

[lucix-aws]

Closing this issue since you appear to be unblocked through use of forcePathStyle.

Your most recent question appears to relate to #848, I've left a response there.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[NikolayMetchev]

@lucix-aws Hi, in the latest version of the AWS SDK forcePathStyle has been removed (f6502eb) which is a workaround mentioned #842 (comment)
Is this workardoung no longer required?

[hantsy]

Updated to 0.35.0-beta, use forcePathStyle, it does not work as expected, I also post a question on discussions.

#1121

The example project config: https://github.com/hantsy/aws-sdk-kotlin-spring-example/blob/master/src/main/kotlin/com/example/demo/internal/AwsConfig.kt#L43-L47

Both endpointProvider and endpointUrl are not working as expected. I only want to use the enpoint url in the a local/test env with LocalStack.