Support for Progress Listener

[sdhuka]

Describe the feature

Add support for Progress Listener on PutObjectRequest, GetObjectRequest and UploadPartRequest. Progress Listener is used to track progress for Upload and Download task from S3.

Is your Feature Request related to a problem?

Amplify SDK relies on this functionality to support progress tracking for S3 Transfer apis.

Proposed Solution

AWS Android SDK support this functionality and could be used as reference.

Describe alternative solutions or features you've considered

No response

Acknowledge

• I may be able to implement this feature request

AWS Kotlin SDK version used

0.9.5-beta

Platform (JVM/JS/Native)

Native

Operating System and version

Android 12

[BassirouRabo]

Hi,
any update regarding this feature? it is implemented using the Java SDK, that would be nice to have it on the kotlin SDK too.
If not what other way can we listen upload progress ?

[aajtodd]

We have a design in place for how we want to expose this. The design requires some other in-progress work to be usable on a per/request basis though.

[aajtodd]

Closing this as both pieces (interceptors + per/op config) are now available as of 0.20.0-beta.

The way to do this is roughly as follows:

1. Create an interceptor that replaces the HTTP body with one that updates your progress listener every time data is read. I've left out a bunch of detail here, but you'll have to deal with various ByteStream variants. More than likely you would want a wrapper type per variant and match that to the actual underlying builder.body type and branch on that.

class AddUploadProgressListenerInterceptor(
    private val progressListener: ProgressListener
): HttpInterceptor {
    override suspend fun modifyBeforeTransmit(
        context: ProtocolRequestInterceptorContext<Any, HttpRequest>
    ): HttpRequest {
        val builder = context.protocolRequest.toBuilder()
        // wrap the body with a custom type that updates `progressListener` every time data is read
        // in practice you need to probably unwrap builder.body and have a progress listener type per/variant
        builder.body = ProgressListenerBody(builder.body, progressListener)
        return builder.build()
    }
}

2. Add this interceptor for specific operations, e.g.

val progressListener = ProgressListener(...)
val response = s3.withConfig { interceptors += AddUploadProgressListener(progressListener) }
    .use { s3WithListener ->
         // this operation will be carried out with the progress listener
         s3WithListener.putObject(...)
    }

It should look roughly the same for response bodies but with a different interceptor hook (e.g. modifyBeforeDeserialization). Start a discussion if you have questions on how to do any of this.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[Conjuror1279]

Hi @aajtodd, is there someplace where I can find a code sample for the ProgressListenerBody class, I am not able to understand how to add a progress listener in the above code. The function modifyBeforeTransmit is only called once and doesnot give me the progress about the upload. I've been stuck at this for quite a while and any help would be much appreciated.

Also, is there a way to receive or generate the presigned url of the uploaded file using this kotlin sdk?

[sdhuka]

@Conjuror1279 You can look at the Amplify source code for both upload and download progress listener reference.

You can use pre-signed API similar to this

[ianbotsf]

Hi @Conjuror1279, I see that you also opened a discussion (#926) and an issue (#922) for this. Have you been able to look at the links provided by @sdhuka and test them out?

[Conjuror1279]

Hi @sdhuka @ianbotsf. Thanks for your reply. I created a issue regarding this and was waiting for a response over there, not realizing that you had responded here. Anyways, I will try the method pointed out by you and see if it works.

Thanks again for your help!

[Conjuror1279]

Hi, I tried the method suggested and it seems that the progress is always coming up to be 8192.0 bytes. I logged the bytes provided from the below code, they come out to be constant. This piece of code is executing multiple times but the bytes returned is always the same
return delegate.read(sink, limit).also {
if (it > 0) {
progressListener?.onProgress(it)
}
}

[sdhuka]

@Conjuror1279 thats correct, you have to add up the progress reported by the interceptor to calculate the overall progress, something similar to this.