Integration with SQS

[maciejwalkowiak]

Integration with SQS must be migrated to AWS SDK v2. This is not meant to be a direct migration, but rather re-thinking how the current one is done. Things to consider:

• must implement Spring Messaging interfaces
• must support both regular queues and FIFO queues
• improve concurrency (Better SQS concurrent message handling #23)
• think how to extends message visibility when processing FIFO messages ([SQS] Exceeding/Changing visibility timeout may lead to messages being processed twice in FIFO queues #196)
• auto extend visibility timeout ([SQS] Auto extend visibility timeout for long tasks spring-attic/spring-cloud-aws#635)
• support different concurrency configuration per listener

Think about all issues listed under:

• SQS issues in awspring
• SQS issues in spring-attic

[tomazfernandes]

Hi @maciejwalkowiak, would you be willing to look into a community contribution for this, or is it something you're planning on getting done by the core team?

I have some ideas and would be willing to open a draft PR maybe mid next week so we could take a look. Then, of course, if you don't like the design, no worries, should at least bring up some interesting discussions.

What do you think?

Thanks.

[maciejwalkowiak]

It's been planned to be done by the core team but since we are not getting into it in upcoming weeks you are welcome to contribute! I don't promise it will be merged as it is but hopefully we will be able to work together on it. I don't have yet a design in mind so I am open for ideas.

[tomazfernandes]

Sounds good! I'll open a draft PR with an MVP of what I have in mind then, and if it looks like a good approach we can work our way up from there.

Design-wise, I still have to take a closer look into a few things, but it seems some of the Spring Messaging interfaces, such as the MessageChannel, are not really designed for the kind of async capabilities that the AWS SDK 2.0 allows for. The Handler family seems to have some more interesting options though.

Also, the Spring Kafka project has a solid design on the assembly phase, so we could probably take a few pages from there.

Then there's the flexibility to have a few strategies for polling, acknowledging, and so on, optimizing either for high throughput or low resource usage, or any mix of both.

How does that sound? Please let me know if you have any further inputs or questions, or we can discuss it further in the draft PR.

Thanks!