feat: Add S3 support for artifacts

[wmsnp]

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

• Closes: Add S3 support to manage artifacts. #555
• Related: feat: Add S3 support for artifacts #556

Problem:
A previous PR attempted to add an Artifact Service but was closed due to merge conflicts and incompatibility with the latest ADK interfaces. While the ADK community plugin repository could offer unofficial implementations, there is currently no official, production-ready, self-hosted Artifact Service.

Solution:

• Introduce S3ArtifactService to provide an self-hosted Artifact storage solution:
  • Supports asynchronous upload and download of S3 artifacts
  • Includes unit tests covering core functionality
  • Depends on aioboto3 (not yet added to pyproject.toml; users must install it manually)
• Fix Windows path handling in FileArtifactService and its tests to ensure cross-platform compatibility

Testing Plan

Unit Tests:

• I have added or updated unit tests for my change.
• All unit tests pass locally.

(.venv) PS > pytest .\tests\unittests\artifacts\                        
=============================================================== test session starts ===============================================================
platform win32 -- Python 3.13.3, pytest-9.0.1, pluggy-1.6.0
configfile: pyproject.toml
plugins: anyio-4.11.0, langsmith-0.4.43, asyncio-1.3.0, mock-3.15.1, xdist-3.8.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
collected 53 items                                                                                                                                 

tests\unittests\artifacts\test_artifact_service.py .......................................                                                   [ 73%]
tests\unittests\artifacts\test_artifact_util.py ..............                                                                               [100%]

=============================================================== 53 passed in 1.80s ================================================================

Manual End-to-End (E2E) Tests:

Please provide instructions on how to manually test your changes, including any
necessary setup or configuration. Please provide logs or screenshots to help
reviewers better understand the fix.

Checklist

• I have read the CONTRIBUTING.md document.
• I have performed a self-review of my own code.
• I have commented my code, particularly in hard-to-understand areas.
• I have added tests that prove my fix is effective or that my feature works.
• New and existing unit tests pass locally with my changes.
• I have manually tested my changes end-to-end.
• Any dependent changes have been merged and published in downstream modules. (Note: depends on aioboto3, not yet in pyproject.toml)

[gemini-code-assist]

Summary of Changes

Hello @wmsnp, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the artifact management capabilities by introducing a new S3-compatible artifact storage service. This addition addresses the need for an official, self-hosted solution, allowing users to store and retrieve artifacts asynchronously in S3 buckets. Furthermore, it includes an important fix for path handling in the existing file-based artifact service, improving its reliability across different operating systems.

Highlights

• New S3 Artifact Service: Introduced a new S3ArtifactService to provide a self-hosted, production-ready solution for storing and retrieving artifacts in Amazon S3 or compatible services. This service supports asynchronous upload and download of artifacts.
• Cross-Platform Path Handling Fix: Resolved an issue with Windows path handling in the FileArtifactService and its associated tests, ensuring better cross-platform compatibility for artifact storage.
• aioboto3 Integration: Integrated aioboto3 for asynchronous interaction with S3, adding it as a test dependency in pyproject.toml. Users will need to install aioboto3 manually for now.
• Comprehensive Test Coverage: Added extensive unit tests for the new S3ArtifactService and updated existing artifact service tests to include the S3 implementation, ensuring robust functionality.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[adk-bot]

Response from ADK Triaging Agent

Hello @wmsnp, thank you for creating this PR!

Could you please provide the manual end-to-end (E2E) test plan in the "Manual End-to-End (E2E) Tests" section of your PR description? This should include instructions on how to manually test your changes, along with any necessary setup or configuration details, logs, or screenshots that can help reviewers better understand the fix.

This information will help reviewers to review your PR more efficiently. Thanks!

[gemini-code-assist]

Code Review

This pull request introduces an S3ArtifactService for artifact storage, which is a great addition. The implementation is comprehensive and includes good test coverage.

My review focuses on a few key areas for improvement:

• There is a critical race condition in save_artifact that could lead to data loss under concurrent writes.
• Several methods that list S3 objects (list_artifact_keys, list_versions, list_artifact_versions) do not handle API pagination, which will lead to incomplete results when more than 1000 items exist.
• There are opportunities to improve performance in delete_artifact and get_artifact_version by using more efficient S3 operations.
• A minor improvement is suggested for the test setup to enhance test isolation.

Overall, this is a solid foundation for S3 support, and addressing these points will make it more robust and performant.

[wmsnp]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces an S3ArtifactService to provide a self-hosted artifact storage solution using S3-compatible services. The implementation is robust, leveraging asynchronous operations, pagination for listing, and atomic uploads. It also includes comprehensive unit tests with detailed mocks for the S3 client. Additionally, a fix for Windows path handling in FileArtifactService is included.

My review includes a few suggestions to improve the new service:

• Properly packaging the aioboto3 dependency as an optional extra for users.
• Increasing the robustness of metadata parsing.
• Improving the clarity of the S3 client lifecycle management.

[gemini-code-assist]

The manual invocation of __aenter__ to initialize the S3 client is a bit unusual and relies on the corresponding __aexit__ call in the close() method. While this is a valid pattern for managing a long-lived client, it's not immediately obvious. To improve code clarity and maintainability for future developers, please add a comment explaining why this manual context management is used (e.g., for performance by reusing the client and its connection pool across multiple calls).

[ryanaiagent]

Hi @wmsnp , Thank you for your contribution through this pull request! This PR has merge conflicts that require changes from your end. Could you please rebase your branch with the latest main branch to address these?
Once this is complete, can you please fix the failing unit tests.

[wmsnp]

@ryanaiagent please check it

[ryanaiagent]

Hi @wmsnp ,Your PR has been received by the team and is currently under review. We will provide feedback as soon as we have an update to share.

[ryanaiagent]

Hi @DeanChensj , can you please review this?

[DeanChensj]

Hi @wmsnp , can we move this to adk-community repo?

[hangfei]

This is really great contribution. It fits well with adk community repo. please move it there: https://github.com/google/adk-python-community. Thanks!