-
Notifications
You must be signed in to change notification settings - Fork 445
ci: enable integ tests for anthropic, cohere, mistral, openai, writer #510
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: enable integ tests for anthropic, cohere, mistral, openai, writer #510
Conversation
Noting that Will need to get a separate PR out just to expose it |
The linting change is failing because of a breaking change made by A2A which switched naming form camel to snake case https://github.com/a2aproject/a2a-python/releases/tag/v0.2.16 |
Description
Currently only Bedrock is being exercised in our integration tests. However, the Strands team has secured API keys for various providers. These keys are stored in Secrets Manager are loaded before all tests. The Secrets Manager Secret name is hidden using GitHub repository secrets. However, the expected form is
Tests are skipped when the environment variable is missing. To prevent regressions there is now an added validation where if any of the following are not set as env vars AND we are executing within a GitHub Actions workflow then we fail
The test in
conformance.py
was not actually being executed because the file did not have atest_
prefix. To address this it was renamed tests_integ/models/test_conformance.py. Additionally, because it was not being executed a bug was left in remaining from the pre 1.0.0 release where the Model ABC was moved from types/ to models/.Related Issues
#237
Type of Change
Other (please describe): Enable testing of all providers
Testing
How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli
Tests were run in a testing repository to verify the workflows succeed
hatch run prepare
Checklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.