Skip to content

Adding reference to an unsupported built-in tool on OpenAI Chat API no longer throws #6276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Apr 10, 2025
Merged

Adding reference to an unsupported built-in tool on OpenAI Chat API no longer throws #6276

merged 2 commits into from
Apr 10, 2025

Conversation

artl93
Copy link
Member

@artl93 artl93 commented Apr 10, 2025 •
edited
Loading

Fixes #6274

  • After removing non-AIFunction calls, we no longer set the result.ToolChoice to Auto, but leave it null.
  • Added a corresponding test to confirm that the function is dropped.

Previously asking for a response would result in:

An exception of type 'System.ClientModel.ClientResultException' occurred in System.Private.CoreLib.dll but was not handled in user code: 'HTTP 400 (invalid_request_error: )
Parameter: tool_choice

Invalid value for 'tool_choice': 'tool_choice' is only allowed when 'tools' are specified.'

Now it just carries on as if no tool was specified.

artl93 added 2 commits April 9, 2025 20:07
@artl93
The OpenAI chat API does not support built-in tool calling
0662eb2 
- Remove the unsupported tool before calling
- Add test
@artl93
Fix naming
f0a360e
@artl93 artl93 requested a review from a team as a code owner April 10, 2025 03:31
@github-actions github-actions bot added the area-ai Microsoft.Extensions.AI libraries label Apr 10, 2025
@artl93
Copy link
Member Author

artl93 commented Apr 10, 2025 •
edited
Loading

@stephentoub - there is the alternative here that we could throw when the requested tool / function counts do not match.

Also note, the "CreateNoneChoice" does not work either - same result as auto. It appears that if there are no tools rendered, setting auto or none have the same effect because it renders a tool_choice into the json.

@artl93 artl93 requested a review from stephentoub April 10, 2025 03:40
@artl93
Copy link
Member Author

artl93 commented Apr 10, 2025

Verified having a valid and built-in tool will prefer the auto tool choice, which works properly.

stephentoub
stephentoub approved these changes Apr 10, 2025
View reviewed changes
Copy link
Member

@stephentoub stephentoub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@stephentoub stephentoub merged commit 69612d9 into dotnet:main Apr 10, 2025
7 checks passed
jeffhandley pushed a commit that referenced this pull request Apr 30, 2025
@artl93 @jeffhandley
Adding reference to an unsupported built-in tool on OpenAI Chat API n…
4ef956f 
…o longer throws (#6276)
jeffhandley pushed a commit that referenced this pull request Apr 30, 2025
@artl93 @jeffhandley
Adding reference to an unsupported built-in tool on OpenAI Chat API n…
b6fcfd1 
…o longer throws (#6276)
jeffhandley added a commit that referenced this pull request Apr 30, 2025
@jeffhandley @peterwald
@jozkee @artl93 @stephentoub @roji @SteveSandersonMS @eiriktsarpalis @MackinnonBuck @rogerbarreto
Staging the MEAI 9.4.3 Release (#6353)
e80caff 
* Merged PR 49569: Cherry pick Evaluation changes for 9.4.3 release

* Update package-lock.json for Evaluations

* Reset package-lock.json to same as main branch

* Merged PR 49585: [9.4.3] [cherry pick] A couple of minor fixes

A couple of minor fixes

1. Adds back tool tip for metrics which are useful in cases where the metric name is long and does not fit entirely in the card
2. Makes the size of the selection buttons on the left of cards consistent with the size of status icons on the right to make the alignment cleaner
3. Implement IEquatable for UrlCacheKey (per Copilot's suggestion)
4. Throw ArgumentException when the number of messages passed to ContentSafetyChatClient does not match expectations (also per Copilot's suggestion)

* Add test for optional parameters being required with RequireAllProperties (#6265)

* Add test for optional parameters being required with RequireAllProperties

* Test both requireAllProperties values

* Use AssertDeepEquals that logs the difference to other tests

* Adding reference to an unsupported built-in tool on OpenAI Chat API no longer throws (#6276)

* Update M.E.AI changelogs (#6269)

* Augment UseDistributedCache XML docs (#6256)

* Augment AIFunctionFactory.Create XML docs (#6255)

* Rename EmbeddingGeneratorExtensions.GenerateEmbedding extension methods (#6295)

To align with the the base method on IEmbeddingGenerator

* Augment FunctionInvokingChatClient's span with token counts (#6296)

* Rename ChatThreadId to ConversationId (#6300)

* In OpenAI responses client, use response ID as ChatThreadId

* Rename ChatThreadId -> ConversationId

* Related renames

* Restore deleted members as obsolete (#6304)

* Restore EmbeddingGeneratorExtensions members as obsolete

* Restore ChatThreadId as obsolete

* Restore ChatResponse.ChatThreadId and ChatResponseUpdate.ChatThreadId as obsolete.

* Remove the tests for obsolete members

* Remove the Embeddings tests for obsolete members

* Support [FromKeyedServices] in AIFunctionFactory (#6310)

* Utilize IServiceProviderIsService in AIFunctionFactory (#6317)

Add AIFunctionFactoryOptions.Services, and use it when examining function parameters to determine whether they should be resolved by default from DI.

* Remove AsChatClient/AsEmbeddingGenerator that were obsoleted in 9.4.0-preview.1.25207.5 (#6327)

* Add ChatOptions.AllowMultipleToolCalls (#6326)

* Add ChatOptions.AllowMultipleToolCalls

* Use it in OpenAI adapter

* Disable default required property schema generation and OpenAI strict mode. (#6285)

* Disable default required property schema generation and OpenAI strict mode.

* Default strictness to OpenAI client defaults.

* Fix a few failing tests.

* Undo a number of additional changes made by #6064

* Revert change to structured output defaults.

* Incorporate more test fixes.

* Address feedback.

* Tweak strict mode signature, fix failing tests.

* Roll back schemaIsStrict flag from ChatResponseFormat.

* Merged PR 49624: [9.4.3] [cherry-pick] Update readmes (#6345)

Includes name of new metric that was added to Quality package.

Also updates some doc comments.

* Bump version to 9.4.3

* Merged PR 49636: [9.4.3] [cherry-pick] Skip messages that have no text when rendering conversations as part of evaluation prompts (#6349)

* Add test for optional parameters being required with RequireAllProperties (#6265)

* Add test for optional parameters being required with RequireAllProperties

* Test both requireAllProperties values

* Use AssertDeepEquals that logs the difference to other tests

* Adding reference to an unsupported built-in tool on OpenAI Chat API no longer throws (#6276)

* Update M.E.AI changelogs (#6269)

* Augment UseDistributedCache XML docs (#6256)

* Augment AIFunctionFactory.Create XML docs (#6255)

* Rename EmbeddingGeneratorExtensions.GenerateEmbedding extension methods (#6295)

To align with the the base method on IEmbeddingGenerator

* Augment FunctionInvokingChatClient's span with token counts (#6296)

* Rename ChatThreadId to ConversationId (#6300)

* In OpenAI responses client, use response ID as ChatThreadId

* Rename ChatThreadId -> ConversationId

* Related renames

* Restore deleted members as obsolete (#6304)

* Restore EmbeddingGeneratorExtensions members as obsolete

* Restore ChatThreadId as obsolete

* Restore ChatResponse.ChatThreadId and ChatResponseUpdate.ChatThreadId as obsolete.

* Remove the tests for obsolete members

* Remove the Embeddings tests for obsolete members

* Support [FromKeyedServices] in AIFunctionFactory (#6310)

* Utilize IServiceProviderIsService in AIFunctionFactory (#6317)

Add AIFunctionFactoryOptions.Services, and use it when examining function parameters to determine whether they should be resolved by default from DI.

* Remove AsChatClient/AsEmbeddingGenerator that were obsoleted in 9.4.0-preview.1.25207.5 (#6327)

* Add ChatOptions.AllowMultipleToolCalls (#6326)

* Add ChatOptions.AllowMultipleToolCalls

* Use it in OpenAI adapter

* Disable default required property schema generation and OpenAI strict mode. (#6285)

* Disable default required property schema generation and OpenAI strict mode.

* Default strictness to OpenAI client defaults.

* Fix a few failing tests.

* Undo a number of additional changes made by #6064

* Revert change to structured output defaults.

* Incorporate more test fixes.

* Address feedback.

* Tweak strict mode signature, fix failing tests.

* Roll back schemaIsStrict flag from ChatResponseFormat.

* Bump version to 9.4.3

* Merged PR 49636: [9.4.3] [cherry-pick] Skip messages that have no text when rendering conversations as part of evaluation prompts (#6349)

* Update chat template dependencies, fix OpenAI/Aspire config, and address build warnings (#6280)

* Update chat template dependencies

* Update test snapshots

* Update Aspire version

* Revert Aspire + update CommunityToolkit.Aspire

* Open README in VS after project creation

* Add Known Issue to Aspire README for Qdrant

* Update survey template URL

* Set the Project Template package version to -preview.2

* Update template baseline

* Update template pinned versions

* Do not append template args to snapshot names

* Fix vector store index in the README. Add an AzureAISearch template test.

* Add a note to the Aspire README for trusting the development certificate

* Use AddOpenAIClient for OpenAI and AddAzureOpenAIClient for Azure OpenAI

* Remove duplicated using System.ClientModel

* Update Aspire README to specify the exception thrown for the known issue

* Augment the Aspire README for more Docker notes for Ollama and Qdrant

* Fix Microsoft.Extensions.Http.Resilience warnings with separate pinned versions

* Update test baseline for: Rename EmbeddingGeneratorExtensions.GenerateEmbedding extension methods (#6295)

* Add --managed-identity to the template developer README

* Apply the Ollama timeout recommendation in the template code

* Call http.RemoveAllResilienceHandlers before adding the handler

* Update template test baseline

---------

Co-authored-by: Jeff Handley <[email protected]>
Co-authored-by: David Cantu <[email protected]>

* Expose AIContent constructor (#6346)

* Add PDF support to OpenAI AsIChatClient (#6344)

* Add PDF support to OpenAI AsIChatClient

* Add missing reference

* Make CreateJsonSchema tolerate JSO inputs that don't have a resolver set. (#6348)

* Update MEAI.Templates to use the just-built version of the libraries

* Enhance Function Invocation Extensibility for Microsoft.Extensions.AI (#6325)

---------

Co-authored-by: Peter Waldschmidt <[email protected]>
Co-authored-by: Peter Waldschmidt <[email protected]>
Co-authored-by: Shyam Namboodiripad <[email protected]>
Co-authored-by: David Cantú <[email protected]>
Co-authored-by: Art Leonard <[email protected]>
Co-authored-by: Stephen Toub <[email protected]>
Co-authored-by: Shay Rojansky <[email protected]>
Co-authored-by: Steve Sanderson <[email protected]>
Co-authored-by: Eirik Tsarpalis <[email protected]>
Co-authored-by: Mackinnon Buck <[email protected]>
Co-authored-by: Roger Barreto <[email protected]>
@github-actions github-actions bot locked and limited conversation to collaborators May 10, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@stephentoub stephentoub stephentoub approved these changes

Assignees

@stephentoub stephentoub

@artl93 artl93

Labels
area-ai Microsoft.Extensions.AI libraries
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Using HostedWebSearchTool causes OpenAI's IChatClient to throws an exception when request sent
2 participants
@artl93 @stephentoub