Move HttpClientFactory to common to expose to other components #4175

zane-neo · 2025-09-17T06:58:09Z

Description

Move HttpClientFactory to common to expose to other components(skills)

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

ylwu-amzn · 2025-09-17T18:17:49Z

common/build.gradle

    compileOnly group: 'com.networknt' , name: 'json-schema-validator', version: '1.4.0'
    // Multi-tenant SDK Client
    compileOnly "org.opensearch:opensearch-remote-metadata-sdk:${opensearch_build}"
+    compileOnly (group: 'software.amazon.awssdk', name: 'netty-nio-client', version: "2.30.18") {


Any impact to other plugins which depend on ml-commons? Like neural-search, flow-framework

For now it's only skills.

@dbwiddis I know you wanted dependent plugins to get informed if in ml-commons we change any dependencies.

Thanks @dhrubo-os .

@zane-neo Highly recommend you use the version from the OpenSearch version catalog for any awssdk components. I believe 2.30.18 is CVE-impacted. (see https://mvnrepository.com/artifact/software.amazon.awssdk/netty-nio-client/2.30.18)

Agree we can use like this: https://github.com/opensearch-project/OpenSearch/blob/a08700c8fd9477c873413306ecb6a88d09ef6ede/plugins/ingestion-kinesis/build.gradle#L56

We have quite a few places using hard coded versions of amazon.awssdk, I prefer we change all these in a separate PR:

(base) zaniu@bcd07465d161:~/Documents/code/ml-commons$ grep 'amazon.awssdk' **/*.gradle common/build.gradle: compileOnly (group: 'software.amazon.awssdk', name: 'netty-nio-client', version: ${versions.aws}) { ml-algorithms/build.gradle: implementation platform('software.amazon.awssdk:bom:2.30.18') ml-algorithms/build.gradle: api 'software.amazon.awssdk:auth:2.30.18' ml-algorithms/build.gradle: implementation 'software.amazon.awssdk:apache-client' ml-algorithms/build.gradle: compileOnly group: 'software.amazon.awssdk', name: 'aws-core', version: "2.30.18" ml-algorithms/build.gradle: compileOnly group: 'software.amazon.awssdk', name: 's3', version: "2.30.18" ml-algorithms/build.gradle: compileOnly group: 'software.amazon.awssdk', name: 'regions', version: "2.30.18" ml-algorithms/build.gradle: resolutionStrategy.force 'software.amazon.awssdk:bom:2.30.18' plugin/build.gradle: implementation group: 'software.amazon.awssdk', name: 'aws-core', version: "2.30.18" plugin/build.gradle: implementation group: 'software.amazon.awssdk', name: 's3', version: "2.30.18" plugin/build.gradle: implementation group: 'software.amazon.awssdk', name: 'regions', version: "2.30.18" plugin/build.gradle: implementation group: 'software.amazon.awssdk', name: 'aws-xml-protocol', version: "2.30.18" plugin/build.gradle: implementation group: 'software.amazon.awssdk', name: 'aws-query-protocol', version: "2.30.18" plugin/build.gradle: implementation group: 'software.amazon.awssdk', name: 'protocol-core', version: "2.30.18" plugin/build.gradle: resolutionStrategy.force 'software.amazon.awssdk:bom:2.30.31' plugin/build.gradle: resolutionStrategy.force 'software.amazon.awssdk:auth:2.30.31' plugin/build.gradle: resolutionStrategy.force 'software.amazon.awssdk:aws-core:2.30.31' plugin/build.gradle: resolutionStrategy.force 'software.amazon.awssdk:aws-query-protocol:2.30.31' plugin/build.gradle: resolutionStrategy.force 'software.amazon.awssdk:aws-xml-protocol:2.30.31' plugin/build.gradle: resolutionStrategy.force 'software.amazon.awssdk:protocol-core:2.30.31' plugin/build.gradle: resolutionStrategy.force 'software.amazon.awssdk:regions:2.30.31' plugin/build.gradle: resolutionStrategy.force 'software.amazon.awssdk:netty-nio-client:2.30.31' plugin/build.gradle: resolutionStrategy.force 'software.amazon.awssdk:s3:2.30.31'

I prefer we change all these in a separate PR:

Didn't this PR fix most of them? https://github.com/opensearch-project/ml-commons/pull/4210/files

We can't release 3.3.0 with these CVE-impacted versions and time is short.

@dbwiddis I've made change to the versions, please take a look again.

ylwu-amzn · 2025-09-17T18:18:00Z

CI failed

zane-neo · 2025-09-17T23:37:50Z

CI failed

The CI failed since opensearch-remote-metadata-sdk used ThreadContextAccess which is not for public use, I've created this PR to fix: opensearch-project/opensearch-remote-metadata-sdk#254

Signed-off-by: zane-neo <[email protected]>

zane-neo · 2025-09-30T06:11:48Z

Failure ITs are flaky

…earch-project#4175) * Move HttpClientFactory to common to expose to other componenets Signed-off-by: zane-neo <[email protected]> * optimize code for better maintainability Signed-off-by: zane-neo <[email protected]> * Optimize code and increase UT coverage Signed-off-by: zane-neo <[email protected]> * Address comments Signed-off-by: zane-neo <[email protected]> * Use amazon aws version from opensearch core Signed-off-by: zane-neo <[email protected]> * address comments Signed-off-by: zane-neo <[email protected]> --------- Signed-off-by: zane-neo <[email protected]>

…earch-project#4175) * Move HttpClientFactory to common to expose to other componenets Signed-off-by: zane-neo <[email protected]> * optimize code for better maintainability Signed-off-by: zane-neo <[email protected]> * Optimize code and increase UT coverage Signed-off-by: zane-neo <[email protected]> * Address comments Signed-off-by: zane-neo <[email protected]> * Use amazon aws version from opensearch core Signed-off-by: zane-neo <[email protected]> * address comments Signed-off-by: zane-neo <[email protected]> --------- Signed-off-by: zane-neo <[email protected]> Signed-off-by: Brian Flores <[email protected]>

* Increment version to 2.19.4-SNAPSHOT Signed-off-by: opensearch-ci-bot <[email protected]> Signed-off-by: Brian Flores <[email protected]> * fix CVE-2025-55163, CVE-2025-48924 (#4298) * address commons-lang3 CVE-2025-48924 Signed-off-by: Brian Flores <[email protected]> * pin netty to 4.2.5.Final version address CVE-2025-55163 Signed-off-by: Brian Flores <[email protected]> * force all subProjects to use updated common-lang3 version Signed-off-by: Brian Flores <[email protected]> --------- Signed-off-by: Brian Flores <[email protected]> * Move HttpClientFactory to common to expose to other components (#4175) * Move HttpClientFactory to common to expose to other componenets Signed-off-by: zane-neo <[email protected]> * optimize code for better maintainability Signed-off-by: zane-neo <[email protected]> * Optimize code and increase UT coverage Signed-off-by: zane-neo <[email protected]> * Address comments Signed-off-by: zane-neo <[email protected]> * Use amazon aws version from opensearch core Signed-off-by: zane-neo <[email protected]> * address comments Signed-off-by: zane-neo <[email protected]> --------- Signed-off-by: zane-neo <[email protected]> Signed-off-by: Brian Flores <[email protected]> * use mainline versions.aws via hardcode Signed-off-by: Brian Flores <[email protected]> * address CVE-2025-58057 Signed-off-by: Brian Flores <[email protected]> * fix code format Signed-off-by: Brian Flores <[email protected]> * empty commit to trigger CI Signed-off-by: Brian Flores <[email protected]> * Fix claude model it (#4167) * fix model it by replace claude v1/v2 Signed-off-by: xinyual <[email protected]> * remove useless change Signed-off-by: xinyual <[email protected]> --------- Signed-off-by: xinyual <[email protected]> Signed-off-by: Brian Flores <[email protected]> * Don't convert schema-defined strings to other types during validation (#3761) Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Brian Flores <[email protected]> * fix Cohere IT (#4174) * fix Cohere IT Signed-off-by: xinyual <[email protected]> * apply spotless Signed-off-by: xinyual <[email protected]> * delete useless it Signed-off-by: xinyual <[email protected]> --------- Signed-off-by: xinyual <[email protected]> Signed-off-by: Brian Flores <[email protected]> --------- Signed-off-by: opensearch-ci-bot <[email protected]> Signed-off-by: Brian Flores <[email protected]> Signed-off-by: zane-neo <[email protected]> Signed-off-by: xinyual <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> Co-authored-by: opensearch-ci-bot <[email protected]> Co-authored-by: zane-neo <[email protected]> Co-authored-by: Xinyuan Lu <[email protected]> Co-authored-by: Daniel Widdis <[email protected]>

zane-neo requested review from HenryL27, Zhangxunmt, austintlee, b4sjoo, dhrubo-os, jngz-es, mingshl, model-collapse, pyek-bot, rbhavna, sam-herman, xinyual and ylwu-amzn as code owners September 17, 2025 06:58

zane-neo had a problem deploying to ml-commons-cicd-env September 17, 2025 06:59 — with GitHub Actions Failure

zane-neo had a problem deploying to ml-commons-cicd-env September 17, 2025 06:59 — with GitHub Actions Error

zane-neo had a problem deploying to ml-commons-cicd-env September 17, 2025 06:59 — with GitHub Actions Failure

ylwu-amzn force-pushed the fix-websearchtool-security-issue branch from f99633b to 4bce041 Compare September 17, 2025 14:59

ylwu-amzn had a problem deploying to ml-commons-cicd-env September 17, 2025 15:01 — with GitHub Actions Error

ylwu-amzn had a problem deploying to ml-commons-cicd-env September 17, 2025 15:01 — with GitHub Actions Failure

ylwu-amzn had a problem deploying to ml-commons-cicd-env September 17, 2025 15:01 — with GitHub Actions Error

ylwu-amzn reviewed Sep 17, 2025

View reviewed changes

ylwu-amzn had a problem deploying to ml-commons-cicd-env September 17, 2025 22:58 — with GitHub Actions Failure

ylwu-amzn had a problem deploying to ml-commons-cicd-env September 17, 2025 22:58 — with GitHub Actions Error

ylwu-amzn had a problem deploying to ml-commons-cicd-env September 17, 2025 22:58 — with GitHub Actions Failure

ylwu-amzn had a problem deploying to ml-commons-cicd-env September 17, 2025 22:58 — with GitHub Actions Error

ylwu-amzn had a problem deploying to ml-commons-cicd-env September 17, 2025 23:46 — with GitHub Actions Failure

address comments

d4df6d6

Signed-off-by: zane-neo <[email protected]>

zane-neo dismissed stale reviews from pyek-bot and dhrubo-os via d4df6d6 September 30, 2025 00:20

zane-neo force-pushed the fix-websearchtool-security-issue branch from 1e0862e to d4df6d6 Compare September 30, 2025 00:20

zane-neo temporarily deployed to ml-commons-cicd-env September 30, 2025 00:22 — with GitHub Actions Inactive

zane-neo had a problem deploying to ml-commons-cicd-env September 30, 2025 00:22 — with GitHub Actions Error

zane-neo had a problem deploying to ml-commons-cicd-env September 30, 2025 00:22 — with GitHub Actions Failure

zane-neo temporarily deployed to ml-commons-cicd-env September 30, 2025 00:22 — with GitHub Actions Inactive

Merge branch 'main' into fix-websearchtool-security-issue

a513f52

zane-neo had a problem deploying to ml-commons-cicd-env September 30, 2025 01:56 — with GitHub Actions Failure

zane-neo had a problem deploying to ml-commons-cicd-env September 30, 2025 01:56 — with GitHub Actions Error

dhrubo-os approved these changes Sep 30, 2025

View reviewed changes

zane-neo temporarily deployed to ml-commons-cicd-env September 30, 2025 04:07 — with GitHub Actions Inactive

zane-neo had a problem deploying to ml-commons-cicd-env September 30, 2025 04:07 — with GitHub Actions Failure

zane-neo had a problem deploying to ml-commons-cicd-env September 30, 2025 04:07 — with GitHub Actions Error

xinyual approved these changes Sep 30, 2025

View reviewed changes

zane-neo had a problem deploying to ml-commons-cicd-env September 30, 2025 05:11 — with GitHub Actions Failure

zane-neo had a problem deploying to ml-commons-cicd-env September 30, 2025 05:11 — with GitHub Actions Error

zane-neo merged commit 5883f54 into opensearch-project:main Sep 30, 2025
13 of 21 checks passed

brianf-aws mentioned this pull request Oct 22, 2025

fix CVE-2025-55163, CVE-2025-48924 #4298

Merged

5 tasks

brianf-aws mentioned this pull request Oct 22, 2025

fix CVE-2025-55163, CVE-2025-48924, CVE-2025-58057 #4339

Merged

5 tasks

Uh oh!

Move HttpClientFactory to common to expose to other components #4175

Move HttpClientFactory to common to expose to other components #4175

Conversation

zane-neo commented Sep 17, 2025

Description

Related Issues

Check List

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dbwiddis Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ylwu-amzn commented Sep 17, 2025

Uh oh!

zane-neo commented Sep 17, 2025

Uh oh!

zane-neo commented Sep 30, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

dbwiddis Sep 26, 2025 •

edited

Loading