Add `pathStyleAccess` to AwsStorageConfigInfo #2012

dimas-b · 2025-07-08T23:46:48Z

This change allows configuring the "path-style" access mode in S3 clients (both in Polaris Servers and Iceberg REST Catalog API clients).

This change is applicable both to AWS storage and to non-AWS S3-compatible storage (#1530).

The Management API change is backward-compatible with clients lenient to unknown JSON properties.

Includes a fix to AWS config object round-trip via generated API classes.

Dev ML discussion: https://lists.apache.org/thread/oxy2yd4mbq06zr4z57lfptgy95c3lyhb

polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsStorageConfigurationInfo.java

eric-maynard · 2025-07-09T03:03:10Z

polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsStorageConfigurationInfo.java

We could use the default here, or just leave it null to leverage the default used in orElse. Better not to proliferate the constants around like this.

changed to use PATH_STYLE_ACCESS_DEFAULT

Now that we have a default at the API level, is this actually nullable?

I had to remove the API default.

spec/polaris-management-service.yml

polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsStorageConfigurationInfo.java

singhpk234

LGTM !

dimas-b · 2025-07-09T14:44:58Z

I had to remove one commit (discussed in the API yaml comment thread). PTAL.

eric-maynard

Let's clarify if this is a breaking change (does it need to target 2.0? Go into API version 2?) and plan accordingly before merging.

snazy

LGTM

dimas-b · 2025-07-14T17:10:12Z

@eric-maynard : I believe I addressed your backward-compatibility concern in the latest push. Could you review again?

This change allows configuring the "path-style" access mode in S3 clients (both in Polaris Servers and Iceberg REST Catalog API clients). This change is applicable both to AWS storage and to non-AWS S3-compatible storage (apache#1530). The Management API change is backward-compatible.

eric-maynard · 2025-07-16T03:59:12Z

spec/polaris-management-service.yml

+              type: boolean
+              description: >- 
+                Whether S3 requests to files in this catalog should use 'path-style addressing for buckets'.
+                Default: false.


nit: With the default in the spec, I don't think we need the default in the doc. Other similar variables don't have it.

removed 👍

Amends apache#1913 and apache#2012

…ess (#2127) This change adds support for endpoint, sts-endpoint, path-style-access to the Polaris Python client. Amends #1913 and #2012

* Add `pathStyleAccess` to AwsStorageConfigInfo This change allows configuring the "path-style" access mode in S3 clients (both in Polaris Servers and Iceberg REST Catalog API clients). This change is applicable both to AWS storage and to non-AWS S3-compatible storage (apache#1530). (cherry picked from commit 987c554)

dimas-b requested review from eric-maynard and snazy July 8, 2025 23:46

github-project-automation bot added this to Basic Kanban Board Jul 8, 2025

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Jul 8, 2025

eric-maynard reviewed Jul 9, 2025

View reviewed changes

polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsStorageConfigurationInfo.java Outdated Show resolved Hide resolved

eric-maynard reviewed Jul 9, 2025

View reviewed changes

spec/polaris-management-service.yml Outdated Show resolved Hide resolved

snazy previously approved these changes Jul 9, 2025

View reviewed changes

polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsStorageConfigurationInfo.java Outdated Show resolved Hide resolved

github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Jul 9, 2025

dimas-b dismissed snazy’s stale review via b889c1e July 9, 2025 14:28

singhpk234 reviewed Jul 9, 2025

View reviewed changes

polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsStorageConfigurationInfo.java Outdated Show resolved Hide resolved

singhpk234 previously approved these changes Jul 9, 2025

View reviewed changes

dimas-b dismissed singhpk234’s stale review via 780f36b July 9, 2025 14:42

dimas-b force-pushed the s3-path-style branch from b889c1e to 780f36b Compare July 9, 2025 14:42

eric-maynard requested changes Jul 10, 2025

View reviewed changes

github-project-automation bot moved this from Ready to merge to PRs In Progress in Basic Kanban Board Jul 10, 2025

snazy previously approved these changes Jul 10, 2025

View reviewed changes

dimas-b dismissed snazy’s stale review via 090b9b6 July 10, 2025 22:50

dimas-b force-pushed the s3-path-style branch from 780f36b to 090b9b6 Compare July 10, 2025 22:50

dimas-b requested review from eric-maynard, singhpk234 and snazy July 10, 2025 22:51

dimas-b force-pushed the s3-path-style branch from 090b9b6 to 09f5d77 Compare July 10, 2025 23:10

dimas-b force-pushed the s3-path-style branch 2 times, most recently from 6157bc0 to 85bbabe Compare July 15, 2025 16:27

dimas-b added 3 commits July 15, 2025 19:03

review: rename to getPathStyleAccess

2db692c

review: PATH_STYLE_ACCESS_DEFAULT, ternary

f21df95

dimas-b added 2 commits July 15, 2025 19:03

fix config round trip, add tests

0f14287

review: add Open API default for pathStyleAccess

47fc30e

dimas-b force-pushed the s3-path-style branch from 85bbabe to 47fc30e Compare July 15, 2025 23:03

eric-maynard previously approved these changes Jul 16, 2025

View reviewed changes

github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Jul 16, 2025

eric-maynard reviewed Jul 16, 2025

View reviewed changes

review: remove default from doc comment for pathStyleAccess

21fe61c

dimas-b dismissed eric-maynard’s stale review via 21fe61c July 16, 2025 14:01

snazy approved these changes Jul 16, 2025

View reviewed changes

dimas-b merged commit 987c554 into apache:main Jul 16, 2025
12 checks passed

dimas-b deleted the s3-path-style branch July 16, 2025 15:46

github-project-automation bot moved this from Ready to merge to Done in Basic Kanban Board Jul 16, 2025

snazy added a commit to snazy/polaris that referenced this pull request Jul 17, 2025

Python client: add support for endpoint, sts-endpoint, path-style-access

449ae9a

Amends apache#1913 and apache#2012

snazy mentioned this pull request Jul 17, 2025

Python client: add support for endpoint, sts-endpoint, path-style-access #2127

Merged

snazy added a commit that referenced this pull request Jul 21, 2025

Python client: add support for endpoint, sts-endpoint, path-style-acc…

3316f4e

…ess (#2127) This change adds support for endpoint, sts-endpoint, path-style-access to the Polaris Python client. Amends #1913 and #2012

dimas-b mentioned this pull request Sep 25, 2025

Add KMS support for S3 #1424

Open

Add pathStyleAccess to AwsStorageConfigInfo #2012

Add pathStyleAccess to AwsStorageConfigInfo #2012

Uh oh!

Conversation

dimas-b commented Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

eric-maynard Jul 9, 2025

Choose a reason for hiding this comment

Uh oh!

dimas-b Jul 9, 2025

Choose a reason for hiding this comment

Uh oh!

eric-maynard Jul 9, 2025

Choose a reason for hiding this comment

Uh oh!

dimas-b Jul 9, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

singhpk234 left a comment

Choose a reason for hiding this comment

Uh oh!

dimas-b commented Jul 9, 2025

Uh oh!

eric-maynard left a comment

Choose a reason for hiding this comment

Uh oh!

snazy left a comment

Choose a reason for hiding this comment

Uh oh!

dimas-b commented Jul 14, 2025

Uh oh!

eric-maynard Jul 16, 2025

Choose a reason for hiding this comment

Uh oh!

dimas-b Jul 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Add `pathStyleAccess` to AwsStorageConfigInfo #2012

Add `pathStyleAccess` to AwsStorageConfigInfo #2012

dimas-b commented Jul 8, 2025 •

edited

Loading