Implement OpaPolarisAuthorizer #2680

sungwy · 2025-09-25T03:03:16Z

Resolves #138

RFC: https://docs.google.com/document/d/1HadMFygjbuZathZZPanO6cFVorx0Ju0FopkICxX1tCE/edit?tab=t.0
Mailing list discussion: https://lists.apache.org/thread/54qdbsxs3j7wwhv3tsccqj6qng5lqgmz

Some followup items highlighted as a part of this PR review:

introduce OpaHttpClientFactory that utilizes PoolingHttpClientConnectionManager to get opa-http-client to be more production ready
publish user-facing docs on OPA authorization
publish json schema for docs and opa server checks
introduce per-realm configuration support

adutra

Thanks @sungwy for this draft PR, I couldn't resist and took a look 😄

This is really interesting imho and a very nice addition to Polaris. We need to start thinking about ways to make Polaris RBAC fully pluggable.

.../auth/opa/impl/src/main/java/org/apache/polaris/extension/auth/opa/OpaPolarisAuthorizer.java

...sions/auth/opa/src/main/java/org/apache/polaris/extension/auth/opa/OpaPolarisAuthorizer.java

runtime/defaults/src/main/resources/application.properties

runtime/service/src/main/java/org/apache/polaris/service/config/AuthorizationConfiguration.java

runtime/service/src/main/java/org/apache/polaris/service/config/ServiceProducers.java

runtime/service/build.gradle.kts

dimas-b

Very interesting idea! Just some preliminary comments below :)

polaris-core/build.gradle.kts

runtime/defaults/src/main/resources/application.properties

flyrain · 2025-09-25T22:32:34Z

Thanks for working on it @sungwy ! Looking forward to the RFC/design doc!

dimas-b · 2025-10-23T15:37:25Z

.../impl/src/main/java/org/apache/polaris/extension/auth/opa/token/FileBearerTokenProvider.java

+    if (Strings.isNullOrEmpty(cachedToken)) {
+      throw new RuntimeException(
+          "Bearer token is unexpectedly empty. This should not happen after successful construction.");
+    }
+    return cachedToken;


technically this is still racy since cachedToken is volatile... it is not likely to materialize as a bug, but the code pattern looks a bit worrisome 😅 why not load the value into a local var, check and return from local?

e.g. return Preconditions.checkNotNull(cachedToken, "Bearer token is unexpectedly empty ...")

I guess this is outdated now with the new commits.

Make token-refresh asynchronous ...

snazy

One nit, everything else beside int-tests LGTM.

For integration-tests, I'm thinking of having non-Quarkus integration tests against an OPA testcontainer in the "main" OPA module.

I've prepared some stuff for tests in this commit (here as well):

Add a starter for an integration test in the "main" module
Updated the existing int tests

Idea is to have the majority of integration-tests in the "main" module and only a smoke-test in runtime-service.

I think we can then get rid of the opa-tests module and can move the "main" module one directory level up.

BTW: Please add an entry for the "main" module to :polaris-bom.

WDYT?

.../impl/src/main/java/org/apache/polaris/extension/auth/opa/token/FileBearerTokenProvider.java

snazy · 2025-10-24T07:53:23Z

.../impl/src/main/java/org/apache/polaris/extension/auth/opa/token/FileBearerTokenProvider.java

+    if (Strings.isNullOrEmpty(cachedToken)) {
+      throw new RuntimeException(
+          "Bearer token is unexpectedly empty. This should not happen after successful construction.");
+    }
+    return cachedToken;


I guess this is outdated now with the new commits.

reduce volatile reads from 2 -> 1 Co-authored-by: Robert Stupp <[email protected]>

sungwy · 2025-10-24T13:58:45Z

For integration-tests, I'm thinking of having non-Quarkus integration tests against an OPA testcontainer in the "main" OPA module.

I've prepared some stuff for tests in this commit (here as well):

Add a starter for an integration test in the "main" module

Updated the existing int tests

Idea is to have the majority of integration-tests in the "main" module and only a smoke-test in runtime-service.

I think we can then get rid of the opa-tests module and can move the "main" module one directory level up.

Thanks again @snazy :) Let me take a look at your draft PR and see if I can get rid of the test module and bring impl back up to main. I had some trouble achieving this because of the dependency on :polaris-runtime-service.

Just so I'm clear: is your suggestion to move the QuarkusIntegrationTest annotated tests back into polaris-runtime-service?

snazy · 2025-10-24T14:52:34Z

Just so I'm clear: is your suggestion to move the QuarkusIntegrationTest annotated tests back into polaris-runtime-service?

Yes and no ;)

Have non-Quarkus integration tests (like OpaAuthorizerIT) in the "main" OPA module and have a minimal set of OPA integration tests in polaris-runtime-service just to verify the integration with Quarkus. I suspect we don't need that many ITs in runtime-service, probably more or less just a "smoke test".
I'm not really familiar with what's worth being integration tested in the non-Quarkus ITs.
Test coverage for the OPA factory and the authorizer itself is pretty good.

dimas-b · 2025-10-24T17:14:53Z

.../impl/src/main/java/org/apache/polaris/extension/auth/opa/token/FileBearerTokenProvider.java

+  }
+
+  private void scheduleRefreshAttempt(Duration delay) {
+    this.refreshTask = asyncExec.schedule(this::refreshTokenAttempt, delay);


Nice reuse of existing functionality 🚀

dimas-b · 2025-10-24T17:24:40Z

.../impl/src/main/java/org/apache/polaris/extension/auth/opa/token/FileBearerTokenProvider.java

+    // In this case we wait for the configured amount of time
+    // (5 seconds in production, much lower in tests).
+    try {
+      return initialTokenFuture.get(initialTokenWaitMillis, TimeUnit.MILLISECONDS);


optional: I do not want to disturb the impl. at this stage as it looks pretty solid to me... but I suppose it could be simplified this way:

keep a volatile reference to a CompletionStage with token data

getToken always calls .get() with a timeout

the initial ref. to the CompletionStage is make in the constructor by scheduling a refresh task

when refresh completes it will update the CompletionStage and reschedule

the CompletionStage will be "complete" in all cases but the initial refresh, so .get() calls will not wait except possibly on the first call.

we have less fields and less ifs (hopefully) in this class (only refreshTask and the CompletionStage)

WDYT?

dimas-b · 2025-10-24T17:27:49Z

polaris-core/build.gradle.kts

  testFixturesApi(libs.jakarta.ws.rs.api)

  compileOnly(libs.jakarta.annotation.api)
+  compileOnly(libs.jakarta.enterprise.cdi.api)


IIRC, we had some prior discussions about avoiding CDI annotations in core as a convention...

If it's required only for @Identifier("internal") on DefaultPolarisAuthorizerFactory, I'd propose remove the annotation from the class and make a producer method for it in ServiceProducers.

Alternatively, we could remove DefaultPolarisAuthorizerFactory completely and update ServiceProducers to "manually" use PolarisAuthorizerImpl if Instance<PolarisAuthorizerFactory> is not resolvable in CDI.

WDYT?

Thanks @dimas-b these are great suggestions. Thank you for giving me the background context on the package discussions.

I've added a Producer method in ServiceProducers and removed the cdi dependency from polaris-core

sungwy · 2025-10-24T23:19:23Z

I suspect we don't need that many ITs in runtime-service, probably more or less just a "smoke test".
I'm not really familiar with what's worth being integration tested in the non-Quarkus ITs.

I actually found it very helpful to have the Quarkus integration tests because it helped verify the smallrye Configs. Maybe that's a matter of me being new to CDIs but I think it's still helpful to have that coverage.

@dimas-b - I would love to get your thoughts on this suggestion as well

…to opa-authorizer

sungwy · 2025-10-26T12:36:31Z

BTW: Please add an entry for the "main" module to :polaris-bom.

I've added the polaris-extensions-auth-opa module to :polaris-bom. I also noticed that the other extensions polaris-extensions-federation-hive and polaris-extensions-federation-hadoop are missing. Should these be added as well? (maybe in a separate PR)

flyrain · 2025-10-27T18:38:13Z

I've added the polaris-extensions-auth-opa module to :polaris-bom. I also noticed that the other extensions polaris-extensions-federation-hive and polaris-extensions-federation-hadoop are missing. Should these be added as well? (maybe in a separate PR)

Yes, we will need them in BOM as well. A separate PR sounds good to me.

dimas-b

Sorry about the delay. This PR LGTM in its current state, except the dep. on CDI annotations in polaris-core.

The current tests layout LGTM. So, if @snazy is ok with that, I think we're pretty close to merging.

Re: CDI in core, we can probably discuss that on the dev ML if you prefer to keep the annotation. The reason I flagged it is not my personal objection, but the fact that IIRC it was discussed before and the agreement in the community was to avoid CDI-specific annotations in core.

sungwy · 2025-10-28T18:35:46Z

@dimas-b - I've addressed the concern regarding the polaris-core dependency. Appreciate the review!

dimas-b

LGTM 👍 @sungwy : Many thanks for the contribution and for bearing with the long review process 😉

dimas-b · 2025-10-28T22:41:05Z

Let's give some more time to @flyrain, @adutra and @snazy for final thoughts / comments. I suppose we could also follow up with test adjustments relocations after merging this PR (it's been in review for a long time 😅 )

sungwy · 2025-10-28T23:57:06Z

Woohoo!

Thank you @dimas-b ! And @snazy @flyrain and @adutra for the reviews as well.

This was a great way for me to get up to speed with the project, so thanks for your interest and patience in reviewing this PR as well.

I'll be following this up with some more PRs (docs, schema, etc) and we'll start integrating this into our platform.

flyrain · 2025-10-30T18:03:55Z

runtime/service/build.gradle.kts


  implementation(libs.jakarta.servlet.api)

+  runtimeOnly(project(":polaris-async-vertx"))


Why do we need this dependency? Is this only for test?

I believe this actually is the CDI implementation of polaris-async-api AsyncExec that's optimized for Quarkus applications. Hence, we need it as a runtimeOnly dependency in polaris-runtime-service

I’m good with using the Vert.x async executor since we’re already on Quarkus. The polaris-async-vertx module currently lives under /persistence/nosql, but it’s really part of the async infrastructure rather than a NoSQL implementation detail. We could consider moving it out of that directory, though it doesn’t need to be done in this PR.

flyrain · 2025-10-31T05:40:49Z

Thanks a lot for working on it, @sungwy ! Thanks @snazy @dimas-b @jbonofre for the review.

sungwy added 9 commits September 20, 2025 17:53

OpaPolarisAuthorizer

be48131

add CDI AuthorizerProducer

317cdc4

inject polarisAuthorizer in ServiceProducers CDI

0739855

add integration test

7be0482

Merge branch 'main' into opa-authorizer

c18d4d2

license

c7701cb

add integration tests

ec3c142

Merge branch 'main' into opa-authorizer

eec60c2

minor fixes

ed6f265

github-project-automation bot added this to Basic Kanban Board Sep 25, 2025

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Sep 25, 2025

adutra reviewed Sep 25, 2025

View reviewed changes

dimas-b reviewed Sep 25, 2025

View reviewed changes

polaris-core/build.gradle.kts Outdated Show resolved Hide resolved

runtime/defaults/src/main/resources/application.properties Show resolved Hide resolved

sungwy added 2 commits September 26, 2025 18:01

adopt review feedback

5caf1f4

remove comment

3935c6a

HonahX mentioned this pull request Sep 29, 2025

Implement Finer Grained Operations and Privileges For Update Table #2697

Merged

sungwy added 7 commits September 30, 2025 02:34

support https and bearer token authz

5ad1030

file token provider and token refresh

0785bdb

Merge branch 'main' into opa-authorizer

4b950e3

fix

85baedc

refactoring

6421275

refactor tests, disable ssl verification in integration tests

36f687c

use http in integration tests

c1ae608

HonahX mentioned this pull request Oct 7, 2025

Add new PolaisAuthorizer.authorizeOrThrow for Fine Grained AuthorizableOperations #2767

Open

sungwy added 2 commits October 8, 2025 01:55

remove properties from initial implementation

6516726

remove unused ssl dependencies

edfe61a

sungwy marked this pull request as ready for review October 8, 2025 02:05

sungwy requested review from adutra and dimas-b October 8, 2025 02:14

dimas-b reviewed Oct 23, 2025

View reviewed changes

sungwy added 2 commits October 23, 2025 22:17

Merge pull request #1 from snazy/opa-authorizer-file-refresh

9f6eda6

Make token-refresh asynchronous ...

thanks snazy

615d731

snazy reviewed Oct 24, 2025

View reviewed changes

thanks snazy!

c2fe9fb

reduce volatile reads from 2 -> 1 Co-authored-by: Robert Stupp <[email protected]>

dimas-b reviewed Oct 24, 2025

View reviewed changes

sungwy added 2 commits October 26, 2025 12:33

add polaris-extensions-auth-opa to bom

306d180

Merge branch 'opa-authorizer' of https://github.com/sungwy/polaris in…

9ba71ec

…to opa-authorizer

dimas-b reviewed Oct 28, 2025

View reviewed changes

sungwy added 2 commits October 28, 2025 17:55

remove cdi dependency in polaris-core

2aeb57b

spotlessApply

32e9951

dimas-b approved these changes Oct 28, 2025

View reviewed changes

dimas-b mentioned this pull request Oct 30, 2025

Move nodeids to nosql package parent #2931

Merged

flyrain reviewed Oct 30, 2025

View reviewed changes

flyrain approved these changes Oct 31, 2025

View reviewed changes

flyrain merged commit 577b32a into apache:main Oct 31, 2025
15 checks passed

github-project-automation bot moved this from Ready to merge to Done in Basic Kanban Board Oct 31, 2025

vchag pushed a commit to vchag/polaris that referenced this pull request Nov 5, 2025

Implement OpaPolarisAuthorizer (apache#2680)

5a161fd

This was referenced Nov 12, 2025

DOCS: OPA as External Policy Decision Point #3030

Open

Use POJOs for OPA JSON schema construction and publish schema #3031

Open


		implementation(libs.jakarta.servlet.api)

		runtimeOnly(project(":polaris-async-vertx"))

Implement OpaPolarisAuthorizer #2680

Implement OpaPolarisAuthorizer #2680

Uh oh!

Conversation

sungwy commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adutra left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

flyrain commented Sep 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

snazy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sungwy commented Oct 24, 2025

Uh oh!

snazy commented Oct 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sungwy commented Oct 24, 2025

Uh oh!

sungwy commented Oct 26, 2025

Uh oh!

flyrain commented Oct 27, 2025

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

sungwy commented Oct 28, 2025

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

dimas-b commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sungwy commented Oct 28, 2025

Uh oh!

flyrain Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

flyrain commented Oct 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

sungwy commented Sep 25, 2025 •

edited

Loading

dimas-b commented Oct 28, 2025 •

edited

Loading

flyrain Oct 30, 2025 •

edited

Loading