Add integration tests with Keycloak #2343

adutra · 2025-08-13T15:25:28Z

No description provided.

adutra · 2025-08-13T15:27:00Z

runtime/test-common/build.gradle.kts

+  }
+  // Keycloak Admin Client brings RESTEasy Classic, which conflicts with Quarkus RESTEasy Reactive;
+  // it must not be present during Quarkus augmentation otherwise Quarkus tests won't start.
+  compileOnly(libs.keycloak.admin.client)


I spent 5 hours today trying to find a better solution for this, but in the end, having both RESTEasy Classic and Reactive on the classpath gets blocked here:

https://github.com/quarkusio/quarkus/blob/072896e711640752ffc3176677591c6d263360e8/extensions/resteasy-reactive/rest-common/deployment/src/main/java/io/quarkus/resteasy/reactive/common/deployment/ResteasyReactiveCommonProcessor.java#L109-L110

Maybe a better solution would be to isolate the Keycloak container in a separate module, and shade RESTEasy Classic.

Having a server runner that launches pre-built jars would help here, I believe... There was a discussion or PR about that somewhere, right?

I recall that RESTEasy classic vs reactive issue from Nessie, where it's resolved this way. But the effect's the same: exclude RESTEasy Classic.

I suggest to leave it this way for now and I'll investigate if shading is an option.

+1, shading feels too much.

shading is too much for this case :) I'd go with a custom runner... not ok "as is" for now

dimas-b

Nice improvement 👍 ... just one minor comment :)

dimas-b · 2025-08-13T23:40:58Z

runtime/service/src/intTest/java/org/apache/polaris/service/it/RestCatalogKeycloakFileIT.java

+    keycloak.createRole(principalRole);
+    keycloak.createUser(
+        principalName,
+        // Use the same password as the client secret


Why? As for me, it detracts from the test's validity because now it is not obvious whether authentication happens locally (client secret) or via Keycloak 🤔

I'm not sure I get your point. This password is never used outside of this class (see method below).

Anyways, I replaced this with a fixed password for all users. Let me know if that's better.

I guessed the password was not used outside, but that's also a reason for not using the same password, right? ;) Thx for changing!

My point was that if the password is the same (with a specific comment) it creates an expectation that it might be reused somewhere... which is what I wanted to avoid 😅

adutra · 2025-08-14T10:14:21Z

...me/test-common/src/main/java/org/apache/polaris/test/commons/keycloak/KeycloakContainer.java

+    tokenEndpoint = issuerUrl.resolve("protocol/openid-connect/token");
+    createRole("ALL");
+    createUser("root");
+    assignRoleToUser("ALL", "root");


FYI I initially forgot to assign PRINCIPAL_ROLE:ALL to root and yet, the tests passed: this is because ActiveRolesProvider automatically infers "all granted roles" when no role is found in the token. This is another reason why I am advocating for merging ActiveRolesProvider into Authenticator.

snazy

LGTM

Letting @dimas-b take another look though.

Restore `PolarisAccessManager` (with default implementation of `IcebergTokenAccessManager`) which was added as an extension point for running Polaris tests in downstream build environments under apache#789, but was mistakenly removed in apache#2343

Restore `PolarisAccessManager` (with default implementation of `IcebergTokenAccessManager`) which was added as an extension point for running Polaris tests in downstream build environments under #789, but was mistakenly removed in #2343

* Fix Keycloak getting-started example (apache#2349) The `polaris-setup` container was erroneously including a non-existent scope when fetching a token from Keycloak. * fix(deps): update dependency com.nimbusds:nimbus-jose-jwt to v10.4.2 (apache#2350) * Use PolarisTaskConstants (apache#2346) * Add a regression test for Catalog Federation (apache#2286) * Add a regression test for Catalog Federation * Install jq dependency * Fix token issues * Update regtests/README.md Co-authored-by: Eric Maynard <[email protected]> * Update README.md --------- Co-authored-by: Eric Maynard <[email protected]> * Modularize federation (Option 2) (apache#2332) * Modularize federation (Option 2) * Move polaris-extensions-federation-hadoop dependency * Change identifier to lowerCase * Change identifiers to constants * Replace CallContext with RealmConfig in enforceFeatureEnabledOrThrow (apache#2348) * Replace CallContext with RealmConfig in CatalogEntity (apache#2336) * chore(deps): update registry.access.redhat.com/ubi9/openjdk-21-runtime docker tag to v1.23-6 (apache#2353) * fix(deps): update dependency com.gradleup.shadow:shadow-gradle-plugin to v9.0.2 (apache#2358) * chore(deps): update postgres docker tag to v17.6 (apache#2354) * Add integration tests with Keycloak (apache#2343) * Fix REST responses for failed Admin operations (apache#2291) * Fix REST responses for failed Admin operations the `boolean` return values of many methods in `PolarisAdminService` were often simply not getting used at all, thus the REST api returned success in those cases even though the `PrivilegeResult` was marked as failed. due to this fix a silently failing test now needs to be adjusted. we return the `PrivilegeResult` instead of a `boolean` to give the client at least some indication of what has gone wrong on the server side. note that some of the other operations were throwing Expcetions already, which are already reported back correctly to the client. * review: use http 400 BAD_REQUEST * Make PolarisAuthorizer RequestScoped (apache#2340) all methods in `PolarisAuthorizer` currently have a `CallContext` parameter. in its only implementation only `CallContext.getRealmConfig` is getting used. so since `PolarisAuthorizer` cant be used outside a request, we can simply make it request-scoped and inject the request-scoped `RealmConfig` directly. * fix(deps): update mockito monorepo to v5.19.0 (apache#2360) * Fix soft-merge conflict on `main` (apache#2364) * feat(docs): Add Getting Stated guide for MinIO (apache#2227) * feat(docs): Add Getting Stated guide for MinIO A simple page of step-by-step instructions for setting up a local environment with Polaris, MinIO and Spark. Closes apache#1530 * IntelliJ: fix project icon in IJ project list (apache#2366) ... copy source has changed * Use asMap property helpers (apache#2347) seems like these helpers existed for a long time but were just not getting used consistently * SigV4 Auth Support for Catalog Federation - Part 2: Connection Config Persistence (apache#2190) * Add SigV4 related DPOs * Rename UserSecretReference to SecretReference and fix some small issues * fix(deps): update dependency software.amazon.awssdk:bom to v2.32.24 (apache#2371) * Rat-check: exclude venv, cleanup excludes, include .svg (apache#2363) * `.svg` files are XML files and can contain a license header * Re-grouped the exclusion rat patterns * Added exclude for `.venv` * Added exclude for `.ruff_cache` * NoSQL: Async-impls: add some safeguards + javadoc spelling * NoSQL: spelling * NoSQL: dependency updates * Last merged commit: 5a7686b --------- Co-authored-by: Alexandre Dutra <[email protected]> Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Christopher Lambert <[email protected]> Co-authored-by: Pooja Nilangekar <[email protected]> Co-authored-by: Eric Maynard <[email protected]> Co-authored-by: Dmitri Bourlatchkov <[email protected]> Co-authored-by: Rulin Xing <[email protected]>

* feat: enforce LIST_PAGINATION_ENABLED (apache#2401) * feat: enforce LIST_PAGINATION_ENABLED The enforcement of the LIST_PAGINATION_ENABLED flag was missed in apache#1938. This change make the flag effective as discussed in apache#2296. Note: this causes a change in the default Polaris behaviour (no pagination by default) with respect to the previous state of `main`. However, there is no behaviour change with respect to 1.0.0 or 1.0.1 as previous releases did not have apache#1938. * Add PolarisMetaStoreManager.loadEntities (apache#2290) * Add PolarisMetaStoreManager.loadEntities currently `PolarisMetaStoreManager.listEntities` only exposes a limited subset of the underlying `BasePersistence.listEntities` functionality. most of the callers have to post-process the `EntityNameLookupRecord` of `ListEntitiesResult` and call `PolarisMetaStoreManager.loadEntity` on the individual items sequentually to transform and filter them. this is bad for the following reasons: - suboptimal performance as we run N+1 queries to basically load every entity twice from the persistence backend - suffering from race-conditions when entities get dropped between the `listEntities` and `loadEntity` call - a lot of repeated code in all the callers (of which only some are dealing with the race-condition by filtering out null values) as a solution we add `PolarisMetaStoreManager.loadEntities` that takes advantage of the already existing `BasePersistence` methods. we rename one of the `listEntities` methods to `loadEntities` for consistency. since many callers dont need paging and want the result as a list, we add `PolarisMetaStoreManager.loadEntitiesAll` as a convenient wrapper. we also remove the `PolarisEntity.nameAndId` method as callers who only need name and id should not be loading the full entity to begin with. note we rework `testCatalogNotReturnedWhenDeletedAfterListBeforeGet` from `ManagementServiceTest` because the simulated race-condition scenario can no longer happen. * review: remove filter + transformer params * review: add TODO in listPolicies * review: improve javadoc * chore: fix Page javadoc (apache#2412) Correct method args (follow-up to apache#2401) * Add 1.0.1-incubating release blog post (apache#2403) * This change fixes: (apache#2395) * use [email protected] for announcement * publish Docker images only when the vote passed (we are not suppose to publish any public artifacts before the vote is completed) * update the vote email accordingly * remove blog post link in the release announcement email to let this to the discretion of the release manager * chore(deps): update actions/setup-java action to v5 (apache#2414) * chore(test): Restore PolarisAccessManager (apache#2413) Restore `PolarisAccessManager` (with default implementation of `IcebergTokenAccessManager`) which was added as an extension point for running Polaris tests in downstream build environments under apache#789, but was mistakenly removed in apache#2343 * Update changelog prior to 1.1.0 release (apache#2406) * fix(deps): update quarkus platform and group to v3.25.4 (apache#2279) * NoSQL: merge related changes * Last merged commit 82cd416 --------- Co-authored-by: Dmitri Bourlatchkov <[email protected]> Co-authored-by: Christopher Lambert <[email protected]> Co-authored-by: JB Onofré <[email protected]> Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Pierre Laporte <[email protected]>

github-project-automation bot added this to Basic Kanban Board Aug 13, 2025

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Aug 13, 2025

adutra commented Aug 13, 2025

View reviewed changes

adutra force-pushed the keycloak-it branch 3 times, most recently from 22eba9a to 64249ed Compare August 13, 2025 19:50

Add integration tests with Keycloak

b1c7adf

adutra force-pushed the keycloak-it branch from 64249ed to b1c7adf Compare August 13, 2025 20:53

dimas-b reviewed Aug 13, 2025

View reviewed changes

adutra added 2 commits August 14, 2025 12:04

review + nits

845537b

nit

22241be

adutra commented Aug 14, 2025

View reviewed changes

snazy approved these changes Aug 14, 2025

View reviewed changes

github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Aug 14, 2025

dimas-b approved these changes Aug 14, 2025

View reviewed changes

adutra merged commit c4fc848 into apache:main Aug 15, 2025
12 checks passed

github-project-automation bot moved this from Ready to merge to Done in Basic Kanban Board Aug 15, 2025

dimas-b mentioned this pull request Aug 20, 2025

chore(test): Restore PolarisAccessManager #2413

Merged

Add integration tests with Keycloak #2343

Add integration tests with Keycloak #2343

Uh oh!

Conversation

adutra commented Aug 13, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Aug 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

snazy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dimas-b Aug 14, 2025 •

edited

Loading