[Improvement] Lack of documentation on non-default backend

[MonkeyCanCode]

By default, Polaris will be using in-memory backend as defined in https://github.com/polaris-catalog/polaris/blob/main/polaris-server.yml#L90. However, there is no documentation on what are the allowed values as well as how to use eclipselink (https://github.com/polaris-catalog/polaris/tree/main/extension/persistence/eclipselink) to switch to non-default backend.

[guitcastro]

Is there a reason why eclipselink is not a dependency of polaris-service? It seems that eclipselink lists polaris-service as a dependency, but it should be the other way around. I downloaded the project, removed eclipselink from the dependencies, and it worked.

When I tried to change the metastore to eclipselink, I encountered the following issue:

2024-07-31 15:12:59 io.dropwizard.configuration.ConfigurationParsingException: polaris-server.yml has an error:
2024-07-31 15:12:59   * Failed to parse configuration at: metaStoreManager; Could not resolve type id 'eclipse-link' as a subtype of `io.polaris.core.persistence.MetaStoreManagerFactory`: known type ids = [in-memory] (for POJO property 'metaStoreManager')
2024-07-31 15:12:59  at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain: io.polaris.service.config.PolarisApplicationConfig["metaStoreManager"])

[MonkeyCanCode]

Is there a reason why eclipselink is not a dependency of polaris-service? It seems that eclipselink lists polaris-service as a dependency, but it should be the other way around. I downloaded the project, removed eclipselink from the dependencies, and it worked.

When I tried to change the metastore to eclipselink, I encountered the following issue:

2024-07-31 15:12:59 io.dropwizard.configuration.ConfigurationParsingException: polaris-server.yml has an error:
2024-07-31 15:12:59   * Failed to parse configuration at: metaStoreManager; Could not resolve type id 'eclipse-link' as a subtype of `io.polaris.core.persistence.MetaStoreManagerFactory`: known type ids = [in-memory] (for POJO property 'metaStoreManager')
2024-07-31 15:12:59  at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain: io.polaris.service.config.PolarisApplicationConfig["metaStoreManager"])

So eclipse-link is not getting loaded into the final JAR (thus not enabled) as of now. A modification of gradle.build is needed to enable it (include it in the final JAR).

Sample PR created for using H2 as backend metastore via eclipselink: #47

[cgpoh]

I tried to use H2 as backend metastore via eclipselink and face this error when I send an auth token request:

Realm is not bootstrapped, please run server in bootstrap mode.

How to run the server in bootstrap mode?

[MonkeyCanCode]

I tried to use H2 as backend metastore via eclipselink and face this error when I send an auth token request:

Realm is not bootstrapped, please run server in bootstrap mode.

How to run the server in bootstrap mode?

@cgpoh , you can do this:

java -jar polaris-service/build/libs/polaris-service-1.0.0-all.jar bootstrap polaris-server.yml

This is documented in the open PR: #43

[cgpoh]

@MonkeyCanCode , thanks! After running the bootstrap command and running ./gradlew runApp, I can't find client id and secret in stdout on initial startup anymore. Without the credentials, how to send OAuth2 requests?

[MonkeyCanCode]

@MonkeyCanCode , thanks! After running the bootstrap command and running ./gradlew runApp, I can't find client id and secret in stdout on initial startup anymore. Without the credentials, how to send OAuth2 requests?

So with the eclipse link extension, the credentials will be stored in the backend metastore after bootstrap (in this case, it will be in h2 files on your local file system...the location will be /tmp/eclipse-link/). You will need h2 client to read the content. It is also possible to replace h2 with other backends (e.g. Postgres), but modifications of Gradle file (used by eclipse link) will be needed as well as the connection info within persistent.xml under resources directory under polaris-service.

[cgpoh]

@MonkeyCanCode , thanks! I'm trying out Postgres, will try to figure out where the credentials are stored for Postgres.

[MonkeyCanCode]

@MonkeyCanCode , thanks! I'm trying out Postgres, will try to figure out where the credentials are stored for Postgres.

For postgres, it will be under xxxxx_credential table.

[cgpoh]

@MonkeyCanCode thanks! Just found out that the secrets are in principal_secrets table. Everything is working properly now.

[MonkeyCanCode]

Closed this issue as #47 is merged.