Helm chart redesign for Quarkus-based runtimes

[adutra]

This is a major redesign of the Helm chart to make it compliant with Quarkus, but also improving a lot of aspects of the existing chart, such as the persistence secret management and the bootstrap jobs, to name a few.

Summary of changes

Persistence

Thanks to the improvement brought by #613, it's not necessary anymore to have an init container create a conf.jar. From now on, the user provides a secret with their persistence.xml, and that file is mounted on each Polaris pod directly.

Enhanced services

The services for Polaris now are separated in 3 categories:

1. Main service: the service that serves API requests; it is usually behind an Ingress or LoadBalancer
2. Management service: the service for metrics and health; this is usually ClusterIP and headless
3. Extra services: if any of the services needs to be exposed with different settings to different consumers, this section can be used.

Each service section has the same structure and allows to configure one or more ports.

Observability

New sections were added for logging, tracing, metrics, with also the ability to create a ServiceMonitor.

Bootstrap Job

The bootstrap section now configures the bootstrap job. Thanks to the Polaris Admin Tool introduced in #605, the jobs now use the tool to bootstrap realms.

I am not convinced personally that this bootstrap job has a huge value, compared to just running the admin tool directly. But I didn't want to remove it.

Advanced Config

A new advancedConfig section can be used to customize Polaris deployments in any possible way, even when the Helm chart does not expose the desired setting.

[adutra]

Note: this is automatically generated by helm-docs.

[MonkeyCanCode]

With https://github.com/apache/polaris/pull/912/files, we will need only the original command without filter out the warning messages due to symbolic link.

[adutra]

Yes! I'm just waiting for #912 to be merged.

[adutra]

#912 is merged! I removed the workaround.

[MonkeyCanCode]

Same as above if we should merge https://github.com/apache/polaris/pull/912/files first then remove the filter part.

[adutra]

Removed.

[MonkeyCanCode]

EclipseLink uses a default of 32 connections for connection pooling. For local testing, this is more than enough. If people want to use it for some load testing, this will suffer when there are many concurrent client connection. Do you think we may want to add an example or add the default value here? So for people who may not be familiar with EclipseLink, they can just change the max value etc.

[adutra]

Good point. I added the following:

          <property name="eclipselink.connection-pool.default.initial" value="1" />
          <property name="eclipselink.connection-pool.default.min" value="1" />
          <property name="eclipselink.connection-pool.default.max" value="1" />

[MonkeyCanCode]

nits: extra space between "list $k"

[MonkeyCanCode]

Should we add a similar comment here for the file listing issue (#907)

[adutra]

Good point, done

[flyrain]

cc @gh-yzou

[gh-yzou]

does that mean by default we will have a connection pool with size 1? maybe we should mention this in the read me, and along with some instruction about how to adjust the size

[adutra]

This is only for CI tests.

[gh-yzou]

new line

[gh-yzou]

where does the private key and public key coming from ? will we be able to generate it on fly?

[adutra]

This is just for tests. They were randomly generated and do not contain any sensitive information. I will add a comment.

[gh-yzou]

is that two ports one for icebergRestCatalog service, and another one for OAuth service?

[adutra]

This file is just for tests and contains "bogus" data that does not necessarily reflect how extraServices will be used in real life, because in these tests we cannot create load balancers, ingresses, etc.

The typical use case for extraServices is:

• User wants port 8181 to be exposed to internal clients (in the same namespace) using a ClusterIP service.
• User also wants to expose port 8181 using a LoadBalancer service, for consumption by external clients (potentially with TLS termination, etc.)

This can be achieved by simply leaving service and managementService untouched, and adding the following config:

extraServices:
  - nameSuffix: "ext"
    type: LoadBalancer
    ports:
    - name: polaris-http
      port: 8181

Assuming the release name is my-release, the above will generate the following services:

• my-release of type ClusterIP for port 8181;
• my-release-ext of type LoadBalancer for port 8181;
• my-release-mgmt service of type ClusterIP (headless) for port 8182.

[adutra]

@MonkeyCanCode @gh-yzou while answering a question from @gh-yzou I realized that I forgot to set ClusterIP: None (headless) by default for the management service. I just fixed it in the last commit. It's common practice to use headless services for healthchecks and metrics scraping. Let me know if you are OK.

[adutra]

@MonkeyCanCode @gh-yzou should we move forward with this? I would suggest to merge unless you see any blockers, then we can always keep improving the chart after the merge. It's not like we are going to do a release of the chart any time soon 😄

[gh-yzou]

@adutra i don't have other further questions, but i am not familiar with this part, so I will let @MonkeyCanCode and @collado-mike to do the final approval on this. You don't have to wait for me to approve this.

[MonkeyCanCode]

@adutra i don't have other further questions, but i am not familiar with this part, so I will let @MonkeyCanCode and @collado-mike to do the final approval on this. You don't have to wait for me to approve this.

Thought I clicked on approved earlier. Approved. Very well done.

[adutra]

Heads up: I am going to merge this now to unblock people looking for node port support, see #982.