Add RealmConfig

[XN137]

getting a config value currently requires quite a ceremony:

ctx.getPolarisCallContext()
   .getConfigurationStore()
   .getConfiguration(ctx.getRealmContext(), "ALLOW_WILDCARD_LOCATION", false))

since a PolarisConfigurationStore cant be used without a RealmContext it makes sense to add a dedicated interface. this allows removal of verbose code and also moves towards injecting that interface via CDI at a request/realm scope in the future.

[snazy]

Overall this is a very nice improvement!

I'd just propose to not unnecessarily repeat "Polaris" in type names.

[eric-maynard]

If we're making this change for ergonomic reasons, I don't think the new invocation is really ideal: x.getRealmConfig().getConfig(y). Is the return value of getRealmConfig the "config" or the return value of getConfig(y)? They both seem to claim they're fetching a config.

[eric-maynard]

Is it really a Realm-specific configuration? This looks more like a configuration store, like PolarisConfigurationStore.

[XN137]

depends on one's definition i guess, currently the interface allows retrieval of configuration values. in my book that makes it a "configuration".

do you want me to add "store" to the javadoc or to also rename the class? if the latter, any concrete name suggestions?

[eric-maynard]

I think calling it a RealmConfigurationStore would be a better name, but that also only makes me think more that this needs to be reconciled with PolarisConfigurationStore

[dimas-b]

I would not mind the proposed renaming, but current name LGTM too.

[eric-maynard]

If a new PolarisConfigurationStore implementation could achieve the same thing, we should try that approach first

[XN137]

the idea of having the interface is to make the life of the callers easier as they most often (/always?) operate in the context of a single realm / CallContext.

whether configuration values are stored by a single store for all/multiple realms or not is an implementation detail that callers should not care about.
so afaict PolarisConfigurationStore can keep its current implementation unless in the future we notice any advantages of "splitting it up".

[eric-maynard]

whether configuration values are stored by a single store for all/multiple realms or not is an implementation detail that callers should not care about.

Isn't that a reason that you'd want to just inject a PolarisConfigurationStore rather than handling something called a RealmConfig?

[dimas-b]

PolarisConfigurationStore is essentially multi-realm (note its explicit realm method parameters). This class acts as a request-scoped facade to automatically contextualize request-specific config lookup with realm IDs and pass those calls to PolarisConfigurationStore.

[eric-maynard]

I'm all for refactoring PolarisConfigurationStore to have both realm-agnostic and realm-gnostic (?) methods. But I do not think a new layer like this over the config store is necessary.

[dimas-b]

The new layer helps to properly delineate application-scoped data (PolarisConfigurationStore and/or it's config source) from request-scoped data (RealmConfig). Effectively, request-scoped code does not need to deal with explicit realm ID parameters, which, IMHO, is beneficial to code clarity and maintainability.

[eric-maynard]

Not having to deal with realms is definitely a great improvement, agreed. But my concern remains that this new class becomes the de facto “Polaris Configuration Store” and this redundancy does not improve clarity. Furthermore, the proposed structure could be interpreted as implying that these configs are set per-realm (while PolarisConfigurationStore confits are not) when in fact this is not the case.

[XN137]

Not having to deal with realms is definitely a great improvement, agreed.

if you agree with that, don't you also agree that having "realm-scoped wrappers" around "multi-realm implementation details" simplifies code that operates strictly within a single realm?

But my concern remains that this new class becomes the de facto “Polaris Configuration Store” and this redundancy does not improve clarity.

it is just an interface for accessing realm-scoped configuration values. RealmConfigImpl is forwarding requests to the actual PolarisConfigurationStore under the hood.
it does not replace or overtake the "storage aspect" of configuration values and is thus not redundant.
would it help address your concern if we rename RealmConfigImpl to StoreForwardingRealmConfig ?
or to RealmScopedConfigurationStore (even though it doesnt store anything by itself) ?

Furthermore, the proposed structure could be interpreted as implying that these configs are set per-realm (while PolarisConfigurationStore confits are not) when in fact this is not the case.

all public methods of PolarisConfigurationStore take a @Nonnull RealmContext parameter as there can be realm-specific overrides which take precedence.
so if we not only look at the name of the class but at its public api as a whole, it does indeed only allow retrieving configuration values per realm currently, as you can not retrieve any configuration value without providing a RealmContext.

if you have concrete ideas how class-naming could improve the "clarity" and "interpretation" of the current suggestion please let me know.

[eric-maynard]

Looks like @snazy merged this, but in response to the above:

don't you also agree that having "realm-scoped wrappers" around "multi-realm implementation details" simplifies code that operates strictly within a single realm?

No, not really -- all callers will now just use RealmConfig, so PolarisConfigurationStore becomes unnecessary. Yes, it is used to implement RealmConfig, but we need to consider which is a better interface to prospective callers who want to load a config. Is it RealmConfig and its methods or PolarisConfigurationStore? Probably the latter, which could have been extended into some RealmConfigurationStore which includes methods to load configs without you having to pass in a realm.

as there can be realm-specific overrides which take precedence.

Is there a case today where we pass in a realm other than the one which would get injected? That is the only situation where I see someone wanting to directly interface with PolarisConfigurationStore now.

[dimas-b]

I agree that PolarisConfigurationStore becomes unnecessary. However, from my POV the main advantage of this PR is detaching request-scoped config activities from explicit realm IDs, so I'd say using RealmConfig is preferable.

[XN137]

I don't think the new invocation is really ideal: x.getRealmConfig().getConfig(y). Is the return value of getRealmConfig the "config" or the return value of getConfig(y)? They both seem to claim they're fetching a config.

PolarisConfigurationStore.getConfiguration is also returning configuration values and not a configuration and certainly not a PolarisConfiguration (which is a parameter not a return value). 😅

but you are right, i also wasnt very convinced of the getConfig name, would you prefer getConfigValue or getValue or maybe just get? would any of those address your concern?

[dimas-b]

Thanks, @XN137 ! The PR LGTM overall. I think it's a nice improvement in request/application-scoped delineation in the Polaris codebase.

[dimas-b]

@eric-maynard : are you ok with merging this PR?

[XN137]

rebased after 19f44d8 due to conflicts:

runtime/service/src/test/java/org/apache/polaris/service/quarkus/catalog/IcebergCatalogTest.java
runtime/service/src/test/java/org/apache/polaris/service/quarkus/catalog/IcebergCatalogViewTest.java
runtime/service/src/test/java/org/apache/polaris/service/quarkus/catalog/PolarisGenericTableCatalogTest.java
runtime/service/src/test/java/org/apache/polaris/service/quarkus/catalog/PolicyCatalogTest.java

[XN137]

rebased due to conflicts with 15f23ca

service/common/src/main/java/org/apache/polaris/service/catalog/io/FileIOUtil.java

[XN137]

rebased (and squashed the rename) due to conflicts after 6ddd148