-
Notifications
You must be signed in to change notification settings - Fork 332
Introduce behavior-change configs as an alternative to feature configs #1124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
eric-maynard
merged 17 commits into
apache:main
from
eric-maynard:behavior-change-configs
Mar 12, 2025
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
081fbe1
builds
eric-maynard fe48ac3
builds, tests fail
eric-maynard 1d51695
autolint
eric-maynard d1fe311
some fixes
eric-maynard 1107448
autolint
eric-maynard f21a621
style
eric-maynard b9f2771
working with duplicated code
eric-maynard 4e63335
autolint
eric-maynard 315a638
working with duplicated code
eric-maynard ab9c902
revert
eric-maynard 41676cd
autolint
eric-maynard f496922
autolint
eric-maynard d46b335
add basic test
eric-maynard ec5894c
autolint
eric-maynard 7dfbd30
better comment
eric-maynard e588123
autolint
eric-maynard 47764d5
resolve conflicts
eric-maynard File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
46 changes: 46 additions & 0 deletions
46
polaris-core/src/main/java/org/apache/polaris/core/config/BehaviorChangeConfiguration.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one | ||
| * or more contributor license agreements. See the NOTICE file | ||
| * distributed with this work for additional information | ||
| * regarding copyright ownership. The ASF licenses this file | ||
| * to you under the Apache License, Version 2.0 (the | ||
| * "License"); you may not use this file except in compliance | ||
| * with the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
| package org.apache.polaris.core.config; | ||
|
|
||
| import java.util.Optional; | ||
|
|
||
| /** | ||
| * Internal configuration flags for non-feature behavior changes in Polaris. These flags control | ||
| * subtle behavior adjustments and bug fixes, not user-facing catalog settings. They are intended | ||
| * for internal use only, are inherently unstable, and may be removed at any time. When introducing | ||
| * a new flag, consider the trade-off between maintenance burden and the risk of an unguarded | ||
| * behavior change. Flags here are generally short-lived and should either be removed or promoted to | ||
| * stable feature flags before the next release. | ||
| * | ||
| * @param <T> The type of the configuration | ||
| */ | ||
| public class BehaviorChangeConfiguration<T> extends PolarisConfiguration<T> { | ||
|
|
||
| protected BehaviorChangeConfiguration( | ||
| String key, String description, T defaultValue, Optional<String> catalogConfig) { | ||
| super(key, description, defaultValue, catalogConfig); | ||
| } | ||
|
|
||
| public static final BehaviorChangeConfiguration<Boolean> VALIDATE_VIEW_LOCATION_OVERLAP = | ||
| PolarisConfiguration.<Boolean>builder() | ||
| .key("STORAGE_CREDENTIAL_CACHE_DURATION_SECONDS") | ||
| .description("If true, validate that view locations don't overlap when views are created") | ||
| .defaultValue(true) | ||
| .buildBehaviorChangeConfiguration(); | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really need different types to support this? I think probably adding a field to
PolarisConfigurationwould be sufficientThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Strictly speaking we don't need a new type, but I chose to add one because I think it's valuable to separate the actual config instances from one another. Besides just having them clearly separated & organized, it means the caller is forced to recognize which type they are relying on (e.g.
BehaviorChangeConfiguration.UNSTABLE_FLAGvsFeatureConfiguration.ACTUAL_FEATURE). It also allows us to override e.g.catalogConfigwhich shouldn't be valid for these flags.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That seems reasonable, but given that we use a builder, can't we just enforce that at construction? I do like having the different config types housed in different namespaces (as in your examples
BehaviorChangeConfiguration.UNSTABLE_FLAGvsFeatureConfiguration.ACTUAL_FEATURE)...I guess I would just like to ensure that the call sites that are checking for config values don't need to know what type the configuration is at runtime. I can just imagine cases where someone accidentally references one type and the config field is another or someone tries to write some generic code that only works for one config type but not the other...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I think the opposite. We should want the caller to have to acknowledge when they write logic that depends on an unstable flag.
Can you say more about this? Are you talking about a situation where the user misconfigures the application.properties?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking of cases where a person writes generic code like (as a very simplistic example)
You can imagine code like this being used to provide bean implementations, to gate certain API implementations, to guard new features, ... if the generic code becomes type-specific, or if it has to downcast to pass the configuration value on to a
Consumer<BehaviorChangeConfiguration>, we could end up having to write that code twice - or possibly have to change everything to accept<? extends PolarisConfiguration>.As I mentioned in the previous comment, I do like having
BehaviorChangeConfigurationas a namespace, so that the feature gates and the risky fix changes are separated. But I think the code that works withPolarisConfigurationtypes should be agnostic to the existence of those subtypes.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking at that example, I think it functions the way we should want it to.
I created a test with a consumer like this:
Where it's used like:
As expected, that last line won't compile because the method expects a PolarisConfiguration<Boolean>. If featureConfig is changed to a FeatureConfiguration, everything works as expected. I'll commit the test in that state now, but let me know how we can adjust it to better suit the use case you had in mind.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That doesn't really address the case I was talking about, but it's fine. As there aren't any concrete issues right now, I'm happy to move forward. The minute I see someone write
<? extends PolarisConfiguration>, though, I'm out ;)I'll defer to @adutra's review above.