-
Notifications
You must be signed in to change notification settings - Fork 332
Implement missing doc "Configuring Polaris for Production" #43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
collado-mike
merged 28 commits into
apache:main
from
eric-maynard:configuring-polaris-for-production
Aug 6, 2024
Merged
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
9a91673
wip
eric-maynard 7c01fec
checkin
eric-maynard c8be49f
check in
eric-maynard 02f320f
rebase
eric-maynard 0b4e492
bootstrapping notes
eric-maynard ba2466a
emphasis
eric-maynard 94b6dee
plug the quickstart in the readme, so there's a path to the other docs
eric-maynard 1b0c049
ensure new links are rendered in web
annafil 06699c3
Merge pull request #1 from polaris-catalog/configuring-polaris-for-pr…
eric-maynard dfaea1d
regenerate web docs
eric-maynard f567b2e
merge
eric-maynard 021db5c
regenerate docs
eric-maynard f195faf
Merge branch 'main' of github.202132.xyz-oss:eric-maynard/polaris into confi…
eric-maynard 160146c
typofix
eric-maynard d543339
regen docs
eric-maynard 1159ec6
merge with main
eric-maynard e9609e8
bootstrap fix
eric-maynard da5c987
regen docs
eric-maynard 9d3b75d
changes per review
eric-maynard cc1d1c7
regen docs
eric-maynard e5ca819
regen docs
eric-maynard 61979f0
refactor
eric-maynard 2a7919f
regen docs
eric-maynard 747020c
merge conflict
eric-maynard 287295c
changes per review
eric-maynard d423b26
regen docs
eric-maynard 4b1855e
style
eric-maynard 135b8d3
regen docs
eric-maynard File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| <!-- | ||
| Copyright (c) 2024 Snowflake Computing Inc. | ||
|
|
||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| --> | ||
|
|
||
|
|
||
| # Configuring Polaris for Production | ||
|
|
||
| The default `polaris-server.yml` configuration is intended for develoment and testing. When deploying Polaris in production, there are several best practices to keep in mind. | ||
|
|
||
| ## Security | ||
|
|
||
| ### Configurations | ||
|
|
||
| Notable configuration used to secure a Polaris deployment are outlined below. | ||
|
|
||
| #### oauth2 | ||
|
|
||
| > [!WARNING] | ||
| > Ensure that the `tokenBroker` setting reflects the token broker specified in `authenticator` below. | ||
|
|
||
| * Configure [OAuth](https://oauth.net/2/) with this setting. Remove the `TestInlineBearerTokenPolarisAuthenticator` option and uncomment the `DefaultPolarisAuthenticator` authenticator option beneath it. | ||
| * Then, configure the token broker. You can configure the token broker to use either [asymmetric](https://github.com/polaris-catalog/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTRSAKeyPair.java#L24) or [symmetric](https://github.com/polaris-catalog/polaris/blob/b482617bf8cc508b37dbedf3ebc81a9408160a5e/polaris-service/src/main/java/io/polaris/service/auth/JWTSymmetricKeyBroker.java#L23) keys. | ||
|
|
||
| #### authenticator.tokenBroker | ||
|
|
||
| > [!WARNING] | ||
| > Ensure that the `tokenBroker` setting reflects the token broker specified in `oauth2` above. | ||
|
|
||
| #### callContextResolver** & **realmContextResolver | ||
| * Use these configurations to specify a service that can resolve a realm from bearer tokens. | ||
| * The service(s) used here must implement the relevant interfaces (i.e. [CallContextResolver](https://github.com/polaris-catalog/polaris/blob/8290019c10290a600e40b35ddb1e2f54bf99e120/polaris-service/src/main/java/io/polaris/service/context/CallContextResolver.java#L27) and [RealmContextResolver](https://github.com/polaris-catalog/polaris/blob/7ce86f10a68a3b56aed766235c88d6027c0de038/polaris-service/src/main/java/io/polaris/service/context/RealmContextResolver.java)). | ||
|
|
||
| ## Metastore Management | ||
|
|
||
| > [!IMPORTANT] | ||
| > The default `in-memory` implementation for `metastoreManager` is meant for testing and not suitable for production usage. Instead, consider an implementation such as `eclipse-link` which allows you to store metadata in a remote database. | ||
|
|
||
| A Metastore Manger should be configured with an implementation that durably persists Polaris entities. Use the configuration `metaStoreManager` to configure a [MetastoreManager](https://github.com/polaris-catalog/polaris/blob/627dc602eb15a3258dcc32babf8def34cf6de0e9/polaris-core/src/main/java/io/polaris/core/persistence/PolarisMetaStoreManager.java#L47) implementation where Polaris entities will be persisted. | ||
|
|
||
| Be sure to secure your metastore backend since it will be storing credentials and catalog metadata. | ||
|
|
||
| ### Configuring EclipseLink | ||
|
|
||
| To use [EclipseLink](https://eclipse.dev/eclipselink/) for metastore management, specify the configuration `metaStoreManager.conf-file` to point to an [EclipseLink `persistence.xml` file](https://eclipse.dev/eclipselink/documentation/2.5/solutions/testingjpa002.htm). This file, local to the Polaris service, will contain information on what database to use for metastore management and how to connect to it. | ||
|
|
||
| ### Bootstrapping | ||
|
|
||
| Before using Polaris when using a metastore manager other than `in-memory`, you must **bootstrap** the metastore manager. This is a manual operation that must be performed **only once** in order to prepare the metastore manager to integrate with Polaris. When the metastore manager is bootstrapped, any existing Polaris entities in the metastore manager may be **purged**. | ||
eric-maynard marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| To bootstrap Polaris, run: | ||
|
|
||
| ```bash | ||
| java -jar /path/to/jar/polaris-service-1.0.0-all.jar bootstrap polaris-server.yml | ||
| ``` | ||
|
|
||
| Afterwards, Polaris can be launched normally: | ||
|
|
||
| ```bash | ||
| java -jar /path/to/jar/polaris-service-1.0.0-all.jar server polaris-server.yml | ||
| ``` | ||
|
|
||
| ## Other Configurations | ||
|
|
||
| When deploying Polaris in production, consider adjusting the following configurations: | ||
|
|
||
| #### featureConfiguration.SUPPORTED_CATALOG_STORAGE_TYPES | ||
| - By default Polaris catalogs are allowed to be located in local filesystem with the `FILE` storage type. This should be disabled for production systems. | ||
| - Use this configuration to additionally disable any other storage types that will not be in use. | ||
|
|
||
|
|
||
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.