-
Notifications
You must be signed in to change notification settings - Fork 332
chore(docs): reorganize getting-started section #2611
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
8bda153
14b4f9a
3b18ee3
2816ec4
ce8445c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| --- | ||
| # | ||
| # Licensed to the Apache Software Foundation (ASF) under one | ||
| # or more contributor license agreements. See the NOTICE file | ||
| # distributed with this work for additional information | ||
| # regarding copyright ownership. The ASF licenses this file | ||
| # to you under the Apache License, Version 2.0 (the | ||
| # "License"); you may not use this file except in compliance | ||
| # with the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, | ||
| # software distributed under the License is distributed on an | ||
| # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| # KIND, either express or implied. See the License for the | ||
| # specific language governing permissions and limitations | ||
| # under the License. | ||
| # | ||
| title: Creating a Catalog | ||
| linkTitle: Creating a Catalog | ||
| type: docs | ||
| weight: 300 | ||
| --- | ||
|
|
||
| The following Object Storage providers can be configured as storage backends for your Polaris catalog: | ||
|
|
||
| - [S3 compatible object stores]({{< ref "s3.md" >}}) | ||
| - [Google Cloud Storage]({{< ref "catalog-gcs.md" >}}) | ||
| - [Azure Blob Storage]({{< ref "catalog-azure.md" >}}) | ||
| - Local file system (By default for testing only) | ||
|
|
||
|
|
||
| ## Create a catalog using polaris CLI | ||
|
|
||
| Check full list of options for the `polaris catalogs create` command [here]({{% ref "../../command-line-interface#create" %}}) | ||
|
|
||
| ### Example | ||
|
|
||
| ```shell | ||
| CLIENT_ID=root \ | ||
| CLIENT_SECRET=s3cr3t \ | ||
| DEFAULT_BASE_LOCATION=s3://example-bucket/my_data \ | ||
| ROLE_ARN=arn:aws:iam::111122223333:role/ExampleCorpRole \ | ||
| ./polaris \ | ||
| --client-id ${CLIENT_ID} \ | ||
| --client-secret ${CLIENT_SECRET} \ | ||
| catalogs \ | ||
| create \ | ||
| --storage-type s3 \ | ||
| --default-base-location ${DEFAULT_BASE_LOCATION} \ | ||
| --role-arn ${ROLE_ARN} \ | ||
| my_catalog | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| --- | ||
| # | ||
| # Licensed to the Apache Software Foundation (ASF) under one | ||
| # or more contributor license agreements. See the NOTICE file | ||
| # distributed with this work for additional information | ||
| # regarding copyright ownership. The ASF licenses this file | ||
| # to you under the Apache License, Version 2.0 (the | ||
| # "License"); you may not use this file except in compliance | ||
| # with the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, | ||
| # software distributed under the License is distributed on an | ||
| # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| # KIND, either express or implied. See the License for the | ||
| # specific language governing permissions and limitations | ||
| # under the License. | ||
| # | ||
| title: Creating a catalog on Azure | ||
| linkTitle: Azure | ||
| type: docs | ||
| weight: 300 | ||
| --- | ||
|
|
||
| For the `polaris catalogs create` [command]({{% ref "../../command-line-interface#create" %}}) there are few `azure` only options | ||
|
|
||
| ```text | ||
| --storage-type azure | ||
| --tenant-id (Required for Azure) A tenant ID to use when connecting to Azure Storage | ||
| --multi-tenant-app-name (Only for Azure) The app name to use when connecting to Azure Storage | ||
| --consent-url (Only for Azure) A consent URL granting permissions for the Azure Storage location | ||
| ``` | ||
|
|
||
| ### example | ||
|
|
||
| ```shell | ||
| CLIENT_ID=root \ | ||
| CLIENT_SECRET=s3cr3t \ | ||
| DEFAULT_BASE_LOCATION=abfss://[email protected] \ | ||
| TENANT_ID=tenant123.onmicrosoft.com \ | ||
| MULTI_TENANT_APP_NAME=myapp \ | ||
| CONSENT_URL=https://myapp.com/consent | ||
| ./polaris \ | ||
| --client-id ${CLIENT_ID} \ | ||
| --client-secret ${CLIENT_SECRET} \ | ||
| catalogs \ | ||
| create \ | ||
| --storage-type azure \ | ||
| --tenant-id ${TENANT_ID} \ | ||
| --multi-tenant-app-name ${MULTI_TENANT_APP_NAME} \ | ||
| --consent-url ${CONSENT_URL} \ | ||
| --default-base-location ${DEFAULT_BASE_LOCATION} \ | ||
| my_azure_catalog | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| --- | ||
| # | ||
| # Licensed to the Apache Software Foundation (ASF) under one | ||
| # or more contributor license agreements. See the NOTICE file | ||
| # distributed with this work for additional information | ||
| # regarding copyright ownership. The ASF licenses this file | ||
| # to you under the Apache License, Version 2.0 (the | ||
| # "License"); you may not use this file except in compliance | ||
| # with the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, | ||
| # software distributed under the License is distributed on an | ||
| # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| # KIND, either express or implied. See the License for the | ||
| # specific language governing permissions and limitations | ||
| # under the License. | ||
| # | ||
| title: Creating a catalog on Google Cloud Storage (GCS) | ||
| linkTitle: GCS | ||
| type: docs | ||
| weight: 200 | ||
| --- | ||
|
|
||
| For the `polaris catalogs create` [command]({{% ref "../../command-line-interface#create" %}}) there are few `gcs` only options | ||
|
|
||
| ```text | ||
| --storage-type gcs | ||
| --service-account (Only for GCS) The service account to use when connecting to GCS | ||
| ``` | ||
|
|
||
| ### example | ||
|
|
||
| ```shell | ||
| CLIENT_ID=root \ | ||
| CLIENT_SECRET=s3cr3t \ | ||
| DEFAULT_BASE_LOCATION=gs://my-ml-bucket/predictions/ \ | ||
| SERVICE_ACCOUNT=serviceAccount:[email protected] \ | ||
| ./polaris \ | ||
| --client-id ${CLIENT_ID} \ | ||
| --client-secret ${CLIENT_SECRET} \ | ||
| catalogs \ | ||
| create \ | ||
| --storage-type gcs \ | ||
| --service-account ${SERVICE_ACCOUNT} \ | ||
| --default-base-location ${DEFAULT_BASE_LOCATION} \ | ||
| my_gcs_catalog | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| --- | ||
| # | ||
| # Licensed to the Apache Software Foundation (ASF) under one | ||
| # or more contributor license agreements. See the NOTICE file | ||
| # distributed with this work for additional information | ||
| # regarding copyright ownership. The ASF licenses this file | ||
| # to you under the Apache License, Version 2.0 (the | ||
| # "License"); you may not use this file except in compliance | ||
| # with the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, | ||
| # software distributed under the License is distributed on an | ||
| # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| # KIND, either express or implied. See the License for the | ||
| # specific language governing permissions and limitations | ||
| # under the License. | ||
| # | ||
| title: Creating a catalog on S3 compatible cloud providers | ||
| linkTitle: S3 | ||
| type: docs | ||
| weight: 100 | ||
| --- | ||
|
|
||
| The following S3 compatible cloud providers can be configured as storage backends for your Polaris catalog: | ||
|
|
||
| - [AWS S3]({{< ref "catalog-aws.md" >}}) | ||
| - [MinIO]({{< ref "catalog-minio.md" >}}) | ||
|
|
||
| For the `polaris catalogs create` [command]({{% ref "../../../command-line-interface#create" %}}) there are few `s3` only options | ||
|
|
||
| ```text | ||
| --storage-type s3 | ||
| --role-arn (Required for S3) A role ARN to use when connecting to S3 | ||
| --region (Only for S3) The region to use when connecting to S3 | ||
| --external-id (Only for S3) The external ID to use when connecting to S3 | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| --- | ||
| # | ||
| # Licensed to the Apache Software Foundation (ASF) under one | ||
| # or more contributor license agreements. See the NOTICE file | ||
| # distributed with this work for additional information | ||
| # regarding copyright ownership. The ASF licenses this file | ||
| # to you under the Apache License, Version 2.0 (the | ||
| # "License"); you may not use this file except in compliance | ||
| # with the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, | ||
| # software distributed under the License is distributed on an | ||
| # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| # KIND, either express or implied. See the License for the | ||
| # specific language governing permissions and limitations | ||
| # under the License. | ||
| # | ||
| title: Creating a catalog on AWS S3 | ||
| linkTitle: AWS | ||
| type: docs | ||
| weight: 100 | ||
| --- | ||
|
|
||
|
|
||
| ### example | ||
|
|
||
| ```shell | ||
| CLIENT_ID=root \ | ||
| CLIENT_SECRET=s3cr3t \ | ||
| DEFAULT_BASE_LOCATION=s3://example-bucket/my_data \ | ||
| ROLE_ARN=arn:aws:iam::111122223333:role/ExampleCorpRole \ | ||
| REGION=us-west-2 \ | ||
| EXTERNAL_ID=12345678901234567890 \ | ||
| ./polaris \ | ||
| --client-id ${CLIENT_ID} \ | ||
| --client-secret ${CLIENT_SECRET} \ | ||
| catalogs \ | ||
| create \ | ||
| --storage-type s3 \ | ||
| --default-base-location ${DEFAULT_BASE_LOCATION} \ | ||
| --role-arn ${ROLE_ARN} \ | ||
| --region ${REGION} \ | ||
| --external-id ${EXTERNAL_ID} \ | ||
| my_aws_catalog | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,9 +17,10 @@ | |
| # specific language governing permissions and limitations | ||
| # under the License. | ||
| # | ||
| Title: Deploying Polaris on MinIO | ||
| title: Creating a catalog on MinIO | ||
| linkTitle: MinIO | ||
| type: docs | ||
| weight: 350 | ||
| weight: 200 | ||
| --- | ||
|
|
||
| In this guide we walk through setting up a simple Polaris Server with local [MinIO](https://www.min.io/) storage. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My thinking was to just give instructions for "manual" setup in this guide (copy from the docker compose file) and may be link to the docker compose example at the end of the guide (with minimal text, just indicating that the docker compose file shows and end-to-end integration example for MinIO). WDYT?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we keep the current version? Using MinIO looks like not a standard path, so I would prefer to save it as is.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Without refactoring this page, the whole Getting Started reorg is not quite complete, IMHO... Still, I'd be ok with merging these doc changes "as is". I can probably make a follow-up PR from my side. I guess it's easier than going through GH comments 😅
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would appreciate it if you cover this part from your side 🙏 |
||
|
|
@@ -42,7 +43,7 @@ docker compose -f getting-started/minio/docker-compose.yml up | |
| ``` | ||
|
|
||
| The compose script will start MinIO on default ports (API on 9000, UI on 9001) | ||
| plus a Polaris Server pre-configured to that MinIO instance. | ||
| plus a Polaris Server pre-configured to that MinIO instance. | ||
|
|
||
| In this example the `root` principal has its password set to `s3cr3t`. | ||
|
|
||
|
|
@@ -86,7 +87,7 @@ mc ls pol/bucket123/ns/t1 | |
| [2025-08-13 18:52:38 EDT] 0B metadata/ | ||
| ``` | ||
|
|
||
| Note: the values of `minio_root`, `m1n1opwd` and `bucket123` are defined in the docker compose file. | ||
| Note: the values of `minio_root`, `m1n1opwd` and `bucket123` are defined in the docker compose file. | ||
|
|
||
| # Notes on Storage Configuation | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| --- | ||
| # | ||
| # Licensed to the Apache Software Foundation (ASF) under one | ||
| # or more contributor license agreements. See the NOTICE file | ||
| # distributed with this work for additional information | ||
| # regarding copyright ownership. The ASF licenses this file | ||
| # to you under the Apache License, Version 2.0 (the | ||
| # "License"); you may not use this file except in compliance | ||
| # with the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, | ||
| # software distributed under the License is distributed on an | ||
| # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| # KIND, either express or implied. See the License for the | ||
| # specific language governing permissions and limitations | ||
| # under the License. | ||
| # | ||
| title: Deploying Polaris on Cloud Providers | ||
| linkTitle: Cloud Providers | ||
| type: docs | ||
| weight: 300 | ||
| --- | ||
|
|
||
| Polaris can be deployed on various cloud providers, including Amazon Web Services (AWS), Azure, and Google Cloud Platform (GCP). | ||
| In the following guides, we will walk you through the process of deploying Polaris on each of these cloud providers. |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This section header is "clickable" in the left-hand pane, but the "Getting Stated" section is not (#2551)... Could you fix that too? I guess it's related to having
render: neverin the latter... Why don't we add some text to it and render it like all the other sections?This can be done later, of course.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, the reason because of
render: never. I will fix it.