Skip to content

Commit 64b46f1

Browse files
Revamp the Quick Start page (#1367)
* First Draft with AWS * try again * try again * try again * try again * try again * try now * should work * AWS First Draft Complete * ensure file changed * Azure First Draft Complete * Azure First Draft, pt. 2 * Azure Completed * GCP First Draft * GCP Verified * File structure fixed * Remove Trino-specific tutorial * Restructured Quick Start * Addresses minor comments from @eric-maynard * Added reference to Deploying Polaris in Production * Fix MD Link Checker --------- Co-authored-by: Adnan Hemani <[email protected]>
1 parent df6e3d6 commit 64b46f1

File tree

24 files changed

+1034
-600
lines changed

24 files changed

+1034
-600
lines changed

getting-started/README.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,12 +33,10 @@ this directory. Each example has detailed instructions.
3333
- [Spark](spark): An example that uses an in-memory metastore, automatically bootstrapped, with
3434
Apache Spark and a Jupyter notebook.
3535

36-
- [Trino](trino): An example that uses Trino with Polaris.
37-
3836
- [Telemetry](telemetry): An example that includes Prometheus and Jaeger to collect metrics and
3937
traces from Apache Polaris. This example automatically creates a `polaris_demo` catalog.
4038

4139
- [Eclipselink](eclipselink): An example that uses an Eclipselink metastore and a Postgres
4240
database. The realm is bootstrapped with the Polaris Admin tool. This example also creates a
43-
`polaris_demo` catalog, and offers the ability to run Spark SQL queries. Finally, it shows how to
41+
`polaris_quickstart` catalog, and offers the ability to run Spark SQL and Trino queries. Finally, it shows how to
4442
attach a debugger to the Polaris server.
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
#
2+
# Licensed to the Apache Software Foundation (ASF) under one
3+
# or more contributor license agreements. See the NOTICE file
4+
# distributed with this work for additional information
5+
# regarding copyright ownership. The ASF licenses this file
6+
# to you under the Apache License, Version 2.0 (the
7+
# "License"); you may not use this file except in compliance
8+
# with the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing,
13+
# software distributed under the License is distributed on an
14+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
# KIND, either express or implied. See the License for the
16+
# specific language governing permissions and limitations
17+
# under the License.
18+
#
19+
20+
EC2_INSTANCE_ID=$(cat /var/lib/cloud/data/instance-id)
21+
22+
DESCRIBE_INSTANCE=$(aws ec2 describe-instances \
23+
--instance-ids $EC2_INSTANCE_ID \
24+
--query 'Reservations[*].Instances[*].{Instance:InstanceId,VPC:VpcId,AZ:Placement.AvailabilityZone}' \
25+
--output json)
26+
27+
CURRENT_VPC=$(echo $DESCRIBE_INSTANCE | jq -r .[0].[0]."VPC")
28+
29+
CURRENT_REGION=$(echo $DESCRIBE_INSTANCE | jq -r .[0].[0]."AZ" | sed 's/.$//')
30+
31+
ALL_SUBNETS=$(aws ec2 describe-subnets \
32+
--region $CURRENT_REGION \
33+
--query 'Subnets[*].{SubnetId:SubnetId}' \
34+
--output json \
35+
| jq -r '[.[]["SubnetId"]] | join(" ")')
36+
37+
RANDOM_SUFFIX=$(head /dev/urandom | tr -dc 'A-Za-z0-9' | head -c 8)
38+
SUBNET_GROUP_NAME="polaris-db-subnet-group-$RANDOM_SUFFIX"
39+
INSTANCE_NAME="polaris-backend-test-$RANDOM_SUFFIX"
40+
41+
aws rds create-db-subnet-group \
42+
--db-subnet-group-name $SUBNET_GROUP_NAME \
43+
--db-subnet-group-description "Apache Polaris Quickstart DB Subnet Group" \
44+
--subnet-ids $ALL_SUBNETS
45+
46+
DB_INSTANCE_INFO=$(aws rds create-db-instance \
47+
--db-instance-identifier $INSTANCE_NAME \
48+
--db-instance-class db.t3.micro \
49+
--engine postgres \
50+
--master-username postgres \
51+
--master-user-password postgres \
52+
--db-name POLARIS \
53+
--db-subnet-group-name $SUBNET_GROUP_NAME \
54+
--allocated-storage 10)
55+
56+
DB_ARN=$(echo $DB_INSTANCE_INFO | jq -r '.["DBInstance"]["DBInstanceArn"]')
57+
58+
DESCRIBE_DB=$(aws rds describe-db-instances --db-instance-identifier $DB_ARN)
59+
60+
until echo $DESCRIBE_DB | jq -e '.["DBInstances"][0] | has("Endpoint")';
61+
do
62+
echo "sleeping 10s to wait for Postgres DB provisioning..."
63+
sleep 10
64+
DESCRIBE_DB=$(aws rds describe-db-instances --db-instance-identifier $DB_ARN)
65+
done
66+
67+
POSTGRES_ADDR=$(echo $DESCRIBE_DB | jq -r '.["DBInstances"][0]["Endpoint"]' | jq -r '"\(.Address):\(.Port)"')
68+
69+
FULL_POSTGRES_ADDR=$(printf '%s\n' "jdbc:postgresql://$POSTGRES_ADDR/{realm}" | sed 's/[&/\]/\\&/g')
70+
sed -i "/jakarta.persistence.jdbc.url/ s|value=\"[^\"]*\"|value=\"$FULL_POSTGRES_ADDR\"|" "getting-started/assets/eclipselink/persistence.xml"
71+
72+
./gradlew clean :polaris-quarkus-server:assemble :polaris-quarkus-admin:assemble \
73+
-PeclipseLinkDeps=org.postgresql:postgresql:42.7.4 \
74+
-Dquarkus.container-image.tag=postgres-latest \
75+
-Dquarkus.container-image.build=true \
76+
--no-build-cache
77+
78+
docker compose -f getting-started/eclipselink/docker-compose-bootstrap-db.yml -f getting-started/eclipselink/docker-compose.yml up -d
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
#
2+
# Licensed to the Apache Software Foundation (ASF) under one
3+
# or more contributor license agreements. See the NOTICE file
4+
# distributed with this work for additional information
5+
# regarding copyright ownership. The ASF licenses this file
6+
# to you under the Apache License, Version 2.0 (the
7+
# "License"); you may not use this file except in compliance
8+
# with the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing,
13+
# software distributed under the License is distributed on an
14+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
# KIND, either express or implied. See the License for the
16+
# specific language governing permissions and limitations
17+
# under the License.
18+
#
19+
20+
CURRENT_REGION=$(curl -H Metadata:true "http://169.254.169.254/metadata/instance?api-version=2021-02-01" | jq -r '.compute.location')
21+
CURRENT_RESOURCE_GROUP=$(curl -H Metadata:true "http://169.254.169.254/metadata/instance?api-version=2021-02-01" | jq -r '.compute.resourceGroupName')
22+
RANDOM_SUFFIX=$(head /dev/urandom | tr -dc 'a-z0-9' | head -c 8)
23+
INSTANCE_NAME="polaris-backend-test-$RANDOM_SUFFIX"
24+
25+
CREATE_DB_RESPONSE=$(az postgres flexible-server create -l $CURRENT_REGION -g $CURRENT_RESOURCE_GROUP -n $INSTANCE_NAME -u postgres -p postgres -y)
26+
27+
az postgres flexible-server db create -g $CURRENT_RESOURCE_GROUP -s $INSTANCE_NAME -d POLARIS
28+
29+
POSTGRES_ADDR=$(echo $CREATE_DB_RESPONSE | jq -r '.host')
30+
31+
FULL_POSTGRES_ADDR=$(printf '%s\n' "jdbc:postgresql://$POSTGRES_ADDR:5432/{realm}" | sed 's/[&/\]/\\&/g')
32+
sed -i "/jakarta.persistence.jdbc.url/ s|value=\"[^\"]*\"|value=\"$FULL_POSTGRES_ADDR\"|" "getting-started/assets/eclipselink/persistence.xml"
33+
34+
./gradlew clean :polaris-quarkus-server:assemble :polaris-quarkus-admin:assemble \
35+
-PeclipseLinkDeps=org.postgresql:postgresql:42.7.4 \
36+
-Dquarkus.container-image.tag=postgres-latest \
37+
-Dquarkus.container-image.build=true \
38+
--no-build-cache
39+
40+
docker compose -f getting-started/eclipselink/docker-compose-bootstrap-db.yml -f getting-started/eclipselink/docker-compose.yml up -d
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
#
2+
# Licensed to the Apache Software Foundation (ASF) under one
3+
# or more contributor license agreements. See the NOTICE file
4+
# distributed with this work for additional information
5+
# regarding copyright ownership. The ASF licenses this file
6+
# to you under the Apache License, Version 2.0 (the
7+
# "License"); you may not use this file except in compliance
8+
# with the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing,
13+
# software distributed under the License is distributed on an
14+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
# KIND, either express or implied. See the License for the
16+
# specific language governing permissions and limitations
17+
# under the License.
18+
#
19+
20+
CURRENT_ZONE=$(curl -H "Metadata-Flavor: Google" "http://169.254.169.254/computeMetadata/v1/instance/zone" | awk -F/ '{print $NF}')
21+
CURRENT_REGION=$(echo $CURRENT_ZONE | sed 's/-[a-z]$//')
22+
VM_INSTANCE_NAME=$(curl -H "Metadata-Flavor: Google" "http://169.254.169.254/computeMetadata/v1/instance/name")
23+
RANDOM_SUFFIX=$(head /dev/urandom | tr -dc 'a-z0-9' | head -c 8)
24+
DB_INSTANCE_NAME="polaris-backend-test-$RANDOM_SUFFIX"
25+
26+
INSTANCE_IP=$(gcloud compute instances describe $VM_INSTANCE_NAME --zone=$CURRENT_ZONE --format="get(networkInterfaces[0].accessConfigs[0].natIP)")
27+
28+
29+
gcloud sql instances create $DB_INSTANCE_NAME \
30+
--database-version=POSTGRES_17 \
31+
--region=$CURRENT_REGION \
32+
--tier=db-perf-optimized-N-4 \
33+
--edition=ENTERPRISE_PLUS \
34+
--root-password=postgres \
35+
--authorized-networks="$INSTANCE_IP/32"
36+
37+
gcloud sql databases create POLARIS --instance=$DB_INSTANCE_NAME
38+
39+
POSTGRES_ADDR=$(gcloud sql instances describe $DB_INSTANCE_NAME --format="get(ipAddresses[0].ipAddress)")
40+
41+
FULL_POSTGRES_ADDR=$(printf '%s\n' "jdbc:postgresql://$POSTGRES_ADDR:5432/{realm}" | sed 's/[&/\]/\\&/g')
42+
sed -i "/jakarta.persistence.jdbc.url/ s|value=\"[^\"]*\"|value=\"$FULL_POSTGRES_ADDR\"|" "getting-started/assets/eclipselink/persistence.xml"
43+
44+
./gradlew clean :polaris-quarkus-server:assemble :polaris-quarkus-admin:assemble \
45+
-PeclipseLinkDeps=org.postgresql:postgresql:42.7.4 \
46+
-Dquarkus.container-image.tag=postgres-latest \
47+
-Dquarkus.container-image.build=true \
48+
--no-build-cache
49+
50+
docker compose -f getting-started/eclipselink/docker-compose-bootstrap-db.yml -f getting-started/eclipselink/docker-compose.yml up -d

getting-started/assets/eclipselink/persistence.xml

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,7 @@
3232
<class>org.apache.polaris.jpa.models.ModelSequenceId</class>
3333
<shared-cache-mode>NONE</shared-cache-mode>
3434
<properties>
35-
<property name="jakarta.persistence.jdbc.url"
36-
value="jdbc:postgresql://postgres:5432/{realm}"/>
35+
<property name="jakarta.persistence.jdbc.url" value="jdbc:postgresql://postgres:5432/{realm}"/>
3736
<property name="jakarta.persistence.jdbc.user" value="postgres"/>
3837
<property name="jakarta.persistence.jdbc.password" value="postgres"/>
3938
<property name="jakarta.persistence.schema-generation.database.action" value="create"/>

getting-started/assets/polaris/create-catalog.sh

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,24 +33,24 @@ echo
3333
echo "Obtained access token: ${token}"
3434

3535
echo
36-
echo Creating a catalog named polaris_demo...
36+
echo Creating a catalog named quickstart_catalog...
3737

3838
curl -s -H "Authorization: Bearer ${token}" \
3939
-H 'Accept: application/json' \
4040
-H 'Content-Type: application/json' \
4141
http://polaris:8181/api/management/v1/catalogs \
4242
-d '{
4343
"catalog": {
44-
"name": "polaris_demo",
44+
"name": "quickstart_catalog",
4545
"type": "INTERNAL",
4646
"readOnly": false,
4747
"properties": {
48-
"default-base-location": "file:///tmp/polaris/"
48+
"default-base-location": "file:///var/tmp/quickstart_catalog/"
4949
},
5050
"storageConfigInfo": {
5151
"storageType": "FILE",
5252
"allowedLocations": [
53-
"file:///tmp"
53+
"file:///var/tmp"
5454
]
5555
}
5656
}

getting-started/eclipselink/README.md

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ This example requires `jq` to be installed on your machine.
3838
2. Start the docker compose group by running the following command from the root of the repository:
3939

4040
```shell
41-
docker compose -f getting-started/eclipselink/docker-compose.yml up
41+
docker compose -f getting-started/eclipselink/docker-compose-postgres.yml -f getting-started/eclipselink/docker-compose-bootstrap-db.yml -f getting-started/eclipselink/docker-compose.yml up
4242
```
4343

4444
3. Using spark-sql: attach to the running spark-sql container:
@@ -71,5 +71,23 @@ This example requires `jq` to be installed on your machine.
7171
7272
```shell
7373
curl -v http://127.0.0.1:8181/api/management/v1/principal-roles -H "Authorization: Bearer $POLARIS_TOKEN"
74-
curl -v http://127.0.0.1:8181/api/catalog/v1/config?warehouse=polaris_demo -H "Authorization: Bearer $POLARIS_TOKEN"
74+
curl -v http://127.0.0.1:8181/api/management/v1/catalogs/polaris_demo -H "Authorization: Bearer $POLARIS_TOKEN"
7575
```
76+
77+
6. Using Trino CLI: To access the Trino CLI, run this command:
78+
```
79+
docker exec -it eclipselink-trino-1 trino
80+
```
81+
Note, `trino-trino-1` is the name of the Docker container.
82+
83+
Example Trino queries:
84+
```
85+
SHOW CATALOGS;
86+
SHOW SCHEMAS FROM iceberg;
87+
SHOW TABLES FROM iceberg.information_schema;
88+
DESCRIBE iceberg.information_schema.tables;
89+
90+
CREATE SCHEMA iceberg.tpch;
91+
CREATE TABLE iceberg.tpch.test_polaris AS SELECT 1 x;
92+
SELECT * FROM iceberg.tpch.test_polaris;
93+
```
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
#
2+
# Licensed to the Apache Software Foundation (ASF) under one
3+
# or more contributor license agreements. See the NOTICE file
4+
# distributed with this work for additional information
5+
# regarding copyright ownership. The ASF licenses this file
6+
# to you under the Apache License, Version 2.0 (the
7+
# "License"); you may not use this file except in compliance
8+
# with the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing,
13+
# software distributed under the License is distributed on an
14+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
# KIND, either express or implied. See the License for the
16+
# specific language governing permissions and limitations
17+
# under the License.
18+
#
19+
20+
services:
21+
polaris-bootstrap:
22+
# IMPORTANT: the image MUST contain the Postgres JDBC driver and EclipseLink dependencies, see README for instructions
23+
image: apache/polaris-admin-tool:postgres-latest
24+
environment:
25+
polaris.persistence.type: eclipse-link
26+
polaris.persistence.eclipselink.configuration-file: /deployments/config/eclipselink/persistence.xml
27+
volumes:
28+
- ../assets/eclipselink/:/deployments/config/eclipselink
29+
command:
30+
- "bootstrap"
31+
- "--realm=POLARIS"
32+
- "--credential=POLARIS,root,s3cr3t"
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
#
2+
# Licensed to the Apache Software Foundation (ASF) under one
3+
# or more contributor license agreements. See the NOTICE file
4+
# distributed with this work for additional information
5+
# regarding copyright ownership. The ASF licenses this file
6+
# to you under the Apache License, Version 2.0 (the
7+
# "License"); you may not use this file except in compliance
8+
# with the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing,
13+
# software distributed under the License is distributed on an
14+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
# KIND, either express or implied. See the License for the
16+
# specific language governing permissions and limitations
17+
# under the License.
18+
#
19+
20+
services:
21+
postgres:
22+
image: postgres:17.4
23+
ports:
24+
- "5432:5432"
25+
# set shared memory limit when using docker-compose
26+
shm_size: 128mb
27+
environment:
28+
POSTGRES_USER: postgres
29+
POSTGRES_PASSWORD: postgres
30+
POSTGRES_DB: POLARIS
31+
POSTGRES_INITDB_ARGS: "--encoding UTF8 --data-checksums"
32+
volumes:
33+
# Bind local conf file to a convenient location in the container
34+
- type: bind
35+
source: ./postgresql.conf
36+
target: /etc/postgresql/postgresql.conf
37+
command:
38+
- "postgres"
39+
- "-c"
40+
- "config_file=/etc/postgresql/postgresql.conf"
41+
healthcheck:
42+
test: "pg_isready -U postgres"
43+
interval: 5s
44+
timeout: 2s
45+
retries: 15

0 commit comments

Comments
 (0)