Trino getting started docker compose

[kevinjqliu]

Description

Add a new getting-started directory, with a trino subdirectory which includes the following

• docker-compose-trino.yml, docker-compose file to spin up Polaris and Trino services, as well as setup script to initialize a catalog in Polaris
• create-polaris-catalog.sh, setup script to initialize a catalog in Polaris
• trino-config/catalog/iceberg.properties file, which configures Trino to register Polaris as an Iceberg REST catalog

To run, use the command:

docker-compose -f getting-started/trino/docker-compose-trino.yml up 

Note

The setup script and Trino both use the credential principal:root;realm:default-realm. This is a workaround so users do not need to fetch credentials from Polaris' output.

To access the Trino CLI,

docker exec -it trino-trino-1 trino

Note, trino-trino-1 is the name docker container.

Run Trino commands via Trino CLI,

SHOW CATALOGS;
SHOW SCHEMAS FROM iceberg;
SHOW TABLES FROM iceberg.information_schema;
DESCRIBE iceberg.information_schema.tables;

CREATE SCHEMA iceberg.tpch;
CREATE TABLE iceberg.tpch.test_polaris AS SELECT 1 x;
SELECT * FROM iceberg.tpch.test_polaris;

Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #22

Reference

• Trino Docker Image
• @ebyhr's Trino Iceberg connector with Polaris catalog

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Test A
• Test B

Test Configuration:

• Firmware version:
• Hardware:
• Toolchain:
• SDK:

Checklist:

Please delete options that are not relevant.

• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules
• If adding new functionality, I have discussed my implementation with the community using the linked GitHub issue
• I have signed and submitted the ICLA and if needed, the CCLA. See Contributing for details.

[flyrain]

Hi @kevinjqliu , is this PR ready for review or not? Can you mark it as draft if not? Thanks.

[kevinjqliu]

Might need to update docs, similar to #44

[kevinjqliu]

whats the stance on using this credential? Since this is a demo, I think it's acceptable

[kevinjqliu]

I can't tell whether this works by design or due to a bug...

[kevinjqliu]

r? @collado-mike / @RussellSpitzer
PTAL

[kevinjqliu]

Can also use Polaris python CLI to create a catalog
https://github.com/polaris-catalog/polaris/blob/main/docs/quickstart.md#defining-a-catalog

[almazrevolut]

Should it also include iceberg.rest-catalog.oauth2.scope? It seems, non scoped requests fail with

org.apache.iceberg.exceptions.ForbiddenException: Forbidden: Principal 'XXX' with activated PrincipalRoles '[catalog]' and activated ids '[]' is not authorized for op XXX

Related trino PR - trinodb/trino#22961

[ebyhr]

I don't think trinodb/trino#22961 is required in case of iceberg.rest-catalog.oauth2.token. The scope is needed for iceberg.rest-catalog.oauth2.credential in my understanding.

[RussellSpitzer]

This worked for me, I'm not enough of a trino expert to understand if anything is off there but since this is just for demoing I think this is probably a good thing to merge. I would request @kevinjqliu we add the instructions to the readme though as well. I think the details you have in the main comment of this pullrequest is basically all we need.

@ebyhr Can you please check out the Trino specific parts and make sure that's all good to go?

[ebyhr]

Looks good except for comments.

[ebyhr]

Using dynamic catalog mode (CATALOG_MANAGEMENT=dynamic) provides more flexibility, but I think a static mode is fine in this PR.

[kevinjqliu]

didn't know about this option, cool. lets leave as is for now since dynamic catalog management requires the user to run an extra CREATE CATALOG statement.

[jbonofre]

In order to not privilege one engine from other, I would propose to create an engine folder in the getting-started one. We can add trino, spark, flink, doris, dremio, etc there.

[kevinjqliu]

sure. is the engine folder necessary? How about just getting-started/trino?

getting-started/
    trino/
    spark/
    ...

[kevinjqliu]

@RussellSpitzer @ebyhr @jbonofre Thanks for the review. I've updated the PR based on your comments. Please take another look.

[flyrain]

LGTM. Do you want to take another look? @ebyhr @jbonofre @RussellSpitzer

[kevinjqliu]

Thanks @flyrain.
@jbonofre I've addressed your previous comments. Do you mind taking another look?

[ebyhr]

Does this context work in your environment? I had do change it to ../../ because Dockerfile exists in the project root directory.

[kevinjqliu]

you're right, this should be ../../, it was accidentally removed when i copied over the new setup in 9c2be7f

[kevinjqliu]

updated! thanks

[kevinjqliu]

Reran docker with clean setup, confirmed working.

[flyrain]

Looks like all comments are resolved. I will merge it if there is no more comment today.

Comments have been resolved.

[flyrain]

Thanks @kevinjqliu for adding Trino docker compose. Thanks @jbonofre @RussellSpitzer @ebyhr @snazy @almazrevolut for the review.