Skip to content

Federation of Catalog-level entities to remote catalogs #540

@dennishuo

Description

@dennishuo

Is your feature request related to a problem? Please describe.

As Iceberg users sometimes find different Iceberg-catalog sources of truth being created organically over time, it can be difficult to consolidate or centralize workloads onto a new target architecture or onboard new catalog implementations like Polaris to unlock its RBAC features without disrupting ongoing workloads.

Being able to have one Catalog instance serve as a combined "pane of glass" serving the contents of a legacy Catalog alongside new datasets enables an incremental migration or even simply enabling different workloads to leverage different catalog features without a stop-the-world migration.

Describe the solution you'd like

Read-through federation of Catalog entities (e.g. Namespaces, Tables, Views) would allow a Polaris instance to link in an existing remote Catalog as a source-of-truth while exposing a Polaris entry-point along with Polaris RBAC features without performing a disruptive migration of Catalog contents.

Ideally, this federation would require minimal setup of the initial catalog connection while still benefiting from all the fine-grained access control features expected from normal Polaris catalogs; it should intelligently pass through API operations but also allow defining privilege grants on entities without requiring prior setup/syncing of those entities into the Polaris federation layer.

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions