Reserved Iceberg metadata table names in Polaris (history, entries, etc.) #1771
Description
Describe the bug
We have seen issues with Polaris and tables named after Iceberg metadata table names.
For example, when we try to create tables named as: entries, history, etc.
It looks like this is an Iceberg problem and not directly a Polaris one.
Upon Creation
Polaris will call Catalog::tableExists(...), which internally calls BaseMetastoreCatalog::loadTable(...). If that call returns an exception then the table is considered non-existent.
The problem, however, is that loadTable will try to resolve the table as a metadata table if it's not found and if the table name matches one of the reserved metadata keywords defined here, it will try to call loadMetadataTable using the namespace as a table name, and this will fail.
This issue is also open on the iceberg community:
apache/iceberg#10550
Possible Fix
I understand that we want to keep tableExists implementation-agnostic and use loadTable in a try/catch, but the problem is that loadTable will try to resolve it as a metadata table if it matches one of these words. Since we do, however, have the IcebergCatalog class, we could simply override tableExists with the following:
public boolean tableExists(TableIdentifier identifier) {
if (isValidIdentifier(identifier)) {
return newTableOps(identifier).current() != null;
}
return false;
}
This would allow us to bypass the problem, since we only care about actual tables when we call this method and not metadata tables.
To Reproduce
Build a table named history
Actual Behavior
Fails with unclear error.
Expected Behavior
Should accept it.
Additional context
Clashes with metadata table names.
System information
No response