[EPIC] feat: support metadata tables

[xxchan]

In Iceberg Spark/Flink there are metadata tables that provide information around the table: https://iceberg.apache.org/docs/latest/spark-queries/#inspecting-tables

Supporting this in iceberg-rust allows other engines (like RisingWave) to support these "standard" metadata tables.

reference implementation:
- https://py.iceberg.apache.org/api/#inspecting-tables
- https://github.com/apache/iceberg-python/blob/main/pyiceberg/table/inspect.py#L58-L90

List of all metadata tables: https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/MetadataTableType.java#L23

• ENTRIES feat: Support metadata table "Entries"  #863
• FILES
• DATA_FILES
• DELETE_FILES
• HISTORY feat: Support metadata table "History" #841
• METADATA_LOG_ENTRIES feat: Support metadata table "Metadata Log Entries" #846
• SNAPSHOTS feat: support metadata table "snapshots" #822
• REFS
• MANIFESTS feat: Support metadata table "Manifests"  #861
• PARTITIONS
• ALL_DATA_FILES
• ALL_DELETE_FILES
• ALL_FILES
• ALL_MANIFESTS
• ALL_ENTRIES
• POSITION_DELETES

[Fokko]

Thanks for working on this @xxchan. Metadata tables are very important when it comes to the metadata operations. For example, the snapshots table in #822 will make it easy to perform operations like expiring snapshots.

[flaneur2020]

i'd like to contribute to some of the implementation of these metadata tables, similar to the work done by @rshkv based on @xxchan's branch.

I plan to start with the MANIFESTS table first. could you kindly let me know if anyone else is currently working on this or has plans to do so? I want to avoid any potential conflicts. thanks.

[rshkv]

@flaneur2020, apologies, I was actually just working on manifests and I should be able to put that up in a few days. If you want to go ahead with entries or files there'll be some overlap and code to reuse but that's fine - happy to rebase.

Ignore the above. I'm looking at entries now. Not working on manifests.

[DeaconDesperado]

I'd like to attempt the FILES metadata table if not already being worked on. Thanks!

[liurenjie1024]

I'd like to attempt the FILES metadata table if not already being worked on. Thanks!

Welcome to contribute!

[felixscherz]

Hi, I'm going to have a look at the PARTITIONS table:)