-
Notifications
You must be signed in to change notification settings - Fork 393
Open
Labels
Description
Feature Request / Improvement
The VariantType is a semi-structured type that's part of V3.
- Hopefully, the Parquet format will be released soon: https://lists.apache.org/thread/mr8krzdy8qslhhy5zbll53fd2hy6cpwg
- Once that's in, I'll be added to Arrow: GH-45937: [C++][Parquet] Variant logical type definition arrow#45375
- I think for PyIceberg, we need to wait for PyArrow support.
There is a Python implementation in Spark that we could vendor into the project, but that will be pretty slow: https://github.com/apache/spark/blob/master/python/pyspark/sql/variant_utils.py
Thoughts?
Original proposal: apache/iceberg#10392