-
-
Notifications
You must be signed in to change notification settings - Fork 153
Closed
Milestone
Description
Description
Discussion and plans for moving to Datafusion 26.
Datafusion 26 comes with lot of exciting features and thus changes to Parseable.
We currently update arrow and datafusion simultaneously for simplicity.
Arrow 40
- Schema field has changed from Vec to Fields ( Arc<[Arc<Field>]> )
- JSON -> RecordBatch flow is now faster and supports direct serialization from serde compatible structs
Datafusion 26
Better support for timestamp based queries. SELECT now() + '1 month' FROM my_table;
Better support for list and struct datatypes. Some insights require to see if schema can be evolved in list of strut case
Stage Data Query:
Since Datafusion supports querying arrows files as well, we can target enabling staging data query.
Moved to separate issues
Metadata
Metadata
Assignees
Labels
No labels