Skip to content

Conversation

@YaroslavLitvinov
Copy link
Contributor

@YaroslavLitvinov YaroslavLitvinov commented Feb 25, 2025

Partially resolve #241
This PR must update hash-commits of updated rust-iceberg and datafusion.

PR in Datafusion repo reverting commit, is merged: Embucket/datafusion#18.
PR in Iceberg-rust repo adds actual support for insert rows count: https://github.com/Embucket/iceberg-rust/pull/18/commits

@YaroslavLitvinov YaroslavLitvinov self-assigned this Feb 27, 2025
@YaroslavLitvinov YaroslavLitvinov merged commit 0b8d0e3 into main Feb 27, 2025
4 of 5 checks passed
@YaroslavLitvinov YaroslavLitvinov deleted the issues/241_return_affected_rows_count_dml branch February 27, 2025 15:41
@github-project-automation github-project-automation bot moved this to Done in embucket Feb 27, 2025
eadgbear added a commit that referenced this pull request Mar 7, 2025
* Initial metastore crate

* Metastore implementation

* Upgrading Slatedb to 0.4, adding snapshot tests

* Checkpoint commit

* Checkpoint before crate re-org

* Fix timestamp displaying format (#279)

* fix timestamp format

* Revert timestamp change

* in login request use database_name instead of warehouse (#277)

* Adding missing licenses + license check (#278)

* Adding missing licenses + license check

* Fixing license config path

* Fix IPC arrow format issue (#280)

* Fix IPC arrow format issue

* Fix IPC arrow format issue

* Fix comments

* Control plane tests (batch 3) (#274)

* Add CP service tests

* Add create/delete session tests

* Add create/delete session tests

* Use converted S3 error

* Move DBT_SERIALIZATION_FORMAT to cli (#286)

* Move dbt_ser_fmt to cli

* Rename the field

* Update query runner with session id (#285)

* Issue 115 Arrow support: replace uints by ints in resulted RecordBatch (#287)

* replace uints by ints in resulted RecordBatch

* cargo clippy --workspace --tests

* rebase & fixes

* Set warehouse location based on storage profile path (#288)

* map float types to real logical_type (#289)

* map float types to real logical_type

* separate sessions tests to avoid blocking issues

* unit test for floats datatype maped into real logical type

* Snowflake timestamp_from_parts udf (#297)

* Inital version

* snowflake timestamp_from_parts udf

* Fix clippy

* Fix example typo

* Support negative values

* Snowflake time_from_parts udf (#298)

* Inital version

* snowflake timestamp_from_parts udf

* snowflake time_from_parts udf

* snowflake time_from_parts udf

* Fix clippy

* Fix clippy

* Fix example typo

* Support negative values

* Merge

* snowflake date_from_parts udf (#299)

* Inital version

* snowflake timestamp_from_parts udf

* snowflake time_from_parts udf

* snowflake time_from_parts udf

* snowflake date_from_parts udf

* Fix clippy

* Implement extraction of the corresponding date part from a date or timestamp (#302)

* Implement extracts the corresponding date part from a date or timestamp

* Add units for visit expressions

* Add geospatial udfs (#304)

* Fix object store for custom endpoint (#295)

AWS S3 builder uses dedicated setting to allow http. Parse endpoint and enable if it isn't https.

* Unified query, execution, catalog

* Adding temporary tables to metastore

* Custom DataFusion Catalog implementation (WIP)

* Adding execution User Session type

* Checkpoint

* Catalog implementation for IB

* Crate re-org complete, writing tests

* Changing metastore crate to use iceberg_rust types

* Metastore crate cleanup

* Create table query fixed

* Fixing copy_into query

* Finished updating query type

* Cleaning up service type

* Wrapping up changes to the execution crate

* Bumping Datafusion git ref

* Fixing up after package reorg

* Fixed DBT handler

* Removing old crates

* Disabling some UI handlers and models until new metastore integration is done

* Disabling extra OpenAPI endpoints and some unit tests while test are fixed

* Deleting manifest files that snuck into the repo (my bad)

* Final compilation and clippy fixes

* `cargo fix`

* `cargo fmt`

* Adding license blocks to new files

* Adding missing licenses + license check (#278)

* Adding missing licenses + license check

* Fixing license config path

* Fix IPC arrow format issue (#280)

* Fix IPC arrow format issue

* Fix IPC arrow format issue

* Fix comments

* Control plane tests (batch 3) (#274)

* Add CP service tests

* Add create/delete session tests

* Add create/delete session tests

* Use converted S3 error

* Move DBT_SERIALIZATION_FORMAT to cli (#286)

* Move dbt_ser_fmt to cli

* Rename the field

* Snowflake timestamp_from_parts udf (#297)

* Inital version

* snowflake timestamp_from_parts udf

* Fix clippy

* Fix example typo

* Support negative values

* Implement extraction of the corresponding date part from a date or timestamp (#302)

* Implement extracts the corresponding date part from a date or timestamp

* Add units for visit expressions

* Add geospatial udfs (#304)

* patch chrono 0.4.40 as it breaks build (#310)

* Added support for `day` and other datetime keywords in DF (#294)

* start here

* no macro, just simple vistor mut

* remvoe redundant trait creation

* integration test add, fixed logic, next unit tests

* activated impl and cargo fmt + clippy

* added unit tests + impl extension

* cargo fmt + clippy

* cargo clippy

* cargo clippy

* removed redundant check

* fix

* merged with Artem PR

* cargo fmt + clippy

* small fixes

* cargo fmt + clippy

* update hash commis for datafusion, iceberg-rust (#301)

* Update tests.yml (#313)

Adding github large runners to tests

* Return hardcoded is_dynamic (#318)

* Add geospatial UDF st_makeline (#315)

* Add geospatial udfs

* Add geospatial st_makeline

* Fix cargo

* Add lic header

* Change allocator to snmalloc (#317)

* Add ST_MakePolygon, st_polygon (#319)

* Add ST_MakePolygon, st_polygon

* Add docs

* Add ST_DIMENSION  (#321)

* Add ST_MakePolygon, st_polygon

* Add docs

* Add ST_dimention

* Add license

* Add ST_PointN and ST_Endpoint (#322)

* Add ST_MakePolygon, st_polygon

* Add docs

* Add ST_dimention

* Add license

* Add ST_Endpoint, ST_PointN

* Reimplement ST_Point to work with snowflake linestring

* Add St_x, st_y, st_srid (#324)

* Add ST_MakePolygon, st_polygon

* Add docs

* Add ST_dimention

* Add license

* Add ST_Endpoint, ST_PointN

* Reimplement ST_Point to work with snowflake linestring

* Add St_x, st_y, st_srid

* Moving UDF registration

* Changing Metastore to use Dashmap

* Adding missing geospatial functions after merge

* Fixing unit test structure, still need to fix some unit tests

* Post-merge cleanup

---------

Co-authored-by: Artem Osipov <[email protected]>
Co-authored-by: Yaroslav Litvinov <[email protected]>
Co-authored-by: Denys Tsomenko <[email protected]>
Co-authored-by: Sergei Turukin <[email protected]>
Co-authored-by: DanCodedThis <[email protected]>
@rampage644 rampage644 removed this from embucket Jun 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Return affected rows count for dml statements

3 participants