Fixing handling of pivot #105
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request refactors the location of the
calculate_fuzzy_match_schema
andpre_calculate_pivot_schema
functions, updates the pivot schema logic, and improves test coverage for pivot-related functionality. The main changes involve moving schema-related logic to a more appropriate module, refining how pivot output columns are named, and adding new tests to ensure correct schema calculation.Refactoring and code organization:
calculate_fuzzy_match_schema
andpre_calculate_pivot_schema
fromflow_data_engine/fuzzy_matching/settings_validator.py
to a newschema_callbacks.py
module for better separation of concerns and maintainability.Pivot schema logic improvements:
pre_calculate_pivot_schema
to generate simpler column names (e.g.,'0'
,'1'
, etc.) when only one aggregation is used, instead of appending the aggregation name (e.g.,'0_sum'
).Testing enhancements:
test_pivot_schema_callback
andtest_schema_callback_in_graph
to verify correct schema calculation for pivot operations and ensure the new schema logic is functioning as intended.