Skip to content

Conversation

@trueleo
Copy link
Contributor

@trueleo trueleo commented Dec 14, 2023

Fixes #572.

Description

While transforming logical plan to add external time field, the column name that was provided is unqualified ( without table reference ).
this caused mismatch in column name in datafusion query runtime.

Error during planning: required columns can't push down, columns: {Column { relation: Some(Bare { table: "integrity" }), name: "p_timestamp" }, Column { relation: None, name: "p_timestamp" }}

In this PR the filter expr start_time_filter and end_time_filter for logical plan is created with table reference so that push down predicate can work properly.


This PR has:

  • been tested to ensure log ingestion and log query works.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added documentation for new or modified features or behaviors.

Copy link
Member

@nitisht nitisht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add few queries in Quest so we catch such failures later.

@nitisht nitisht requested a review from theteachr December 14, 2023 09:01
@trueleo
Copy link
Contributor Author

trueleo commented Dec 14, 2023

Closes #572

@nitisht
Copy link
Member

nitisht commented Dec 15, 2023

Let's add few queries in Quest so we catch such failures later.

parseablehq/quest#48

Copy link
Member

@nitisht nitisht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM and tested locally

@nitisht nitisht merged commit 8b16014 into parseablehq:main Dec 15, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Dec 15, 2023
@trueleo trueleo deleted the add-relation branch December 19, 2023 06:30
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Query execution fails for any query that uses p_timestamp

2 participants