Skip to content

Convert NULL type columns to Julia Missing type #237

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

TacHawkes
Copy link

At the moment columns which are NULL type are returned as Julia type "Any" which is suboptimal for filtering missing values when reading values iteratively.
With this PR a NULL type column is handled as a Missing type in Julia.

At the moment columns which are NULL type are returned as Julia type "Any" which is suboptimal for filtering missing values when reading values iteratively.
With this PR a NULL type column is handled as a Missing type in Julia.
Copy link
Member

@quinnj quinnj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @TacHawkes!

@quinnj
Copy link
Member

quinnj commented Mar 25, 2021

It looks like one of the tests is failing; would you mind taking a look? https://github.com/JuliaDatabases/SQLite.jl/blob/master/test/runtests.jl#L107

@TacHawkes
Copy link
Author

It looks like one of the tests is failing; would you mind taking a look? https://github.com/JuliaDatabases/SQLite.jl/blob/master/test/runtests.jl#L107

Sure, the problem is the testcase itself in my opinion. Without the PR, the test "empty query has correct schema and return type:" finds the following types for all_tbl and empty_tbl and the corresponding type comparison:

empty_tbl type, all_tbl type
Union{Missing, Int64}, Int64 -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, Int64}, Union{Missing, Int64} -> true
Any, String -> true
Any, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true

With the pull request:

empty_tbl type, all_tbl type
Union{Missing, Int64}, Int64 -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, Int64}, Union{Missing, Int64} -> true
Missing, String -> false
Missing, String -> false
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true
Union{Missing, String}, String -> true

As Missing >: String is false. Simple solution (but valid!?):

@test all(col -> eltype(empty_tbl[col]) == Missing || eltype(empty_tbl[col]) >: eltype(all_tbl[col]), propertynames(all_tbl))

What do you think?

@quinnj
Copy link
Member

quinnj commented Mar 25, 2021

Yeah, that sounds better to me!

@quinnj
Copy link
Member

quinnj commented Oct 20, 2021

Put the fix in a PR w/ the test fix here: #261

@quinnj quinnj closed this Oct 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants