Skip to content

Conversation

@gatorsmile
Copy link
Member

When calling get_json_object for the following two cases, both results are "null":

    val tuple: Seq[(String, String)] = ("5", """{"f1": null}""") :: Nil
    val df: DataFrame = tuple.toDF("key", "jstring")
    val res = df.select(functions.get_json_object($"jstring", "$.f1")).collect()
    val tuple2: Seq[(String, String)] = ("5", """{"f1": "null"}""") :: Nil
    val df2: DataFrame = tuple2.toDF("key", "jstring")
    val res3 = df2.select(functions.get_json_object($"jstring", "$.f1")).collect()

Fixed the problem and also added a test case.

@SparkQA
Copy link

SparkQA commented Nov 27, 2015

Test build #46827 has finished for PR 10018 at commit 54edc84.

  • This patch fails from timeout after a configured wait of 250m.
  • This patch merges cleanly.
  • This patch adds no public classes.

@gatorsmile
Copy link
Member Author

retest this please

@SparkQA
Copy link

SparkQA commented Nov 28, 2015

Test build #46832 has finished for PR 10018 at commit 54edc84.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@gatorsmile
Copy link
Member Author

@rxin @davies Could you help me review the code changes for fixing get_json_object? The bug was discovered in the PR: #9977.

Thank you very much!

@davies
Copy link
Contributor

davies commented Nov 28, 2015

LGTM

asfgit pushed a commit that referenced this pull request Nov 28, 2015
…the value is null literals

When calling `get_json_object` for the following two cases, both results are `"null"`:

```scala
    val tuple: Seq[(String, String)] = ("5", """{"f1": null}""") :: Nil
    val df: DataFrame = tuple.toDF("key", "jstring")
    val res = df.select(functions.get_json_object($"jstring", "$.f1")).collect()
```
```scala
    val tuple2: Seq[(String, String)] = ("5", """{"f1": "null"}""") :: Nil
    val df2: DataFrame = tuple2.toDF("key", "jstring")
    val res3 = df2.select(functions.get_json_object($"jstring", "$.f1")).collect()
```

Fixed the problem and also added a test case.

Author: gatorsmile <[email protected]>

Closes #10018 from gatorsmile/get_json_object.

(cherry picked from commit 149cd69)
Signed-off-by: Davies Liu <[email protected]>
@asfgit asfgit closed this in 149cd69 Nov 28, 2015
@gatorsmile gatorsmile deleted the get_json_object branch December 5, 2015 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants