Python SQL Example Code #2521

jyotiska · 2014-09-24T16:30:47Z

SQL example code for Python, as shown on SQL Programming Guide

SparkQA · 2014-09-24T16:34:22Z

QA tests have started for PR 2521 at commit 0b46148.

This patch merges cleanly.

SparkQA · 2014-09-24T16:35:21Z

QA tests have finished for PR 2521 at commit 0b46148.

This patch fails unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2014-09-24T16:35:22Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20754/

SparkQA · 2014-09-24T16:49:23Z

QA tests have started for PR 2521 at commit c90502a.

This patch merges cleanly.

SparkQA · 2014-09-24T16:50:25Z

QA tests have finished for PR 2521 at commit c90502a.

This patch fails unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2014-09-24T16:50:26Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20756/

SparkQA · 2014-09-24T16:59:22Z

QA tests have started for PR 2521 at commit 306667e.

This patch merges cleanly.

davies · 2014-09-24T17:41:18Z

examples/src/main/python/sql.py

This assume that this script will be run at SPARK_HOME, it will be broken if user run it at SPARK_HOME/examples/src/main/python

In that case, should the JSON file be supplied as sys.argv[1]?

How about

path = os.environ['SPARK_HOME'] + "examples/src/main/resources/people.json"

davies · 2014-09-24T17:45:05Z

This example only demonstrate jsonFile(), it will more powerful if it could have some usage of inferSchema() and applySchema().

SparkQA · 2014-09-24T18:07:17Z

QA tests have finished for PR 2521 at commit 306667e.

This patch passes unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2014-09-24T18:07:21Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20757/

SparkQA · 2014-09-25T07:19:29Z

QA tests have started for PR 2521 at commit 83d5ab7.

This patch merges cleanly.

SparkQA · 2014-09-25T07:20:33Z

QA tests have finished for PR 2521 at commit 83d5ab7.

This patch fails unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-09-25T07:20:34Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20783/

SparkQA · 2014-09-25T07:34:21Z

QA tests have started for PR 2521 at commit 43fd10a.

This patch merges cleanly.

SparkQA · 2014-09-25T07:35:22Z

QA tests have finished for PR 2521 at commit 43fd10a.

This patch fails unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-09-25T07:35:23Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20785/

AmplabJenkins · 2014-09-25T07:37:21Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20784/

SparkQA · 2014-09-25T08:54:23Z

QA tests have started for PR 2521 at commit b25e436.

This patch merges cleanly.

SparkQA · 2014-09-25T09:46:18Z

QA tests have finished for PR 2521 at commit b25e436.

This patch fails unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-09-25T09:46:21Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20796/

jyotiska · 2014-09-25T10:30:15Z

Jenkins, retest this please.

SparkQA · 2014-09-25T10:34:29Z

QA tests have started for PR 2521 at commit b25e436.

This patch merges cleanly.

SparkQA · 2014-09-25T11:43:06Z

QA tests have finished for PR 2521 at commit b25e436.

This patch passes unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-09-25T11:43:09Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20798/

SparkQA · 2014-09-25T11:49:22Z

QA tests have started for PR 2521 at commit 1471dcb.

This patch merges cleanly.

SparkQA · 2014-09-25T12:41:13Z

QA tests have finished for PR 2521 at commit 1471dcb.

This patch fails unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-09-25T12:41:16Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20801/

jyotiska · 2014-09-25T12:45:11Z

Jenkins, retest this please.

SparkQA · 2014-09-25T12:49:21Z

QA tests have started for PR 2521 at commit 1471dcb.

This patch merges cleanly.

SparkQA · 2014-09-25T13:59:13Z

QA tests have finished for PR 2521 at commit 1471dcb.

This patch passes unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-09-25T13:59:17Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20806/

jyotiska · 2014-09-25T14:00:39Z

@davies I have made the changes you asked for. Can you verify this?

davies · 2014-10-01T20:42:33Z

@jyotiska Cool, look good to me, thanks!

marmbrus · 2014-10-01T20:53:12Z

Thanks! I've merged this to master.

jyotiska added 2 commits September 24, 2014 21:55

added python sql example

8f67b5b

fixed appname for python sql example

0b46148

jyotiska added 2 commits September 24, 2014 22:13

added new line at end for python style

4939a70

fixed new line

c90502a

replaced blank line with end line

306667e

davies reviewed Sep 24, 2014
View reviewed changes

added inferschema and applyschema to the demo

83d5ab7

jyotiska added 2 commits September 25, 2014 12:56

removed blank lines

b4fdf4e

lines broken to maintain 80 char limit

43fd10a

pep 8 compliance

b25e436

added imports for sql

1471dcb

asfgit closed this in 17333c7 Oct 1, 2014

Python SQL Example Code #2521

Python SQL Example Code #2521

Uh oh!

Conversation

jyotiska commented Sep 24, 2014

Uh oh!

SparkQA commented Sep 24, 2014

Uh oh!

SparkQA commented Sep 24, 2014

Uh oh!

SparkQA commented Sep 24, 2014

Uh oh!

SparkQA commented Sep 24, 2014

Uh oh!

SparkQA commented Sep 24, 2014

Uh oh!

SparkQA commented Sep 24, 2014

Uh oh!

SparkQA commented Sep 24, 2014

Uh oh!

davies Sep 24, 2014

Choose a reason for hiding this comment

Uh oh!

jyotiska Sep 24, 2014

Choose a reason for hiding this comment

Uh oh!

davies Sep 24, 2014

Choose a reason for hiding this comment

Uh oh!

davies commented Sep 24, 2014

Uh oh!

SparkQA commented Sep 24, 2014

Uh oh!

SparkQA commented Sep 24, 2014

Uh oh!

SparkQA commented Sep 25, 2014

Uh oh!

SparkQA commented Sep 25, 2014

Uh oh!

AmplabJenkins commented Sep 25, 2014

Uh oh!

SparkQA commented Sep 25, 2014

Uh oh!

SparkQA commented Sep 25, 2014

Uh oh!

AmplabJenkins commented Sep 25, 2014

Uh oh!

AmplabJenkins commented Sep 25, 2014

Uh oh!

SparkQA commented Sep 25, 2014

Uh oh!

SparkQA commented Sep 25, 2014

Uh oh!

AmplabJenkins commented Sep 25, 2014

Uh oh!

jyotiska commented Sep 25, 2014

Uh oh!

SparkQA commented Sep 25, 2014

Uh oh!

SparkQA commented Sep 25, 2014

Uh oh!

AmplabJenkins commented Sep 25, 2014

Uh oh!

SparkQA commented Sep 25, 2014

Uh oh!

SparkQA commented Sep 25, 2014

Uh oh!

AmplabJenkins commented Sep 25, 2014

Uh oh!

jyotiska commented Sep 25, 2014

Uh oh!

SparkQA commented Sep 25, 2014

Uh oh!

SparkQA commented Sep 25, 2014

Uh oh!

AmplabJenkins commented Sep 25, 2014

Uh oh!

jyotiska commented Sep 25, 2014

Uh oh!

davies commented Oct 1, 2014