Skip to content

[SPARK-15820][PySpark][SQL]Add Catalog.refreshTable into python API #13558

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 2 commits into from
Closed

[SPARK-15820][PySpark][SQL]Add Catalog.refreshTable into python API #13558

wants to merge 2 commits into from

Conversation

@WeichenXu123
Copy link
Contributor

@WeichenXu123 WeichenXu123 commented Jun 8, 2016

What changes were proposed in this pull request?

Add Catalog.refreshTable API into python interface for Spark-SQL.

How was this patch tested?

Existing test.

@WeichenXu123 WeichenXu123 changed the title [SPARK-15820][pyspark][SQL] update python sql interface refreshTable [SPARK-15820][PySpark][SQL]Add Catalog.refreshTable into python API Jun 8, 2016
@SparkQA
Copy link

SparkQA commented Jun 8, 2016

Test build #60159 has finished for PR 13558 at commit eedb961.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WeichenXu123
Copy link
Contributor Author

WeichenXu123 commented Jun 8, 2016

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Jun 8, 2016

Test build #60162 has finished for PR 13558 at commit eedb961.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WeichenXu123
Copy link
Contributor Author

WeichenXu123 commented Jun 8, 2016

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Jun 8, 2016

Test build #60169 has finished for PR 13558 at commit eedb961.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WeichenXu123
Copy link
Contributor Author

WeichenXu123 commented Jun 8, 2016

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Jun 8, 2016

Test build #60173 has finished for PR 13558 at commit eedb961.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WeichenXu123
Copy link
Contributor Author

WeichenXu123 commented Jun 8, 2016

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Jun 8, 2016

Test build #60179 has finished for PR 13558 at commit eedb961.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WeichenXu123
Copy link
Contributor Author

WeichenXu123 commented Jun 9, 2016

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Jun 9, 2016

Test build #60220 has finished for PR 13558 at commit eedb961.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WeichenXu123 WeichenXu123 force-pushed the update_python_sql_interface_refreshTable branch from eedb961 to 8d87c0f Compare June 9, 2016 07:16
@SparkQA
Copy link

SparkQA commented Jun 9, 2016

Test build #60226 has finished for PR 13558 at commit 8d87c0f.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WeichenXu123 WeichenXu123 mentioned this pull request Jun 9, 2016
Closed
@WeichenXu123
Copy link
Contributor Author

WeichenXu123 commented Jun 10, 2016

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Jun 10, 2016

Test build #60266 has finished for PR 13558 at commit 8d87c0f.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WeichenXu123
Copy link
Contributor Author

WeichenXu123 commented Jun 13, 2016

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Jun 13, 2016

Test build #60378 has finished for PR 13558 at commit 8d87c0f.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@andrewor14
Copy link
Contributor

andrewor14 commented Jun 13, 2016

Looks good. Jenks retest this please

@SparkQA
Copy link

SparkQA commented Jun 13, 2016

Test build #60446 has finished for PR 13558 at commit 8d87c0f.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WeichenXu123
Copy link
Contributor Author

WeichenXu123 commented Jun 14, 2016

@andrewor14 It looks strange, I test on my own machine and it is all OK. If it is the test server's problem?

@MLnick
Copy link
Contributor

MLnick commented Jun 17, 2016

jenkins retest this please

@SparkQA
Copy link

SparkQA commented Jun 17, 2016

Test build #60701 has finished for PR 13558 at commit 8d87c0f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

MLnick
MLnick reviewed Jun 17, 2016
View reviewed changes
python/pyspark/sql/catalog.py Outdated

@since(2.0)
def refreshTable(self, tableName):
"""Invalidate and refresh all the cached the metadata of the given table."""
Copy link
Contributor

@MLnick MLnick Jun 17, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an extra "the" in this comment, should be refresh all the cached metadata

Copy link
Contributor Author

@WeichenXu123 WeichenXu123 Jun 17, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MLnick Done. I also correct the comment with extra "the" in the scala code. Thanks!

@MLnick MLnick mentioned this pull request Jun 17, 2016
Closed
@SparkQA
Copy link

SparkQA commented Jun 17, 2016

Test build #60705 has finished for PR 13558 at commit 87fcb7c.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WeichenXu123
Copy link
Contributor Author

WeichenXu123 commented Jun 19, 2016

@andrewor14 The PR is OK now.

@WeichenXu123
Copy link
Contributor Author

WeichenXu123 commented Jun 27, 2016

cc @liancheng Thanks!

@liancheng
Copy link
Contributor

liancheng commented Jun 30, 2016

LGTM, merging to master and branch-2.0.

@asfgit asfgit closed this in 5344bad Jun 30, 2016
asfgit pushed a commit that referenced this pull request Jul 1, 2016
@WeichenXu123 @liancheng
[SPARK-15820][PYSPARK][SQL] Add Catalog.refreshTable into python API
80a7bff 
## What changes were proposed in this pull request?

Add Catalog.refreshTable API into python interface for Spark-SQL.

## How was this patch tested?

Existing test.

Author: WeichenXu <[email protected]>

Closes #13558 from WeichenXu123/update_python_sql_interface_refreshTable.

(cherry picked from commit 5344bad)
Signed-off-by: Cheng Lian <[email protected]>
@WeichenXu123 WeichenXu123 deleted the update_python_sql_interface_refreshTable branch July 31, 2016 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@WeichenXu123 @SparkQA @andrewor14 @MLnick @liancheng