Skip to content

Commit 5344bad

Browse files
WeichenXu123liancheng
authored andcommitted
[SPARK-15820][PYSPARK][SQL] Add Catalog.refreshTable into python API
## What changes were proposed in this pull request? Add Catalog.refreshTable API into python interface for Spark-SQL. ## How was this patch tested? Existing test. Author: WeichenXu <[email protected]> Closes #13558 from WeichenXu123/update_python_sql_interface_refreshTable.
1 parent 5320adc commit 5344bad

File tree

2 files changed

+6
-1
lines changed

2 files changed

+6
-1
lines changed

python/pyspark/sql/catalog.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -232,6 +232,11 @@ def clearCache(self):
232232
"""Removes all cached tables from the in-memory cache."""
233233
self._jcatalog.clearCache()
234234

235+
@since(2.0)
236+
def refreshTable(self, tableName):
237+
"""Invalidate and refresh all the cached metadata of the given table."""
238+
self._jcatalog.refreshTable(tableName)
239+
235240
def _reset(self):
236241
"""(Internal use only) Drop all existing databases (except "default"), tables,
237242
partitions and functions, and set the current database to "default".

sql/core/src/main/scala/org/apache/spark/sql/catalog/Catalog.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -214,7 +214,7 @@ abstract class Catalog {
214214
def clearCache(): Unit
215215

216216
/**
217-
* Invalidate and refresh all the cached the metadata of the given table. For performance reasons,
217+
* Invalidate and refresh all the cached metadata of the given table. For performance reasons,
218218
* Spark SQL or the external data source library it uses might cache certain metadata about a
219219
* table, such as the location of blocks. When those change outside of Spark SQL, users should
220220
* call this function to invalidate the cache.

0 commit comments

Comments
 (0)