Skip to content

Commit 8c087f1

Browse files
committed
[SPARK-54065][PYTHON][TESTS] Enable test_in_memory_data_source in Python 3.14
### What changes were proposed in this pull request? This PR aims to enable `test_in_memory_data_source` in Python 3.14. ### Why are the changes needed? After upgrading to the latest `cloudpickle` 3.1.2, I verified that the test passed on Python 3.14.0. - #52964 ``` $ python/run-tests --parallelism 1 --testnames pyspark.sql.tests.test_python_datasource --python-executables python3 Running PySpark tests. Output is in /Users/dongjoon/APACHE/spark-merge/python/unit-tests.log Will test against the following Python executables: ['python3'] Will test the following Python tests: ['pyspark.sql.tests.test_python_datasource'] python3 python_implementation is CPython python3 version is: Python 3.14.0 Starting test(python3): pyspark.sql.tests.test_python_datasource (temp output: /Users/dongjoon/APACHE/spark-merge/python/target/e771105c-5304-4f4f-bea6-b8d896b1b250/python3__pyspark.sql.tests.test_python_datasource__2wyrv5n4.log) Finished test(python3): pyspark.sql.tests.test_python_datasource (29s) Tests passed in 29 seconds ``` ### Does this PR introduce _any_ user-facing change? No. This is a test change. ### How was this patch tested? Pass the CIs and manual tests. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #52967 from dongjoon-hyun/SPARK-54065. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 79d0c42) Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 0fb91ff commit 8c087f1

File tree

1 file changed

+0
-1
lines changed

1 file changed

+0
-1
lines changed

python/pyspark/sql/tests/test_python_datasource.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -258,7 +258,6 @@ def test_read_with_invalid_return_row_type(self):
258258
with self.assertRaisesRegex(PythonException, "DATA_SOURCE_INVALID_RETURN_TYPE"):
259259
df.collect()
260260

261-
@unittest.skipIf(sys.version_info > (3, 13), "SPARK-54065")
262261
def test_in_memory_data_source(self):
263262
class InMemDataSourceReader(DataSourceReader):
264263
DEFAULT_NUM_PARTITIONS: int = 3

0 commit comments

Comments
 (0)