You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-41989][PYTHON] Avoid breaking logging config from pyspark.pandas
### What changes were proposed in this pull request?
See https://issues.apache.org/jira/browse/SPARK-41989 for in depth explanation
Short summary: `pyspark/pandas/__init__.py` uses, at import time, `logging.warning()` which might silently call `logging.basicConfig()`.
So by importing `pyspark.pandas` (directly or indirectly) a user might unknowingly break their own logging setup (e.g. when based on `logging.basicConfig()` or related). `logging.getLogger(...).warning()` does not trigger this behavior.
### Does this PR introduce _any_ user-facing change?
User-defined logging setups will be more predictable.
### How was this patch tested?
Manual testing so far.
I'm not sure it's worthwhile to cover this with a unit test
Closes#39516 from soxofaan/SPARK-41989-pyspark-pandas-logging-setup.
Authored-by: Stefaan Lippens <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
0 commit comments