Spark DataFrame Viewer lets you inspect Spark DataFrames directly in PyCharm.
In the debugger, right-click a variable and select View Spark DataFrame to open the data in PyCharm's built-in DataFrame viewer.
The plugin works by executing your Spark query with a configurable row limit, converting the result to a pandas DataFrame using .toPandas(), and displaying it in PyCharm's DataFrame viewer.
-
Get it from the JetBrain Marketplace
-
Using the IDE built-in plugin system:
Settings/Preferences > Plugins > Marketplace > Search for "Spark DataFrame Viewer" > Install
-
Manually:
Download the latest release and install it manually using Settings/Preferences > Plugins > ⚙️ > Install plugin from disk...
This project is licensed under the MIT license.