Skip to content

Commit b51f058

Browse files
shivaramRobert Kruszewski
authored andcommitted
[SPARKR][PYSPARK] Fix R source package name to match Spark version. Remove pip tar.gz from distribution
## What changes were proposed in this pull request? Fixes name of R source package so that the `cp` in release-build.sh works correctly. Issue discussed in apache#16014 (comment) Author: Shivaram Venkataraman <[email protected]> Closes apache#16221 from shivaram/fix-sparkr-release-build-name.
1 parent e0d924f commit b51f058

File tree

1 file changed

+9
-0
lines changed

1 file changed

+9
-0
lines changed

dev/make-distribution.sh

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -222,11 +222,14 @@ fi
222222
# Make R package - this is used for both CRAN release and packing R layout into distribution
223223
if [ "$MAKE_R" == "true" ]; then
224224
echo "Building R source package"
225+
R_PACKAGE_VERSION=`grep Version $SPARK_HOME/R/pkg/DESCRIPTION | awk '{print $NF}'`
225226
pushd "$SPARK_HOME/R" > /dev/null
226227
# Build source package and run full checks
227228
# Install source package to get it to generate vignettes, etc.
228229
# Do not source the check-cran.sh - it should be run from where it is for it to set SPARK_HOME
229230
NO_TESTS=1 CLEAN_INSTALL=1 "$SPARK_HOME/"R/check-cran.sh
231+
# Make a copy of R source package matching the Spark release version.
232+
cp $SPARK_HOME/R/SparkR_"$R_PACKAGE_VERSION".tar.gz $SPARK_HOME/R/SparkR_"$VERSION".tar.gz
230233
popd > /dev/null
231234
else
232235
echo "Skipping building R source package"
@@ -238,6 +241,12 @@ cp "$SPARK_HOME"/conf/*.template "$DISTDIR"/conf
238241
cp "$SPARK_HOME/README.md" "$DISTDIR"
239242
cp -r "$SPARK_HOME/bin" "$DISTDIR"
240243
cp -r "$SPARK_HOME/python" "$DISTDIR"
244+
245+
# Remove the python distribution from dist/ if we built it
246+
if [ "$MAKE_PIP" == "true" ]; then
247+
rm -f $DISTDIR/python/dist/pyspark-*.tar.gz
248+
fi
249+
241250
cp -r "$SPARK_HOME/sbin" "$DISTDIR"
242251
# Copy SparkR if it exists
243252
if [ -d "$SPARK_HOME"/R/lib/SparkR ]; then

0 commit comments

Comments
 (0)