Skip to content

Commit 80a824d

Browse files
committed
[SPARK-12152][PROJECT-INFRA] Speed up Scalastyle checks by only invoking SBT once
Currently, `dev/scalastyle` invokes SBT four times, but these invocations can be replaced with a single invocation, saving about one minute of build time. Author: Josh Rosen <[email protected]> Closes #10151 from JoshRosen/speed-up-scalastyle.
1 parent 49efd03 commit 80a824d

File tree

1 file changed

+11
-8
lines changed

1 file changed

+11
-8
lines changed

dev/scalastyle

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -17,14 +17,17 @@
1717
# limitations under the License.
1818
#
1919

20-
echo -e "q\n" | build/sbt -Pkinesis-asl -Phive -Phive-thriftserver scalastyle > scalastyle.txt
21-
echo -e "q\n" | build/sbt -Pkinesis-asl -Phive -Phive-thriftserver test:scalastyle >> scalastyle.txt
22-
# Check style with YARN built too
23-
echo -e "q\n" | build/sbt -Pkinesis-asl -Pyarn -Phadoop-2.2 scalastyle >> scalastyle.txt
24-
echo -e "q\n" | build/sbt -Pkinesis-asl -Pyarn -Phadoop-2.2 test:scalastyle >> scalastyle.txt
25-
26-
ERRORS=$(cat scalastyle.txt | awk '{if($1~/error/)print}')
27-
rm scalastyle.txt
20+
# NOTE: echo "q" is needed because SBT prompts the user for input on encountering a build file
21+
# with failure (either resolution or compilation); the "q" makes SBT quit.
22+
ERRORS=$(echo -e "q\n" \
23+
| build/sbt \
24+
-Pkinesis-asl \
25+
-Pyarn \
26+
-Phive \
27+
-Phive-thriftserver \
28+
scalastyle test:scalastyle \
29+
| awk '{if($1~/error/)print}' \
30+
)
2831

2932
if test ! -z "$ERRORS"; then
3033
echo -e "Scalastyle checks failed at following occurrences:\n$ERRORS"

0 commit comments

Comments
 (0)