Skip to content

[SPARK-12530][Build] Fix build break at Spark-Master-Maven-Snapshots from #1293 #10488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

[SPARK-12530][Build] Fix build break at Spark-Master-Maven-Snapshots from #1293 #10488

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,10 @@ case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInput
* the hash length is not one of the permitted values, the return value is NULL.
*/
@ExpressionDescription(
usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
"input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
"to 256",
usage =
"""_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the input.
SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256."""
,
Copy link
Contributor

@hvanhovell hvanhovell Dec 28, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit style. I guess this was needed to stay within 100 charaters?

Copy link
Member Author

@kiszk kiszk Dec 28, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this keeps within 100 characters at a line.

Copy link
Contributor

@vectorijk vectorijk Dec 28, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style below make newline display more reasonable.

s"""first line
  | second line
""".stripMargin

Copy link
Contributor

@hvanhovell hvanhovell Dec 28, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that work under 2.11?

Copy link
Contributor

@hvanhovell hvanhovell Dec 28, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have just tried this; it doesn't work unfortunately.

Copy link
Member Author

@kiszk kiszk Dec 28, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yhuai , the above line did not cause the same issue. This is because its length is less than 100 and it has only one concatenation.

I checked by inserting // scalastyle:off before the annotation and // scalastyle:on after the annotation as follows:

// scalastyle:off
@ExpressionDescription(
  usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
    "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256",
  extended = "> SELECT _FUNC_('Spark', 0);\n " +
    "'529bc3b07127ecb7e53a4dcf1991d9152c24537d919178022b2c42657f79a26b'")
// scalastyle:on

Copy link
Contributor

@yhuai yhuai Dec 28, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kiszk I see. How about we remove concatenation from both usage and extended? We can just use multi-line string.

Copy link
Member

@gatorsmile gatorsmile Dec 29, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use the multi-line string, we need to be careful when adding \n. This does not work. I already hit this issue when I do the fix in another related PR: #10418

Copy link
Member Author

@kiszk kiszk Dec 29, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yhuai, it sounds good to use multi-line string for usage and extended. How about this policy?

  1. First, we use raw string literal to remove concatenations.
  2. If we still want to use a line with more than 100 characters, // scalastyle:off and // scalastyle:on can be used.

Copy link
Member Author

@kiszk kiszk Dec 29, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In another way, I am trying to reproduce this compilation error in a standalone test case. However, I cannot reproduce this failure (i.e. compilations are successfully finished).

$ cat ExpressionDescription.java 
import java.lang.annotation.*;

@Retention(RetentionPolicy.RUNTIME)
public @interface ExpressionDescription {
    String usage() default "_FUNC_ is undocumented";
    String extended() default "No example for _FUNC_.";
}
$ cat test.scala
@ExpressionDescription(
  usage = "_FUNC_(input, bitLength) - Returns a checksum of SHA-2 family as a hex string of the " +
    "input. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent " +
    "to 256",
  extended = "> SELECT _FUNC_('Spark', 0);\n " +
    "'529bc3b07127ecb7e53a4dcf1991d9152c24537d919178022b2c42657f79a26b'")
class test {
  def func() : Unit = {
    val str = "abc" + "123" + "XYZ"
  }
}
$ javac ExpressionDescription.java 
$ ~/scala-2.11.7/bin/scalac test.scala 
$

extended = "> SELECT _FUNC_('Spark', 0);\n " +
"'529bc3b07127ecb7e53a4dcf1991d9152c24537d919178022b2c42657f79a26b'")
case class Sha2(left: Expression, right: Expression)
Expand Down