Skip to content

[SPARK-3412][SQL]add missing row api #2529

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 5 commits into from
Closed

[SPARK-3412][SQL]add missing row api #2529

wants to merge 5 commits into from

Conversation

@adrian-wang
Copy link
Contributor

@adrian-wang adrian-wang commented Sep 25, 2014

@chenghao-intel assigned this to me, check PR #2284 for previous discussion

@SparkQA
Copy link

SparkQA commented Sep 25, 2014

QA tests have started for PR 2529 at commit 1614493.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 25, 2014

QA tests have finished for PR 2529 at commit 1614493.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Sep 25, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20776/

marmbrus
marmbrus reviewed Sep 26, 2014
View reviewed changes
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Row.scala Outdated
Copy link
Contributor

@marmbrus marmbrus Sep 26, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there is any reason to have this function. The update call works independent of type already.

@marmbrus
Copy link
Contributor

marmbrus commented Sep 26, 2014

Thanks for working on this! A few minor comments.

@SparkQA
Copy link

SparkQA commented Sep 28, 2014

QA tests have started for PR 2529 at commit 4c18c29.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 28, 2014

Tests timed out after a configured wait of 120m.

@AmplabJenkins
Copy link

AmplabJenkins commented Sep 28, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20934/

@adrian-wang
Copy link
Contributor Author

adrian-wang commented Sep 30, 2014

retest this please.

@AmplabJenkins
Copy link

AmplabJenkins commented Sep 30, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21021/

@SparkQA
Copy link

SparkQA commented Sep 30, 2014

QA tests have started for PR 2529 at commit 4c18c29.

  • This patch merges cleanly.

@adrian-wang
Copy link
Contributor Author

adrian-wang commented Sep 30, 2014

retest this please.

@AmplabJenkins
Copy link

AmplabJenkins commented Sep 30, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21023/

@SparkQA
Copy link

SparkQA commented Sep 30, 2014

QA tests have started for PR 2529 at commit 4c18c29.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 30, 2014

QA tests have finished for PR 2529 at commit 4c18c29.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 30, 2014

QA tests have finished for PR 2529 at commit 4c18c29.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

marmbrus
marmbrus reviewed Oct 1, 2014
View reviewed changes
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SpecificRow.scala Outdated
Copy link
Contributor

@marmbrus marmbrus Oct 1, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't want the first .asInstanceOfhere as it can cause errors.

scala> new SpecificMutableRow(IntegerType :: Nil).getAs[Int](0)
java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.MutableInt cannot be cast to org.apache.spark.sql.catalyst.expressions.MutableAny

Since all MutableValue classes have a .value method I think you can just leave it out.

Copy link
Contributor

@marmbrus marmbrus Oct 1, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, looks like this file has always been named wrong. Would you mind fixing that (to SpecificMutableRow.scala) while you are at it?

@marmbrus
Copy link
Contributor

marmbrus commented Oct 1, 2014

Only one small change then this LGTM.

@SparkQA
Copy link

SparkQA commented Oct 8, 2014

QA tests have started for PR 2529 at commit 7b7e6e3.

  • This patch merges cleanly.

@AmplabJenkins
Copy link

AmplabJenkins commented Oct 8, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21448/Test FAILed.

@adrian-wang
Copy link
Contributor Author

adrian-wang commented Oct 8, 2014

retest this please.

@SparkQA
Copy link

SparkQA commented Oct 8, 2014

QA tests have started for PR 2529 at commit 7b7e6e3.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Oct 8, 2014

QA tests have finished for PR 2529 at commit 7b7e6e3.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Oct 8, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21449/Test PASSed.

@SparkQA
Copy link

SparkQA commented Oct 8, 2014

QA tests have finished for PR 2529 at commit 7b7e6e3.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Oct 8, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21451/Test FAILed.

@adrian-wang
Copy link
Contributor Author

adrian-wang commented Oct 8, 2014

retest this please.

@SparkQA
Copy link

SparkQA commented Oct 8, 2014

QA tests have started for PR 2529 at commit 7b7e6e3.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Oct 8, 2014

QA tests have finished for PR 2529 at commit 7b7e6e3.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Oct 8, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21453/Test PASSed.

marmbrus
marmbrus reviewed Oct 9, 2014
View reviewed changes
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SpecificMutableRow.scala Outdated
Copy link
Contributor

@marmbrus marmbrus Oct 9, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't this just be override def getAs[T](i: Int): T = values(i).boxed.asInstanceOf[T]?

Copy link
Contributor

@marmbrus marmbrus Oct 9, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I'm sorry for going back again. I should have said boxed instead of value in my previous comment.

Copy link
Contributor Author

@adrian-wang adrian-wang Oct 9, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for miss that, I didn't notice .boxed method there...

@SparkQA
Copy link

SparkQA commented Oct 9, 2014

QA tests have started for PR 2529 at commit c6594b2.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Oct 9, 2014

QA tests have finished for PR 2529 at commit c6594b2.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Oct 9, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21510/Test PASSed.

@marmbrus
Copy link
Contributor

marmbrus commented Oct 9, 2014

Thanks! Merged to master.

@asfgit asfgit closed this in 0c0e09f Oct 9, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@adrian-wang @SparkQA @AmplabJenkins @marmbrus