Skip to content

[SPARK-23046][ML][SparkR] Have RFormula include VectorSizeHint in pipeline #20238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 2 commits into from
Closed

[SPARK-23046][ML][SparkR] Have RFormula include VectorSizeHint in pipeline #20238

wants to merge 2 commits into from

Conversation

@MrBago
Copy link
Contributor

@MrBago MrBago commented Jan 11, 2018

What changes were proposed in this pull request?

Including VectorSizeHint in RFormula piplelines will allow them to be applied to streaming dataframes.

How was this patch tested?

Unit tests.

@MrBago MrBago changed the title Have RFormula include VectorSizeHint in pipeline [SPARK-23046][ML] Have RFormula include VectorSizeHint in pipeline Jan 11, 2018
@MrBago MrBago changed the title [SPARK-23046][ML] Have RFormula include VectorSizeHint in pipeline [SPARK-23046][ML][SparkR] Have RFormula include VectorSizeHint in pipeline Jan 11, 2018
@SparkQA
Copy link

SparkQA commented Jan 11, 2018

Test build #85979 has finished for PR 20238 at commit 09f0d60.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jan 11, 2018

Test build #85981 has finished for PR 20238 at commit ff4d1cf.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WeichenXu123 WeichenXu123 mentioned this pull request Jan 11, 2018
Closed
@jkbradley
Copy link
Member

jkbradley commented Jan 11, 2018

Reviewing now

@jkbradley
Copy link
Member

jkbradley commented Jan 11, 2018

LGTM
I'm going to merge this with master and backport it to branch-2.3 since this fixes 1 of 2 bugs in RFormulaModel for use with Structured Streaming.

Thanks @MrBago !

WeichenXu123
WeichenXu123 approved these changes Jan 11, 2018
View reviewed changes
Copy link
Contributor

@WeichenXu123 WeichenXu123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. thanks!

asfgit pushed a commit that referenced this pull request Jan 11, 2018
@MrBago @jkbradley
[SPARK-23046][ML][SPARKR] Have RFormula include VectorSizeHint in pip…
2ec3026 
…eline

## What changes were proposed in this pull request?

Including VectorSizeHint in RFormula piplelines will allow them to be applied to streaming dataframes.

## How was this patch tested?

Unit tests.

Author: Bago Amirbekian <[email protected]>

Closes #20238 from MrBago/rFormulaVectorSize.

(cherry picked from commit 186bf8f)
Signed-off-by: Joseph K. Bradley <[email protected]>
@asfgit asfgit closed this in 186bf8f Jan 11, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@WeichenXu123 WeichenXu123 WeichenXu123 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@MrBago @SparkQA @jkbradley @WeichenXu123