[SPARK-7545][mllib] Added check in Bernoulli Naive Bayes to make sure that both training and predict features have values of 0 or 1 #6073
Conversation
…del to save them for user access
leahmcguire
changed the title
|
Can one of the admins verify this patch? |
There was a problem hiding this comment.
Fix Scala style for multi-line closure, and branch a little less:
if (!brzData.forall(v => v == 0.0 || v == 1.0)) {
throw new SparkException(
s"Bernoulli Naive Bayes requires feature values 0 or 1 but found feature vector $testData.")
}
|
@leahmcguire Thanks for the quick update! I think the 2nd comment is for a bug, but it should be an easy fix. |
|
add to whitelist |
|
test this please |
|
Merged build triggered. |
|
Merged build started. |
|
Test build #32530 has started for PR 6073 at commit |
|
Test build #32530 has finished for PR 6073 at commit
|
|
Merged build finished. Test FAILed. |
|
Test FAILed. |
There was a problem hiding this comment.
Could you please move the if (modelType == "Bernoulli") check here so that only one of these require() methods is called?
|
Looks good other than those 2 minor items. Thank you! |
|
Merged build triggered. |
|
Merged build started. |
|
Test build #32555 has started for PR 6073 at commit |
There was a problem hiding this comment.
This does need to be called for the Bernoulli model in the createCombiner. Can you please make this an if-else?
There was a problem hiding this comment.
Sorry! I meant "does not" but clearly you understood
|
Test build #32555 has finished for PR 6073 at commit
|
|
Merged build finished. Test PASSed. |
|
Test PASSed. |
|
Merged build triggered. |
|
Merged build started. |
|
Test build #32569 has started for PR 6073 at commit |
|
Test build #32569 has finished for PR 6073 at commit
|
|
Merged build finished. Test FAILed. |
|
Test FAILed. |
|
spurious failure, retesting |
|
Test build #804 has started for PR 6073 at commit |
|
LGTM pending tests |
|
Test build #804 has finished for PR 6073 at commit
|
|
Test build #805 has started for PR 6073 at commit |
|
Test build #805 has finished for PR 6073 at commit
|
|
Merging into master and branch-1.4 @leahmcguire Thank you! |
…e that both training and predict features have values of 0 or 1 Author: leahmcguire <[email protected]> Closes #6073 from leahmcguire/binaryCheckNB and squashes the following commits: b8442c2 [leahmcguire] changed to if else for value checks 911bf83 [leahmcguire] undid reformat 4eedf1e [leahmcguire] moved bernoulli check 9ee9e84 [leahmcguire] fixed style error 3f3b32c [leahmcguire] fixed zero one check so only called in combiner 831fd27 [leahmcguire] got test working f44bb3c [leahmcguire] removed changes from CV branch 67253f0 [leahmcguire] added check to bernoulli to ensure feature values are zero or one f191c71 [leahmcguire] fixed name 58d060b [leahmcguire] changed param name and test according to comments 04f0d3c [leahmcguire] Added stats from cross validation as a val in the cross validation model to save them for user access (cherry picked from commit 61e05fc) Signed-off-by: Joseph K. Bradley <[email protected]>
…e that both training and predict features have values of 0 or 1 Author: leahmcguire <[email protected]> Closes apache#6073 from leahmcguire/binaryCheckNB and squashes the following commits: b8442c2 [leahmcguire] changed to if else for value checks 911bf83 [leahmcguire] undid reformat 4eedf1e [leahmcguire] moved bernoulli check 9ee9e84 [leahmcguire] fixed style error 3f3b32c [leahmcguire] fixed zero one check so only called in combiner 831fd27 [leahmcguire] got test working f44bb3c [leahmcguire] removed changes from CV branch 67253f0 [leahmcguire] added check to bernoulli to ensure feature values are zero or one f191c71 [leahmcguire] fixed name 58d060b [leahmcguire] changed param name and test according to comments 04f0d3c [leahmcguire] Added stats from cross validation as a val in the cross validation model to save them for user access
…e that both training and predict features have values of 0 or 1 Author: leahmcguire <[email protected]> Closes apache#6073 from leahmcguire/binaryCheckNB and squashes the following commits: b8442c2 [leahmcguire] changed to if else for value checks 911bf83 [leahmcguire] undid reformat 4eedf1e [leahmcguire] moved bernoulli check 9ee9e84 [leahmcguire] fixed style error 3f3b32c [leahmcguire] fixed zero one check so only called in combiner 831fd27 [leahmcguire] got test working f44bb3c [leahmcguire] removed changes from CV branch 67253f0 [leahmcguire] added check to bernoulli to ensure feature values are zero or one f191c71 [leahmcguire] fixed name 58d060b [leahmcguire] changed param name and test according to comments 04f0d3c [leahmcguire] Added stats from cross validation as a val in the cross validation model to save them for user access
…e that both training and predict features have values of 0 or 1 Author: leahmcguire <[email protected]> Closes apache#6073 from leahmcguire/binaryCheckNB and squashes the following commits: b8442c2 [leahmcguire] changed to if else for value checks 911bf83 [leahmcguire] undid reformat 4eedf1e [leahmcguire] moved bernoulli check 9ee9e84 [leahmcguire] fixed style error 3f3b32c [leahmcguire] fixed zero one check so only called in combiner 831fd27 [leahmcguire] got test working f44bb3c [leahmcguire] removed changes from CV branch 67253f0 [leahmcguire] added check to bernoulli to ensure feature values are zero or one f191c71 [leahmcguire] fixed name 58d060b [leahmcguire] changed param name and test according to comments 04f0d3c [leahmcguire] Added stats from cross validation as a val in the cross validation model to save them for user access
5 participants
No description provided.