Skip to content

[SPARK-2358][MLLIB] Add an option to include native BLAS/LAPACK loader in the build #1295

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[SPARK-2358][MLLIB] Add an option to include native BLAS/LAPACK loader in the build #1295

wants to merge 1 commit into from
+13 −0

Conversation

@mengxr
Copy link
Contributor

@mengxr mengxr commented Jul 4, 2014

It would be easy for users to include the netlib-java jniloader in the spark jar, which is LGPL-licensed. We can follow the same approach as ganglia support in Spark, which could be enabled by turning on "-Pganglia-lgpl" at build time. We can use "-Pnetlib-lgpl" flag for this.

@AmplabJenkins
Copy link

AmplabJenkins commented Jul 4, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jul 4, 2014

Merged build started.

@AmplabJenkins
Copy link

AmplabJenkins commented Jul 4, 2014

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

AmplabJenkins commented Jul 4, 2014

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16325/

@mengxr
Copy link
Contributor Author

mengxr commented Jul 10, 2014

Updated PR to use profile only. User needs to enable -Pnetlib-lgpl at build time to include the jni loaders.

@AmplabJenkins
Copy link

AmplabJenkins commented Jul 10, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jul 10, 2014

Merged build started.

@AmplabJenkins
Copy link

AmplabJenkins commented Jul 10, 2014

Merged build finished.

@AmplabJenkins
Copy link

AmplabJenkins commented Jul 10, 2014

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16515/

@mengxr
Copy link
Contributor Author

mengxr commented Jul 10, 2014

Jenkins, retest this please.

@AmplabJenkins
Copy link

AmplabJenkins commented Jul 10, 2014

Merged build triggered.

@AmplabJenkins
Copy link

AmplabJenkins commented Jul 10, 2014

Merged build started.

@SparkQA
Copy link

SparkQA commented Jul 10, 2014

QA tests have started for PR 1295. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16519/consoleFull

@SparkQA
Copy link

SparkQA commented Jul 10, 2014

QA results for PR 1295:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16519/consoleFull

@AmplabJenkins
Copy link

AmplabJenkins commented Jul 10, 2014

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16519/

@SparkQA
Copy link

SparkQA commented Jul 11, 2014

QA tests have started for PR 1295. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16540/consoleFull

@SparkQA
Copy link

SparkQA commented Jul 11, 2014

QA results for PR 1295:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16540/consoleFull

@rxin
Copy link
Contributor

rxin commented Jul 11, 2014

Merging in master. Thanks!

@asfgit asfgit closed this in 2f59ce7 Jul 11, 2014
gzm55 pushed a commit to MediaV/spark that referenced this pull request Jul 18, 2014
@mengxr
[SPARK-2358][MLLIB] Add an option to include native BLAS/LAPACK loade…
5e1a3fa 
…r in the build

It would be easy for users to include the netlib-java jniloader in the spark jar, which is LGPL-licensed. We can follow the same approach as ganglia support in Spark, which could be enabled by turning on "-Pganglia-lgpl" at build time. We can use "-Pnetlib-lgpl" flag for this.

Author: Xiangrui Meng <[email protected]>

Closes apache#1295 from mengxr/netlib-lgpl and squashes the following commits:

aebf001 [Xiangrui Meng] add a profile to optionally include native BLAS/LAPACK loader in mllib
@JoshRosen
Copy link
Contributor

JoshRosen commented Aug 12, 2014

It would be great to mention this option in the MLlib documentation.

xiliu82 pushed a commit to xiliu82/spark that referenced this pull request Sep 4, 2014
@mengxr @conviva-zz
[SPARK-2358][MLLIB] Add an option to include native BLAS/LAPACK loade…
c6e7327 
…r in the build

It would be easy for users to include the netlib-java jniloader in the spark jar, which is LGPL-licensed. We can follow the same approach as ganglia support in Spark, which could be enabled by turning on "-Pganglia-lgpl" at build time. We can use "-Pnetlib-lgpl" flag for this.

Author: Xiangrui Meng <[email protected]>

Closes apache#1295 from mengxr/netlib-lgpl and squashes the following commits:

aebf001 [Xiangrui Meng] add a profile to optionally include native BLAS/LAPACK loader in mllib
wangyum pushed a commit that referenced this pull request May 26, 2023
@xingchaozh
[CARMEL-6633] Reduce Skew Join Split Size Considering Expand Node (#1295
2973a31 
)

* [CARMEL-6633] Reduce Skew Join Split Size Considering Expand Node

* Add UT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@mengxr @AmplabJenkins @SparkQA @rxin @JoshRosen