Fixing AWS instance type information based upon current EC2 data #1156
Conversation
|
Can one of the admins verify this patch? |
|
Jenkins, test this please. |
|
Merged build triggered. |
|
Merged build started. |
|
According to the table, it seems like all of the types you changed support both hvm and pvm? |
|
Yes. But pvm has better performance than hvm. PS, hvm AMIs seem to have problems working along with Apache modules. |
|
Merged build finished. All automated tests passed. |
|
All automated tests passed. |
|
Okay - I think we should merge this into master and then backport it into branch-1.0. Except in the backport I'm not going to change the existing instance types from hvm-to-pvm, this could cause problems for users who are relying on the existing behavior. |
Fixed a problem in previous file in which some information regarding AWS instance types were wrong. Such information was updated base upon current AWS EC2 data. Author: Zichuan Ye <[email protected]> Closes #1156 from jerry86/master and squashes the following commits: ff36e95 [Zichuan Ye] Fixing AWS instance type information based upon current EC2 data (cherry picked from commit 62d4a0f) Conflicts: ec2/spark_ec2.py
|
@jerry86 actually it turns out PVM had some performance issues for PySpark that caused a regression during the 1.1 release testing. So I think we'll stick with HVM for at least some of these. |
During regression tests of Spark 1.1 we discovered perf issues with PVM instances when running PySpark. This reverts a change added in apache#1156 which changed the default type for m3 instances to PVM.
During regression tests of Spark 1.1 we discovered perf issues with PVM instances when running PySpark. This reverts a change added in #1156 which changed the default type for m3 instances to PVM. Author: Patrick Wendell <[email protected]> Closes #2244 from pwendell/ec2-hvm and squashes the following commits: 1342d7e [Patrick Wendell] SPARK-3358: [EC2] Switch back to HVM instances for m3.X.
During regression tests of Spark 1.1 we discovered perf issues with PVM instances when running PySpark. This reverts a change added in #1156 which changed the default type for m3 instances to PVM. Author: Patrick Wendell <[email protected]> Closes #2244 from pwendell/ec2-hvm and squashes the following commits: 1342d7e [Patrick Wendell] SPARK-3358: [EC2] Switch back to HVM instances for m3.X.
Fixed a problem in previous file in which some information regarding AWS instance types were wrong. Such information was updated base upon current AWS EC2 data. Author: Zichuan Ye <[email protected]> Closes apache#1156 from jerry86/master and squashes the following commits: ff36e95 [Zichuan Ye] Fixing AWS instance type information based upon current EC2 data
During regression tests of Spark 1.1 we discovered perf issues with PVM instances when running PySpark. This reverts a change added in apache#1156 which changed the default type for m3 instances to PVM. Author: Patrick Wendell <[email protected]> Closes apache#2244 from pwendell/ec2-hvm and squashes the following commits: 1342d7e [Patrick Wendell] SPARK-3358: [EC2] Switch back to HVM instances for m3.X.
|
@pwendell I think this seems to be the best choice at this moment. Not sure if Amazon has made or will make any improvements to PVM that fits PySpark. |
3 participants
Fixed a problem in previous file in which some information regarding AWS instance types were wrong. Such information was updated base upon current AWS EC2 data.