Skip to content

Spark 1532- provide option to specify allowed IPs for EC2 instance when creating EC2 cluster #453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

Spark 1532- provide option to specify allowed IPs for EC2 instance when creating EC2 cluster #453

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8112708
Provide option to set more restrictive Source in the security groups.
Apr 14, 2014
0bf9a9d
Merge branch 'access_mask_option' into spark-1532
Apr 19, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 16 additions & 14 deletions ec2/spark_ec2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,8 @@ def parse_args():
help="When destroying a cluster, delete the security groups that were created")
parser.add_option("--use-existing-master", action="store_true", default=False,
help="Launch fresh slaves, but use an existing stopped master if possible")
parser.add_option("--ips-allowed", default="0.0.0.0/0",
help="IP addresses allowed to access the machine")
Copy link
Contributor

@pwendell pwendell Jun 4, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the format for specifying multiple IP addresses (is it comma separated?). It would be good to put that format in this comment. I'd also mention that these are CIDR ranges and not individual addresses.

  parser.add_option("--ips-allowed", default="0.0.0.0/0",
      help="CIDR IP ranges allowed to access the machine. Multiple ranges can be separated with commas.")

(this example is probably not PEP8 complaint :P, it would be good to see what the right way to break the line is)


(opts, args) = parser.parse_args()
if len(args) != 2:
Expand Down Expand Up @@ -230,24 +232,24 @@ def launch_cluster(conn, opts, cluster_name):
if master_group.rules == []: # Group was just now created
master_group.authorize(src_group=master_group)
master_group.authorize(src_group=slave_group)
master_group.authorize('tcp', 22, 22, '0.0.0.0/0')
master_group.authorize('tcp', 8080, 8081, '0.0.0.0/0')
master_group.authorize('tcp', 19999, 19999, '0.0.0.0/0')
master_group.authorize('tcp', 50030, 50030, '0.0.0.0/0')
master_group.authorize('tcp', 50070, 50070, '0.0.0.0/0')
master_group.authorize('tcp', 60070, 60070, '0.0.0.0/0')
master_group.authorize('tcp', 4040, 4045, '0.0.0.0/0')
master_group.authorize('tcp', 22, 22, opts.ips_allowed)
master_group.authorize('tcp', 8080, 8081, opts.ips_allowed)
master_group.authorize('tcp', 19999, 19999, opts.ips_allowed)
master_group.authorize('tcp', 50030, 50030, opts.ips_allowed)
master_group.authorize('tcp', 50070, 50070, opts.ips_allowed)
master_group.authorize('tcp', 60070, 60070, opts.ips_allowed)
master_group.authorize('tcp', 4040, 4045, opts.ips_allowed)
if opts.ganglia:
master_group.authorize('tcp', 5080, 5080, '0.0.0.0/0')
master_group.authorize('tcp', 5080, 5080, opts.ips_allowed)
if slave_group.rules == []: # Group was just now created
slave_group.authorize(src_group=master_group)
slave_group.authorize(src_group=slave_group)
slave_group.authorize('tcp', 22, 22, '0.0.0.0/0')
slave_group.authorize('tcp', 8080, 8081, '0.0.0.0/0')
slave_group.authorize('tcp', 50060, 50060, '0.0.0.0/0')
slave_group.authorize('tcp', 50075, 50075, '0.0.0.0/0')
slave_group.authorize('tcp', 60060, 60060, '0.0.0.0/0')
slave_group.authorize('tcp', 60075, 60075, '0.0.0.0/0')
slave_group.authorize('tcp', 22, 22, opts.ips_allowed)
slave_group.authorize('tcp', 8080, 8081, opts.ips_allowed)
slave_group.authorize('tcp', 50060, 50060, opts.ips_allowed)
slave_group.authorize('tcp', 50075, 50075, opts.ips_allowed)
slave_group.authorize('tcp', 60060, 60060, opts.ips_allowed)
slave_group.authorize('tcp', 60075, 60075, opts.ips_allowed)

# Check if instances are already running in our groups
existing_masters, existing_slaves = get_existing_cluster(conn, opts, cluster_name,
Expand Down