Respect externalIP setting in spark-submit for jar upload to NodePort service

[ash211]

For a use case @mccheah and I are building, I think we'll need to support NodePort's externalIp metadata. This comes in two places:

• when set the spark-submit script will place the given IP as the external IP on the NodePort service created on the driver (for jar uploads, webui, etc)
• when uploading local jars to the driver pod for execution, spark-submit will use the external IP if present on the service, rather than randomly selecting a pod IP and using that (otherwise default behavior)

A strawman name for the new flag would be spark.kubernetes.driver.externalIP

https://kubernetes.io/docs/tutorials/stateless-application/expose-external-ip-address/

[ash211]

After chatting with @mccheah, actually creating an Ingress is more valuable for our workflows than the external IP setting is. So I'll close this issue and open a separate one for Ingress support.

If anyone needs externalIP support in the NodePort proxy, please ping here and let's reopen to discuss.