Spark with native support for Nomad?

[Hushpar]

Spark 2.3.0 now have native K8S support http://blog.kubernetes.io/2018/03/apache-spark-23-with-native-kubernetes.html
Any plans to do the same with Nomad?

[Lwelch45]

There was a thread a while back around adding native support for nomad. It seemed to not go anywhere, due in partly to politics but also because they want to move to a model where anyone can add support for their resource management system. I don't have the ticket on hand but it was closed a while ago. Given the pace of the hasicorp team I expect that they will add it soon.

[rkettelerij]

@Lwelch45 I think you're referring to this ticket: https://issues.apache.org/jira/browse/SPARK-19700

[cgbaker]

The hope is to get this into upstream Spark or using a plugin system like in the link above; we will keep this ticket up-to-date regarding our progress in that.

[OneCricketeer]

Duplicates apache#18209

[cgbaker]

We have decided to focus our ecosystem efforts on other projects; this project is now deprecated. I have updated the README with information to this effect.