Gang Scheduling Support in Kubernetes

[kerthcet]

Enhancement Description

• One-line enhancement description (can be used as a release note): Support gang scheduling primitive in Kubernetes
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/4671-gang-scheduling/README.md (google doc)
  Previous proposal: https://docs.google.com/document/d/1q4a8uB_he2gx_lB2YFxsGaSVqMtF6CGJBhFkvArwnd4/edit?
• Primary contact (assignee): @wojtek-t @erictune
• Responsible SIGs: sig-scheduling
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.35
  • Beta release target (x.y): 1.36
  • Stable release target (x.y): 1.38
• Alpha
  • KEP (k/enhancements) update PR(s): KEP-4671 : Gang Scheduling #5558
  • Code (k/k) update PR(s):
    • tracked issue: [Umbrella] KEP-4671: Gang Scheduling for v1.35 kubernetes#134471
    • KEP-4671: Implement Gang scheduling in kube-scheduler kubernetes#134722
    • KEP-4671: Add Workload API kubernetes#134564
  • Docs (k/website) update PR(s):
    • KEP-4671 Add docs for Workload API and Gang scheduling website#53296
    • KEP-4671, KEP-5598: Introducing Workload Aware Scheduling blog post website#53012

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[kerthcet]

/sig scheduling
/assign

[kerthcet]

Some designs are based on #3370.

[alculquicondor]

cc @cs20

[utam0k]

Recently, @sanposhiho and I have opened our gang scheduler plugin, which we actually use in our cluster. Since I believe our approach is different from co-scheduling, it is valuable to take it into consideration. This plugin doesn't require custom resources such as PodGroups. Perhaps it has tips to improve this proposal.
https://github.com/pfnet/scheduler-plugins/tree/master/plugins/gang

[kerthcet]

Thanks @utam0k I'll take a look but the link is invalid.

[utam0k]

Thanks @utam0k I'll take a look but the link is invalid.

Sorry, I have fixed it

[sanposhiho]

@kerthcet
Our company used an custom plugin similar to co-scheduling (design-wise), and hit challenges.
(I actually haven't followed this topic though,) if you're planning to introduce the current co-scheduling plugin implementation almost as is, we'd definitely hit the same too.

These are major challenges from our experience -

• Inefficient scheduling: waiting Pods reserve too large space in a cluster.
  • Let's say a group has 5 Pods; 4 Pods go through the scheduling cycle and get on waitOnPermit, while 1 Pod is rejected. 4 Pods reserves the places until the last Pod goes thru the scheduling cycle too (or reach timeout).
  • As the number of groups in the cluster grows, it likely gets worse situation like a dead lock; many Pods would be unschedulable while the cluster has plenty space because a few Pods in each group reserve space and wait for the rest of Pods, while the rest of Pods cannot go through the scheduling cycle as many space is reserved.
• Difficulty in requeueing: For efficient requeueing, we should requeue a group of Pod when all Pods are ready to get schedule.
  • Let's say a group has 5 Pods; one Pod is rejected by resource fit, another Pod is rejected by NodeAffinity, three Pods are schedulable (they have to wait for the other 2 Pods). In this case, we have to requeue all 5 Pods when both resource fit plugin's failure for Pod-1 and NodeAffinity plugin failure for Pod-2 are solved.

---

Our gang plugin overcomes those challenges, which is worth taking a look for you, hopefully :)
But, on the other hand, I'm not saying, then we should follow our plugin's design. At its design phase, I had to solve them only by what is allowed within plugins; we didn't want to fork the scheduler.
So, it should be much easier/simpler for sure if we could introduce changes in the scheduling framework itself to wisely support scheduling a group of Pods.

[kerthcet]

Thanks for the feedbacks @sanposhiho

Took a briefly look at the https://github.com/pfnet/scheduler-plugins/tree/master/plugins/gang, as the README.md highlights, the gang scheduler enhanced at:

• Simple configuration - Simplify is what we chased for but annotation is arbitrary, not good for validation and it's not that official, I may not follow this, but I got your points.
• Enhanced requeueing - This is something we should consider when we're dedicated to solve the performance problem.

Let's say a group has 5 Pods; 4 Pods go through the scheduling cycle and get on waitOnPermit, while 1 Pod is rejected. 4 Pods reserves the places until the last Pod goes thru the scheduling cycle too (or reach timeout).

If 4 Pods passed the scheduling cycle, shouldn't we wait for the last Pod's scheduling result? Sorry, I didn't get your point here, but regarding to co-scheduling, it has similar logic as https://github.com/kubernetes-sigs/scheduler-plugins/blob/531a1831bdda0bdad6057f7d00f3cafacdb93d86/pkg/coscheduling/coscheduling.go#L152.

After all, I do agree we should reject the podGroup scheduling ASAP if we're 100% sure that the podGroup will not succeed at last.

As the number of groups in the cluster grows, it likely gets worse situation like a dead lock;

Yes, we should make sure the group Pods are queued up together. I may take a deep look of your approach about your implementations.

Difficulty in requeueing: For efficient requeueing, we should requeue a group of Pod when all Pods are ready to get schedule.

This is a more fine-gained approach focused on performance.

Our gang plugin overcomes those challenges, which is worth taking a look for you, hopefully :)

Definitely I will, thanks for sharing.

[sftim]

Kubernetes already supports pluggable schedulers. Should this feature be delivered in-tree?