Skip to content

Commit f7d43dd

Browse files
yuliao0214KAGA-KOKO
authored andcommitted
tick/broadcast: Make takeover of broadcast hrtimer reliable
Running the LTP hotplug stress test on a aarch64 machine results in rcu_sched stall warnings when the broadcast hrtimer was owned by the un-plugged CPU. The issue is the following: CPU1 (owns the broadcast hrtimer) CPU2 tick_broadcast_enter() // shutdown local timer device broadcast_shutdown_local() ... tick_broadcast_exit() clockevents_switch_state(dev, CLOCK_EVT_STATE_ONESHOT) // timer device is not programmed cpumask_set_cpu(cpu, tick_broadcast_force_mask) initiates offlining of CPU1 take_cpu_down() /* * CPU1 shuts down and does not * send broadcast IPI anymore */ takedown_cpu() hotplug_cpu__broadcast_tick_pull() // move broadcast hrtimer to this CPU clockevents_program_event() bc_set_next() hrtimer_start() /* * timer device is not programmed * because only the first expiring * timer will trigger clockevent * device reprogramming */ What happens is that CPU2 exits broadcast mode with force bit set, then the local timer device is not reprogrammed and CPU2 expects to receive the expired event by the broadcast IPI. But this does not happen because CPU1 is offlined by CPU2. CPU switches the clockevent device to ONESHOT state, but does not reprogram the device. The subsequent reprogramming of the hrtimer broadcast device does not program the clockevent device of CPU2 either because the pending expiry time is already in the past and the CPU expects the event to be delivered. As a consequence all CPUs which wait for a broadcast event to be delivered are stuck forever. Fix this issue by reprogramming the local timer device if the broadcast force bit of the CPU is set so that the broadcast hrtimer is delivered. [ tglx: Massage comment and change log. Add Fixes tag ] Fixes: 989dcb6 ("tick: Handle broadcast wakeup of multiple cpus") Signed-off-by: Yu Liao <[email protected]> Signed-off-by: Thomas Gleixner <[email protected]> Cc: [email protected] Link: https://lore.kernel.org/r/[email protected]
1 parent 59dbee7 commit f7d43dd

File tree

1 file changed

+23
-0
lines changed

1 file changed

+23
-0
lines changed

kernel/time/tick-broadcast.c

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1141,13 +1141,36 @@ void tick_broadcast_switch_to_oneshot(void)
11411141
#ifdef CONFIG_HOTPLUG_CPU
11421142
void hotplug_cpu__broadcast_tick_pull(int deadcpu)
11431143
{
1144+
struct tick_device *td = this_cpu_ptr(&tick_cpu_device);
11441145
struct clock_event_device *bc;
11451146
unsigned long flags;
11461147

11471148
raw_spin_lock_irqsave(&tick_broadcast_lock, flags);
11481149
bc = tick_broadcast_device.evtdev;
11491150

11501151
if (bc && broadcast_needs_cpu(bc, deadcpu)) {
1152+
/*
1153+
* If the broadcast force bit of the current CPU is set,
1154+
* then the current CPU has not yet reprogrammed the local
1155+
* timer device to avoid a ping-pong race. See
1156+
* ___tick_broadcast_oneshot_control().
1157+
*
1158+
* If the broadcast device is hrtimer based then
1159+
* programming the broadcast event below does not have any
1160+
* effect because the local clockevent device is not
1161+
* running and not programmed because the broadcast event
1162+
* is not earlier than the pending event of the local clock
1163+
* event device. As a consequence all CPUs waiting for a
1164+
* broadcast event are stuck forever.
1165+
*
1166+
* Detect this condition and reprogram the cpu local timer
1167+
* device to avoid the starvation.
1168+
*/
1169+
if (tick_check_broadcast_expired()) {
1170+
cpumask_clear_cpu(smp_processor_id(), tick_broadcast_force_mask);
1171+
tick_program_event(td->evtdev->next_event, 1);
1172+
}
1173+
11511174
/* This moves the broadcast assignment to this CPU: */
11521175
clockevents_program_event(bc, bc->next_event, 1);
11531176
}

0 commit comments

Comments
 (0)