System timer handling with low-frequency timers

[carlescufi]

Nordic ICs use a low-power, 32KHz timer to driver the Zephyr kernel.
This results in several precision errors that have an effect on real-life application timer handling.

Sub-issues:
#12332 Undesirable behaviour of z_clock_announce() when ticks argument is greater by 1
#16195 k_uptime_delta(): Defective by design

Closed issues related:
#16238 k_cycle_get_32() API is useless in some Zephyr configurations
#12509: Fix rounding in _ms_to_ticks()
#12542 nrf timers unstable with ticks faster than 100 Hz
#9708: tests/kernel/tickless/tickless_concept fails on nRF5x with 1000msec boot delay
#8159: tests/kernel/fifo/fifo_timeout fails on nrf51_pca10028 and nrf52_pca10040
#9843: tests sched/deadline fails on nRF boards

[pizi-nordic]

First, I would like to present some background information about this complex problem:

Issue 1: Tick time calculation.
The sys_clock_hw_cycles_per_tick, which is used as a reference for RTC programming, is calculated as (CONFIG_SYS_CLOCK_HW_CYCLES_PER_SEC / sys_clock_ticks_per_sec), where sys_clock_ticks_per_sec equals to CONFIG_SYS_CLOCK_TICKS_PER_SEC.

In nRF5x series such approach leads to the following kernel settings (assuming CONFIG_SYS_CLOCK_TICKS_PER_SEC = 100):

• sys_clock_hw_cycles_per_tick = 32768 / 100 = 327.
• Real tick time = 327/32768 s = 9.979 ms

Problem: CONFIG_SYS_CLOCK_TICKS_PER_SEC is no longer true. In fact we have 32768/32 = 100.2 ticks per second. All code using CONFIG_SYS_CLOCK_TICKS_PER_SEC for timing calculations directly or indirectly produces invalid output as the fractional part is not taken under account. However many tests still calculates timeouts basing on this value and hidden assumption that CONFIG_SYS_CLOCK_HW_CYCLES_PER_SEC divides by CONFIG_SYS_CLOCK_TICKS_PER_SEC without remainder.

Of course, we could try to mitigate this problem, for example by setting CONFIG_SYS_CLOCK_TICKS_PER_SEC to 128, however this will not eliminate bigger issue described below.

Issue 2: Tick <-> ms conversion

• Situation before merging kernel: Do not use sys_clock_ticks_per_sec in _ms_to_ticks() #9073:
  Tick <-> ms conversion is based on CONFIG_SYS_CLOCK_TICKS_PER_SEC: 1000 ms is equivalent of 100 ticks and 100 ticks equals to 1000 ms. However as ticks are in fact shorter than 10ms, we have the following problems:

  • Tick and Cycle based timing API gives different results (k_uptime_get() vs k_cycle_get_32()).
  • Whole kernel API sleeps shorter than requested which is very bad. POSIX tests detects the situation: tests/posix/timer.test_timer fails on nrf51_pca10028 #7891

• Situation after merging kernel: Do not use sys_clock_ticks_per_sec in _ms_to_ticks() #9073:
  Tick <-> ms conversion is based on sys_clock_hw_cycles_per_tick and CONFIG_SYS_CLOCK_HW_CYCLES_PER_SEC: 1000 ms is equivalent of 101 ticks (100.2 rounded up) and 101 ticks equals to 1007 ms (1007.9ms rounded down). Rounding existed in the code for ages, however as all tests used "nice" timing values so it was not visible. Such approach fixes POSIX (we are always sleeping longer than expected, but this is normal in OS environment), however it introduces the following issues:

  • Many tests are failing as now kernel sleeps much longer than expected. Example: tests/kernel/fifo/fifo_timeout fails on nrf51_pca10028 and nrf52_pca10040 #8159 (and in this case it is not trivial to update the test).
  • Users are complaining that kernel sleeps longer than requested (this is something we should expect from loaded OS, however in our case this is happening on idle system).
  • tick2ms and ms2tick conversion is no longer commutative, which leads to problems like the Failed test: kernel.sched.slice_reset and kernel.sched.slice_scheduling (tests/kernel/sched/schedule_api/kernel.sched) on nrf51/nrf52 #8896, Failed test: kernel.tickless.tickless_slice on nrf51/nrf52 #8897 and nRF52_PCA10040: Failing test_slice_reset #9310.

• The latest update in the area (kernel: Improve precision of ticks and ms conversions #9745) reintroduces CONFIG_SYS_CLOCK_TICKS_PER_SEC to the tick<->ms conversion reverting us back to the situation before kernel: Do not use sys_clock_ticks_per_sec in _ms_to_ticks() #9073 merge.

Issue 3: Tickless mode
This is not even touched as I planned to fix simpler tick-based mode first :)

Summary:
I see only one solution to fix timing issues on nRF5x platform. However it is drastic one: We have to eliminate tick idea from Zephyr kernel (this include removal of tick-based operation as well as using system clock cycles as timing unit in tickles mode).

BTW: We can also "nicely" hide all of these issues just by rounding up sys_clock_hw_cycles_per_tick calculation. The tick will be longer than 10ms (POSIX will be happy), and the difference will be small enough to allow us pass all tests, as they usually use relatively small timeouts ;-)

[pizi-nordic]

@ioannisg, @nashif : Could you please look at this? Maybe you will see better solution than the one above.

[carlescufi]

I see only one solution to fix timing issues on nRF5x platform. However it is drastic one: We have to eliminate tick idea from Zephyr kernel (this include removal of tick-based operation as well as using system clock cycles as timing unit in tickles mode).

CC @andrewboie @andyross

[ioannisg]

I see only one solution to fix timing issues on nRF5x platform. However it is drastic one: We have to eliminate tick idea from Zephyr kernel (this include removal of tick-based operation as well as using system clock cycles as timing unit in tickles mode).

This is a bit confusing to me. The system clock is the tick-based "soft" clock (as opposed to the 32-bit HW clock). "System clock cycles as timing unit": you mean 32-bit HW clock cycles as timing unit?

[pizi-nordic]

I see only one solution to fix timing issues on nRF5x platform. However it is drastic one: We have to eliminate tick idea from Zephyr kernel (this include removal of tick-based operation as well as using system clock cycles as timing unit in tickles mode).

This is a bit confusing to me. The system clock is the tick-based "soft" clock (as opposed to the 32-bit HW clock). "System clock cycles as timing unit": you mean 32-bit HW clock cycles as timing unit?

Even in tickless mode, the system still is using "ticks" to manage time slices

#ifdef CONFIG_TICKLESS_KERNEL
#define sys_clock_ticks_per_sec \
                (1000000 / (CONFIG_TICKLESS_KERNEL_TIME_UNIT_IN_MICRO_SECS))
(...)

Without eliminating whole idea of ticks, we will always end fighting with ms<->tick conversion issues.

[andyross]

I agree with @carlescufi for the most part. There's an idea floating around locally here to try to make TICKLESS_KERNEL the default on all platforms that support it, which will go most of the way to what you want.

Note that there's still a need for something like a "tick". There are two time units internal to the Zephyr kernel: the high speed one returned by k_cycle_get_32(), and a lower speed one that is needed for scheduling timeouts (because the cycle counter rolls over too fast) -- that's our "tick" right now, and it will still be around in any design that needs to use a 32 bit timeout counter, even if we don't call it a "tick".

[carlescufi]

@andyross I assume you meant that you agree with @pizi-nordic and his comments here and here

[andyross]

Heh, indeed. Misread the attribution in your quote :)

[pdunaj]

@pizi-nordic maybe correction applied once per second (or per n ticks), that would make one of the ticks shorter (or longer)? Should be relatively cheap.