driver callbacks and user mode

[andrewboie]

Is your enhancement proposal related to a problem? Please describe.
It's common to want to run some application code in response to driver events. Many of our driver subsystems (and our k_timer() API, which is essentially a timer driver callback) allow the user to install a callback handler (and in many cases a context pointer) to run when the event happens. The event is typically in the context of an ISR.

In our current MPU-based protection model, we've specifically forbidden providing access to driver callback installation from user mode, because such callback code runs in supervisor mode in the context of an ISR. We can't allow user mode to install program text which doesn't itself run in user mode for any reason.

The current workaround for this is as follows: in the early stages of the application, like in main() which starts in supervisor mode, is to install these callbacks here, before the application drops its privileges. The callback running under its parent ISR can access any memory it likes, including user memory; it's possible, for example, for the callback to fill user-accessible buffers with data. A user thread which has those buffers in its memory domain may later access them, perhaps using a k_sem shared between the user thread and the callback to synchronize.

Describe the solution you'd like
I would like an efficient way for a user application to set up driver callbacks for kernel events that originate from ISRs. The callback should be able to access memory in the process that installed the callback. The callback should run with the CPU in user mode, with the ability to sleep or make syscalls.

Internally, I imagine infrastructure such that a callback has two parts:

1. The actual code that runs on behalf of the ISR, which obtains any relevant data and then releases an IPC object (like a k_sem or something) to wake up some user thread for further processing.
2. The user part, which wakes up and runs the actual callback function logic, and has full access to the data posted by the ISR.

For the particular case of data buffers, we'd like to minimize the amount of copying done. But since the relevant user address space might not even be active, we'd either have to do copies or come up with some mechanism for mapping these buffers on the kernel side. I'm really waving my arms here, more contemplation of the specifics is needed.

I'd like the application-facing parts of this to be very simple to use. Particularly since this has implications for our driver APIs. Since this looks like it will require additional considerations to the current callback APIs we have, like mapping buffer memory, IPC objects, handler threads, etc, I'm wondering if the interface for this could be something like a workqueue API which takes a driver's callback installation API as a parameter.

Describe alternatives you've considered
This currently isn't a full solution spec, more like a problem statement with a few nebulous notions on how to think about addressing it. Open to ideas.

[jukkar]

I would like an efficient way for a user application to set up driver callbacks for kernel events that originate from ISRs. The callback should be able to access memory in the process that installed the callback. The callback should run with the CPU in user mode, with the ability to sleep or make syscalls.

Just brainstorming here, this might be totally stupid idea. Instead of function callback that is called from ISR, could we pass data from ISR to user mode application via IPC like k_msgq, fifo, queue etc. That would be very inefficient of course but would avoid direct function call.

[pabigot]

sys_notify is intended to do this for non-user-mode applications. The model assumes that the structure may be contained within another structure which the callback can access, which doesn't work with vrfy wrappers that copy data into/out of user memory. A way to integrate this with any new user mode callback infrastructure would be nice: then we could use this generic API to support async operations regardless of how the caller wants to receive the result.

[andrewboie]

@jukkar I think there may be a subset of current callback APIs where providing an IPC object, instead of a function, may actually be sufficient for everyone. But need to study this really carefully. I want to make sure that for users who are not using memory protection, there are no performance or footprint implications.

@pabigot thanks I'll study sys_notify and assess how it could fit into a solution for this.

Miscellaneous side note: Microkernels take this one step further, where almost the entirety of ISR processing is bounced down to thread context. EOI is not sent to the interrupt controller until the thread-level processing completes. Zephyr is not a microkernel but there may be some value in allowing control on when EOI is sent. Something to think about anyway.

Another side note: need to consider how Meta-IRQs fit in here.

[andrewboie]

With the upcoming planned changes to introduce virtual memory semantics to systems that have an MMU, some of the assumptions in this workaround fall over.

The current plan for MMU support is to have a single view of memory from an ISR perspective, and this problem goes away. Details in #27819.