non-overlapping MPU gap-filling needs to be optional

[andrewboie]

ARMv8, and ARC MPU v3 are newer MPUs which allow for great flexibility on MPU region sizes, but regions are not allowed to overlap.

Currently, MPU regions are set up to control supervisor mode access, so that supervisor mode reads/writes to invalid memory generate faults. Implementing overlapping regions for memory domain requires that these base regions be split.

Unfortunately, this gap-filling has drawbacks:

1. The number of MPU regions available to applications is cut roughly in half, due to the need for additional regions to fill in the gaps
2. The gap-filling introduces overhead, currently during context switch

The overhead can be looked at (see #15223) but halving the number of free regions may in a lot of cases be a dealbreaker for applications.

This gap-filling is NOT needed to properly support user mode. This is a feature to catch stray memory access in supervisor mode and is orthogonal to CONFIG_USERSPACE requirements.

Add a Kconfig option for these MPUs which allows this to be disabled, maximizing the number of memory domain partitions available to applications.

[andrewboie]

@ioannisg would this be something you could look at, at least for ARMv8?

[ioannisg]

This gap-filling is NOT needed to properly support user mode. This is a feature to catch stray memory access in supervisor mode and is orthogonal to CONFIG_USERSPACE requirements.

@andrewboie in ARMv8-M, the reason for the gap-filling is to prevent supervisor mode to (accidentally) execute from SRAM (where, normally, it should have RW access only). We need MPU to block execution - the background policy unfortunately allows it.
So, yes, it is orthogonal to USERSPACE; it is just a "security" issue; are we ok with not preventing SUP mode to accidentally execute from SRAM?

Also it is a nice debug issue that catches bugs, e.g. wrong pointer dereferencing (this is not a hard requirement of course).

[andrewboie]

This is now done on ARM, still needs a PR for ARC.