[RFC] Preprocessor-Based Arithmetic and Condition Evaluation

[soburi]

Problem

Currently, DeviceTree macros face limitations in setting flexible compile-time conditions. Specifically, COND_CODE_1 evaluates whether a given argument is 1 or not, but deriving this 1 using only macros can be challenging in certain cases.

Proposed Solution (Overview)

Introduce the capability to perform "calculations" using the preprocessor, allowing for flexible condition evaluation. The following example demonstrates the proposed functionality:

/* (1 + 2 == 9) is not true, so the 3rd argument will remains. */
COND_CODE_1(
    EXPR_IS_EQ(EXPR_ADD(EXPR_HEX(0x1), EXPR_HEX(0x2)), EXPR_HEX(0x9)),
    (zassert_unreachable("unreachable")),
    (zassert_true(1))
);

Proposed Solution (Details)

To address the problem, implement 32-bit arithmetic calculations with macros. Numbers will be represented as lists of 0 and 1 arguments (referred to as "bit args"). Using these bit args, the following operations will be provided:

• Comparison operations:
  • EQ(= ,equal)
  • GT(>, greater than)
  • LT(<, less than)
  • GTE(>=, greater than equal)
  • LTE(<=, less than equal)
• Bitwise operations:
  • OR(|)
  • AND(&)
  • XOR(^)
  • NOT(~)
• Arithmetic operations:
  • ADD(+, addition)
  • SUB(-, subtraction)
  • MUL(*, multiplication)
  • DIV(/, division)
  • MOD(%, modulus)

This enables complex conditional expressions to be evaluated entirely within the preprocessor.
In other words, it is a proposal to implement a simple computer using only macros.

• Integration with DeviceTree

  This functionality is designed to work alongside DeviceTree macros.
  My suggestion is to process devicetree_generated.h and automatically generate definitions that
  correspond to decimal numbers that appear; it is possible to include values ​​that can be obtained
  through the DeviceTree API in the processing. I think this covers the main use cases.

Considerations

1. Performance

   Multiplication and division are inherently slower operations. For performance reasons, we may need to be avoided in CI environments.

2. Limitations of Number Representations

   Due to the preprocessor's constraints, decimal and hexadecimal literals cannot be handled directly. They must first be converted into the internal bit args representation.
   Lookup tables containing up to 4096 entries for decimal and hexadecimal numbers have been defined to facilitate this. However, this approach has limitations in computation resources.

Internally, generating hexadecimal literals from bit args is easy, so outputting results in hexadecimal is unrestricted.

The limitations can be summarized as follows:

Format	Input Support	Output Support
bit args	✓	✓
Hexadecimal	Table-dependent	✓
Decimal	Table-dependent	Table-dependent

Benefits

This feature enables flexible conditional branching and complex calculations at compile time, enhancing the power and versatility of DeviceTree macros.

• note
  Changes to some common parts have been grouped into a separate PR.
  I will rebase as appropriate.

#84191
#84190

[bjarki-andreasen]

We currently have a bit of this implemented, but spread out in IS_EQ(), UTIL_INC(), UTIL_ADD etc. It would make sense to add the missing expressions, and move the existing expressions to the same namespace, like EXPR_EQ(), EXPR_OR() etc.

It will add many thousands of lines of code, but if we hide these macros behind a header like zephyr/expr/mul.h, zephyr/expr/div.h, zephyr/expr/eq.h etc. and not include these in zephyr/kernel.h or similar, it should not impact build times unless the macros are explicitly needed/included :)

[teburd]

I have concerns, though not likely worth -1ing over as we have already been going this direction for some time.

What happens if there's a typo or mistake in the macro? What kind of errors are reported by gcc? How do you go about debugging these easily?

Basically I'd like to see what the debugging story is for this sort of thing. Our biggest hurdle today for new people are macros which are not very easy to debug. This leads to our yearly developer survey showing DT/Kconfig/macros being a hurdle to using Zephyr. I find myself avoiding these things in Zephyr as I've personally burned hours debugging macros, and I'm constantly using and developing Zephyr as a maintainer!

At what point are we going to be concerned about our developer experience in Zephyr with all of these things?

[bjarki-andreasen]

I have concerns, though not likely worth -1ing over as we have already been going this direction for some time.

What happens if there's a typo or mistake in the macro? What kind of errors are reported by gcc? How do you go about debugging these easily?

Basically I'd like to see what the debugging story is for this sort of thing. Our biggest hurdle today for new people are macros which are not very easy to debug. This leads to our yearly developer survey showing DT/Kconfig/macros being a hurdle to using Zephyr. I find myself avoiding these things in Zephyr as I've personally burned hours debugging macros, and I'm constantly using and developing Zephyr as a maintainer!

At what point are we going to be concerned about our developer experience in Zephyr with all of these things?

I do agree many macros in zephyr are near impossible to follow "intuitively", stuff like having to use IS_EQ() in a COND_CODE_1() because it has to expand to a constant with the value 1 is not exactly obvious, or the last to args of COND_CODE_1() having to be in brackets etc. etc. Both of these quirks cost me hours to :) But what to do?

[carlescufi]

Architecture WG:

• @bjarki-andreasen states that we already have some of those, but scattered around, and it would be good to have them in one place, until we have a better solution
• @gmarull asks what the compiler output will be in case of error, because with Devicetree we already have a serious issue with this.
• @nordicjm tells us that there was an earlier attempt at this, which at the time was rejected because of its complexity: Include: Simple preprocessing math util macros #75380

[teburd]

I have concerns, though not likely worth -1ing over as we have already been going this direction for some time.
What happens if there's a typo or mistake in the macro? What kind of errors are reported by gcc? How do you go about debugging these easily?
Basically I'd like to see what the debugging story is for this sort of thing. Our biggest hurdle today for new people are macros which are not very easy to debug. This leads to our yearly developer survey showing DT/Kconfig/macros being a hurdle to using Zephyr. I find myself avoiding these things in Zephyr as I've personally burned hours debugging macros, and I'm constantly using and developing Zephyr as a maintainer!
At what point are we going to be concerned about our developer experience in Zephyr with all of these things?

I do agree many macros in zephyr are near impossible to follow "intuitively", stuff like having to use IS_EQ() in a COND_CODE_1() because it has to expand to a constant with the value 1 is not exactly obvious, or the last to args of COND_CODE_1() having to be in brackets etc. etc. Both of these quirks cost me hours to :) But what to do?

The answer is straightforward but likely many will not like where it goes. Not using the C preprocessor for this stuff. Going a step further... encoding more information in this not-C-preprocessor, enough such that typos and errors don't get punted to the next build step until at long last you get a linker error that all goes back to a typo you had in some metadata file describing you wanting to enable a hardware peripheral and its driver. Basically re-look at all this stuff from orbit rather from the anthill on the ground trying to solve the next micro problem like this that comes along.

[bjarki-andreasen]

I have concerns, though not likely worth -1ing over as we have already been going this direction for some time.
What happens if there's a typo or mistake in the macro? What kind of errors are reported by gcc? How do you go about debugging these easily?
Basically I'd like to see what the debugging story is for this sort of thing. Our biggest hurdle today for new people are macros which are not very easy to debug. This leads to our yearly developer survey showing DT/Kconfig/macros being a hurdle to using Zephyr. I find myself avoiding these things in Zephyr as I've personally burned hours debugging macros, and I'm constantly using and developing Zephyr as a maintainer!
At what point are we going to be concerned about our developer experience in Zephyr with all of these things?

I do agree many macros in zephyr are near impossible to follow "intuitively", stuff like having to use IS_EQ() in a COND_CODE_1() because it has to expand to a constant with the value 1 is not exactly obvious, or the last to args of COND_CODE_1() having to be in brackets etc. etc. Both of these quirks cost me hours to :) But what to do?

The answer is straightforward but likely many will not like where it goes. Not using the C preprocessor for this stuff. Going a step further... encoding more information in this not-C-preprocessor, enough such that typos and errors don't get punted to the next build step until at long last you get a linker error that all goes back to a typo you had in some metadata file describing you wanting to enable a hardware peripheral and its driver. Basically re-look at all this stuff from orbit rather from the anthill on the ground trying to solve the next micro problem like this that comes along.

Well, the "what we don't want" is straightforward, no C-preprocessor, the "what to replace it with", that's the answer I'm interested in :) The preprocessor is solving the not having to store every node, child, prop, etc. in C structures (like Linux), which I'm fairly sure is not an option. What else can we do?

[pdgendt]

I've been intrigued by https://gcc-python-plugin.readthedocs.io/en/latest/index.html to create some custom pre-processor shenanigans, but I think it's GCC only.

[github-actions]

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.