Device Tree support overhaul

[carlescufi]

This is an umbrella issue for a series of changes to the way Zephyr uses Device Tree.
The following items need to be addressed in order to consider this issue closed:

ELCE meeting path forward issue: #10821

• Describe what is the scope of Device Tree. i.e. What should go into Device Tree and what into Kconfig
  • PR doc: dt: Clarify the relationship between DT and Kconfig #8507
  • Issue The guidelines for whether something should be in DTS or Kconfig are too vague #7230
• Run Device Tree before Kconfig
  • PR Shift DTS before Kconfig #12108
  • Issue DTS metadata should be made available to Kconfig #7302
• Device Tree improvements
  • Issue Generate DTS 'compatible' based compilation flags #9406: Generate DTS 'compatible' based compilation flags
• edts database generation (cut-out of scripts: add Zephyr inline code generation with Python #6762)
  • PR script/dts: Generate Extended Device Tree database #9876
  • https://github.com/zephyrproject-rtos/zephyr/tree/topic-EDTS branch
  • PR dts: Add new DTS/binding parser #17660
• Code generation for instancing of devices
  • PR Codegen: Generated driver instantiation prototype  #8561, scripts: inline code generation with cogeno (and edts) #10885, drivers: use codegen for device instantiation #10888 Jinja: [RFC] Template prototype using jinja2 templating engine #10957
  • Issue Requirements for driver devices generation using device tree #10904
  • https://github.com/zephyrproject-rtos/zephyr/tree/topic-EDTS branch
• (optional) Integrate DT support in menuconfig
  • PR
  • Issue

[SebastianBoe]

@erwango : I have replaced the codegen PR with your prototype PR under "Code generation for instancing of devices".

Is the prototype PR supposed to be a part of this effort? If so it should take into account the repercussions of running device tree before Kconfig. Or else we risk having to rewrite the whole thing again.

[erwango]

@SebastianBoe

Is the prototype PR supposed to be a part of this effort?

From what we discussed last week, one of the (/the) first step was to demonstrate the feasibility of using codegen to provide and acceptable solution for driver instantiation (from zephyr user point of view, the end user API). This is what #8561 aims at doing.
This task was supposed to go straight to the point, so I took whatever was available and tried to come up with a demonstrator. If there is a general agreement this is acceptable, then everything should be done the right way.

Doing so, there will be some additional efforts, but at least a plan is there and I hope showing something concrete will help people agreeing this is a desirable thing to have and efforts should be done to achieve the plan. (Rather than endless debates that happened so far in the various PR/issues)

[erwango]

Added #9406 as bullet of 'Run Device Tree before Kconfig '

#9604 proposes a way to provide a DTS generated set of flag that could be handled by Kconfig for driver build control.

[erwango]

Comment about: " Code generation for instancing of devices ":

Current PR for this point is #8561.
#8561 is based on #6762 that introduces Codegen support. It adds a friendly device declaration module on top of Codegen. It aims at hiding python code from the user in the driver code by providing a nearly python agnostic API.