qemu_x86_64 leaks system headers into the build process

[pfalcon]

Any comments on the following? From https://app.shippable.com/github/zephyrproject-rtos/zephyr/runs/33725/4/tests

ninja: Entering directory `/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/out-3nd-pass/qemu_x86_64/tests/posix/fs/portability.posix'
[1/86] Preparing syscall dependency handling

[2/86] Generating include/generated/driver-validation.h
[3/86] Generating include/generated/kobj-types-enum.h, include/generated/otype-to-str.h
[4/86] Generating include/generated/syscall_macros.h
[5/86] Generating misc/generated/syscalls_subdirs.trigger
[6/86] Generating misc/generated/syscalls.json
[7/86] Generating include/generated/syscall_dispatch.c, include/generated/syscall_list.h
[8/86] Building C object zephyr/CMakeFiles/offsets.dir/arch/x86_64/core/offsets/offsets.c.obj
[9/86] Generating include/generated/offsets.h
[10/86] Building C object CMakeFiles/app.dir/src/main.c.obj
FAILED: ccache /usr/bin/gcc -DBUILD_VERSION=zephyr-v1.13.0-4395-g17d5b48d02 -DKERNEL -D_FORTIFY_SOURCE=2 -D__LINUX_ERRNO_EXTENSIONS__ -D__ZEPHYR__=1 -I/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/include -I/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/kernel/include -I/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/arch/x86_64/include -I/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/include/drivers -Izephyr/include/generated -I/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/soc/x86_64/x86_64 -I/opt/sdk/zephyr-sdk-0.9.5/sysroots/x86_64-zephyr-elf/usr/include -I/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/tests/include -I/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/tests/ztest/include -I/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/ext/fs/fat/include -isystem /usr/lib/gcc/x86_64-linux-gnu/6/include -isystem /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed -Os -g -Wall -Wformat -Wformat-security -Wno-format-zero-length -imacros /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/out-3nd-pass/qemu_x86_64/tests/posix/fs/portability.posix/zephyr/include/generated/autoconf.h -ffreestanding -Wno-main -fno-common --sysroot /opt/sdk/zephyr-sdk-0.9.5/sysroots/x86_64-zephyr-elf/usr -Werror -Wno-deprecated-declarations -fno-asynchronous-unwind-tables -fno-pie -fno-pic -fno-strict-overflow -Wno-pointer-sign -Wno-unused-but-set-variable -fno-reorder-functions -fno-defer-pop -Werror=implicit-int -Wpointer-arith -ffunction-sections -fdata-sections -mno-sse -mno-red-zone -mx32 -std=c99 -MD -MT CMakeFiles/app.dir/src/main.c.obj -MF CMakeFiles/app.dir/src/main.c.obj.d -o CMakeFiles/app.dir/src/main.c.obj   -c /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/tests/posix/fs/src/main.c
In file included from /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed/syslimits.h:7:0,
                 from /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed/limits.h:34,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/include/kernel_includes.h:18,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/include/kernel.h:17,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/include/zephyr.h:18,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/tests/ztest/include/ztest.h:49,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/tests/posix/fs/src/test_fs.h:7,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/tests/posix/fs/src/main.c:7:
/usr/lib/gcc/x86_64-linux-gnu/6/include-fixed/limits.h:168:61: error: no include path in which to search for limits.h
 #include_next <limits.h>  /* recurse down to the real one */
                                                             ^
In file included from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/include/misc/__assert.h:83:0,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/include/kernel_includes.h:22,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/include/kernel.h:17,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/include/zephyr.h:18,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/tests/ztest/include/ztest.h:49,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/tests/posix/fs/src/test_fs.h:7,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/tests/posix/fs/src/main.c:7:
/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/include/misc/printk.h:14:22: fatal error: inttypes.h: No such file or directory
 #include <inttypes.h>
                      ^
compilation terminated.
ninja: build stopped: subcommand failed.

[pfalcon]

Just to give a hint where I lead: I'd be looking for a commit similar to c2c9265 to be applied to tests/posix/ , because in its current shape, qemu_x86_64 isn't compatible with Zephyr POSIX subsystem (because it's itself a posix thingy, and 2 posixes don't mix well.)

[andyross]

I don't get that failure. Are you trying to build something against newlib? IIRC this missing limits.h is the first point of failure if you try that. But the root cause is that your toolchain (the host toolchain on linux) simply doesn't have the library you're asking for.

As far as "system headers" being included, I don't see any in this error -- those are all toolchain headers, not linux/glibc things.

[pfalcon]

I don't get that failure.

@andyross, believe or not, but I'm in the same situation as you. What I do is: there's a POSIX subsystem is Zephyr, but is a toy one, not suitable for running any real-world POSIX app. I'm trying to improve on that part, but taking one random app/lib and making needed changes to Zephyr to run it. I then post my changes to CI, as passing CI is a requirement for merging changes. And I see totally random build errors like that from niche platform/archs. I try to workaround 1, 2, 10, 20 issues, but they keep popping up, and I find that I have zero progress on the original task - improving Zephyr POSIX subsys, - and instead firefighting issues like that.

Hence I propose to skip that part, and come to specifics: do you agree that qemu_x86_64 isn't developed enough so far (doesn't have own toolchain, own libs, etc, etc.), and it's worth to apply to posix tests the same changes as were applied to cmsis ones:
c2c9265 . If yes, I'll prepare patch immediately. If not, I'm working on changes which are effectively not upstreamable. There's a hope that something may change later, but it's unclear who and how will reap benefits of those changes, because once I've done with the current task (porting that random lib to Zephyr's POSIX), I likely will work on other tasks, just as effortful and all-encompassing as this one.

[pfalcon]

I seems that this ticket doesn't mention that the failures discussed (of which here presented only 1) come from #12984

[andyross]

I'm not a zealot about platform whitelisting and frankly the posix tests don't provide a lot of coverage anyway. But (see comments in the linked bug, too), there is a root cause here, it is understood, and those tests do build, run and pass on x86_64 right now.

So turning them off just because you're changing include file dependencies around seems like a bad trade. Let's fix the root cause, which is that Zephyr is failing to provide headers needed by POSIX and is relying on its SDK to magically have them. Non-SDK platforms will want POSIX too, x86_64 is just a particularly weird one.

[pfalcon]

But (see comments in the linked bug, too), there is a root cause here, it is understood, and those tests do build, run and pass on x86_64 right now.

I've run thru a couple of dozen of such "root causes" and think I've had enough for now. Someone certainly will fix it later, maybe me, maybe not.

[pfalcon]

Let's fix the root cause, which is that Zephyr is failing to provide headers needed by POSIX and is relying on its SDK to magically have them.

Heh, so running thru those couple of dozen causes, I've come to the exactly opposite conclusion. And that's the direction I'm exploring now, and even ring to related parties so they were in loop (and maybe randomly say "yes" right away ;-) ).

So that again shows we won't be able to find a solution now, except for the most obvious, temporary solution proposed.

[andyross]

Gonna close this one. The root cause is just that you were trying to configure newlib on a platform that doesn't have it. In fact the error above is the opposite of the way it's described: the toolchain headers are trying to find limits.h and inttype.h in the environment and failing to locate them.