Create a way for offline building

[kloenk]

Some build systems (like nix) do not allow network access while building, but then provide a download phase, where network access is allowed.
For this to work with cargo, we would have to create a cargo vendor dir, and call cargo with the flags, to not update the index and use the vendor dir.

[ojeda]

Agreed! Offline building will be supported (it is one of the requirements).

At the moment, you may try prefetching the dependencies in the download phase.

[kloenk]

I saw in the docs something about charge fetch. But for nix it would be best I can do it out of tree and link the folder in tree later.
To download I woul like to just copy one Cargo.lock

[kloenk]

I will try to build it via nix next week. If that succeeds, I'm happy to close this issue.
Currently I also fail with this error. Did someone get this kernel to build?

elf.c: In function ‘read_symbols’:
elf.c:378:8: error: implicit declaration of function ‘gelf_getsymshndx’; did you mean ‘gelf_getsym’? [-Werror=implicit-function-declaration]
  378 |   if (!gelf_getsymshndx(symtab->data, shndx_data, i, &sym->sym,
      |        ^~~~~~~~~~~~~~~~
      |        gelf_getsym
elf.c:378:8: error: nested extern declaration of ‘gelf_getsymshndx’ [-Werror=nested-externs]

[kloenk]

I will try to build it via nix next week. If that succeeds, I'm happy to close this issue.
Currently I also fail with this error. Did someone get this kernel to build?

elf.c: In function ‘read_symbols’:
elf.c:378:8: error: implicit declaration of function ‘gelf_getsymshndx’; did you mean ‘gelf_getsym’? [-Werror=implicit-function-declaration]
  378 |   if (!gelf_getsymshndx(symtab->data, shndx_data, i, &sym->sym,
      |        ^~~~~~~~~~~~~~~~
      |        gelf_getsym
elf.c:378:8: error: nested extern declaration of ‘gelf_getsymshndx’ [-Werror=nested-externs]

Seems like i had an to old version of some dependencies

[ojeda]

I will try to build it via nix next week. If that succeeds, I'm happy to close this issue.

If you have the time to try, it would be great! The issue may remain opened since building in general (and offline building in particular) is not yet set in stone.

Currently I also fail with this error. Did someone get this kernel to build?

The branch is intended to always be building and working (CI was just added to keep it that way, too). Of course, mistakes could happen.

[kloenk]

I'm currently seeing this warning, is ../include/config/auto.conf something introduced by this fork?

linux-config> sed: can't read ../include/config/auto.conf: No such file or directory

(sorry, it's my first time working on the kernel)

[ojeda]

No, it is something that appears the first time when you haven't configured or built the kernel previously (so after a make mrproper too). If you are using a config file from somewhere else, potentially from older versions of the kernel, you can run make oldconfig && make prepare (it will ask for new options if the old config didn't know about them).

(Don't worry about asking!)

[kloenk]

I'm currently seeing this warning, is ../include/config/auto.conf something introduced by this fork?

linux-config> sed: can't read ../include/config/auto.conf: No such file or directory

This line seems to be the problem if you build with O=$something

linux/scripts/Makefile.lib

Line 232 in f60edb2

rustc_cfg_flags = $(shell sed -nE 's/^(CONFIG_[^=]+)=(y|m)$$/--cfg \1/p' $(srctree)/include/config/auto.conf | xargs)

[kloenk]

Not sure what I did, but I got it to die with this error:

scripts/kconfig/conf  --oldconfig Kconfig
make[1]: scripts/kconfig/conf: Argument list too long
make[1]: *** [scripts/kconfig/Makefile:71: oldconfig] Error 127
make: *** [Makefile:627: oldconfig] Error 2

[kloenk]

What about changing from --cfg to std::env!()? This would maybe produce more binary code, but it also would enable us to get values inside of rust.
Furthermore, auto.conf seems to be configurable where it will be placed. And it may get so big, that it exceeds the max argument length of a Linux shell.

@ojeda What do you think about this? should I open an Issue/PR for this?

[ojeda]

This line seems to be the problem if you build with O=$something

linux/scripts/Makefile.lib

Line 232 in f60edb2

rustc_cfg_flags = $(shell sed -nE 's/^(CONFIG_[^=]+)=(y|m)$$/--cfg \1/p' $(srctree)/include/config/auto.conf | xargs)

Good catch, we need to avoid reading the file when not needed.

What about changing from --cfg to std::env!()? This would maybe produce more binary code, but it also would enable us to get values inside of rust.

Do you mean avoiding conditional compilation altogether?

Furthermore, auto.conf seems to be configurable where it will be placed. And it may get so big, that it exceeds the max argument length of a Linux shell.

Does rustc support other ways of passing cfg variables? If not, the best would be to have that support added (even if we didn't hit issues with length etc., it is not nice to see pages of arguments when building verbosely).

@ojeda What do you think about this? should I open an Issue/PR for this?

Definitely! Please open the issue.

[kloenk]

What about changing from --cfg to std::env!()? This would maybe produce more binary code, but it also would enable us to get values inside of rust.

Do you mean avoiding conditional compilation altogether?

Yes, that was my idea. We maybe could create special cases, like some debug values or so, but I think the should be opt-in

Furthermore, auto.conf seems to be configurable where it will be placed. And it may get so big, that it exceeds the max argument length of a Linux shell.

Does rustc support other ways of passing cfg variables? If not, the best would be to have that support added (even if we didn't hit issues with length etc., it is not nice to see pages of arguments when building verbosely).

I'm currently trying to figure this out and ask in the rust zulipchat.

@ojeda What do you think about this? should I open an Issue/PR for this?

Definitely! Please open the issue.

Opened as #22

[ojeda]

Yes, that was my idea. We maybe could create special cases, like some debug values or so, but I think the should be opt-in

No, conditional compilation must work -- nobody will settle on something so fundamental (myself included!). It is not just about bloat or performance (which are both a big no-no), but also about complexity (we'd need to define what subset goes in), divergence (developers expect all CONFIG_* options to be available), etc.

I'm currently trying to figure this out and ask in the rust zulipchat.

Opened as #22

Thanks a lot x2!