Support testing with ARM FVP in the CI

[stephanosio]

Summary

Support running the Zephyr tests using the ARM FVP (fixed virtual platform; emulation software) in the CI.

Problem description

Some boards and ARM features (e.g. Ethos neural coprocessors) can only be tested using the ARM FVP emulation platform (QEMU does not currently support emulating them).

Proposed change

1. Include ARM FVP as part of the CI Docker image private CI Docker image.

   • We need to check if ARM FVP EULA allows this. this is allowed since we are not redistributing anything.

2. Update CI workflows to run tests using the ARM FVP included in the CI Docker image.

   • Set ARMFVP_BIN_PATH environment variable.

List of required FVPs

• FVP_BaseR_AEMv8R by fvp_baser_aemv8r
• FVP_Base_RevC-2xAEMvA by fvp_base_revc_2xaemv8a
• FVP_Corstone_SSE-300_Ethos-U55 by mps3_an547

[carlocaione]

For emulation FVP should be more advanced and truthful to the hardware than QEMU, so maybe we can avoid all the usual icount issues with QEMU.

[povergoing]

While the annoying issue may be You are not permitted to redistribute the 'Fixed Virtual Platforms (FVPs)' Software (in whole or in part) or any derivatives thereof.. I am not an expert on Licenses, but it's a kind of dangerous thing I think. I already contacted the relevant guy half a year ago. But still no response...

[povergoing]

For emulation FVP should be more advanced and truthful to the hardware than QEMU, so maybe we can avoid all the usual icount issues with QEMU.

That's true, I 100% agree that including FVP into Docker img. Anyway, I'll try again. Fortunately, some high-level one in Arm is aware of this thing, so let's see.

[stephanosio]

@povergoing The clause you quoted falls under (b) if you are receiving a Non-Commercial Use License or version (as applicable) of the Arm Tools:, which I suppose is referring to those who have received a non-commercial license of the normally paid tools.

The (a) if you are receiving the Arm Tools free of charge: does not seem to explicitly prohibit redistribution, but it does not expressly permit redistribution either.

[Weichen81]

Just some personal suggestion:
another thing that need to be considered is that the instance to accept "EULA" must be a human, so the click or enter "OK" behaviors should not be done by a "BOT".
For Cortex-A series, an architecture level FVP "Armv-A Base RevC AEM FVP" (https://developer.arm.com/tools-and-software/simulation-models/fixed-virtual-platforms/arm-ecosystem-models) can be download without "EULA". So maybe you
can integrate the URL in docker image, but not FVP binary. But for Cortex-R, current we still haven't a proper way to bypass "EULA" click.

[povergoing]

@povergoing The clause you quoted falls under (b) if you are receiving a Non-Commercial Use License or version (as applicable) of the Arm Tools:, which I suppose is referring to those who have received a non-commercial license of the normally paid tools.

The (a) if you are receiving the Arm Tools free of charge: does not seem to explicitly prohibit redistribution, but it does not expressly permit redistribution either.

I am also very confused. This is totally out of my knowledge... But IMO, since the only way to get the FVP is to "register on arm website, accept "EULA" then download" (Arm wants to record who uses FVP?), there must be some restrictions on redistribution. In another world, if the binary is included in CI docker IMG or zephyr SDK, that means anyone can download FVP without accepting the EULA so that Arm cannot record.

Or is there any way that the host machine keeps the FVP binary instead of the docker IMG, mapping the FVP binary path to docker when running CI? In this way, we use FVP as a service and certainly do not redistribute FVP, so we don't need to care about the restrictions of redistribution.

[stephanosio]

Or is there any way that the host machine keeps the FVP binary instead of the docker IMG, mapping the FVP binary path to docker when running CI?

There is a long-term plan to implement something like that to get testing with the proprietary toolchains working in the CI, but it will take some time to get that implemented (no timeline yet).

[povergoing]

Hi @stephanosio @carlocaione,
Update here: the FVP_BaseR_AEMv8R on page can download without any logins or registers.

The tricky part is how we avoid redistribution? That means the docker file should not contain the fvp bins (since anyone can easily get the docker image). Can we download the FVP at runtime triggered by some event? like, run ci -> trigger download fvp -> run ci test?
Any suggestions?

[stephanosio]

Thanks for the update.

Can we download the FVP at runtime triggered by some event?

While that should work, doing so would incur a significant amount of traffic to the ARM website and also, more importantly, will slow down the CI depending on the file provider server load.

I think the best way to handle this would be to create a private CI Docker image, based on the public one, that is only available to the authenticated Zephyr CI runners (and of course, cannot be downloaded publicly).

[povergoing]

I think the best way to handle this would be to create a private CI Docker image, based on the public one, that is only available to the authenticated Zephyr CI runners (and of course, cannot be downloaded publicly).

Agreed, looks like it is the best way. So, is there anything we can help with? :)