Documentation for Multi FPGA support

[georgiosddimou]

The documentation does not seem to describe how the multiple FPGAs are all interconnected in the larger F2 x4 and x8 systems. Do those instances simply contain 8 single-FPGA boards that can only communicate over PCIe through the host, or could one potentially split a larger design over multiple FPGA instances? If yes, what interface capabilities exist for direct FPGA-to-FPGA communication?

I suspect this may be a common question, so some reference in the high-level HDK documentation might be helpful to others as well.

[kristopk]

Hi @georgiosddimou , thanks for reaching out with questions.
We do not have F2 x4 and x8 systems.
F2 instance sizes are 6xlarge, 12xlarge and 48xlarge. Documented at EC2 instance types and DevKit User Guide.
Direct FPGA-to-FPGA communication is not a supported feature, which is why no documentation exists.
Each FPGA is connected to the host CPU via PCIe gen4 x8 as documented in the AWS Shell Interface Specification
Does your design require FPGA-to-FPGA communication ?

[georgiosddimou]

Apologies for the lack of clarity. I meant F2.12xlarge, which has x4 FPGAs and the F2.48xlarge with 8x FPGAs. We were considering partitioning a large design over multiple FPGAs, which, for performance reasons, would benefit from FPGA-to-FPGA high-bandwidth interfaces. But from your response it sounds like this will have to be done of the PCIe Gen4 interface through the host. Thank you very much for the quick response and clear explanation (and references).