main branch cannot build mtl/ofi on RHEL7 like

[ggouaillardet]

configure passes but make fails:

how should this be resolved?

• simply bump the OFI version requirements?
• test the existence of these macros and do not build mtl/ofi if they are not defined?
• test the existence of these macros and #ifdef protect some parts so mtl/ofi can be built?

FWIW

$ cat /etc/redhat-release 
CentOS Linux release 7.9.2009 (Core)
$ rpm -q libfabric
libfabric-1.7.2-1.el7.x86_64

In file included from /home/gilles/src/ompi-master/ompi/mca/mtl/ofi/mtl_ofi.c:13:0:
/home/gilles/src/ompi-master/ompi/mca/mtl/ofi/mtl_ofi.h: In function 'ompi_mtl_ofi_register_buffer':
/home/gilles/src/ompi-master/ompi/mca/mtl/ofi/mtl_ofi.h:324:17: error: 'struct fi_mr_attr' has no member named 'iface'
             attr.iface = FI_HMEM_CUDA;
                 ^
/home/gilles/src/ompi-master/ompi/mca/mtl/ofi/mtl_ofi.h:324:26: error: 'FI_HMEM_CUDA' undeclared (first use in this function)
             attr.iface = FI_HMEM_CUDA;
                          ^
/home/gilles/src/ompi-master/ompi/mca/mtl/ofi/mtl_ofi.h:324:26: note: each undeclared identifier is reported only once for each function it appears in
/home/gilles/src/ompi-master/ompi/mca/mtl/ofi/mtl_ofi.h:325:46: error: 'struct fi_mr_attr' has no member named 'device'
             opal_accelerator.get_device(&attr.device.cuda);
                                              ^
/home/gilles/src/ompi-master/ompi/mca/mtl/ofi/mtl_ofi.h:327:17: error: 'struct fi_mr_attr' has no member named 'iface'
             attr.iface = FI_HMEM_ROCR;
                 ^
/home/gilles/src/ompi-master/ompi/mca/mtl/ofi/mtl_ofi.h:327:26: error: 'FI_HMEM_ROCR' undeclared (first use in this function)
             attr.iface = FI_HMEM_ROCR;
                          ^
/home/gilles/src/ompi-master/ompi/mca/mtl/ofi/mtl_ofi.h:328:46: error: 'struct fi_mr_attr' has no member named 'device'
             opal_accelerator.get_device(&attr.device.cuda);

[bwbarrett]

Previously, we had a check for HMEM and would #ifdef those parts of code. It looks like we broke that. The HMEM requirements are pretty new in Libfabric world; I don't think we should bump up the required version of Libfabric; the old version of Libfabric should still just work with selective #ifdefs.

[ggouaillardet]

The configure absolute minimum requirement for OFI is 1.5.

But on v5.0.x, the requirement is bumped to 1.9 when CUDA is detected.
There is no such thing in main.

mtl_ofi_component.c seems to requires 1.9 at runtime, but there is no configure check.

Should we just simply require 1.9 at configure time?

[jsquyres]

@ggouaillardet Has this been fixed yet? It would seem to be a problem if we can't compile out-of-the-box with the RHEL 7 libfabric.

[janciesko]

Seeing the same issue on our cluster at HEAD @ main. (Red Hat Enterprise Linux Server release 7.6 (Maipo))

[jsquyres]

FYI @open-mpi/ofi team. Are you comfortable requiring libfabric >= v1.9, especially given that that will disqualify RHEL7 inbox libfabric?

[hppritcha]

Not really @jsquyres . I'd prefer an ifdef approach too.

[jsquyres]

It would be great if someone in @open-mpi/ofi would fix this, then. 😇

[hppritcha]

yes i'm checking this out on an old rhel7+libfab 1.7 system.