Apple linker does not accept `-commons use_dylibs` flag anymore

[fxcoudert]

Background information

What version of Open MPI are you using? 5.0.2

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

Installed from released sources as part of Homebrew build (Homebrew/homebrew-core#166807)

Please describe the system on which you are running

• Operating system/version: macOS 14.4
• Computer hardware: Apple M1
• Network type: not relevant

---

Details of the problem

Compiling any Fortran MPI code with mpifort hellof.f90 -o hellof with Xcode 15.3 gives:

  ld: warning: -commons use_dylibs is no longer supported, using error treatment instead
  ld: common symbol '_mpi_fortran_argv_null_' from '/private/tmp/cclH6ubZ.o' conflicts with definition from dylib '_mpi_fortran_argv_null_' from '/opt/homebrew/Cellar/open-mpi/5.0.2_1/lib/libmpi_usempi_ignore_tkr.40.dylib'
  collect2: error: ld returned 1 exit status

That is because -commons use_dylibs is now ignored (giving the warning), which leads to the symbol being rejected as defined twice.

[fxcoudert]

I have reported the regression (compared to Xcode 14 and earlier linkers) to Apple as FB13194355.

[neoh54]

similar problem I have encountered when I was trying to install super-dist package using spack and using apple-clang@15.0.0

ld: warning: ignoring duplicate libraries: '-lemutls_w', '-lgcc', '-lgfortran', '-lmpi', '-lmpi_mpifh', '-lmpi_usempi_ignore_t
             kr', '-lmpi_usempif08', '-lquadmath'
             
ld: warning: -commons use_dylibs is no longer supported, using error treatment instead

ld: warning: ignoring duplicate libraries: '-lemutls_w', '-lgcc', '-lgfortran', '-lmpi', '-lmpi_mpifh', '-lmpi_usempi_ignore_t
             kr', '-lmpi_usempif08', '-lquadmath'
             
ld: common symbol '_mpi_fortran_argv_null_' from '/private/var/folders/pd/9hc154y94k9_t_rb4lw0vcw00000gn/T/neoh/spack-stage/sp
             ack-stage-superlu-dist-8.2.1-h3rdwb66k3wb4s6gjglymknnv4xor3nf/spack-build-h3rdwb6/FORTRAN/CMakeFiles/f_pddrive.dir/f_pddrive.F
             90.o' conflicts with definition from dylib '_mpi_fortran_argv_null_' from '/Users/neoh/spack/opt/spack/darwin-sonoma-m2/apple-
             clang-15.0.0/openmpi-5.0.2-ja66vwemf6adckixrk6njhmaglqwck6v/lib/libmpi_usempi_ignore_tkr.40.dylib'

Does this warning treated differently in earlier versions of apple-clang?

[ggouaillardet]

It might be worth trying LDFLAGS=-ld_classic

Not sure if this is related to this issue thougth

If this fixes the issue, all the credit should go to @jeffhammond
https://twitter.com/science_dot/status/1772314603692626154

[jsquyres]

Is Open MPI getting these flags from GNU Libtool? I.e., is this actually a Libtool issue?

[fxcoudert]

Is Open MPI getting these flags from GNU Libtool? I.e., is this actually a Libtool issue?

No x 2:

ompi/config/ompi_setup_fc.m4

Line 236 in 984944d

LDFLAGS="$LDFLAGS -Wl,-commons,use_dylibs"

[jeffhammond]

FYI I have verified the following works with XCode 15.3 on Sonoma 14.4, which is the workaround Apple gave me.

I also confirmed it works when gfortran is used to initiate the linker, if -Wl,-ld_classic -Wl,-commons,use_dylibs is used.

% gcc -fPIC -shared extern2.c -o libxxx.so && \
  gfortran -c extern.F90 && ld extern.o libxxx.so \
  -L/opt/homebrew/Cellar/gcc/13.2.0/lib/gcc/current/ -lgfortran \
  -o extern -ld_classic -commons use_dylibs && \
  ./extern ; nm extern | grep MPI
ld: warning: -commons use_dylibs is no longer supported, using error treatment instead
MPIR_F08_MPI_IN_PLACE=0 &MPIR_F08_MPI_IN_PLACE=0x102818000 &MPIR_F08_MPI_IN_PLACE=4337008640
 LOC(MPI_IN_PLACE)=           4337008640
 LOC(buf)=           6134510240
sendbuf=0x102818000, sendbuf=4337008640
sendbuf is MPI_IN_PLACE? yes
recvbuf=0x16da532a0, recvbuf=6134510240
*count=1, *datatype=2, *op=3, *comm=4
         911
                 U _MPIR_F08_MPI_IN_PLACE
                 U _MPI_Allreduce

// extern2.c
#include <stdio.h>
#include <stdint.h>

int MPIR_F08_MPI_IN_PLACE;

void p(void)
{
    printf("MPIR_F08_MPI_IN_PLACE=%d &MPIR_F08_MPI_IN_PLACE=%p &MPIR_F08_MPI_IN_PLACE=%zu\n",
            MPIR_F08_MPI_IN_PLACE,   &MPIR_F08_MPI_IN_PLACE,   (intptr_t)&MPIR_F08_MPI_IN_PLACE);
}

void MPI_Allreduce(void ** sendbuf, void ** recvbuf,
                   int * count, int * datatype,
                   int * op, int * comm, int * ierror)
{
    printf("sendbuf=%p, sendbuf=%zu\n", sendbuf, (intptr_t)sendbuf);
    printf("sendbuf is MPI_IN_PLACE? %s\n",
           (intptr_t)sendbuf==(intptr_t)&MPIR_F08_MPI_IN_PLACE ? "yes" : "no");
    printf("recvbuf=%p, recvbuf=%zu\n", recvbuf, (intptr_t)recvbuf);
    printf("*count=%d, *datatype=%d, *op=%d, *comm=%d\n",
            *count, *datatype, *op, *comm);
    *ierror = 911;
}

! extern.F90
module mpi
    use iso_c_binding
    !type(c_ptr), bind(C,name="MPI_F_IN_PLACE") :: MPI_IN_PLACE
    integer(c_int), bind(C, name="MPIR_F08_MPI_IN_PLACE"), target :: MPI_IN_PLACE
    interface
        subroutine p() bind(C,name="p")
        end subroutine
    end interface
    interface
        SUBROUTINE MPI_ALLREDUCE(SENDBUF, RECVBUF, COUNT, DATATYPE, OP, COMM, IERROR) &
                   bind(C,name="MPI_Allreduce")
            use iso_c_binding
            import :: MPI_IN_PLACE
            !DEC$ ATTRIBUTES NO_ARG_CHECK :: sendbuf,recvbuf
            !GCC$ ATTRIBUTES NO_ARG_CHECK :: sendbuf,recvbuf
            !$PRAGMA IGNORE_TKR sendbuf,recvbuf
            !DIR$ IGNORE_TKR sendbuf,recvbuf
            !IBM* IGNORE_TKR sendbuf,recvbuf
            INTEGER(kind=c_int) :: SENDBUF(*), RECVBUF(*)
            INTEGER(kind=c_int) :: COUNT, DATATYPE, OP, COMM, IERROR
        END SUBROUTINE MPI_ALLREDUCE
    end interface
end module mpi

program main
    use mpi
    implicit none
    real :: buf(100)
    integer :: ierror
    call p
    buf = 17
    print*,'LOC(MPI_IN_PLACE)=',LOC(MPI_IN_PLACE)
    print*,'LOC(buf)=',LOC(buf)
    call MPI_ALLREDUCE(MPI_IN_PLACE,buf,1,2,3,4,ierror)
    print*,ierror
end program main

[fxcoudert]

I am trying to do the same for Homebrew: Homebrew/homebrew-core#166807

My original analysis was that -ld_classic was not effective anymore, because of the weird warning. But in spite of the warning, the classic linker can still be called that way.

[neoh54]

this worked for me:

spack install superlu-dist ldflags=-ld_classic

thanks @jeffhammond @ggouaillardet
my specs are Sonoma 14.2.1 and apple-clang@15.0.0 (Xcode 15.3)

[waveman68]

Oddly enough this worked for me:

brew install gcc-13
../configure --prefix=/opt/extlib/openmpi/5.0.2/gcc/13.2.0 \
        --with-libevent=internal \
        --enable-mpi1-compatibility \
        --enable-static \
        --enable-pmix-timing \
        CC=gcc-13 CXX=g++-13 FC=gfortran-13
make clean
make -j 8
make check
sudo make install

I was unable to install open-mpi 5.0.3 with the same method.

[jsquyres]

Is Open MPI getting these flags from GNU Libtool? I.e., is this actually a Libtool issue?

No x 2:

ompi/config/ompi_setup_fc.m4

Line 236 in 984944d

LDFLAGS="$LDFLAGS -Wl,-commons,use_dylibs"

I'm sorry for the huge delay here. Thanks for the citation of ompi_setup_fc.m4.

I have Sonoma 14.4.1 with XCode 15.3, and Homebrew gfortran

$ gfortran --version
GNU Fortran (Homebrew GCC 13.2.0) 13.2.0

But I don't see these warnings when I compile with the homebrew gfortran.

$ mpifort --showme  
gfortran -I/Users/jsquyres/bogus/include -Wl,-flat_namespace -Wl,-commons,use_dylibs -I/Users/jsquyres/bogus/lib -L/Users/jsquyres/bogus/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi
$ mpifort hello_usempif08.f90 -o hello  
$

What is different between my setup and yours?

[fxcoudert]

But I don't see these warnings when I compile with the homebrew gfortran.

We have re-enabled the "classic linker" in Homebrew gfortran at some point.