What to do with system-cxx-std-lib?

[hsyl20]

system-cxx-std-lib is a virtual cabal package used to indicate that a package wants to link with C++ stdlibs.

It is virtual because GHC's build system generates directly the following package configuration for it (see mk/system-cxx-std-lib-1.0.conf.in):

name:                 system-cxx-std-lib
version:              1.0
visibility:           public
id:                   system-cxx-std-lib-1.0
key:                  system-cxx-std-lib-1.0
license:              BSD-3-Clause
synopsis:             A placeholder for the system's C++ standard library implementation.
description:          Building against C++ libraries requires that the C++ standard
                      library be included when linking. Typically when compiling a C++
                      project this is done automatically by the C++ compiler. However,
                      as GHC uses the C compiler for linking, users needing the C++
                      standard library must declare this dependency explicitly.
                      .
                      This "virtual" package can be used to depend upon the host system's
                      C++ standard library implementation in a platform agnostic manner.
category:             System
abi:                  00000000000000000000000000000000
exposed:              True
exposed-modules:
extra-libraries:      @CXX_STD_LIB_LIBS@
library-dirs:         @CXX_STD_LIB_LIB_DIRS@
dynamic-library-dirs: @CXX_STD_LIB_DYN_LIB_DIRS@       

Issues

• we use a C compiler to link C++ code: it is brittle
• the configuration is statically determined at GHC build time (or maybe at GHC installation time too?):
  • the package db with this system-cxx-std-lib package isn't portable (e.g. from a platform with Clang++ to a platform with g++)
  • if the host C++ toolchain is updated it will likely break

How we could fix it

• we need to have this package otherwise cabal-install will complain that it is missing when some package depends on it
• but we could keep this package empty (no extra-libraries field, etc.)
• GHC could detect the C++ configuration at link time and inject it into the package configuration when it loads it
• if we do this lazily, it should only be used when linking C++ code, which is already slow so it won't be noticeable
• alternatively we could switch to using the C++ compiler to link when system-cxx-std-lib is in the dependencies

[angerman]

We know when we need this. As soon as there is (lib)(std)c++ in the link dependencies, we should just switch from cc to cxx to link. Maybe the heuristic needs to check for c++abi as well? But there can't be that many, once the user specifies that c++ is needed (because they link against the standard lib with their c++ stuff), we can just use cxx instead of cc.

[hsyl20]

We already have a C++ compiler listed in GHC's settings. I've just tried to switch to it when system-cxx-std-lib is in the package dependencies, and it passes the tests locally!

[hsyl20]

I've opened https://gitlab.haskell.org/ghc/ghc/-/issues/25767 to suggest doing this upstream to see the reactions.

[angerman]

Just to re-iterate why I think system-cxx-std-lib is unnecessary complexity.

Why do we have system-cxx-std-lib in the first place?

This can be fairly well illustrated with the text package (which depends on some c++ code):

if impl(ghc >= 9.4)
      build-depends: system-cxx-std-lib == 1.0
    elif os(darwin) || os(freebsd)
      extra-libraries: c++
    elif os(openbsd)
      extra-libraries: c++ c++abi pthread
    elif os(windows)
      -- GHC's Windows toolchain is based on clang/libc++ in GHC 9.4 and later
      if impl(ghc < 9.3)
        extra-libraries: stdc++
      else
        extra-libraries: c++ c++abi
    elif arch(wasm32)
      cpp-options: -DSIMDUTF_NO_THREADS
      cxx-options: -fno-exceptions
      extra-libraries: c++ c++abi
    else
      extra-libraries: stdc++

Prior to the introduction of system-cxx-std-lib in GHC 9.4 (which here is also pinned to an exact version 🙄). We did not have a good way to link c++ projects as the c++ compiler will inject stdc++ or similar to the linker to link. As GHC had no support for this, and effecively used $CC to link, people started adding this into the cabal file to support as many platforms as they could to pass the necessary flags to $CC to behave like $CXX

system-cxx-std-lib's package contains the following information

extra-libraries:      @CXX_STD_LIB_LIBS@
library-dirs:         @CXX_STD_LIB_LIB_DIRS@
dynamic-library-dirs: @CXX_STD_LIB_DYN_LIB_DIRS@

which will then be threaded through to $CC. Just adding one more indirection.

Treating $CC as $CXX is bad and fragile.

As can be seen above if all you have is $CC, and you really want $CXX you end up with a hopelessly convoluted mess across platforms (and technically also across c++ compilers). If we want to link c++ code, we should defer to the c++ compiler that knows by design how to do this.

system-cxx-std-lib sits at the wrong level

From the text example above we can see that system-cxx-std-lib does appear to simplify the situation by just having one build-depends package. Which is virtual and will inject the platform necessary flags. This however means that we need to have one system-cxx-std-lib package for each and every platform we want to ever support (and the platforms c++ compiler already knows how to deal with this!). We also bake these value into build or install time of GHC. If the c++ compiler/toolchain is updated after the installation of GHC things will break in fun ways.

What do we want?

Fundamentally we want a signal to ghc when we want to link c++ code. Luckily this is not blind, we have this signal, even if somewhat crude. People will put c++ or stdc++ in their extra-libraries (or now system-cxx-std-lib in their build-depends 🙄). Which tells us that they really want to link against c++, and thus we should use the c++ compiler.

system-cxx-std-lib is unnecessary complexity

As just outlined, we already have the signal to use $CXX instead of $CC, but now we have two different ones. The situation didn't get any better (old packages will not have system-cxx-std-lib but still some form of c++ or stdc++). Now we have multiple signals across multiple paths (extra-libraries, build-depends).

This will help with other languages.

There is no known (to me) language where this will help. We have pretty much collectively agreed that the c abi is the lingua franca for linking. And other languages come with their own package managers that will happily produce static archives with the necessary dependencies (and not a confusing set of stdlibs like c++ compilers) or properly linked DSOs. If they need standard lirbaries or runtimes those will be linked into the their static archive, or DSO. (ghc even has a similar trick up it's sleve with -staticlib). At that point these libraries can be treated like any other and don't need any specific treatment like the C++ one. I've only remotely heard about some issues with CUDA + C++ + X being tricky. This seems such an esoteric case for haskell today, that I'm sure the handful of people who do this today can work around this with the options in cabal to pass flags to the linker, ghc, ... and others as needed. And even then I'm not convinced that the system-cuda-cxx-std-lib solution would be the right direction to go.

And now?

Given the current mixed state, we now need to have GHC ignore system-css-std-lib in the build-depends as well as c++ stdc++ in the extra-libraries, and treat those just a signals to switch from using $CC as our linker to $CXX as our linker.

---

Ultimately, the complexity of system-cxx-std-lib outweighs its benefits. By treating c++ or stdc++ in extra-libraries as a signal to use $CXX, GHC could simplify C++ linking, align with existing conventions, and remain flexible for future toolchain changes—all while minimizing disruption to existing packages.

[Ericson2314]

And other languages come with their own package managers that will happily produce static archives with the necessary dependencies (and not a confusing set of stdlibs like c++ compilers) or properly linked DSOs.

To be clear, the C++ thing in question is:

1. Create static archive / object files for the C++ code that is morally in this library
2. Such code depends on an external C++ library which is a DSO

That IMO is not such a weird situation. If there is some foreign code that is in a hot path, it makes sense we would want it right in the the same library as the haskell code calling it, not in a separate DSO. As to whether the standard library is a DSO or statically linked, I have no problem statically linking it, but I could see situations if it is big bulky where it happens to be dynamically linked.

Our general view is standard libraries are or should be "just libraries" --- that is why we like e.g. reinstallable base, cross compilation base on demand, etc. system-cxx-std-lib in my view is recognizing that the C++ standard library is just another library, and thus we ought to be able to explicitly depend on that library for sake of any C++ files we have.

The current logic with the many branches is, yes, not scalable to more C++ implementations. But an configure script, rather than "big expert system of guesses", that programmatically ran $CXX and looked at how invoked the linker, would do the right thing, and do the right thing everywhere.

My hope is that eventually C++ and all language implementations are encouraged to do their own version of "reinstallable base" and provide .pc files (or equivalent) for all their standard libraries, especially the DSO ones. Then we can replace system-cxx-std-lib with a bog-standard pkgconfig-depends and the problem goes away!

[hsyl20]

This should be discussed with upstream as it impacts the whole ecosystem.