[CMake] Support Macros in Linux #68082

rintaro · 2023-08-23T06:10:42Z

For compiling codes required for macro support, we now need swiftc compiler in the build machine.

Unlike Darwin OSes, where swiftCore runtime is guaranteed to be present in /usr/lib, Linux doesn't have ABI stability and the stdlib of the build machine is not at the specific location. So the built compiler cannot relies on the shared object in the system.

rintaro · 2023-08-23T06:16:08Z

swiftlang/swift-syntax#2090
@swift-ci Please smoke test

rintaro · 2023-08-23T06:17:46Z

SwiftCompilerSources/CMakeLists.txt

This is to avoid mtime of HeaderDependencies.cpp being updated every time cmake configuration happens.

rintaro · 2023-08-23T06:21:31Z

cmake/modules/AddSwift.cmake

RUNPATH is now like $ORIGIN/../lib/swift/linux:/opt/swift/5.8.1/usr/lib/swift/linux
So after the swiftCore is built in the build directory, that's used.

rintaro · 2023-08-23T06:24:11Z

cmake/modules/AddSwiftUnittests.cmake

e.g. runtime tests is a "target" test, not "host" test. "target" test should not link with host libraries.

rintaro · 2023-08-23T06:26:45Z

cmake/modules/SwiftImplicitImport.cmake

Linux swiftc reads the output file after compiling (for e.g. autolink-extract). Since /dev/null is not a valid object file, this failed.
All we want to check here is if the frontend accepts the command line option, so -parse should be enough.

rintaro · 2023-08-23T06:27:51Z

lib/CMakeLists.txt

This is really meant to be a "build" directory where earlyswiftsyntax's .so files exist.

rintaro · 2023-08-23T06:29:57Z

lib/CMakeLists.txt

SwiftSyntax libraries (.so and .swiftmodule) are now being installed from swift build directory, but not earlyswiftsyntax build directory. So we can centralize the RUNPATH mutation logic in this repository.

rintaro · 2023-08-23T06:32:00Z

lib/Macros/CMakeLists.txt

Linux's RUNPATH doesn't inherits executable's RUNPATH. So each .so must have its own complete RUNPATH.

rintaro · 2023-08-23T17:32:01Z

swiftlang/swift-syntax#2090
@swift-ci Please smoke test

finagolfin · 2023-08-23T19:25:02Z

I notice you are stripping builder runpaths. I haven't look into why that's needed in detail, but maybe it won't be needed if -no-toolchain-stdlib-rpath is passed in so the Swift compiler doesn't set that builder runpath and you set it manually with CMake's BUILD_RPATH instead, which is automatically removed before installation.

That may be easier than removing builder runpaths yourself before installation, as you seem to be doing here? I could be way off since I didn't look deeply into why you're doing this.

rintaro · 2023-08-23T20:59:54Z

swiftlang/swift-syntax#2090
@swift-ci Please smoke test

rintaro · 2023-08-23T23:16:43Z

swiftlang/swift-syntax#2090
@swift-ci Please smoke test

For compiling codes required for macro support, we now need swiftc compiler in the build machine. Unlike Darwin OSes, where swiftCore runtime is guaranteed to be present in /usr/lib, Linux doesn't have ABI stability and the stdlib of the build machine is not at the specific location. So the built compiler cannot relies on the shared object in the toolchain.

rintaro · 2023-08-24T17:44:40Z

swiftlang/swift-syntax#2090
@swift-ci Please smoke test

bnbarham · 2023-08-24T20:37:46Z

I notice you are stripping builder runpaths. I haven't look into why that's needed in detail, but maybe it won't be needed if -no-toolchain-stdlib-rpath is passed in so the Swift compiler doesn't set that builder runpath and you set it manually with CMake's BUILD_RPATH instead, which is automatically removed before installation.

That may be easier than removing builder runpaths yourself before installation, as you seem to be doing here? I could be way off since I didn't look deeply into why you're doing this.

The TLDR is that we need to cherry-pick this change to 5.9.0 and thus would like as few changes as possible.

-no-toolchain-stdlib-rpath is actually being used for the "pure swift" libs and tools in the compiler, but only so that we can add the host path to the end rather than the start. We could split BUILD/INSTALL_RPATH here rather than using BUILD_WITH_INSTALL_RPATH and then stripping, but that's a much larger change (especially if we want to do it for the non-"pure swift" cases).

We also need to do this for the copied-in "early" swift-syntax libs, which currently aren't part of the swift build itself. Since they're built separately, we can't do the same BUILD/INSTALL_RPATH split. I have a PR up to build swift-syntax in the same tree (#66043), but will get back to that after this is in and 5.9 is released (I ran into some issues with the bootstrapping case, which I haven't had the time to look into). I also have a local patch that does most of the BUILD/INSTALL_RPATH splitting that I'll put up as well.

Once that's all in, I definitely agree that using BUILD/INSTALL_RPATH here rather than the manual stripping is the way to go.

rintaro · 2023-08-24T23:08:21Z

swiftlang/swift-syntax#2090
@swift-ci Please smoke test

Many shared libs and executables are only run after stdlib/runtime are built. They don't need to link with builders stdlib at all.

rintaro · 2023-08-24T23:10:48Z

swiftlang/swift-syntax#2090
@swift-ci Please smoke test

and Swift parser integration is enabled. If swift parser integration is enabled, SwiftSyntax libraries are always built with host tools. Other SwiftCompilerSources modules must use the same runtime with parser libraries.

rintaro · 2023-08-24T23:32:10Z

swiftlang/swift-syntax#2090
@swift-ci Please smoke test

rintaro · 2023-08-25T17:22:15Z

swiftlang/swift-syntax#2090
@swift-ci Please Build Toolchain Linux Platform

rintaro · 2023-08-25T17:22:42Z

swiftlang/swift-syntax#2090
@swift-ci Please Build Toolchain macOS Platform

bnbarham · 2023-08-25T18:41:40Z

cmake/modules/AddPureSwift.cmake

+    get_filename_component(swift_bin_dir ${CMAKE_Swift_COMPILER} DIRECTORY)
+    get_filename_component(swift_dir ${swift_bin_dir} DIRECTORY)
+    set(host_lib_dir "${swift_dir}/lib/swift/${SWIFT_SDK_${SWIFT_HOST_VARIANT_SDK}_LIB_SUBDIR}")


Are these used? I guess this is currently working because ASTGen is statically linked and the pure tools would all use FALSE now anyway, so this function is only used by the plugins. But those aren't used except for in tests and thus host isn't actually needed anywhere that uses this right now?

They are leftover. Just removed.

bnbarham · 2023-08-25T18:43:15Z

cmake/modules/AddSwift.cmake


  if (ASHL_SHARED AND ASHL_HAS_SWIFT_MODULES)
-      _add_swift_runtime_link_flags(${name} "." "")
+      _add_swift_runtime_link_flags(${name} "." "" FALSE)


Hmm, maybe required_for_minimal_compiler doesn't make sense as the name. Since these ... are required, it's just they don't have any Swift in them.

I don't think any shared (ASHL_SHARED) libraries are required for building stdlib.

Ah this is only shared, true. Having said that, if we did move to shared libraries more, it would be the case that they are "required for minimal" but this would still be FALSE because they wouldn't have any Swift. And that was more my point.

bnbarham · 2023-08-25T18:48:04Z

cmake/modules/SwiftStripBuilderRpath.cmake

+  if((NOT SWIFT_SWIFT_PARSER) AND NOT(BOOTSTRAPPING_MODE STREQUAL "HOSTTOOLS"))
+      return()
+  endif()


Should this be an OR? ie. We just want !(linux && swift_swift_parser && bootstrapping == hosttools) but this is !swift_swift_parser && bootstrapping != hosttools

My intent was !(linux && (swift_swift_parser || boostrapping==hosttools)) because boostrapping==hosttools alone also needs to strip builder's RUNPATH.

bootstrapping==hosttools without early swift syntax shouldn't have the host in the RUNPATH anyway though right?

It should, otherwise which runtime does the SwiftCompilerSources library link with (before it builds stdlib/runtime)?

Ah true, forgot about SwiftCompilerSources here 😅 Thanks, sorry for the run around!

bnbarham · 2023-08-25T18:59:09Z

tools/SourceKit/cmake/modules/AddSwiftSourceKit.cmake

+      # So there's no need to add RUNPATH to builder's runtime libraries.

    elseif(ASKD_BOOTSTRAPPING_MODE STREQUAL "BOOTSTRAPPING")
      get_bootstrapping_swift_lib_dir(bs_lib_dir "")


Lines 169-191 are very odd, given that there is no bootstrapping for SK. I know we're trying to make minimal changes, but IMO we should at least:

Delete 159 which is just plain wrong

Delete the else branch, since we can't get into there for Linux anyway

Remove the bootstrapping check, since it's always empty

Delete 161-165 since that's handled just above this comment

Ah, (4) adds host. IMO I'd move that to inside the ifs here since the if there is wrong anyway (it should be specifically linux/android/openbsd). So ie.

if(SWIFT_SWIFT_PARSER) file(RELATIVE_PATH relative_hostlib_path "${path}" "${SWIFTLIB_DIR}/host") list(APPEND RPATH_LIST "$ORIGIN/${relative_hostlib_path}") endif()

bnbarham · 2023-08-25T19:00:44Z

tools/driver/CMakeLists.txt

    HAS_SWIFT_MODULES
    BOOTSTRAPPING 0
    THINLTO_LD64_ADD_FLTO_CODEGEN_ONLY
+    REQUIRED_FOR_MINIMAL_COMPILER


Could skip adding this and the next, since they're only used for bootstrapping which we're not doing.

bnbarham · 2023-08-25T19:01:54Z

tools/libSwiftScan/CMakeLists.txt

+    # Ensure that we can find the host shared libraries.
+    set_property(
+      TARGET libSwiftScan
+      APPEND PROPERTY INSTALL_RPATH "@loader_path/swift/host")


Could just append them both at once with : but doesn't really matter.

bnbarham · 2023-08-25T19:04:24Z

utils/build-presets.ini

 install-llbuild
 install-swiftpm
 install-swift-driver
-install-swiftsyntax


Could just leave this in since the produce is changed. There's a lot of these that would need to be removed anyway.

bnbarham · 2023-08-25T19:06:12Z

utils/build-presets.ini

+# Linux only support 'hosttools' with early-swiftsyntax.
+bootstrapping=hosttools


We're going to need to add this in quite a few places. Maybe we should override and set to hosttools when early swiftsyntax is enabled, since that's the only thing that works? Bit weird but...

Updated to force setting set(BOOTSTRAPPING_MODE "HOSTTOOLS") when SWIFT_SWIFT_PARSER is enabled.

bnbarham · 2023-08-25T19:06:54Z

utils/swift_build_support/swift_build_support/products/earlyswiftsyntax.py

-        """
-        self.install_with_cmake(["install"], self.host_install_destdir(host_target))
+        # No-op.
+        None


pass is the Python-ism you want here

bnbarham · 2023-08-25T19:07:28Z

Thanks Rintaro!

Swift Build Toolchain Ubuntu 20.04 (x86_64) — Done

:O!

SourceKit components are never built while bootstrapping stages. So it doen't need to have stage 0 or 1 handling.

rintaro · 2023-08-25T22:42:07Z

swiftlang/swift-syntax#2090
@swift-ci Please smoke test

rintaro · 2023-08-25T22:42:32Z

swiftlang/swift-syntax#2090
@swift-ci Please Build Toolchain

`$ORIGIN` is only supported in Linux, BSD, etc. (i.e. not in Windows)

rintaro · 2023-08-25T23:48:28Z

swiftlang/swift-syntax#2090
@swift-ci Please smoke test

rintaro · 2023-08-28T17:38:45Z

swiftlang/llvm-project#7331
swiftlang/swift-syntax#2090
@swift-ci Please Build Toolchain

libSwiftScan is built in 'lib' but installed in 'lib/swift/host' RUNPATH should have correct '$ORIGIN/../{platform}' to load 'swiftCore' runtime library.

swift-compatibility-symbols, swift-def-to-strings-converter, and swift-serialize-diagnostics don't use any Swift modules. But when SWIFT_SWIFT_PARSER was enabled, they are linked with swiftCore. But these binaries can be executed before the runtime is being built. We need to stop them linking with swiftCore.

rintaro · 2023-08-28T19:14:24Z

swiftlang/llvm-project#7331
swiftlang/swift-syntax#2090
@swift-ci Please smoke test

rintaro · 2023-08-28T19:14:35Z

swiftlang/llvm-project#7331
swiftlang/swift-syntax#2090
@swift-ci Please Build Toolchain

rintaro · 2023-08-28T22:29:59Z

swiftlang/llvm-project#7331
swiftlang/swift-syntax#2090
@swift-ci Please Build Toolchain Windows platform

RPATH_SET is not available until cmake 3.21.0. Use RPATH_CHANGE instead.

rintaro · 2023-08-29T01:13:28Z

swiftlang/swift-syntax#2090
@swift-ci Please smoke test

rintaro · 2023-08-29T01:13:41Z

swiftlang/swift-syntax#2090
@swift-ci Please Build Toolchain

rintaro · 2023-08-29T01:55:45Z

swiftlang/swift-syntax#2090
@swift-ci Please Build Toolchain

rintaro · 2023-08-29T02:11:56Z

swiftlang/swift-syntax#2090
@swift-ci Please Build Toolchain

Some non-stdlib thing e.g. swift-backtrace still might be built before the runtime is built. For building Swift code in Linux "hosttools", always set 'LD_LIBRARY_PATH' to the runtime in the builder.

rintaro · 2023-08-29T16:40:20Z

swiftlang/swift-syntax#2090
@swift-ci Please Build Toolchain

rintaro · 2023-08-29T16:40:30Z

swiftlang/swift-syntax#2090
@swift-ci Please smoke test

finagolfin · 2023-08-30T10:19:53Z

I see a handful of runpath issues with the last CI-built toolchain for linux after this pull, I'll look into the fixes:

> ./usr/bin/lldb-server
./usr/bin/lldb-server: error while loading shared libraries: libswiftCore.so: cannot open shared object file: No such file or directory
> readelf -d ./usr/bin/lldb-server |ag runpath
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../lib:$ORIGIN/swift/linux:$ORIGIN/../lib/swift/host]

That second relative path is wrong, hence the runtime error.

bnbarham · 2023-08-30T16:22:11Z

Thanks for checking @finagolfin! Looks like https://github.com/apple/llvm-project/blob/63a082e326d053e778a05a899514ba101d7db5f6/lldb/cmake/modules/AddLLDB.cmake#L225 needs a ${reldir}lib

finagolfin · 2023-08-30T18:48:19Z

tools/libSwiftScan/CMakeLists.txt

+  elseif(SWIFT_HOST_VARIANT_SDK MATCHES "LINUX|ANDROID|OPENBSD|FREEBSD")
+    set_property(
+      TARGET libSwiftScan
+      APPEND PROPERTY INSTALL_RPATH "$ORIGIN/swift/host")


@rintaro, is this needed for some intermediate build step or something? Because the final installed library also has this path, which is wrong:

> readelf -d ./usr/lib/swift/host/lib_InternalSwiftScan.so |ag runpath 0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN:$ORIGIN/./swift/linux:$ORIGIN/../linux:$ORIGIN/swift/host:$ORIGIN/../host]

Yeah, see the many FIXME above. This all needs to be split into BUILD/INSTALL_RPATH + built into host rather than into lib, but that's too much for 5.9. We'll fix this up after all the 5.9 PRs are in as well. We could remove it, but it doesn't really hurt either.

rintaro commented Aug 23, 2023

View reviewed changes

rintaro force-pushed the swift-swift-linux branch from 800b056 to 4c10475 Compare August 23, 2023 17:31

rintaro force-pushed the swift-swift-linux branch from 4c10475 to 37216ab Compare August 23, 2023 20:57

bnbarham mentioned this pull request Aug 24, 2023

Install .swiftdoc files into the toolchain swiftlang/swift-syntax#2097

Closed

rintaro force-pushed the swift-swift-linux branch from 37216ab to 9c9010e Compare August 24, 2023 17:33

Avoid adding redundant temporary RUNPATH to builder's stdlib

2a2787b

Many shared libs and executables are only run after stdlib/runtime are built. They don't need to link with builders stdlib at all.

rintaro force-pushed the swift-swift-linux branch from 26cc7d5 to 40841b3 Compare August 24, 2023 23:10

[CMake] Fatal error when bootstrapping is not "hosttools"

0f0c492

and Swift parser integration is enabled. If swift parser integration is enabled, SwiftSyntax libraries are always built with host tools. Other SwiftCompilerSources modules must use the same runtime with parser libraries.

rintaro force-pushed the swift-swift-linux branch from 40841b3 to 0f0c492 Compare August 24, 2023 23:32

bnbarham reviewed Aug 25, 2023

View reviewed changes

rintaro added 2 commits August 25, 2023 21:58

[CMake] Update for review comments

9017ef5

[CMake] Fix weiredness in swift runtime linking for SourceKit libraries

914a6fd

SourceKit components are never built while bootstrapping stages. So it doen't need to have stage 0 or 1 handling.

[CMake] Further cleanup for RUNPATH setting

602604b

`$ORIGIN` is only supported in Linux, BSD, etc. (i.e. not in Windows)

rintaro added 2 commits August 28, 2023 18:45

[CMake] Fix RUNPATH of lib_InternalSwiftScan in Linux

4ccc866

libSwiftScan is built in 'lib' but installed in 'lib/swift/host' RUNPATH should have correct '$ORIGIN/../{platform}' to load 'swiftCore' runtime library.

rintaro force-pushed the swift-swift-linux branch from ab0142b to dc68773 Compare August 28, 2023 19:13

rintaro changed the title ~~[WIP][CMake] Support Macros in Linux~~ [CMake] Support Macros in Linux Aug 28, 2023

rintaro marked this pull request as ready for review August 28, 2023 21:52

rintaro requested review from ahoppen, eeckstein and hamishknight as code owners August 28, 2023 21:52

[CMake] Use RPATH_CHANGE instead RPATH_SET

f645069

RPATH_SET is not available until cmake 3.21.0. Use RPATH_CHANGE instead.

rintaro force-pushed the swift-swift-linux branch from b4e7d09 to f645069 Compare August 29, 2023 01:12

[CMake] Always use builder's runtime for building Swift code in Linux

0aa0aac

Some non-stdlib thing e.g. swift-backtrace still might be built before the runtime is built. For building Swift code in Linux "hosttools", always set 'LD_LIBRARY_PATH' to the runtime in the builder.

rintaro mentioned this pull request Aug 29, 2023

[5.9][CMake] Support Macros in Linux #68190

Merged

bnbarham approved these changes Aug 30, 2023

View reviewed changes

bnbarham merged commit 0988236 into swiftlang:main Aug 30, 2023

finagolfin reviewed Aug 30, 2023

View reviewed changes

This was referenced Aug 30, 2023

[CMake][SourceKit] Add RPATH to 'lib/swift/host' #68219

Merged

[CMake] Update host platform check for macro support #68224

Merged

finagolfin mentioned this pull request Aug 31, 2023

[CMake] Tweak path separators in new Swift runpaths #68238

Merged

		# Linux only support 'hosttools' with early-swiftsyntax.
		bootstrapping=hosttools

[CMake] Support Macros in Linux #68082

[CMake] Support Macros in Linux #68082

Uh oh!

Conversation

rintaro commented Aug 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rintaro commented Aug 23, 2023

Uh oh!

rintaro Aug 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rintaro commented Aug 23, 2023

Uh oh!

finagolfin commented Aug 23, 2023

Uh oh!

rintaro commented Aug 23, 2023

Uh oh!

rintaro commented Aug 23, 2023

Uh oh!

rintaro commented Aug 24, 2023

Uh oh!

bnbarham commented Aug 24, 2023

Uh oh!

rintaro commented Aug 24, 2023

Uh oh!

rintaro commented Aug 24, 2023

Uh oh!

rintaro commented Aug 24, 2023

Uh oh!

rintaro commented Aug 25, 2023

Uh oh!

rintaro commented Aug 25, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bnbarham Aug 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

rintaro commented Aug 23, 2023 •

edited

Loading

rintaro Aug 23, 2023 •

edited

Loading

bnbarham Aug 25, 2023 •

edited

Loading

bnbarham commented Aug 30, 2023 •

edited

Loading