From 44e8243c2f68f09d6dcf3d86edddc17ea47739d0 Mon Sep 17 00:00:00 2001 From: Aidan Hobson Sayers Date: Thu, 21 May 2015 16:45:38 +0100 Subject: [PATCH 1/5] Rename 'link-args' to 'advanced-linking', add intro --- src/doc/trpl/SUMMARY.md | 2 +- src/doc/trpl/{link-args.md => advanced-linking.md} | 9 ++++++++- 2 files changed, 9 insertions(+), 2 deletions(-) rename src/doc/trpl/{link-args.md => advanced-linking.md} (76%) diff --git a/src/doc/trpl/SUMMARY.md b/src/doc/trpl/SUMMARY.md index ca3381ffba465..f1e51591aea0e 100644 --- a/src/doc/trpl/SUMMARY.md +++ b/src/doc/trpl/SUMMARY.md @@ -63,7 +63,7 @@ * [No stdlib](no-stdlib.md) * [Intrinsics](intrinsics.md) * [Lang items](lang-items.md) - * [Link args](link-args.md) + * [Advanced linking](advanced-linking.md) * [Benchmark Tests](benchmark-tests.md) * [Box Syntax and Patterns](box-syntax-and-patterns.md) * [Slice Patterns](slice-patterns.md) diff --git a/src/doc/trpl/link-args.md b/src/doc/trpl/advanced-linking.md similarity index 76% rename from src/doc/trpl/link-args.md rename to src/doc/trpl/advanced-linking.md index ee5159afb8e6f..2b38bd3f15968 100644 --- a/src/doc/trpl/link-args.md +++ b/src/doc/trpl/advanced-linking.md @@ -1,4 +1,11 @@ -% Link args +% Advanced Linking + +The common cases of linking with Rust have been covered earlier in this book, +but supporting the range of linking possibilities made available by other +languages is important for Rust to achieve seamless interaction with native +libraries. + +# Link args There is one other way to tell rustc how to customize linking, and that is via the `link_args` attribute. This attribute is applied to `extern` blocks and From f3f3e864224e98d5e93d644cd653a722d0005c5c Mon Sep 17 00:00:00 2001 From: Aidan Hobson Sayers Date: Thu, 21 May 2015 16:47:41 +0100 Subject: [PATCH 2/5] Note possiblities of empty extern blocks (based on #12575) --- src/doc/reference.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/doc/reference.md b/src/doc/reference.md index c19aec78de2e4..ee65004664083 100644 --- a/src/doc/reference.md +++ b/src/doc/reference.md @@ -1620,6 +1620,10 @@ The type of a function declared in an extern block is `extern "abi" fn(A1, ..., An) -> R`, where `A1...An` are the declared types of its arguments and `R` is the declared return type. +It is valid to add the `link` attribute on an empty extern block. You can use +this to satisfy the linking requirements of extern blocks elsewhere in your code +(including upstream crates) instead of adding the attribute to each extern block. + ## Visibility and Privacy These two terms are often used interchangeably, and what they are attempting to From 354026cf830de80262dcf481ec3c975ad924e42b Mon Sep 17 00:00:00 2001 From: Aidan Hobson Sayers Date: Thu, 21 May 2015 19:55:51 +0100 Subject: [PATCH 3/5] Static linking --- src/doc/trpl/advanced-linking.md | 126 +++++++++++++++++++++++++++++++ 1 file changed, 126 insertions(+) diff --git a/src/doc/trpl/advanced-linking.md b/src/doc/trpl/advanced-linking.md index 2b38bd3f15968..3b2bc0d7cb7e3 100644 --- a/src/doc/trpl/advanced-linking.md +++ b/src/doc/trpl/advanced-linking.md @@ -30,3 +30,129 @@ meaning. It is highly recommended to *not* use this attribute, and rather use the more formal `#[link(...)]` attribute on `extern` blocks instead. +# Static linking + +Static linking refers to the process of creating output that contain all +required libraries and so don't need libraries installed on every system where +you want to use your compiled project. Rust libraries are statically linked by +default so you can use Rust-created binaries and libraries without installing +the Rust runtime everywhere. By contrast, native libraries are dynamically +linked by default, but sometimes it can be useful to change this. + +Linking is a very platform dependant topic - on some platforms, static linking +may not be possible at all! This section assumes some basic familiarity with +linking on your platform on choice. + +## Linux + +By default, all Rust programs on Linux will link to the system libc along with +a number of other libraries Let's look at an example on a 64-bit linux machine +with GCC and glibc (by far the most common libc on Linux): + +``` text +$ cat example.rs +fn main() {} +$ rustc example.rs +$ ldd example + linux-vdso.so.1 => (0x00007ffd565fd000) + libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa81889c000) + libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa81867e000) + librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fa818475000) + libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa81825f000) + libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa817e9a000) + /lib64/ld-linux-x86-64.so.2 (0x00007fa818cf9000) + libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa817b93000) +``` + +Dynamic linking on Linux can be undesirable if you wish to target older +machines as applications compiled aginst newer versions glibc are not +guaranteed to run against older versions. + +You can examine Rust linking arguments with an option to rustc. Newlines have +been added for readability: + +``` text +$ rustc example.rs -Z print-link-args +"cc" + "-Wl,--as-needed" + "-m64" + [...] + "-o" "example" + "example.o" + "-Wl,--whole-archive" "-lmorestack" "-Wl,--no-whole-archive" + "-Wl,--gc-sections" + "-pie" + "-nodefaultlibs" + [...] + "-Wl,--whole-archive" "-Wl,-Bstatic" + "-Wl,--no-whole-archive" "-Wl,-Bdynamic" + "-ldl" "-lpthread" "-lrt" "-lgcc_s" "-lpthread" "-lc" "-lm" "-lcompiler-rt" +``` + +Arguments with a `-L` before them set up the linker search path and arguments +ending with `.rlib` are linking Rust crates statically into your application. +Neither of these are relevent for static linking so have been ommitted. + +The first step in being able to statically link is to obtain an object file. +This can be achieved with `rustc --emit obj example.rs`, and creates a file +called `example.o`, which you can see being passed in the command line above - +rustc automatically deletes it when finished with it by default. As you now have +the object file, you should be able to run the link command obtained with +`print-link-args` to create perform the linking stage yourself. + +In order to statically link, there are a number of changes you must make. Below +is the command required to perform a static link; we will go through them each +in turn. + +``` text +$ rustc example.rs -Z print-link-args +"cc" + "-static" + "-m64" + [...] + "-o" "example" + "example.o" + "-Wl,--whole-archive" "-lmorestack" "-Wl,--no-whole-archive" + "-Wl,--gc-sections" + "-nodefaultlibs" + [...] + "-Wl,--whole-archive" + "-Wl,--no-whole-archive" + "-ldl" "-lpthread" "-lrt" "-lgcc_eh" "-lpthread" "-lc" "-lm" "-lcompiler-rt" +``` + + - `-static` was added - this is the signal to the compiler to use a static + glibc, among other things + - `-Wl,--as-needed` was removed - this can be left in, but is unnecessary + as it only applies to dynamic librares + - `-pie` was removed - this is not compatible with static binaries + - both `-Wl,-B*` options were removed - everything will be linked statically, + so informing the linker of how certain libraries should be linked is not + appropriate + - `-lgcc_s` was changed to `-lgcc_eh` - `gcc_s` is the GCC support library, + which Rust uses for unwinding support. This is only available as a dynamic + library, so we must specify the static version of the library providing + unwinding support. + +By running this command, you will likely see some warnings like + +``` text +warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking +``` + +These should be considered carefully! They indicate calls in glibc which +*cannot* be statically linked without significant extra effort. An application +using these calls will find it is not as portable as 'static binary' would imply. +Rust supports targeting musl as an alternative libc to be able to fully +statically link these calls. + +As we are confident that our code does not use these calls, we can now see the +fruits of our labour: + +``` +$ ldd example + not a dynamic executable +``` + +This binary can be copied to virtually any 64-bit Linux machine and work +without requiring external libraries. From 7e334ebcabcbdfb8ca99a234696accd238c05402 Mon Sep 17 00:00:00 2001 From: Aidan Hobson Sayers Date: Mon, 15 Jun 2015 18:54:37 +0100 Subject: [PATCH 4/5] musl static linking not glibc --- src/doc/trpl/advanced-linking.md | 144 ++++++++++++++----------------- 1 file changed, 67 insertions(+), 77 deletions(-) diff --git a/src/doc/trpl/advanced-linking.md b/src/doc/trpl/advanced-linking.md index 3b2bc0d7cb7e3..adb5749838953 100644 --- a/src/doc/trpl/advanced-linking.md +++ b/src/doc/trpl/advanced-linking.md @@ -41,12 +41,12 @@ linked by default, but sometimes it can be useful to change this. Linking is a very platform dependant topic - on some platforms, static linking may not be possible at all! This section assumes some basic familiarity with -linking on your platform on choice. +linking on your platform of choice. ## Linux By default, all Rust programs on Linux will link to the system libc along with -a number of other libraries Let's look at an example on a 64-bit linux machine +a number of other libraries. Let's look at an example on a 64-bit linux machine with GCC and glibc (by far the most common libc on Linux): ``` text @@ -68,91 +68,81 @@ Dynamic linking on Linux can be undesirable if you wish to target older machines as applications compiled aginst newer versions glibc are not guaranteed to run against older versions. -You can examine Rust linking arguments with an option to rustc. Newlines have -been added for readability: +Static linking supported via an alternative libc, musl - this must be enabled +at rust compile-time with some prerequisites available. You can compile your +own version of rust with musl enabled and install it into a custom directory +with the instructions below: -``` text -$ rustc example.rs -Z print-link-args -"cc" - "-Wl,--as-needed" - "-m64" - [...] - "-o" "example" - "example.o" - "-Wl,--whole-archive" "-lmorestack" "-Wl,--no-whole-archive" - "-Wl,--gc-sections" - "-pie" - "-nodefaultlibs" - [...] - "-Wl,--whole-archive" "-Wl,-Bstatic" - "-Wl,--no-whole-archive" "-Wl,-Bdynamic" - "-ldl" "-lpthread" "-lrt" "-lgcc_s" "-lpthread" "-lc" "-lm" "-lcompiler-rt" ``` - -Arguments with a `-L` before them set up the linker search path and arguments -ending with `.rlib` are linking Rust crates statically into your application. -Neither of these are relevent for static linking so have been ommitted. - -The first step in being able to statically link is to obtain an object file. -This can be achieved with `rustc --emit obj example.rs`, and creates a file -called `example.o`, which you can see being passed in the command line above - -rustc automatically deletes it when finished with it by default. As you now have -the object file, you should be able to run the link command obtained with -`print-link-args` to create perform the linking stage yourself. - -In order to statically link, there are a number of changes you must make. Below -is the command required to perform a static link; we will go through them each -in turn. - -``` text -$ rustc example.rs -Z print-link-args -"cc" - "-static" - "-m64" - [...] - "-o" "example" - "example.o" - "-Wl,--whole-archive" "-lmorestack" "-Wl,--no-whole-archive" - "-Wl,--gc-sections" - "-nodefaultlibs" - [...] - "-Wl,--whole-archive" - "-Wl,--no-whole-archive" - "-ldl" "-lpthread" "-lrt" "-lgcc_eh" "-lpthread" "-lc" "-lm" "-lcompiler-rt" +$ mkdir musldist +$ PREFIX=$(pwd)/musldist +$ +$ # Build musl +$ wget http://www.musl-libc.org/releases/musl-1.1.10.tar.gz +[...] +$ tar xf musl-1.1.10.tar.gz +$ cd musl-1.1.10/ +musl-1.1.10 $ ./configure --disable-shared --prefix=$PREFIX +[...] +musl-1.1.10 $ make +[...] +musl-1.1.10 $ make install +[...] +musl-1.1.10 $ cd .. +$ du -h musldist/lib/libc.a +2.2M musldist/lib/libc.a +$ +$ # Build libunwind.a +$ wget http://llvm.org/releases/3.6.1/llvm-3.6.1.src.tar.xz +$ tar xf llvm-3.6.1.src.tar.xz +$ cd llvm-3.6.1.src/projects/ +llvm-3.6.1.src/projects $ svn co http://llvm.org/svn/llvm-project/libcxxabi/trunk/ libcxxabi +llvm-3.6.1.src/projects $ svn co http://llvm.org/svn/llvm-project/libunwind/trunk/ libunwind +llvm-3.6.1.src/projects $ sed -i 's#^\(include_directories\).*$#\0\n\1(../libcxxabi/include)#' libunwind/CMakeLists.txt +llvm-3.6.1.src/projects $ mkdir libunwind/build +llvm-3.6.1.src/projects $ cd libunwind/build +llvm-3.6.1.src/projects/libunwind/build $ cmake -DLLVM_PATH=../../.. -DLIBUNWIND_ENABLE_SHARED=0 .. +llvm-3.6.1.src/projects/libunwind/build $ make +llvm-3.6.1.src/projects/libunwind/build $ cp lib/libunwind.a $PREFIX/lib/ +llvm-3.6.1.src/projects/libunwind/build $ cd cd ../../../../ +$ du -h musldist/lib/libunwind.a +164K musldist/lib/libunwind.a +$ +$ # Build musl-enabled rust +$ git clone https://github.com/rust-lang/rust.git muslrust +$ cd muslrust +muslrust $ ./configure --target=x86_64-unknown-linux-musl --musl-root=$PREFIX --prefix=$PREFIX +muslrust $ make +muslrust $ make install +muslrust $ cd .. +$ du -h musldist/bin/rustc +12K musldist/bin/rustc ``` - - `-static` was added - this is the signal to the compiler to use a static - glibc, among other things - - `-Wl,--as-needed` was removed - this can be left in, but is unnecessary - as it only applies to dynamic librares - - `-pie` was removed - this is not compatible with static binaries - - both `-Wl,-B*` options were removed - everything will be linked statically, - so informing the linker of how certain libraries should be linked is not - appropriate - - `-lgcc_s` was changed to `-lgcc_eh` - `gcc_s` is the GCC support library, - which Rust uses for unwinding support. This is only available as a dynamic - library, so we must specify the static version of the library providing - unwinding support. - -By running this command, you will likely see some warnings like +You now have a build of a musl-enabled rust! Because we've installed it to a +custom prefix we need to make sure our system can the binaries and appropriate +libraries when we try and run it: -``` text -warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking +``` +$ export PATH=$PREFIX/bin:$PATH +$ export LD_LIBRARY_PATH=$PREFIX/lib:$LD_LIBRARY_PATH ``` -These should be considered carefully! They indicate calls in glibc which -*cannot* be statically linked without significant extra effort. An application -using these calls will find it is not as portable as 'static binary' would imply. -Rust supports targeting musl as an alternative libc to be able to fully -statically link these calls. - -As we are confident that our code does not use these calls, we can now see the -fruits of our labour: +Let's try it out! ``` +$ echo 'fn main() { println!("hi!"); panic!("failed"); }' > example.rs +$ rustc --target=x86_64-unknown-linux-musl example.rs $ ldd example not a dynamic executable +$ ./example +hi! +thread '
' panicked at 'failed', example.rs:1 ``` -This binary can be copied to virtually any 64-bit Linux machine and work -without requiring external libraries. +Success! This binary can be copied to almost any Linux machine with the same +machine architecture and run without issues. + +`cargo build` also permits the `--target` option so you should be able to build +your crates as normal. However, you may need to recompile your native libraries +against musl before they can be linked against. From 6a56a450d259d6e410650f4f23988f844ffcd53e Mon Sep 17 00:00:00 2001 From: Aidan Hobson Sayers Date: Mon, 15 Jun 2015 18:58:28 +0100 Subject: [PATCH 5/5] Additional notes to link-args --- src/doc/trpl/advanced-linking.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/doc/trpl/advanced-linking.md b/src/doc/trpl/advanced-linking.md index adb5749838953..8eaa3878ae51f 100644 --- a/src/doc/trpl/advanced-linking.md +++ b/src/doc/trpl/advanced-linking.md @@ -22,10 +22,12 @@ extern {} Note that this feature is currently hidden behind the `feature(link_args)` gate because this is not a sanctioned way of performing linking. Right now rustc -shells out to the system linker, so it makes sense to provide extra command line +shells out to the system linker (`gcc` on most systems, `link.exe` on MSVC), +so it makes sense to provide extra command line arguments, but this will not always be the case. In the future rustc may use LLVM directly to link native libraries in which case `link_args` will have no -meaning. +meaning. You can achieve the same effect as the `link-args` attribute with the +`-C link-args` argument to `rustc`. It is highly recommended to *not* use this attribute, and rather use the more formal `#[link(...)]` attribute on `extern` blocks instead.