Problems with linking with system-wide installed libraries

[jauhien]

Short description

Rust linking system has one big problem now, that blocks possibility to have a given rust package to be installed and to be developed on the same system.

tl;dr: when you have a library installed system wide and you are developing package that contains this library (and uses other system wide installed libs) you'll have problems with linking because of

error: multiple dylib candidates for `hello_world` found

Example projects

Consider the minimal example, think we have those projects:

1. A library
2. A project that has executables + libraries

A library project looks like this:

jauhien@zcj linking % cat hello_lib/hello_lib.rs 
#![crate_type = "dylib"]

pub fn hello(name: &str) -> String {
    "Hello, ".to_string() + name + "!".to_string()
}

A project with binary and library looks like this:

jauhien@zcj linking % cat hello_world/hello_world.rs 
#![crate_type = "dylib"]

extern crate hello_lib;

use hello_lib::hello;

pub fn hello_name(name: &str, surname: &str) -> String {
    hello((name.to_string() + " " + surname).as_slice())
}

jauhien@zcj linking % cat hello_world/hello.rs 
#![crate_type = "bin"]

extern crate hello_world;

use hello_world::hello_name;

fn main() {
   println!("{}", hello_name("Test", "Test"))
}

Steps to reproduce the error

I use 'image' directory as root here.

1. Compile and install hello_lib library

   jauhien@zcj hello_lib % rustc hello_lib.rs 
   jauhien@zcj hello_lib % cp libhello_lib.so ../image/usr/lib 
   

2. Compile and install hello_world project that uses hello_lib

   jauhien@zcj hello_world % rustc hello_world.rs -L ../image/usr/lib
   jauhien@zcj hello_world % rustc hello.rs -L . -L ../image/usr/lib 
   jauhien@zcj hello_world % cp libhello_world.so ../image/usr/lib 
   jauhien@zcj hello_world % cp hello ../image/usr/bin 
   

   Test if it works:

   jauhien@zcj hello_world % LD_LIBRARY_PATH=../image/usr/lib ../image/usr/bin/hello
   Hello, Test Test!         
   

3. Continue developing of hello_world project and try to recompile it

   jauhien@zcj hello_world % rustc hello_world.rs -L ../image/usr/lib
   jauhien@zcj hello_world % rustc hello.rs -L . -L ../image/usr/lib 
   hello.rs:3:1: 3:26 error: multiple dylib candidates for `hello_world` found
   hello.rs:3 extern crate hello_world;
              ^~~~~~~~~~~~~~~~~~~~~~~~~
   hello.rs:3:1: 3:26 note: candidate #1: /home/jauhien/work/rust/linking/hello_world/libhello_world.so
   hello.rs:3 extern crate hello_world;
              ^~~~~~~~~~~~~~~~~~~~~~~~~
   hello.rs:3:1: 3:26 note: candidate #2: /home/jauhien/work/rust/linking/image/usr/lib/libhello_world.so
   hello.rs:3 extern crate hello_world;
              ^~~~~~~~~~~~~~~~~~~~~~~~~
   error: aborting due to previous error
   

Conclusions

So we have problem with current linking system. It is bad because

• it makes not possible to develop rust software on the machine that has already installed version of this software (as rust infrastructure will grow, nearly every project will use libraries from system)
• it leads to problems with packaging rust software (and rust compiler itself), see e.g. Build with --llvm-root=$DIR fails when Rust is already installed into $DIR #13421, https://github.com/Heather/gentoo-rust/issues/19 and Cannot link to third-party Rust libraries in the same prefix rustc was installed to #13733

I think that this issue can be solved if some algorithm that chooses which dynamic library to use depending on the order of -L flags is introduced to linking code. That means, e.g. use libraries found under the earliest -L path and ignore others.

[alexcrichton]

I talked with @brson about this yesterday, and we think that we should try out the following scheme:

1. If two libraries are found in two different locations (via -L), the first mentioned on the command line (far left) is chosen silently. This mirrors what gcc/clang do.
2. If two libraries are found in the same location, an error is emitted.

The current behavior is a relic of an era since past, so it's certainly ripe for changing.

[jauhien]

Yes, this is reasonable solution. It would be nice if this would be implemented.

[Siosm]

According to me this should be marked as in need of a fix before the 1.0 release.

[bheesham]

I'm getting a similar error when trying to build with openssl = "~0.2.2". Full output: https://clbin.com/19aX8

Steps to reproduce:

1. cargo new test --bin
2. Add openssl = "~0.2.2" as a dependency.
3. cargo build

[msierks]

I am also getting this error. Is there a workaround ?

[jauhien]

@msierks, @bheesham: the workaround is installation of the Rust libraries not in the system location. You can use --libdir configure switch for this. Or you can install the whole Rust in different location (and if you are compiling it by hand, I would highly recommend install it in /usr/local as any software compiled by hand).

The first installation way is used e.g. here: https://github.com/Heather/gentoo-rust/tree/master/dev-lang/rust

[msierks]

@jauhien: worked, thanks.