1.30 -> 1.31 dylib late-binding regression with GNU binutils 2.28 or older.

[eddyb]

Originally reported as #60593, and appears to be caused by #54592, this is my reduction:

• common.rs

pub static STATIC: u8 = 0;
pub fn get() -> *const u8 { &STATIC }

• plugin.rs

extern crate common;
#[no_mangle]
pub fn run() {
    println!("plugin: {:p}", common::get as *const ());
    assert_eq!(common::get(), &common::STATIC as *const _);
}

• driver.rs

extern crate common;
fn main() {
    println!("driver: {:p}", common::get as *const ());
    // You can comment this line to make it work.
    assert_eq!(common::get(), &common::STATIC as *const _);
    unsafe {
        extern "C" {
            fn dlopen(filename: *const u8, flag: i32) -> *mut u8;
            fn dlsym(handle: *mut u8, symbol: *const u8) -> *const u8;
        }
        std::mem::transmute::<*const u8, fn()>(dlsym(
            dlopen(b"./libplugin.so\0".as_ptr(), 1),
            b"run\0".as_ptr(),
        ))()
    }
}

How to run it:

RELEASE=nightly-2018-10-11
rustup toolchain add $RELEASE

echo 'System toolchain versions:'
ldd --version | head -n1 | sed 's/^ldd //'
ld.bfd --version | head -n1
ld.gold --version | head -n1
echo

rustc +$RELEASE common.rs --crate-type=lib
# You can change this from dylib to cdylib to make it work.
rustc +$RELEASE plugin.rs --crate-type=dylib -L.
# You can uncomment -Z plt=yes to make it work.
rustc +$RELEASE driver.rs -L. # -Z plt=yes
./driver

I was able to try this out with NixOS 18.03 and 18.09 releases, via:

NIX_PATH=nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-18.03.tar.gz \
    nix-shell -p stdenv --run ./test.sh
NIX_PATH=nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-18.09.tar.gz \
    nix-shell -p stdenv --run ./test.sh

Example of failure (using a NixOS 18.03 toolchain):

info: syncing channel updates for 'nightly-2018-10-12-x86_64-unknown-linux-gnu'

  nightly-2018-10-12-x86_64-unknown-linux-gnu unchanged - rustc 1.31.0-nightly (77af31408 2018-10-11)

System toolchain versions:
(GNU libc) 2.26
GNU ld (GNU Binutils) 2.28.1
GNU gold (GNU Binutils 2.28.1) 1.14

driver: 0x55d244a55820
plugin: 0x55d244a55820
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `0x55d244a9e7ac`,
 right: `0x7f87a69b3f19`', plugin.rs:5:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Example of success (by switching to NixOS 18.09):

info: syncing channel updates for 'nightly-2018-10-12-x86_64-unknown-linux-gnu'

  nightly-2018-10-12-x86_64-unknown-linux-gnu unchanged - rustc 1.31.0-nightly (77af31408 2018-10-11)

System toolchain versions:
(GNU libc) 2.27
GNU ld (GNU Binutils) 2.30
GNU gold (GNU Binutils 2.30) 1.15

driver: 0x5649e7dbf4f0
plugin: 0x7fca6df40930

Other ways to get it to succeed:

• use a Rust version before Support for disabling PLT for better function call performance #54592 (e.g. RELEASE=nightly-2018-10-11)
• pass -Z plt=yes when compiling driver
  • restores pre-Support for disabling PLT for better function call performance #54592 behavior
• change the crate type of plugin to cdylib (instead of dylib)
  • presumably common::get is private in that case?
  • not a general fix AFAIK, but I'm not sure how to repro with cdylib
• don't call common::get in driver
  • common::get won't get looked up early, which means driver's copy won't override plugin's (within the context of affected toolchains)
• set RTLD_DEEPBIND when calling dlopen
  • common::get from driver is ignored when looking up the use from plugin

cc @alexcrichton @rust-lang/compiler

[eddyb]

Looks like relro support was added in GNU binutils 2.28, so maybe it had some bugs?
There isn't any 2.29 binutils build on the official NixOS build servers so I can't easily test that version.

[nagisa]

How confident are we that PLT entirely negates this issue other than just accidentally preventing it from triggering in this particular case?

To me it seems that exactly what happens with PLT here is prevention and the core issue still remains regardless of whether PLT is enabled or not.

Not opposed to flipping back the default for now, though.

[eddyb]

I agree, I have no idea what the actual bug is yet, just what affects this specific reduction.
Note that in all cases that work, common::get is not deduplicated, as opposed to both common::get and common::STATIC being deduplicated.

What I suspect is happening with -Z plt=yes is bypassing a buggy linker feature.
But I'd have to bisect binutils to prove that. Which I might still do, I just hope it's easier than glibc.

[nagisa]

@eddyb an easy way to (dis-)prove linker fault is to attempt linking with lld.

[pnkfelix]

triage: we need to discuss this at today's meeting before I'll be comfortable assigning a priority to it.

[pnkfelix]

discussed in triage meeting. P-high. Removing nomination tag. Assigning to @nagisa and myself.