ThreadSanitizer: data race in _$LT$std..sys_common..lazy_box..LazyBox$LT$T$GT$$u20$as$u20$core..ops..drop..Drop$GT$::drop::hfc9babc43b28f06f+0x20

[ksolana]

WARNING: ThreadSanitizer: data race (pid=57940)
  Read of size 8 at 0x00010b003920 by thread T212:
    #0 _$LT$std..sys_common..lazy_box..LazyBox$LT$T$GT$$u20$as$u20$core..ops..drop..Drop$GT$::drop::hfc9babc43b28f06f <null> (rayon-e96c7d70efaed326:arm64+0x100358f6c)
    #1 core::ptr::drop_in_place$LT$rayon_core..registry..ThreadInfo$GT$::h18ce66d027b40f67 <null> (rayon-e96c7d70efaed326:arm64+0x10035979c)
    #2 _$LT$alloc..vec..Vec$LT$T$C$A$GT$$u20$as$u20$core..ops..drop..Drop$GT$::drop::h019ec8b68d410a24 <null> (rayon-e96c7d70efaed326:arm64+0x100359c70)
    #3 alloc::sync::Arc$LT$T$C$A$GT$::drop_slow::hf216276d743cfe6a <null> (rayon-e96c7d70efaed326:arm64+0x10035b528)
    #4 core::ptr::drop_in_place$LT$rayon_core..registry..WorkerThread$GT$::hc131c84a1338943b <null> (rayon-e96c7d70efaed326:arm64+0x100353bbc)
    #5 rayon_core::registry::ThreadBuilder::run::h69a3357875d12a8f <null> (rayon-e96c7d70efaed326:arm64+0x100354838)
    #6 std::sys::backtrace::__rust_begin_short_backtrace::hb32d90315d0702ae <null> (rayon-e96c7d70efaed326:arm64+0x10035bbb4)
    #7 std::panicking::try::do_call::he3df66b3d7d28e9c <null> (rayon-e96c7d70efaed326:arm64+0x10035d6e8)
    #8 __rust_try <null> (rayon-e96c7d70efaed326:arm64+0x10035e3d8)
    #9 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h017394cf0fb12bb1 <null> (rayon-e96c7d70efaed326:arm64+0x10035d890)
    #10 std::sys::pal::unix::thread::Thread::new::thread_start::hbb9155e18f2e6be5 <null> (rayon-e96c7d70efaed326:arm64+0x10038832c)

  Previous atomic write of size 8 at 0x00010b003920 by thread T211:
    #0 std::sys_common::lazy_box::LazyBox$LT$T$GT$::initialize::h6621bd616e1a7576 <null> (rayon-e96c7d70efaed326:arm64+0x1003df960)
    #1 std::sync::mutex::Mutex$LT$T$GT$::lock::hcdbfd517f3ef4443 <null> (rayon-e96c7d70efaed326:arm64+0x10035ab1c)
    #2 _$LT$rayon_core..latch..LockLatch$u20$as$u20$rayon_core..latch..Latch$GT$::set::hf31cf24770f2ca7b <null> (rayon-e96c7d70efaed326:arm64+0x100354500)
    #3 rayon_core::registry::ThreadBuilder::run::h69a3357875d12a8f <null> (rayon-e96c7d70efaed326:arm64+0x1003547c4)
    #4 std::sys::backtrace::__rust_begin_short_backtrace::hb32d90315d0702ae <null> (rayon-e96c7d70efaed326:arm64+0x10035bbb4)
    #5 std::panicking::try::do_call::he3df66b3d7d28e9c <null> (rayon-e96c7d70efaed326:arm64+0x10035d6e8)
    #6 __rust_try <null> (rayon-e96c7d70efaed326:arm64+0x10035e3d8)
    #7 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h017394cf0fb12bb1 <null> (rayon-e96c7d70efaed326:arm64+0x10035d890)
    #8 std::sys::pal::unix::thread::Thread::new::thread_start::hbb9155e18f2e6be5 <null> (rayon-e96c7d70efaed326:arm64+0x10038832c)

  Location is heap block of size 704 at 0x00010b003900 allocated by thread T210:
    #0 malloc <null> (librustc-nightly_rt.tsan.dylib:arm64+0x2e4fc)
    #1 alloc::raw_vec::RawVec$LT$T$C$A$GT$::try_allocate_in::ha01d4d87e478e2e9 raw_vec.rs:229 (rayon-e96c7d70efaed326:arm64+0x100281564)
    #2 alloc::vec::in_place_collect::_$LT$impl$u20$alloc..vec..spec_from_iter..SpecFromIter$LT$T$C$I$GT$$u20$for$u20$alloc..vec..Vec$LT$T$GT$$GT$::from_iter::hac01835e69b52987 in_place_collect.rs:245 (rayon-e96c7d70efaed326:arm64+0x1002169b8)
    #3 rayon_core::registry::Registry::new::h01a3bb2a460606a0 registry.rs:262 (rayon-e96c7d70efaed326:arm64+0x1001a1ee0)
    #4 rayon_core::thread_pool::ThreadPool::build::h482935b73b6f5da7 mod.rs:69 (rayon-e96c7d70efaed326:arm64+0x1002fe1d4)
    #5 rayon::range::test_usize_i64_overflow::h00cf5ed90294ff5b range.rs:435 (rayon-e96c7d70efaed326:arm64+0x100258a30)
    #6 core::ops::function::FnOnce::call_once::he4ecca1f999eed86 function.rs:250 (rayon-e96c7d70efaed326:arm64+0x100273928)
    #7 test::__rust_begin_short_backtrace::h94dab995366ed857 <null> (rayon-e96c7d70efaed326:arm64+0x10034d954)

  Thread T212 (tid=7382174, running) created by thread T210 at:
    #0 pthread_create <null> (librustc-nightly_rt.tsan.dylib:arm64+0x7bb4)
    #1 std::sys::pal::unix::thread::Thread::new::hff18a13ebab4f619 <null> (rayon-e96c7d70efaed326:arm64+0x100388190)
    #2 _$LT$rayon_core..registry..DefaultSpawn$u20$as$u20$rayon_core..registry..ThreadSpawn$GT$::spawn::hbcecc71bc650aa86 <null> (rayon-e96c7d70efaed326:arm64+0x1003549dc)
    #3 rayon_core::registry::Registry::new::h01a3bb2a460606a0 registry.rs:303 (rayon-e96c7d70efaed326:arm64+0x1001a235c)
    #4 rayon_core::thread_pool::ThreadPool::build::h482935b73b6f5da7 mod.rs:69 (rayon-e96c7d70efaed326:arm64+0x1002fe1d4)
    #5 rayon::range::test_usize_i64_overflow::h00cf5ed90294ff5b range.rs:435 (rayon-e96c7d70efaed326:arm64+0x100258a30)
    #6 core::ops::function::FnOnce::call_once::he4ecca1f999eed86 function.rs:250 (rayon-e96c7d70efaed326:arm64+0x100273928)
    #7 test::__rust_begin_short_backtrace::h94dab995366ed857 <null> (rayon-e96c7d70efaed326:arm64+0x10034d954)

  Thread T211 (tid=7382173, running) created by thread T210 at:
    #0 pthread_create <null> (librustc-nightly_rt.tsan.dylib:arm64+0x7bb4)
    #1 std::sys::pal::unix::thread::Thread::new::hff18a13ebab4f619 <null> (rayon-e96c7d70efaed326:arm64+0x100388190)
    #2 _$LT$rayon_core..registry..DefaultSpawn$u20$as$u20$rayon_core..registry..ThreadSpawn$GT$::spawn::hbcecc71bc650aa86 <null> (rayon-e96c7d70efaed326:arm64+0x1003549dc)
    #3 rayon_core::registry::Registry::new::h01a3bb2a460606a0 registry.rs:303 (rayon-e96c7d70efaed326:arm64+0x1001a235c)
    #4 rayon_core::thread_pool::ThreadPool::build::h482935b73b6f5da7 mod.rs:69 (rayon-e96c7d70efaed326:arm64+0x1002fe1d4)
    #5 rayon::range::test_usize_i64_overflow::h00cf5ed90294ff5b range.rs:435 (rayon-e96c7d70efaed326:arm64+0x100258a30)
    #6 core::ops::function::FnOnce::call_once::he4ecca1f999eed86 function.rs:250 (rayon-e96c7d70efaed326:arm64+0x100273928)
    #7 test::__rust_begin_short_backtrace::h94dab995366ed857 <null> (rayon-e96c7d70efaed326:arm64+0x10034d954)

  Thread T210 (tid=7382172, finished) created by main thread at:
    #0 pthread_create <null> (librustc-nightly_rt.tsan.dylib:arm64+0x7bb4)
    #1 std::sys::pal::unix::thread::Thread::new::hff18a13ebab4f619 <null> (rayon-e96c7d70efaed326:arm64+0x100388190)
    #2 std::sys::backtrace::__rust_begin_short_backtrace::h034156b15463edba backtrace.rs:154 (rayon-e96c7d70efaed326:arm64+0x1002da6c8)
    #3 std::rt::lang_start::_$u7b$$u7b$closure$u7d$$u7d$::hbfa217990800bed2 <null> (rayon-e96c7d70efaed326:arm64+0x1002cfab4)
    #4 std::rt::lang_start_internal::h5b9912862e0fda83 <null> (rayon-e96c7d70efaed326:arm64+0x10037c140)
    #5 <null> <null> (0x00018cfec274)
    #6 <null> <null> (0x00018cfec274)

SUMMARY: ThreadSanitizer: data race (rayon-e96c7d70efaed326:arm64+0x100358f6c) in _$LT$std..sys_common..lazy_box..LazyBox$LT$T$GT$$u20$as$u20$core..ops..drop..Drop$GT$::drop::hfc9babc43b28f06f+0x20

To repro on Mac

RUSTFLAGS=-Zsanitizer=thread cargo +nightly-2024-08-08 test --profile release-with-debug --target aarch64-apple-darwin

diff --git a/Cargo.toml b/Cargo.toml
index 761c03c..70e863f 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,3 +1,8 @@
+[profile.release-with-debug]
+inherits = "release"
+debug = true
+

[cuviper]

LazyBox is an implementation detail of the standard Mutex (on non-futex platforms). It's possible that we could be causing a race between initialization and destruction of the Mutex due to unsafe code, but I don't think it can happen in this case.

• The "read" backtrace is coming through the Registry drop via Arc drop.
• The "write" backtrace is in the registry main_loop, which holds one of the Arc counts.
• Therefore, the Registry drop can't happen until the synchronized Arc drop completes.

I think ThreadSanitizer is really complaining about the atomic write vs non-atomic read, but that doesn't understand the Rust semantics that the final Arc handle gets an exclusive &mut to drop its contents, which doesn't need atomic operations anymore.