Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deadlock on Windows when unloading libraries in several threads #111

Open
aseyboldt opened this issue Apr 25, 2023 · 1 comment
Open

Deadlock on Windows when unloading libraries in several threads #111

aseyboldt opened this issue Apr 25, 2023 · 1 comment

Comments

@aseyboldt
Copy link
Contributor

aseyboldt commented Apr 25, 2023

While working on the rust backend we noticed problems that appear when stan modules are loaded and unloaded in separate threads.

On windows we see a deadlock under the following conditions:

Start three threads. Load stan module A in the first thread, while the other threads wait. Load module B in the second thread. Then, unload module A in the first thread and exit the thread. And last, try to load module C in the third thread. This will deadlock. From some debugging it seems that the second thread actually died somewhere while exiting.
This exact behavior seems to depend on the compiler or tbb version, because if we compile the stan model with clang or msvc instead of mingw and a recent TBB, we do not see the same issue (see #105 (comment))

This might be related to TBB, we found some issues that look at bit similar to this on the TBB bugtracker (#105 (comment)).

A reproducer using the rust backend can be found here: https://github.com/aseyboldt/bridgestan/blob/main/rust/tests/create_models.rs#L236

There also seem to be some sporadic issues involving library unloading on linux as well however. With compilation settings O=0 and enabled library unloading I observed a couple of sporadic segfaults when I ran the rust test. In a debugger it looked like the regular expression here was not correctly initialized. I don't know for sure that this is related to unloading the libraries, but I've never seen that issue without it. And I think that since c++11 initialization of static variables should actually be thread safe.

(cc @WardBrian Feel free to add or modify the summary as you like)

@aseyboldt
Copy link
Contributor Author

I guess if I had checked the libloading issues a bit more thoroughly I might have saved myself quite a bit of time...

nagisa/rust_libloading#41
nagisa/rust_libloading#5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants