Skip to content

Commit

Permalink
Add a page about library loading
Browse files Browse the repository at this point in the history
  • Loading branch information
serenity4 committed Jul 1, 2024
1 parent 87f1c24 commit 8485a5e
Show file tree
Hide file tree
Showing 4 changed files with 45 additions and 3 deletions.
1 change: 1 addition & 0 deletions docs/make.jl
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ makedocs(;
"Explanations" => [
"Motivations" => "about/motivations.md",
"Extension mechanism" => "about/extension_mechanism.md",
"Library loading" => "about/library_loading.md",
],
"API" => "api.md",
"Utility" => "utility.md",
Expand Down
41 changes: 41 additions & 0 deletions docs/src/about/library_loading.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Library loading

Owing to its extensible architecture, Vulkan may require additional libraries to be available during runtime. That will be
notably the case of every layer, and of most [WSI (Window System Integration)](https://registry.khronos.org/vulkan/specs/1.3-extensions/html/chap34.html)
instance extensions which require hooking into the OS' windowing system.

It is important to know where these libraries come from, to avoid crashes and ensure correct behavior.
A notable case for failure is when some code uses a new function exposed in a recent library release, but the loaded library is too old.
In particular, this may occur after updating Vulkan drivers, or upgrading the OS (which in turn updates OS-specific libraries and possibly the Vulkan loader).
Other than that, libraries are generally backward compatible, and compatibility issues are fairly rare.

In Julia, there are two notable systems that may provide them:
- Your operating system, using whatever is available, as matched first by the linker depending on configuration. Version suffixes (e.g. `libvulkan.so.1`) may be be used to provide weak compatibility guarantees.
- Pkg's [artifact system](https://pkgdocs.julialang.org/v1/artifacts/), providing libraries and binaries with set versions and stronger compatibility guarantees with semantic versioning.

When a library is required by a Vulkan feature, extension or layer, it will use the first one available.
That may be an artifact, or a system library. Relying on either comes with caveats:
- Relying on an artifact may incorrectly interface with OS-specific functionality, which requires to match system libraries.
- Relying on system libraries may cause compatibility issues when using artifact libraries that require specific versions.

The artifact system explicitly uses libraries from other artifacts, *and not from the system*. Keep that in mind especially if you rely on artifacts for application-level functionality (e.g. GLFW).

**Vulkan** may be redirected to use a specific system or artifact library. It can be attempted by:
- Forcing the system linker to preload a specific library (e.g. `LD_PRELOAD` for `ld` on linux).
- Emulating such preload using `Libdl.dlopen` before the corresponding library is loaded; that is, before `using Package` where `Package` depends on artifacts (artifacts tend to `dlopen` their library dependencies during [module initialization](https://docs.julialang.org/en/v1/manual/modules/#Module-initialization-and-precompilation)).
- Loading an artifact (either directly or indirectly), triggering the loading of its dependent libraries (which may be redirected too, see below).

**Artifacts** may only be redirected to use a specific library by using preferences:

```julia-repl
julia> using Xorg_libxcb_jll
julia> Xorg_libxcb_jll.set_preferences!(Xorg_libxcb_jll, "libxcb_path" => "/usr/lib/libxcb.so")
# Restart Julia to trigger precompilation, updating artifact settings.
julia> using Xorg_libxcb_jll
```

Note that every artifact may provide many library products, and each one of them will require an explicit preference to opt out of the artifact system. For instance, `Xorg_libxcb_jll` provides `libxcb.so`, but also `libxcb-render.so`, `libxcb-xkb.so`, and many more; `libxcb_path` only affects `libxcb.so`, and to affect these other libraries there exist similar preferences `libxcb_render_path`, `libxcb_xkb_path`, etc.

If you stumble upon an error during instance creation and wonder if it's related to compatibility issues, these tend to show up when the `VK_LOADER_DEBUG=all` option is set; see [Internal API errors](@ref).
4 changes: 2 additions & 2 deletions docs/src/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@ If using VSCode, you can set it for the integrated terminal (e.g. `terminal.inte

## Internal API errors

If you encounter the error `INITIALIZATION_FAILED` or similar errors with Julia, which you do not encounter with other languages (e.g. C/C++) or with your system Vulkan utilities, then it may be due to `libstdc++` version requirements (see [this tip](@ref libstdc)).
If you encounter the error `INITIALIZATION_FAILED` or similar errors with Julia, which you do not encounter with other languages (e.g. C/C++) or with your system Vulkan utilities, then it may be due to `libstdc++` version requirements (see [this tip](@ref libstdc)) or [incompatibilities in library loading](@ref Library-loading).

If the bug is encountered in a function from the loader (e.g. via a function that operates on an `Instance`, and not a `Device`), and you use the official [Vulkan-Loader](https://github.com/KhronosGroup/Vulkan-Loader) you can turn on logging via setting the environment variable `VK_LOADER_DEBUG=all`. This should help you understand the cause (see [Debug environment variables](https://github.com/KhronosGroup/Vulkan-Loader/blob/master/docs/LoaderInterfaceArchitecture.md#table-of-debug-environment-variables=) for more options).
If the bug is encountered in a function from the loader (e.g. via a function that operates on an `Instance`, and not a `Device`), and if you are using [Vulkan-Loader](https://github.com/KhronosGroup/Vulkan-Loader) (which is most likely the case), it is recommended to enable additional logging by setting the environment variable `VK_LOADER_DEBUG=all`. See [the loader's debug environment variables](https://github.com/KhronosGroup/Vulkan-Loader/blob/master/docs/LoaderInterfaceArchitecture.md#table-of-debug-environment-variables) for more options.

## 0-based vs 1-based indexing

Expand Down
2 changes: 1 addition & 1 deletion docs/src/tutorial/minimal_working_compute.jl
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@ unwrap(queue_submit(compute_q, [SubmitInfo([], [], [cbuf], [])]))
# that for example the pipeline and buffer objects are still used and that
# there's a dependency with these variables until the command returns, so we
# tell it manually.
GC.@preserve buff dsl pl p const_buf spec_consts begin
GC.@preserve buffer dsl pl p const_buf spec_consts begin
unwrap(queue_wait_idle(compute_q))
end

Expand Down

0 comments on commit 8485a5e

Please sign in to comment.