Node-API (N-API) for Rust
Replies: 1 comment
-
|
i try to use valgrind to run a small example: #![deny(clippy::all)]
#[macro_use]
extern crate napi_derive;
#[napi(object)]
#[derive(Debug)]
pub struct SomeOptions {
pub one: Vec<String>,
pub two: String,
pub three: u32,
}
#[napi(object)]
pub struct SomeResponse {
pub success: bool,
pub data: Vec<String>
}
#[napi]
pub fn empty_function(opt: SomeOptions) -> SomeResponse {
println!("opt=>{:?}", opt);
let response = SomeResponse{
success: false,
data: vec!["some".to_string()],
};
return response;
}const thread = require('worker_threads');
if (thread.isMainThread) {
const worker = new thread.Worker(__filename, {});
worker.on('message', (msg) => {
// console.log(`#${i}:`, msg);
worker.terminate();
})
}else{
const addon = require('./index');
const resuslt = addon.emptyFunction({
one: ["one", "one", "one", "one", "one", "one", "one", "one", "one", "one", "one", "one",],
two:"emptyFunctionemptyFunctionemptyFunctionemptyFunctionemptyFunctionemptyFunctionemptyFunction",
three: 122222222
});
thread.parentPort.postMessage(resuslt);
// process.exit();
}i got this: i dont know why, did use thread work to run napi addon is support? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I have a NodeJS application that primarily parses large text files into a more readable format. Message reception is handled on the Node side. Upon receiving a message, I save it as a file in a temporary directory and pass the file location to a Rust program built with napi-rs via a Node addon method. The Rust program parses the text, which may require downloading multiple symbol files (around 800MB in total) using reqwest, extracting them to a specified temporary directory, and then using these symbol files to convert the text into a readable format. The parsed text is saved as a file, and its location is returned to the Node side. Upon receiving the result, Node deletes both the original and parsed files.
One particular aspect is that directly calling the Node addon might block the main thread, so I use a new thread (work_thread) on the Node side to call the Rust program, and then return the result to the main thread before closing the new thread.
Additionally, in the Rust program, due to the long call chain, I store the parameters passed from Node in a static Mutex struct initialized only once, which can be accessed deep in the call chain.
One aspect to note is that the Rust application, initially designed as a CLI application, has a verbose option for outputting debug information. For Node, I can't output directly, so I use a static Mutex<Vec> to store the log messages, which are then returned to the Node side. However, under high concurrency, the log messages received by the Node side are very large and contain logs from non-current processing flows. After some investigation, I realized that since the static Mutex<Vec> object is shared across threads unless the process ends, I wrapped it with thread_local so that each independent thread has its own log Vec.
When deploying the program to the server, the Node application experienced rapid memory leaks, consuming around 32GB of memory within an hour. This issue was caused by the Rust program repeatedly downloading and deleting symbol files. After some adjustments and optimizations, I found that the memory usage still increased, though not as quickly.
Can you help me analyze where the problem might be? I am out of ideas. Alternatively, could you guide me on how to debug the memory usage of a napi addon?
Beta Was this translation helpful? Give feedback.
All reactions