Running multiple worker threads starts failing at the second worker thread

[Sir-hennihau]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow.js): Yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu
• TensorFlow.js installed from (npm or script link): npm
• TensorFlow.js version (use command below): 4.21

Describe the current behavior
When running tensorflow js from a worker thread multiple times, the first worker thread will run successfully. However, the second and every after will fail.

Describe the expected behavior
All worker threads should succeed.

Standalone code to reproduce the issue

import { createTsWorker } from "./ts-worker-loader";

/**
 * Creates a worker that loads TensorFlow.js and handles messages from the worker.
 */
function startTensorFlowWorker() {
  return new Promise((resolve, reject) => {
    const worker = createTsWorker("./tf-worker.ts");

    worker.on(
      "message",
      (message: {
        status: string;
        tensorValues?: number[];
        error?: string;
      }) => {
        if (message.status === "success") {
          console.log("TensorFlow.js initialized in worker thread.");
          console.log("Tensor Values:", message.tensorValues);
        } else if (message.status === "error") {
          console.error("Error from worker:", message.error);
        }
      }
    );

    worker.on("error", (error: Error) => {
      console.error("Worker error:", error);
    });

    worker.on("exit", (code: number) => {
      resolve(null);
      console.log("exit code", code);
    });
  });
}

(async () => {
  // Runs successfully
  await startTensorFlowWorker();

  // All of those will fail
  await startTensorFlowWorker();
  await startTensorFlowWorker();
  await startTensorFlowWorker();
  await startTensorFlowWorker();
  await startTensorFlowWorker();
})();

// ts-worker-loader.js
import { Worker } from "worker_threads";
const path = require("path");

// This function creates a new worker that runs a TypeScript file.
export function createTsWorker(filePath: any) {
  const tsNodePath = require.resolve("ts-node/register");
  const absoluteFilePath = path.resolve(filePath);

  return new Worker(
    `
        require('${tsNodePath}');
        require('${absoluteFilePath}');
    `,
    { eval: true }
  );
}

module.exports = { createTsWorker };

// tf-worker.js
const { parentPort } = require("worker_threads");
const tf = require("@tensorflow/tfjs-node");

/**
 * Initializes TensorFlow.js and sends a message back to the main thread.
 */
async function initializeTensorFlow() {
  const model = tf.sequential();
  model.add(tf.layers.dense({ units: 1, inputShape: [200] }));
  model.compile({
    loss: "meanSquaredError",
    optimizer: "sgd",
    metrics: ["MAE"],
  });

  // Generate some random fake data for demo purposes.
  const xs = tf.randomUniform([10, 200]);
  const ys = tf.randomUniform([10, 1]);
  const valXs = tf.randomUniform([10, 200]);
  const valYs = tf.randomUniform([10, 1]);

  // Start model training process.
  async function train() {
    await model.fit(xs, ys, {
      epochs: 1,
      validationData: [valXs, valYs],
      // Add the tensorBoard callback here.
      callbacks: tf.node.tensorBoard("/tmp/fit_logs_1"),
    });
  }
  await train();

  const xsTest = tf.randomUniform([10, 200]);

  const result = model.predict(xsTest);
  //console.log("result", result);
}

initializeTensorFlow();

I made a small repo where you can reproduce the error
https://github.com/Sir-hennihau/tfjs-multiple-worker-threads-error

Other info / logs

sir-hennihau@Cyberminifred:~/Workspace/tfjs-multiple-worker-threads$ npm start

> [email protected] start
> ts-node index.ts

2024-09-27 17:32:04.956732: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-09-27 17:32:04.981745: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Epoch 1 / 1

46ms 4606us/step - MAE=0.305 loss=0.156 val_MAE=0.396 val_loss=0.299 
exit code 0
Worker error: node:internal/modules/cjs/loader:1454
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: Module did not self-register: '/home/sir-hennihau/Workspace/tfjs-multiple-worker-threads/node_modules/@tensorflow/tfjs-node/lib/napi-v8/tfjs_binding.node'.
    at Object..node (node:internal/modules/cjs/loader:1454:18)
    at Module.load (node:internal/modules/cjs/loader:1208:32)
    at Function._load (node:internal/modules/cjs/loader:1024:12)
    at Module.require (node:internal/modules/cjs/loader:1233:19)
    at require (node:internal/modules/helpers:179:18)
    at Object.<anonymous> (/home/sir-hennihau/Workspace/tfjs-multiple-worker-threads/node_modules/@tensorflow/tfjs-node/dist/index.js:72:16)
    at Module._compile (node:internal/modules/cjs/loader:1358:14)
    at node:internal/modules/cjs/loader:1416:10
    at Object.require.extensions.<computed> [as .js] (/home/sir-hennihau/Workspace/tfjs-multiple-worker-threads/node_modules/ts-node/src/index.ts:1608:43)
    at Module.load (node:internal/modules/cjs/loader:1208:32) {
  code: 'ERR_DLOPEN_FAILED'
}
exit code 1
Worker error: node:internal/modules/cjs/loader:1454
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

All after the first will fail (shortened logs here to improve readability). It might be related to using TypeScript and importing workers threads. This is basically the follow up for this issue: #2079

I'm not quite sure if tfjs correctly implements the C++ addons requirements of node?
https://nodejs.org/api/addons.html#addons_worker_support
I didn't find anything relevant in the tfjs source code yet. Would be great to solve this issue, since this is a workflow that might impact a lot of node developers. Using worker threads is even recommended in the docs.