Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dsl)!: support other JS runtimes #3355

Merged
merged 5 commits into from
May 24, 2024
Merged

Conversation

ObserverOfTime
Copy link
Member

@ObserverOfTime ObserverOfTime commented May 8, 2024

Problem

The generate command currently only supports Node out of the box and Bun via a shim that calls bun -.

Solution

  • Replace the Node-specific global with the universal globalThis.
  • Replace the Node-specific require() with the universal await import().
    • This also adds support for ES modules but no longer works in Node out of the box.
  • Adapt the environment & stdout functions depending on the runtime.
  • Set the required arguments for known runtimes (Node, Bun, Deno).
    • The user can still specify a different command or the full path for custom runtimes.

Notes

  • In Deno, the DSL only needs --allow-env --allow-read but we use --allow-all since grammars may need more.
    • The other runtimes are not restrictive anyway.
  • QuickJS does not support reading input from stdin at the moment.
  • MozJS does not support reading input from stdin as a module at the moment.

@ObserverOfTime ObserverOfTime added this to the 0.23 milestone May 8, 2024
@ObserverOfTime
Copy link
Member Author

We could also support QuickJS & MozJS by writing a temporary file, but loading node modules (if needed) would be tricky.

@ObserverOfTime
Copy link
Member Author

Cross does not seem to support --input-type. Is it using a version of Node older than 12.0?

Windows is, as usual, weird:

 TypeError [ERR_INVALID_MODULE_SPECIFIER]: Invalid module "\\?\D:\a\tree-sitter\tree-sitter\test\fixtures\test_grammars\uses_current_column\grammar.js" is not a valid package name imported from D:\a\tree-sitter\tree-sitter\cli\[eval1]

@ObserverOfTime
Copy link
Member Author

ObserverOfTime commented May 8, 2024

Is it using a version of Node older than 12.0?

Right, of course it is… Surely cross-rs/cross#973 will be merged soon…

@ObserverOfTime
Copy link
Member Author

Note to self: add these back if QuickJS and MozJS are ever supported.

--- a/cli/src/generate/dsl.js
+++ b/cli/src/generate/dsl.js
@@ -459,6 +459,8 @@ function checkPrecedence(value) {
 function getEnv(name) {
   if (globalThis.process) return process.env[name]; // Node/Bun
   if (globalThis.Deno) return Deno.env.get(name); // Deno
+  if (globalThis.std) return std.getenv(name); // QuickJS
+  if (globalThis.os?.getenv) return os.getenv(name); // MozJS
   throw Error("Unsupported JS runtime");
 }
 
@@ -482,6 +484,10 @@ if (globalThis.process) { // Node/Bun
   process.stdout.write(output);
 } else if (globalThis.Deno) { // Deno
   Deno.stdout.writeSync(new TextEncoder().encode(output));
+} else if (globalThis.std) { // QuickJS
+  std.out.puts(output);
+} else if (globalThis.putstr) { // MozJS
+  putstr(output);
 } else {
   throw Error("Unsupported JS runtime");
 }

@amaanq amaanq merged commit 055d0cb into tree-sitter:master May 24, 2024
12 checks passed
@github-actions github-actions bot removed the request for review from amaanq May 24, 2024 20:53
@ObserverOfTime ObserverOfTime deleted the jsrt branch May 24, 2024 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants