Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inserting data via Arrow fails in DuckDB 1.1.0 #120

Open
marvold-mw opened this issue Sep 18, 2024 · 3 comments · May be fixed by #125
Open

Inserting data via Arrow fails in DuckDB 1.1.0 #120

marvold-mw opened this issue Sep 18, 2024 · 3 comments · May be fixed by #125

Comments

@marvold-mw
Copy link

Using the following code (basically the sample code from https://duckdb.org/docs/api/nodejs/overview.html#inserting-data-via-arrow), DuckDB 1.1.0 failed to register an Arrow buffer via the Node bindings:

const arrow = require("apache-arrow");
const duckdb = require("duckdb");
const db = new duckdb.Database(":memory:");

const jsonData = [
  { userId: 1, id: 1, title: "delectus aut autem", completed: false },
  {
    userId: 1,
    id: 2,
    title: "quis ut nam facilis et officia qui",
    completed: false,
  },
];

// note; doesn't work on Windows yet
db.exec(`INSTALL arrow; LOAD arrow;`, (err) => {
  if (err) {
    console.warn(err);
    return;
  }

  const arrowTable = arrow.tableFromJSON(jsonData);
  db.register_buffer(
    "jsonDataTable",
    [arrow.tableToIPC(arrowTable)],
    true,
    (err, res) => {
      if (err) {
        console.warn(err);
        return;
      }

      // `SELECT * FROM jsonDataTable` would return the entries in `jsonData`
    }
  );
});
$ node ./test.js
[Error: Binder Error: No function matches the given name and argument types 'scan_arrow_ipc(STRUCT(ptr BIGINT, size INTEGER)[])'. You might need to add explicit type casts.
	Candidate functions:
	scan_arrow_ipc(STRUCT(ptr UBIGINT, size UBIGINT)[])

LINE 1: ...RY VIEW jsonDataTable AS SELECT * FROM scan_arrow_ipc([{'ptr': 5099265536, 'si...
                                                  ^] {
  errno: -1,
  code: 'DUCKDB_NODEJS_ERROR',
  errorType: 'Binder'
}

This code failed with NPM packages apache-arrow 17.0.0 and duckdb 1.1.0, but succeeded with apache-arrow 17.0.0 and duckdb 1.0.0.

@marvold-mw
Copy link
Author

(I should note that this was on an Apple Silicon Mac running macOS Sequoia and Node 20.17.0, in case that proves relevant. This also happened with slightly different code on a GitHub runner running Ubuntu 22.04 and Node 20.17.0; I suspect it's a general issue, but I ought to mention as much as I can.)

@SanYann
Copy link

SanYann commented Sep 23, 2024

Same issue here

@nshiab
Copy link

nshiab commented Sep 25, 2024

I have the same problem.

carlopi added a commit to carlopi/duckdb-node that referenced this issue Oct 1, 2024
@carlopi carlopi linked a pull request Oct 1, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants