Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connection#createArrayOf performance worse than passing in array literals #68

Open
StephenFlavin opened this issue Aug 23, 2024 · 0 comments

Comments

@StephenFlavin
Copy link

I originally commented on the MR that implements this.

I was excited to see support for Connection#createArrayOf was added so I decided to build locally to try it out and I came across one issue and a question on performance.

Before I saw that this had been implemented I passed arrays as literals from java then casting the parameter to the array type in the sql, it seems passing the array literal is many times faster than using Connection#createArrayOf, I tried similar on postgres in the past and the speed improvement using Connection#createArrayOf was significant.
Quick test code (using spring)

    public static void main(String[] args) throws SQLException {
        var jdbcClient = JdbcClient.create(new SingleConnectionDataSource("jdbc:duckdb:", true));

        jdbcClient.sql("create table foo(i int)")
                        .update();

        String arrayLiteral = IntStream.range(0, 10000).mapToObj(String::valueOf).collect(Collectors.joining(",", "[", "]"));

        System.out.println("ArrayLiteral");

        for (int i = 0; i < 100; i++) {
            final var now2 = Instant.now();
            jdbcClient.sql("insert into foo select unnest(:arr::int[])")
                    .param("arr", arrayLiteral)
                    .update();
            final var duration = Duration.between(now2, Instant.now());
            System.out.println(duration.toNanos() / 1000000d);
        }

        jdbcClient.sql("truncate table foo")
                .update();

        System.out.println("`Connection#createArrayOf`");

        final var elements = IntStream.range(0, 10000).boxed().toArray();

        try (final var connection = new SingleConnectionDataSource("jdbc:duckdb:", true).getConnection()) {
            for (int i = 0; i < 100; i++) {
                final var now = Instant.now();
                jdbcClient.sql("insert into foo select unnest(:arr::int[])")
                        .param("arr", connection.createArrayOf("int4", elements))
                        .update();
                final var duration = Duration.between(now, Instant.now());
                System.out.println(duration.toNanos() / 1000000d);
            }
        }
    }

output on M1 mac showing 1.3ms for array literal vs 6.4ms for Connection#createArrayOf, is this expected as a first implementation or do you think something is a rye?

unfortunately, I've not got the time to dig into potential performance improvements on the java side and I don't know c++ so can't help there 😓

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant