fix: avoid globbing non-dynamic paths #121
Conversation
|
ci is failing, will debug tomorrow |
|
Could you also test if there are improvements with the Seems now it went down to 2min: https://github.com/yao-pkg/pkg/actions/runs/11675697212/job/32510712730?pr=121#step:7:20 :) I think there is still range of improvement here. Could you try enabling that test also for windows and mac? Just remove this lines: pkg/test/test-80-compression-node-opcua/main.js Lines 17 to 21 in 8913dff |
robertsLando
changed the title
|
I also created #122, I would like to compare performances of tests in this two PR |
|
In fact seems this is faster 👍🏼 |
| if (isDynamicPattern(pattern)) { | ||
| patterns.push(pattern); | ||
| } else if (existsSync(pattern)) { | ||
| paths.push(pattern); | ||
| } | ||
| } | ||
|
|
||
| if (patterns.length !== 0) { | ||
| return [...paths, ...globSync(patterns, { absolute: true, dot: true })]; | ||
| } | ||
|
|
||
| return paths; |
There was a problem hiding this comment.
I'm wondering why tinyglobby cannot use isDynamicPattern his own by default before starting parsing a pattern?
There was a problem hiding this comment.
consistency shenanigans, but mainly just that i don't think it will be needed once its optimization pr gets fixed and lands
|
okay i see why at least some tests fail, it's due to this project using directory expansion, which means that globbing |
|
@SuperchupuDev Yep understood, could you fix that? |
|
trying |
|
okay, fixing it would just make all patterns dynamic, making performance not change whatsoever. we need a better solution. there needs to be a refactor in the walker logic so that all patterns are pushed into an array and then when all of them are collected call |
|
@SuperchupuDev Makes sense, agreee |
fixes #119
the problem with glob usage in this project is that
globSyncis called multiple times which isn't optimal, as it forcestinyglobbyto traverse the filesystem multiple times. in theory,globSynccould be called just once during the walking process, but that would be a bigger refactor, although a possible oneit looks like under a default config most patterns passed to
globSyncaren't even globs, so this PR avoids unnecessaryglobSynccalls when none of the patterns are globsscreenshot of patterns passed to tinyglobby in 6.1.0, each log is a different glob call
locally (windows) this change makes it faster than 5.15.0, with the reproduction from #119 taking 25s in 5.15.0 and 13s in latest with the change (latest without the change took too long to measure)