Provide example FZF_DEFAULT_COMMAND for find alternative to speed up filtering with a cache

[harrypotterIUP]

Checklist

• I have read through the manual page (man fzf)
• I have searched through the existing issues
• For bug reports, I have checked if the bug is reproducible in the latest version of fzf

Output of fzf --version

0.55.0 (fc69308)

OS

• Linux
• macOS
• Windows
• Etc.

Shell

• bash
• zsh
• fish

Problem / Steps to reproduce

❤️ First of all, love fzf, thank you very much for your hard work!

Unfortunately, it's kinda slow on my old laptop due to lots of files and folders. It's much quicker in subsequent runs. This has come up in previous issues a lot. As far as I understand, fzf parses find by default, which explains the observed behavior. However, it would be possible to replace the default FZF_DEFAULT_COMMAND with a smarter program, which uses a cache to speed up things. Here are some references from previous threads:

If you can come up with such a command, you can specify it in your $FZF_CTRL_T_COMMAND, so the binding uses it instead of the default command.

#1284

It's beyond the scope of this project. For CTRL-T, it simply reads the output of the command specified in FZF_CTRL_T_COMMAND which defaults to a find command.

Set FZF_CTRL_T_COMMAND to a smarter command that caches the result.

#763

fzf is a text filter like grep. Caching is beyond the scope of the project.

my-smart-listing-program-that-implements-caching-mechanism | fzf

# or
export FZF_DEFAULT_COMMAND="my-smart-listing-program-that-implements-caching-mechanism"

#2550

I understand ripgrep and/or fd might help but I struggle to configure my fzf settings accordingly. Would it still be out of scope for fzf docs to provide a simple, first draft example how to set an alternative FZF_DEFAULT_COMMAND to speed things up by using a cache? I'd certainly appreciate it. And judging by the amount of previous threads I found, I might not be the only one who'd appreciate it. 🙏

[LangLangBart]

As far as I understand, fzf parses find by default

When fzf is started without STDIN, it utilizes a built-in directory walker, starting from version 0.47.0¹.
The built-in walker is used only when FZF_DEFAULT_COMMAND is not set.

# Reading input from STDIN
seq 10 | fzf

# Using fzf's built-in directory walker
fzf

Unfortunately, it's kinda slow on my old laptop due to lots of files and folders

Version 0.48.0² added built-in directory walker options that can be added to your FZF_DEFAULT_OPTS or to your FZF_DEFAULT_OPTS_FILE if you prefer the latter. Could you experiment with them and see if it helps?

# Using fzf's built-in directory walker excluding some folders
fzf  --walker-skip=.git,Photos,Library

Footnotes

1. 0.47.0 fzf/CHANGELOG.md ↩

2. 0.48.0 fzf/CHANGELOG.md ↩

[harrypotterIUP]

Thanks for the pointers! 👍 I'm beginning to think the issue must be something else, too. If I run time find ~ | wc -l it only takes a couple of seconds and counts about 500,000 lines. However I'm just realizing that vim ~/**<tab> or simply fzf isn't just slow - it actually seems to run forever and while I'm not sure what the progress nnnnn/mmmmm exactly means, both numbers exceed the number of 500,000 lines from find, which seems surprising. It's using 100% CPU and I aborted the program when it got close to exhausting free memory of approximately 8 GB. Not sure exactly where to start investigating, but I'll try to look into it.

[harrypotterIUP]

Could you experiment with them and see if it helps?

# Using fzf's built-in directory walker excluding some folders
fzf  --walker-skip=.git,Photos,Library

Same result, yes. Runs forever.

[harrypotterIUP]

fzf --walker=file,hidden,follow

Runs forever.

fzf --walker=file,hidden

Finishes. (Very quick even.)

Sooo, something goes wrong with symlinks.

[harrypotterIUP]

Investigating possible responsible subfolders which contain symlinks:

find . -type l | sed 's/.\/\([^/]\+\).*/\1/p' | uniq

[harrypotterIUP]

Closed in favor of #4015