Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Directory Exclusion #27

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

bleachedsleet
Copy link

@bleachedsleet bleachedsleet commented Nov 18, 2023

Recursive directory exclusion (#9) and user configurable through config.ts. A list of excluded parent directories is read into an array to be checked against during the crawling process and skipped if found. Pretty quick and dirty and could probably use some optimization.

Tidied up some related comments as well to make things a little clearer.

Recursive directory exclusion added
@vaibhavkumar-sf
Copy link

Preparing review...

// Iterate over each directory in the exclude array of the config object
for (const dir of config.exclude) {
// If the URL contains the current directory, return false
if (url.includes(dir)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for consistency sake, can we make these glob expressions? similar to include. we can use minimatch for testing against it

Copy link

@ollieanwyll ollieanwyll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but agree the global expression changes being needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants