Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

robots.txt - Disallow: /search #52

Open
ghost opened this issue Apr 4, 2019 · 2 comments
Open

robots.txt - Disallow: /search #52

ghost opened this issue Apr 4, 2019 · 2 comments
Assignees

Comments

@ghost
Copy link

ghost commented Apr 4, 2019

@DeniseColbert Any reason we're not blocking search engine indexing for /search URL's? (Lemon noticed bingbot traffic - unless there is a known reason for allowing /search URL's to be indexed, best practices would discourage search engine spiders from this portion of the site)

@DeniseColbert
Copy link
Collaborator

No reason I'm aware of... @PatReynolds do you know of any?

@DeniseColbert
Copy link
Collaborator

This has been raised again: We're having an (ongoing) issue with users landing on https://www.freeukgenealogy.org.uk/search and trying to search for their ancestors using the search bar, rather than going to the projects via the buttons. We think they're coming from:
image

Gray's options:

  1. Remove /search and all pages under /search from Google's index (removes all traces of /search from Google results)
  2. Create a separate page to direct to ancestry searches and tell Google that /search should actually refer to that URL (index gets updated to /ancestry-search or whatever the new URL is)
  3. Mix of 1 & 2 - leave /search in Google's index (pointing to the new URL) and remove pages under /search from the index Gray's preference

However we don't (/can't?) have one page where you can search your ancestors; our records are on three separate websites.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants