GWT Crawl Errors

[baldidiot]

I guess it isn't really SEO but I figured this was (possibly) the most appropriate forum for it.

When I first put up my site I had a folder that had an incorrect name and was immediately changed, however in the crazy short period between it going up and being changed google crawled a bunch of pages, which now show up as crawl errors on my webmaster tools.

Is their anyway to tell google that these links dont exist anymore? It's been about 3 months and they're still showing up.

[allfreechips]

As long as your server shows a 401 page not found error google will drop them.

[Chips]

As long as they 404 you can also enter each URL in webmaster tools to be removed from the index. It has to be done one at a time and is rather time consuming, but it can be done.

Site Configuration>>Crawler Access>>Remove URL

[baldidiot]

As long as they 401 you can also enter each URL in webmaster tools to be removed from the index. It has to be done one at a time and is rather time consuming, but it can be done.

Site Configuration>>Crawler Access>>Remove URL

Awesome, thanks. They've returned a 404 for about 3 months now, so google is taking it's sweet time to remove them. They're driving me insane though so I might do them one by one...

[Chips]

I did it myself last year, had about 120 urls to enter but I guess it was a good investment in my time. I hate to see any errors in WMT too.

Meant to say as long as the return 404, (not 401)... don't know where that came from.

[baldidiot]

Today the number has gone up to 400

I think it might be time to spend a couple of hours removing urls!

[best.hands]

When I first put up my site I had a folder that had an incorrect name and was immediately changed, however in the crazy short period between it going up and being changed google crawled a bunch of pages, which now show up as crawl errors on my webmaster tools.

I wonder how google will react to this if you have made the following code in a .htaccess file. This code will give hint to crawlers that what they first observed before was changed and move to the correct one. Additionally, humans who happen to access the incorrect folder will be redirected the the updated and correct one, and not be left with those page error display.

RewriteRule ^incorrectfolder/?(.*) http://www.yourdomain.com/correctfolder/$1 [R=301,L]

Where $1 indicates all that follows after the dir/ will be permanently redirected to. ex. domain.com/incorrectfolder/filename.extension redirects to domain.com/correctfolder/filename.extension

If I am not mistaken, this will avoid those error responses caused by not existing file due to changed/updated directory name. And I also think that google is good enough to understand that.

One thing I am not sure from here though is if google found out it was changed and redirected permanently, will it remove those error logs in the google webmaster console. I have not stumble to the same error yet, so I really can't tell if that's really the outcome of it. However, if this results a positive, then it would save you time from manually deleting each of those errors.

[baldidiot]

I wonder how google will react to this if you have made the following code in a .htaccess file. This code will give hint to crawlers that what they first observed before was changed and move to the correct one. Additionally, humans who happen to access the incorrect folder will be redirected the the updated and correct one, and not be left with those page error display.

RewriteRule ^incorrectfolder/?(.*) http://www.yourdomain.com/correctfolder/$1 [R=301,L]

Where $1 indicates all that follows after the dir/ will be permanently redirected to. ex. domain.com/incorrectfolder/filename.extension redirects to domain.com/correctfolder/filename.extension

If I am not mistaken, this will avoid those error responses caused by not existing file due to changed/updated directory name. And I also think that google is good enough to understand that.

One thing I am not sure from here though is if google found out it was changed and redirected permanently, will it remove those error logs in the google webmaster console. I have not stumble to the same error yet, so I really can't tell if that's really the outcome of it. However, if this results a positive, then it would save you time from manually deleting each of those errors.

Oh awesome! Most of the links have been removed already but it'll definitely help with sorting this mess out!

Thanks!

[dhayman]

Not only will mis-formed URL's from your site appear in WMT, but any links to your site, that reference your domain, but that are mis-formed URL's, will also appear in WMT report. Unfortunately, short of asking the offending webmaster to make the change or removal, they will continue to appear.