Strange 404's caused by Google

[universal4]

I discovered some strange 404 errors recently and wanted to see if anyone has seen these.

We all know that Google will warn webmasters when they try to follow a link and see that the page returns a 404.

Well I found that they were trying to go to a page or link mobile/
When that didn't work they went to m/

They claimed these pages were linked from my home page, well they certainly are NOT!

So I wrote a few entries in htaccess to fix it and notified google the errors were fixed.

I then proceeded to log in to 10 or 12 more websites and sure enough they ALL showed that each site had the same 404's.

This is NOT a wordpress issue, nor is it a dreamweaver or front page issue. 3 different development environments and in fact some of the sites are on different servers. Some are wordpress and some are straight html sites.

If google in their infinite wisdom determines you should have a specific page residing on your website, they will claim your site has errors if it does not have those pages.

Sorry Mr. Page and Mr. Brin, website owners should have the right to determine what page names are used on their sites and you should only follow and warn them of links that actually exist on this planet and stop making stuff up because it suits your engineers.

Rick
Universal4

[dfiocch]

Rick, I'm noticing a similar behavior.
G is crawling a lot of old pages now coming back with 404 errors.
90% of those pages are very old pages (2006, 200.
Looks like G is doing some heavy "housecleaning"

Edit: just discovered a new one... some pages with querystrings! WTF! G is crawling my backend (blocked also by robots.txt)

[golfbettingsystem]

Yep, saw exactly the same on my straight html sites probably a fortnight ago or more. Marked them as fixed rather than amend .htaccess myself and they've not reappeared, put it down to something related to the mobile alogrithm update in some shape or form.

[slotplayer]

I've been seeing a lot of stuff like that lately.
I see 1 from a /m and another from /mobile, dated 4-13-15
Looks like Google did a deep crawl because I had a post from cap dated 2007 where I put an x in front of the http: of a strange link to my site and G reported it as a 404. It looks like putting the xx in from of the http: doesn't work any more.

Does anyone disallow their sites' 404 error page in their robots.txt? I had mine disallowed but it still shows up in Google search but says the desc is blocked by the robots.txt file. I didn't want it to be crawled at all.

I might add I see some strange stuff when I look at the linked from.

[wonderpunter]

yes, I have a lot of warnings about affiliate links which i have disallowed in robots.txt anyhow.. telling me how they are not mobile compatible etc.. treating them like pages, when they are A. just links, B. shouldn't be crawled anyhow

[xecutable]

I see links poping from like 2010. I don't even know where they are coming from. Not enough info in WMT. Just says error 404 and that's all. Im pretty sure I had those cleaned up, but I've seen 404 reported if anyone externally linked to those posts as well.

[universal4]

I too have been seeing 404's from old pages long since gone (years) but in those cases the pages "used to exist".

It's aggravating and actually disgusting that Google would attempt to crawl pages (from what it looks like on every website on the planet) that do not exist and just because some other site named pages /m or /mobile does not mean they should attempt to crawl them on the rest of the planet.

Follow the links and read the damn site maps and stop making crap up.

Rick
Universal4

[Sherlock]

Universal - are you 100% sure they never existed? Especially if you use some 3rd party CMS system like WP, with all the plugins etc., maybe they were generated at some point in the past even without you knowing that.

And yes, it happens now to pages that were deleted long ago. I have those warnings on domains that I bought and redesigned.

[universal4]

Yes Sherlock,

I am 100% sure without a doubt that those pages NEVER existed, and they damn sure did NOT exist on more than 15 sites and since many other webmasters are seeing the exact same thing, it is pretty close to 100% proof that Google is attempting to go to pages at every website on the planet named /mobile and /m.

By the way some of the sites are straight html sites and are not wordpress, although some were.

Rick
Universal4

[Topboss]

I am seeing the exact same thing - especially the /mobile and /m which are appearing for practically every site in my Google Webmasters tools regardless of the server.

[tehsik]

same thing here too.... This leads me on thinking that this is another reason why old pages are important and have better value than the new ones..... So when you decide to remake a page and change its url think on doing a 301 redirect too ... always ...

[tehsik]

and for your response universal, use this : https://pro-sitemaps.com/ , and if this shows OK than don't worry and keep marking pages as fixed .

[universal4]

You see, although I do appreciate the link, I do not need a spider to show me that I have NEVER in the history of more than 15 sites that showed this issue that I certainly do not now nor have I EVER had pages names /m or /mobile, and if I had I certainly didn't have them on 15 websites.

And for the sake of the argument, let's say I did.

Did dozens, hundreds or maybe even tens of thousands of others also name their pages the same?

As far as the sitemaps that I actually submitted, since by Google's terms is what they should follow, NONE of them listed any pages that do not exist. Some site maps I have created manually, some I have used a php program, and a few I have used a wp plugin, and I have checked them all.

I have never solicited links for the non-existent pages, and in the first few cases I specifically search the web looking to see if some screwball out there was creating and publishing thee links, but have found zero occurrences.

This leads me to speculate that once again Google is following a totally and completely incorrect standard that actually goes against their own stated terms and standards.

IE: They are attempting follow links and look for pages that have never existed and that no one on the internet links to. I have seen posts and comments about this same issue appearing on occasion throughout seo discussions in recent weeks so it is clearly not an isolated incident.

Rick
Universal4