300k Website! How do i get the whole site indexed??

[gillies8888]

My site is over 300k pages now.

Any ideas on getting the whole lot to go through google?

Alot of the team stats pages are not indexing.

Any ideas would be great

[eenzoo]

Check your internal linkbuilding, probably you have a lot of "orphaned" sites without any incoming internal links. Signup with Google webmaster tools and submit a sitemap. Do you use a special CMS? I can´t see your source code properly, Google maybe has the same problem!

[Christiaan]

300k pages is a lot. And they all look similar. Google doesn't want to spend all its sources to 1 website. Therefore they only index part of your website. Especially if the pages look similar.
Only if your site becomes a highly important website Google will index all of your pages.

The tips eenzoo gives like a Signup to GWT and submit a sitemap can help a bit but they still won't index all the pages.

Your sourcecode looks fine to me, it is minified which is even better!

[wonderpunter]

How old are the pages? 300,000 is a lot to crawl... you could ping.. sitemap should give you what you need..

[DanHorvat]

A single Sitemap is limited to 50.000 URLs and 10 MB.

[wonderpunter]

A single Sitemap is limited to 50.000 URLs and 10 MB.

Thats a Google limit you can create your own sitemap for personal use as big as you want, Then he can extract the information and ping.

[DanHorvat]

Thats a Google limit you can create your own sitemap for personal use as big as you want, Then he can extract the information and ping.

That's the Sitemap protocol limit and is therefore universal. The solution is to use a sitemap index.

[gillies8888]

Looks like i need to give this some thought. Work out a new plan. Would love it all 2 index, but will just keep playing.

[Doolally]

Google doesn't include pages where it sees little value or duplicated content. Stats pages may well reflect this.

[gillies8888]

Looks like i need to give this some thought. Work out a new plan. Would love it all 2 index, but will just keep

[sweetbet]

Google doesn't include pages where it sees little value or duplicated content. Stats pages may well reflect this.

I agree. Doolally has hit the nail on the head.

[wonderpunter]

Google doesn't include pages where it sees little value or duplicated content. Stats pages may well reflect this.

It may not show or rank.. but should still index.. 300,000 pages is a lot.. were they just produced or have they been there for a while? and are they dynamic?

[Moonlight Cat]

Alot of the team stats pages are not indexing.

Any ideas would be great

Check your site here: https://developers.google.com/speed/pagespeed/insights/

Your site loaded very slow 9/100. May be this is the reason, why google don't
indexing all pages.

[wettFORM.AT]

wow that´s a lot. I would not send stats pages to the index. G can see it as low content or low quality without any kind of written text on it. Do you think that all pages have the change to rank? Are these pages of a good quality in terms of G guidelines?

[gillies8888]

Wettform you will be shocked at the amount of people looking for these stats. But will have to look at other options. Possibly create a stats site on its own.

[wettFORM.AT]

Hi Gillies,

I am sure that people look for stats sites but how can you manage that search engines see these sites as quality content? We also have some sites with good quality and useful information for our visitors but we do not send them to the index because we don´t have written content on it. Does this makes sense?