Need help on google index

[jack87]

I have a new site which is 1 month old from the day we launch the site. until now my site index in google are still 5 pages which are the old content from the themes. i don't know if there are something i missed from google search console.

[universal4]

If the console shows pages with content that doesn't match, have you looked at the source code of those pages to make sure that content is not lurking there still?

If you mean you have a few pages or posts that appear in the index with lorem ipsum content that should clear over time, but make sure none of that exists anywhere.

If you do not have an seo plugin installed such as yoast, install one and make sure there are no metas on any pages or posts you may have "edited" instead of deleting when you first started building the site,

If you would give more specific examples you would likely get more specific answers or things to look at.

Rick
Universal4

[chaumi]

If I'm understanding correctly, it sounds to me like you put your site live too early...and Google indexed some pages that you weren't ready for them to index. You've now repopulated those pages...so when they get crawled again (which unfortunately might be some time, given it was a new site) the new content should get picked up and indexed.

The problem you might have....if the content that got indexed was rubbish and/or flawed in other ways, that could affect (negatively) the amount of time it will take for the crawler to come back again.

I'd be making sure everything is technically OK with the site, you have the content on the pages as good as you can get it, then either just wait or try asking for reindexing (of the offending pages) in GSC (and that might have no impact anyway).

If you want, name the site and you'll probably get some comments on whether it at least looks OK.

[jack87]

Hi Guys, Thank you all for the help - I will look in to it all the suggested reasons you've given. and will go back here to inform you guys. Name of my site is jack87 com "Hi admin permission to share my site name"

[chaumi]

OK I don't see anything that's obviously technically wrong with pages.

I see what you mean though. You come up first on a Google search of "Asianbookie Tips, Odds and Casino Games Review with Jack87"

But the snippet says "1 Free Spin credited for every $1 deposit. Up to 200 Free Spins valued at $0.30 each on Book. 16 games."

I'll assume that's obviously what you meant and was the template content on the theme.

I don't see that in the code, so it's definitely gone.

It's pretty safe to assume this was all down to the early mistake of putting the pages live too early, and the new content will get picked up on the next proper crawl.

There's a plug-in you can install that shows you the last bot visit, if you want to track what's happening....I've had it installed for a good few months, no problems and it didn't appear to have any negative page speed impacts....

https://en-gb.wordpress.org/plugins/...oglebot-visit/

[wonderpunter]

2 things your robot txt you have disallowed the site at the root, adding disallow without any command will keep you deindexed

# XML Sitemap & Google News version 5.3.3 -

User-agent: *
Disallow:

Also 165,000 do follow links, i would disavow or remove before you decide to index the site.. it will never make it through the algorithim

[chaumi]

It does appear to be indexed though, WP. Well, the home page at least, but indexed with the original (unwanted) content.

[dannyx]

I was indexed some time ago by a page that I did not manage to publish and GSC marked it as 404. Immediately after that I published the page and submitted it for re-verification. The Pending verification status lasted for over a month.
I guess everything takes quite a long time now, especially with new sites.

[wonderpunter]

Delete the disallow and 24 to 48 hours your impressions will explode

[pokeraussie]

Submt a sitemap to Google and obtain backlinks you will be crawled more frequently.

[jack87]

Hi all, thanks to all opinions and help. It's seems Google search console needs to request URL for indexing my new page before it will be indexed in google.

[wonderpunter]

I already told you what the problem is.. You have your robot txt set to disallow everything, if there is not /visit/ instruction or anything else it will just go back down the root so / would disallow after http:// and if you have nothing then it will disallow prior to that which is pretty much nothing (http://) , if you had fixed this the other day you would be seeing indexing.. Such an easy fix but many complicated answers

Hi all, thanks to all opinions and help. It's seems Google search console needs to request URL for indexing my new page before it will be indexed in google.

[chaumi]

I don't understand, WP.

If it's set to disallow everything, then how did the pages get indexed in the first place?

Or is it possible that the robots.txt was changed after the initial indexing? So it would be stopping any further indexing visits now?

[universal4]

I don't understand, WP.

If it's set to disallow everything, then how did the pages get indexed in the first place?

Or is it possible that the robots.txt was changed after the initial indexing? So it would be stopping any further indexing visits now?

One way is because robots and spiders mostly do not follow instructions of robots.txt or the wp setting to "discourage search engines".

Then the scrapers that refuse to follow the spider directions, often publish the data found, even if obscure pages, then since"their" sites are indexed the engines find the content there.

Put up a test site on an ip for example, turn off all the stuff in robots,txt etc, and links on the pages (even marked nofollow) will be followed. If you have affiliate links on the pages, you will often see the clicks in some of the affiliate programs from the test site or pages, easy to track using campaigns.

Now Jack's issue is a bit different, and he should change his robots.txt file as suggested by Womderpunter. The sitemap and other suggestions of a link or two would also help clear his issue faster I suspect.

Rick
Universal4

[chaumi]

Ah, I get it now. I'd just assumed it was a human error (in allowing indexing before the pages were properly populated) that allowed it to get indexed.

So Jack87, if it's still not clear.....

Sounds like you did have 'discourage search engines' enabled right from the start (and still do, or at least your Robots.txt is configured that way)

Rick is dead on. Sometimes discouraging doesn't work, the page or pages get found and noticed/indexed by search engines (probably via one or more external backlinks on other sites)

I'm of the opinion that since G has indexed it already, then it will come back again. But possibly not (I guess it could have crawled, indexed, and noted the discourage command at that point), and other search engines may not come at all.

You can see what you have by navigating to /robots.txt


Side note : When I put a site live with discourage enabled once, it was a big mistake. It was noted, not indexed.....and when I did enable it, it took G around 4-5 months to come back again even with sitemap submissions and links.


*********

PS : And note what Wonderpunter said about existing backlinks. You have a pretty huge number (you could say an oddly huge number for a new site), many from Chinese sites that look to me to be some sort of sitewide distribution. They might hurt you, although most do look to be betting-related.

Indeed, thinking about it with better clarity now, it was probably one or more of these that got you indexed in the first place. If they are OK links, it might be these that'll help you get crawled again.