Googlebot stalking me

[saviourdlima]

I know Googlebot is following me since I have just started a forum and it is already in the sandbox. When I log in, I can see that there are no online users. I can just see myself. After a few seconds, I can see 2 users online. When I check out the “Currently Active Users” page, I can see Googlebot as the other visitor. Then when I roam around the forum and come back, I see that Googlebot viewed the same page that I had been viewing. Googlebot seems to be there everywhere. I find it strange. Please help!

[universal4]

Having the googlebot in your site is a good thing....

The "lack" of that is not....

Rick
Universal4

[slotplayer]

you can run but you can not hide!

[best.hands]

you can run but you can not hide!

This is funny but true.

However, if you don't want googlebot to follow you on specific directories of your site, you declare nofollow, noindex using robot.txt for that specific directory you don't want bots in.

You can visit the following pages for guide, including the pros and cons:
- http://www.thesitewizard.com/archive/robotstxt.shtml
- http://www.searchtools.com/robots/robots-txt.html

[slotplayer]

However, if you don't want googlebot to follow you on specific directories of your site, you declare nofollow, noindex using robot.txt for that specific directory you don't want bots in.

Aren't those for use with for meta tags for individual pages.
<meta name="googlebot" content="noindex">

I think it is Allow/Disallow for files or directories in the robots.txt.

[best.hands]

Aren't those for use with for meta tags for individual pages.
<meta name="googlebot" content="noindex">

I think it is Allow/Disallow for files or directories in the robots.txt.

Yeah, that's right. Thank you for clearing that out.