I was googling and came across an Seo site that looks decent. Does anyone have any experience with semalt.com?
Printable View
I was googling and came across an Seo site that looks decent. Does anyone have any experience with semalt.com?
Caution might be needed if any of these comments are true.
http://clicky.com/forums/?id=16360
http://www.straight.com/blogra/59320...y-flags-semalt
http://thegrinningskull.wordpress.co...what-fk-is-it/
It seems they started being seen as early as January, and a number of people are not impressed.
Rick
Universal4
I recommend staying far away.
https://twitter.com/hashtag/semalt
Rick
Universal4
What I find interesting, and disgusting at the same time, is that they do NOT use a clearly defined user agent for their bot. If they did we could easily block the user agent....that says a lot about them. Most respectable crawlers will have an about crawler page, and will define the user agent so that anyone that wants to block them can.
I also found it quite interesting that no one has been able to pin down a set of ip's or subnets that they always use, again would make blocking easier. They must jump around and bounce off proxies or??
Rick
Universal4
I thought I had finally found someone who had pinned down their ip
http://logorrhoea.net/2014/01/how-to...sing-htaccess/
217.23.11.15 but it turns out that is only the ip of their website, and although they may run the crawler from there, from what I am reading I don't think it will help.
The absolute BEST comment I have read so far is
(bolding mine)Quote:
Hi Seth,
I would not recommend going to their website and submitting anything. I have read numerous times that this does not work and some believe they are using that as a way to confirm websites and people that have done that have found the crawling to intensify. The people running this bot have no respect for webmasters. I read on Twitter where someone had managed to redirect the semalt crawlers back to the semalt server and then the owner of the bot accused this person of launching a DDOS attack against them, when it was actually their own bots coming back at them, #semalt on twitter will bring it up.
that had me laughing a fair amount so I will be sure to visit the http://logorrhoea.net again to see what other entertaining information they might have.
What I have read so far on other sites in comments etc, many claimed the ip's were all over the place different countries etc....
Rick
Universal4
I only did the trial but I did run my site through. I'm not going to bug out about it, there isn't anything I can do now.
Their bot is actually on my sites daily and I have been wondering about them just the other day...
Same mate, I was wondering what it was but just ignored it.Quote:
Originally Posted by sportsfreak
It may be referral spam, or the bot may also be scraping and databasing content for later.
I have read a number of reports that it ignores robots.txt and also changes the user agent after being blocked by such methods as htaccess. These actions are bad behavior by a bot.
They do have a remove database, but I have read many reports of them returning, (sometime with more activity), after asking for removal and that also should be considered bad behavior.
Rick
Universal4
ah think my blog is affected by this! What should I do?? :D
I blocked semalt referrer spam with adding this to htaccess
Code:RewriteEngine on
RewriteCond %{HTTP_REFERER} semalt\.com [OR]
RewriteCond %{HTTP_REFERER} semalt\.semalt\.com [NC]
RewriteRule .* - [F]
http://semalt.com/project_crawler.php - says you can have your site removed from their crawl list, just tried, will see if it works.
Thanks for that. I'm going to add it to one of my sites that has been getting hit quite a bit by semalt.Quote:
Originally Posted by Sergej_AGR
I just added my site to the list, hopefully no harm was done in the first place.
looks like it worked, no semalt bot on my sites in the last hours ;)
I am interested if this continues for a number of weeks, especially after someone runs a compare with your site.
Rick
Universal4
Hi folks,Quote:
Originally Posted by universal4
Full disclosure - I work for an SEO agency and have been tracking semalt on both personal and work fronts.
I received an update from google about this thread and wanted to drop in with some details.
I tried their crawler removal tool on a few sites, it does stop briefly, then comes back - blocking it is the best method.
Here is a post I have done with a lot of research and information, with 2 options to block them at the bottom.
kymodo(dot)com(dot)au what-is-semalt-doing-on-your-website/
If you have any questions, please feel free to ask either on the post or on twitter - I'm KarenDFrancis
Hope that sheds some light.
How did you receive a update from google about this thread kfau ?
Thanks for asking, sorry I didn't explain that previously.Quote:
Originally Posted by edgarf76
I use google alerts - google(dot)com/alerts and one of the terms I have been tracking recently is semalt.
I gotchaQuote:
Originally Posted by kfau