How often do you use crawls of your site to guide you in making site updates?

[MichaelCorfman]

Crawling sites to check for errors is a common webmaster practice. It might be that you crawl you own sites directly, using a tool like screaming frog to look for issues. Or you might use a service that will crawl your site and produce reports with feedback about what you could improve such as semrush, ahrefs or moz.

Auditing your website with tools like these can provide valuable feedback. And that feedback can help you improve your site, at least with respect to the metrics they measure.

For this week's poll, I ask how often you look at data based on crawls of your site and use that data to guide you in making site improvements. Besides voting in the poll, please share in a post the tools you use, and how they influence the work you do on your sites.

Michael

[MichaelCorfman]

We make heavy use of screaming frog to crawl our sites. We monitor 17 of the more important sites on a regular basis, and those sites have many hundreds of thousands of pages between them. We cycle through sites, crawling them and looking for changes from the last crawl to make sure they seem reasonable, and looking for different types of issues. We typically crawl a few sites every week. Especially we crawl sites after development projects that I think have a noticeable risk of unintended side effects.

We usually find a few new issues to address every week.

And we also feed screaming frog lists of urls in our databases, and then make corrections based on what we find. For example, the website address for a site we link to might add or drop a www prefix. Or a site might switch from http to https, so we should update the protocol used in our links to the site. And, of course, there is plenty of link rot where external sites are simply not available anymore. So far this month we have checked links to over 20,000 websites, and we found over 1,000 links that required updating. All of those were external links that sites of ours linked to.

Michael

[newcustomeroffer]

Broken links once a quarter or so, other SEO stuff more regularly. Hummingbird is a nice plugin for wordpress in terms of core vitals/speed.

[Wannas Official]

we usually do it once a week since our website is new, so that we can see if there is any broken links or duplicate meta description. I also try to be involved with our seo team when conducting audits i make sure that everything is working fine by the way we use scream frog.

[PaulEchere]

I usually do a full crawl of my websites using screaming from once every 4-6 months.

[onlinebetdotcom]

I'm using mostly ahrefs for this but am helping a few other sites as the site I manage doesn't have much to crawl. Just check Google analytics almost daily