Do you analyze your log files?

Ages ago I used to rely on log file analysis to understand traffic to my websites. Then, along came Google analytics and after a while I stopped paying any attention to log files, relying on Google Analytics and Google Search Console (well, Google Webmaster Tools at the time) for insights about the traffic to my sites.

Over the past few years I've added the Screaming Frog spider into the mix, and found it critical to understanding what my sites look like and to rapidly identify and fix issues.

Lately, though, I'm thinking it might make a lot of sense to add log file analysis back into the mix. I've known for a while that in addition to their spider that I love, Screaming Frog came out with the first version of their log analysis tool nearly five years ago now (It was announced on 11 April 2016 in the following blog post: Announcing The Screaming Frog Log File Analyser). The most recent major release was announced on 25 March 2020 just as the COVID-19 pandemic was causing serious havoc and was codenamed Stay At Home: Screaming Frog Log File Analyser Update – Version 4.0. Just over a week ago a new update, version 4.3 was announced.

So, based on my interest in trying this tool out, for this week's poll I ask if you have used a log file analyzer recently, or if you think you should. If you have used one recently, what analyzer have you used and what do you think of it.

Michael

i personally only use it to check for attacks and things like that. should do i frequently but in reality i mostly only do it in case i am facing issues with the server..

Never used it. Maybe, I need to change my point of view on that. Thanks for the hint!

Assuming you are taking about server side logging.
the only reason to open server side logs is to identify attacks,but there's almost nothing you can do about it so I don't see why to bother. But that's server side logs.

Client side log, also know as pixel, on the other hand is crucial for monitoring, monetization and conversions.

We need to track page views, clicks, conversion, net revenue.
It's 2021 and most iGaming affiliate do not monetize!

Sent from my SM-N981B using Tapatalk

I never used either. But looks interesting.

For wordpress sites, I use the Defender plugin which monitors attempted attacks and login attempts as well as 404 errors. Gives some real insight into the sheer volume of bots sniffing around websites.

Never checked log files. Always use other tools like Analytics and Search Console. Might be worth checking the log files too. Thanks for the suggestion.

A log file can grow infinitely in size, if the level of errors is not reduced to critical only. It could even weigh gigabytes. I have set up a cron job that removes it once a month.

To deal with larger log file sizes, you can tell the server to do daily logs instead of a single or monthly log. At that point to can use a cron to zip the logs up weekly, monthly or whatever if you prefer, or delete the logs older then a certain time frame of you prefer.

Rick
Universal4

I wonder what you are doing with the information from the log files.
It's a plain text file, not indexed and has web server information.
It's like Google analytics. Everyone does it but only few know what they can do with it (with affiliate website it's even more complicated )

I started analyzing logfiles from a SEO point of view: you can understand many things about googlebot having problems spidering your site, searching for very old pages and interesting stuff like that.

To be honest, never checked the log files. Maybe it's a good time to start. Thanks for the suggestion.

For anyone who finds hard/doesnt want to go through the filterin/hassle i use this sometimes

sunlightmetrics com

not affiliates in any way

Quote:

---

Originally Posted by fsc

For anyone who finds hard/doesnt want to go through the filterin/hassle i use this sometimes

sunlightmetrics com

not affiliates in any way

---

Thanks for the tip, but unfortunately they don't accept new users.

No, I never analyze my log files, at least not for many years, but think I should.

And thanks, I think I got a new topic to study