Web troubleshooting is fun and can be frustrating if you are not equipped with right tools. If you are supporting heavy traffic website then often you need to analyze and monitor web servers logs for performance & capacity planning. This is essential for web engineer. Checking smaller log size manually is ok, but if you have the large file, then it wouldn’t be fun to go through millions of lines to find the metrics. That’s why you need tools to facilitate administrator job and make it more productive. GoAccess is a lightweight open-source log analyzer which supports multiple log format and can be used with any of the following.
Nginx Apache HTTP AWS ELB, S3, CloudFront Google cloud storage
What metrics can you analyze with GoAccess?
Nearly everything you capture in the logs. To give you an idea:
Time is taken to serve the request Visitor IP, DNS, host Visitor’s browser & Operating System details 404 not found details Top requests/visitor Bandwidth Static files Geo Location Status Code and more..
Looking for these metrics to be monitored of your site? Good!
On which OS you can install?
GoAccess got only one dependency – ncurses. If you can install, you can use it any OS. It’s available in distribution package for:
Ubuntu Debian Fedora CentOS FreeBSD/OpenBSD Slackware Arch Linux Gentoo MacOS Windows through Cygwin
However, you can also build from the source or use with Docker. If you are new to Docker, I would recommend taking this Docker Mastery course.
Installing GoAccess on Ubuntu
Login to Ubuntu server with the root privilege Use apt-get to install as below
Easy.
Installing on CentOS
Log in to the server and execute yumcommand
Installing using Source on CentOS/Ubuntu
Love compiling from source? Here are the steps.
Install the following dependencies if using CentOS
If using Ubuntu
Download the latest package using wget
Extract the downloaded file
Go to newly created folder, which you got after extract
Compile with the below command
Well done, you have installed GoAccess and all set to analyze the logs.
Verify Installation
Once installed, just execute goaccess on the command prompt and it should print the usage like below.
Analyzing Nginx & Apache with GoAccess
One of the quickest ways to analyze access.log is by using-fparameter. Ex: Above, I am instructing to open the file access.log. This will show you the overall dashboard and the following 15 sections.
Unique visitors per day Requested files Static requests (fonts, image, pdf, etc) Not found (404) requests Visitor’s IP/host details Visitor’s OS Browser details Time distribution Referrer HTTP status code Geo location
If the chosen file is getting updated in a real-time then you will notice metrics get updated on the terminal. Here, you can go through the metrics you need to analyze.
Real-time Monitoring over HTTP(s)
GoAccess let you redirect the output to HTML file which you can use as a real-time monitoring. This is handy when you don’t want to login to the server each time you need to verify some metrics. Above, I am redirecting output to real-time.html file which is available under htdocs. Since it’s htdocs, I can access this file from https://geekflare.com/real-time.html whenever I need to see the metrics.
A beautiful dashboard! However, I won’t recommend doing this way in production. I am sure you don’t want someone to read your web server logs and you may want to apply the following restriction.
Protect the file with user and password Allow accessing only from your IP Use other URL with custom port and put that behind a firewall so only allowed IP/users can access
GoAccess looks powerful open-source logs analyzer. It’s lightweight and FREE so go ahead and give a try. You may also be interested to check out cloud-based log analyzer.
