[TOP TIP] Multitail (live multi-file log display)

So, you are one of those weird people who keep a separate monitor, just for looking at live logs, endlessly scrolling? I've got news for you! This little shell tool is the swiss knife of log monitoring, if you haven't heard of multitail then prepare to be wow'ed with its awesomeness!

Multitail has many features and endless combinations of using those features, so I'm going to focus on just the basics.

Install

If you are on CentOS 7 then please add the epel repository first with:

yum install epel-release

Then install multitail with:

yum install multitail

Monitor /var/log/maillog

In CentOS, everything related to mail is being saved in the /var/log/maillog file, including daemons like dovecot, postfix, opendkim, spamd, policyd-spf, etc. Multitail allows us to monitor live such a log file but only focusing on the relevant parts, since multitail can pass the output of the log file through multiple regular expressions.

Here is how we may monitor parts of postfix only:

* -CS postfix = use the "postfix" scheme for coloured output

• -ev "…" = remove lines that match this regular expression

• -e "…" = only include lines that match this regular expression

• /path/to/file = log file to monitor

multitail -cS postfix -ev " postfix/dnsblog" -ev " postfix/anvil" -e " postfix" -i /var/log/maillog

In the above command, I'm removing output from "dnsblog" and "anvil" because they are irrelevant to me and clutter the log output.

Monitor /var/log/audit/audit.log

Here is how we may monitor the audit log for AVC errors in SELinux-enabled systems:

multitail -cS audit -e " avc:" -i /var/log/audit/audit.log

Monitor /var/log/httpd/error_log

It is typical to monitor the apache error log, in case something important comes up, but at the same time you want to hide all the minor warnings.

multitail -cS apache_error -ev "AH01753" -ev "AH02033"  -ev "AH01630" -ev "AH01797" -i /var/log/httpd/error_log

In the above example, I remove common warnings about unresolvable addresses, unsupported SNI and htaccess blocks.

So far so good… now lets get creative and see the awesomeness…

Split-screen multi file monitoring

Lets take the above examples and merge them together in a 3-part split screen:

multitail -cS postfix -ev " postfix/dnsblog" -ev " postfix/anvil" -e " postfix" -i /var/log/maillog -cS audit -e " avc:" -i /var/log/audit/audit.log -cS apache_error -ev "AH01753" -ev "AH02033"  -ev "AH01630" -ev "AH01797" -i /var/log/httpd/error_log

WOW! nice right?

But what if we don't like the split screen method? Could we just MERGE all logs into one output? Sure.. change the "-i" parameter into "-I" for all subsequent logs:

multitail -cS postfix -ev " postfix/dnsblog" -ev " postfix/anvil" -e " postfix" -i /var/log/maillog -cS audit -e " avc:" -I /var/log/audit/audit.log -cS apache_error -ev "AH01753" -ev "AH02033"  -ev "AH01630" -ev "AH01797" -I /var/log/httpd/error_log

Optional parameters

There are a number of optional parameters that some may find useful, for example:

* -D = remove the status bar

• -N 5000 = use a 5000 line scrollback buffer (hit "b" to scroll back, page-up/down to scroll pages)

• -s 3 = vertically split the screen in 3 parts

Is that it? I want MORE!

Sure, multitail doesn't only monitor text files, so how about executing shell commands?

multitail -rc 3 -l "ss -t -s" -rc 3 -l "top -b -n 1 | head"

This command will execute ss and top every 3 seconds, in split screens.

Enjoy!