Monitor log files for specific patterns

In this tutorial, we will learn how to create a Bash script for monitoring log files for specific patterns. This task can be particularly useful when you’re dealing with large amounts of logs and want to catch specific events or issues in real time.

While there are dedicated tools for log monitoring like Logstash or Fluentd, sometimes a simple Bash script can do the job just fine.

Let’s get started.

Step 1: Understand the commands

We will base our solution on the tail -F and grep commands. The tail -F command allows us to continuously follow the content of a file as it grows, while grep allows us to filter this content based on a pattern.

For instance, tail -F /var/log/syslog | grep "error" will print every new line containing the word “error” in the system log.

Step 2: The Basic Script

The first version of our script will take a log file path and a pattern as arguments:

#!/bin/bash

LOG_FILE=$1
PATTERN=$2

tail -F $LOG_FILE | grep --line-buffered "$PATTERN"

You can save this script as log_monitor.sh and run it with the log file and the pattern as arguments, like ./log_monitor.sh /var/log/syslog error.

Step 3: Adding Custom Options

We want to make our script more flexible by adding custom options. For instance, we may want to ignore case when matching the pattern, or we may want to send an email when the pattern is found.

We will use the getopts built-in function of Bash for parsing options. Our options will be:

• -i for ignoring case.
• -e for specifying an email to send alerts.

The updated script will look as follows:

#!/bin/bash

IGNORE_CASE=""
EMAIL=""

while getopts "i:e:" opt; do
  case ${opt} in
    i )
      IGNORE_CASE="-i"
      PATTERN=$OPTARG
      ;;
    e )
      EMAIL=$OPTARG
      ;;
    \? )
      echo "Invalid option: $OPTARG" 1>&2
      exit 1
      ;;
    : )
      echo "Invalid option: $OPTARG requires an argument" 1>&2
      exit 1
      ;;
  esac
done
shift $((OPTIND -1))

LOG_FILE=$1

if [ -n "$EMAIL" ]; then
  tail -F $LOG_FILE | grep --line-buffered $IGNORE_CASE "$PATTERN" | while read line; do
    echo "$line" | mail -s "Log Monitor Alert" $EMAIL
  done
else
  tail -F $LOG_FILE | grep --line-buffered $IGNORE_CASE "$PATTERN"
fi

Now you can run the script like ./log_monitor.sh -i error -e me@example.com /var/log/syslog. This will monitor the system log for lines containing the word “error” (ignoring case), and send an email to me@example.com for each matching line.

Note: The mail command must be properly configured on your system for sending email alerts.