The complete guide: logrotate utility on Linux

Learn everything about logrotate utility in Linux. This post explains what is it, why to use it, and its configuration steps.

What is logrotate?

First of all, but an obvious introduction to logrotate tool. It is a utility which rotates, compress, purge, email logs once they match specified criteria like size, an age defined in the configuration. In short, its an automated log management tool.

It offers service (log specific) policies that can be set and it will handle logs accordingly. Conditions to trigger logrotate can be set on a size basis or time basis like daily, weekly, or monthly.

Why to rotate logs ?

Another obvious question is why to rotate logs? Logs are files that grow over time. Since they keep on filling over time. If you don’t maintain them they will end up filling your mount point which you never want to see in production!

Means, to save your disk space logs should be rotated. Logs are always useful for troubleshooting but yes you don’t need pretty old logs keeping your disk space on toss!  Logs can be purged, compressed, or moved manually. But to save your time from manual tasks you can automate log management using this tool.

Logrotate Installation:

Let’s walk through logrotate configuration. How to install it, how to configure it as per your requirement etc.

First of all, you need to install logrotate utility on your machine like below (Red Hat, CentOS, Fedora):

For Debian, ubuntu use : apt-get install logrotate

# yum install logrotate
Loaded plugins: amazon-id, rhui-lb, security
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package logrotate.x86_64 0:3.7.8-26.el6_7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================
 Package                        Arch                        Version                              Repository                                             Size
=============================================================================================================================================================
Installing:
 logrotate                      x86_64                      3.7.8-26.el6_7                       rhui-REGION-rhel-server-releases                       58 k

Transaction Summary
=============================================================================================================================================================
Install       1 Package(s)

Total download size: 58 k
Installed size: 87 k
Is this ok [y/N]: y
Downloading Packages:
logrotate-3.7.8-26.el6_7.x86_64.rpm                                                                                                   |  58 kB     00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : logrotate-3.7.8-26.el6_7.x86_64                                                                                                           1/1
  Verifying  : logrotate-3.7.8-26.el6_7.x86_64                                                                                                           1/1

Installed:
  logrotate.x86_64 0:3.7.8-26.el6_7

Complete!

Confirm if its installed

# rpm -q logrotate
logrotate-3.7.8-26.el6_7.x86_64

Once logrotate is confirmed installed move on to configurations.

Logrotate configuration :

/etc/logrotate.conf is the main configuration file. Individual service-specific configuration files can be kept under directory /etc/logrotate.d

To use these service-specific individual config files you must include below the line in /etc/logrotate.conf

include /etc/logrotate.d

Let’s configure logrotate to manage /var/log/messages (Syslog in Linux). To keep things clean, its recommended to create individual configs under /etc/logrotate.d. We will create rotate_syslog.conf file under this directory and add the below code.

/var/log/messages {
daily
rotate 3
size 10M
compress
delaycompress
}

Now understand each filed specified in conf file –

1. daily: Rotate logs on daily basis
2. rotate 3: Means keep last 3 rotated logs. Older copies to be purged.
3. size 10M: Min size for rotation is 10MB i.e. logs won’t be rotated unless they grow more than 10MB
4. compress: Compress rotated logs
5. delaycompress: Do not compress current log and last rotated log.

Above all options in config file tells logrotate to rotate /var/log/messages logs when they grow over 10MB. After rotation keeps any extra logs than the last 3 one. Also, compress logs except current and last rotated one. Check for size daily and if found >10MB then rotate.

logroate testing :

We will dry run the above config file to check how it will work in the actual run. -d option dry runs utility but doesn’t rotate logs in actual. We have 21MB /var/log/messages file on the machine.

# logrotate -d /etc/logrotate.d/rotate_syslog.conf
reading config file /etc/logrotate.d/rotate_syslog.conf
reading config info for /var/log/messages

Handling 1 logs

rotating pattern: /var/log/messages  10485760 bytes (3 rotations)
empty log files are rotated, old logs are removed
considering log /var/log/messages
  log needs rotating
rotating log /var/log/messages, log->rotateCount is 3
dateext suffix '-20170224'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
previous log /var/log/messages.1 does not exist
renaming /var/log/messages.3.gz to /var/log/messages.4.gz (rotatecount 3, logstart 1, i 3),
renaming /var/log/messages.2.gz to /var/log/messages.3.gz (rotatecount 3, logstart 1, i 2),
renaming /var/log/messages.1.gz to /var/log/messages.2.gz (rotatecount 3, logstart 1, i 1),
renaming /var/log/messages.0.gz to /var/log/messages.1.gz (rotatecount 3, logstart 1, i 0),
renaming /var/log/messages to /var/log/messages.1
removing old log /var/log/messages.4.gz
error: error opening /var/log/messages.4.gz: No such file or directory

In above output you can see, it rotate logs, compress them (gz extensions), renames them to maintain iterations, try to purge older than 3 logs

When I dry run on file with 9MB size, it gave me below output :

# logrotate -d /etc/logrotate.d/rotate_syslog.conf
reading config file /etc/logrotate.d/rotate_syslog.conf
reading config info for /var/log/messages

Handling 1 logs

rotating pattern: /var/log/messages  10485760 bytes (3 rotations)
empty log files are rotated, old logs are removed
considering log /var/log/messages
  log does not need rotating

Since we specified 10MB size to rotate log and our file is less than 10MB, logrotate reported no need to rotate logs!

You can observe logrotate entry being added to /etc/cron.daily directory too.

# cd /etc/cron.daily
# ll
total 32
-rwx------. 1 root root  118 Jun 10  2015 cups
-rwx------. 1 root root  180 Jul  9  2015 logrotate
-rwxr-xr-x. 1 root root  905 Nov 16  2012 makewhatis.cron
-rwx------. 1 root root  189 Jan 26  2015 mlocate.cron
-rwxr-xr-x. 1 root root 2126 Jul 19  2013 prelink
-rwxr-xr-x. 1 root root  563 Oct 21  2013 readahead.cron
-rwx------. 1 root root  256 Jun  9  2015 rhsmd
-rwxr-xr-x. 1 root root  416 Oct 14  2015 tmpwatch

logroatete available options :

Below is a list of logrotate options that can be used in configuration files along with their meaning.

1. daily,monthly, weekly: when to rotate logs (if conditions meet)
2. rotate N: Purge all older logs keeping the last N rotated logs on the server
3. size X: Rotate logs when they grow more than X size
4. compress: Compress rotated logs using gzip (there are more options to choose compression commands, extensions, etc if you want)
5. delaycompress: Don’t compress current and last rotated log file
6. copy: Make a copy of the current log file
7. create mode user group: After rotation create an empty log file with permission (mode), user and group specified
8. dateext: Append the date to rotated file names
9. mail: Mail rotated logs to email before deletion (more options available with this)
10. ifempty: Rotate log even if it’s empty
11. maxage N: Rotate logs older than N days
12. maxsize N: Rotate when they have grown beyond N size even before run-time daily, weekly or monthly.

There are many other options available that can be used. I have listed here only frequently used ones. A detailed list can be obtained on the man page of logrotate.

All the above outputs are from the RHEL6 box. Let me know if you have any queries in the comments or if you are looking for specific options to rotate logs with your requirement.