Build Syslog server in Linux for centralized log management

Step by step guide to configure Syslog Server in a Linux environment. Learn how to enable remote Syslog logging in Linux for centralized log management.

In many It infrastructure environments, clients choose to have one centralized Syslog server in which all logs from remote systems can be collected. It then easier to filter, monitor, verify a report in a single location rather than querying all systems in infra. In this post, we will be seeing how to configure Linux machine to act as a Syslog server.

In the configuration, there are two parts. First server-side configuration to be done on the Linux machine which will act as Syslog server. Secondly, client-side configuration to be done on a remote system that will be sending logs to the Syslog server.

Server side configurations:

A machine which will be acting as Syslog server should have below pre-requisites done :

1. syslog daemon i.e. syslogd should be up and running
2. portmap and xinetd services should be running
3. Targeted client machine’s IP range should be able to reach the Syslog server over network.

# service syslog status
syslogd (pid  3774) is running...
klogd (pid  3777) is running...
# service portmap  status
portmap (pid 3891) is running...
# service xinetd  status
xinetd (pid  4410) is running...

Once you make sure all related services are running, proceed to edit syslogd configuration file i.e. /etc/syslog.conf. You need to add -r option in the configuration file which will enable daemon to receive logs from remote machines.

# cat /etc/sysconfig/syslog

# Options to syslogd
# -m 0 disables 'MARK' messages.
# -r enables logging from remote machines
# -x disables DNS lookups on messages recieved with -r
# See syslogd(8) for more details
SYSLOGD_OPTIONS="-m 0"
# Options to klogd
# -2 prints all kernel oops messages twice; once for klogd to decode, and
#    once for processing with 'ksymoops'
# -x disables all klogd processing of oops messages entirely
# See klogd(8) for more details
KLOGD_OPTIONS="-x"
#
SYSLOG_UMASK=077
# set this to a umask value to use for all log files as in umask(1).
# By default, all permissions are removed for "group" and "other".

Here you can see a row with parameter SYSLOGD_OPTIONS="-m 0". This needs to be added with -r option like  SYSLOGD_OPTIONS="-r -m 0". 

Edit the conf file with a text editor like vi and add -r option as stated above. To take up these new changes restart Syslog service.

# service syslog restart
Shutting down kernel logger:          [  OK  ]
Shutting down system logger:          [  OK  ]
Starting system logger:               [  OK  ]
Starting kernel logger:               [  OK  ]

Now your server Syslog daemon is ready to accept logs from remote machines. All messages from remote machines and Syslog server’s own Syslog will be logged in /var/log/messages on Syslog server. Its own messages will be having “localhost” in 2nd field after the date and remote machine logs will be having IP/hostname instead of localhost in the 2nd field.

It should look like below once it starts populating remote machine’s logs too. First entry beings its own and second one being remote server’s log.

Nov 10 12:34:44 localhost syslogd 1.4.1: restart (remote reception).
Nov 10 12:34:44 server3  snmpd[4380]: Connection from UDP: [10.100.49.125]:55234

Client side configurations:

In client machine, you need to edit Syslog configuration file /etc/syslog.conf. Here you need to instruct Syslog daemon to send logs to remote Syslog server.

Open /etc/syslog.conf configuration file and append user.* @[ server IP] to end of it. In which server IP is your Syslog server IP. If you have mentioned Syslog server IP in /etc/hosts of client machine then you can give hostname in above entry instead of IP.

user.* defines the type of log messages to be sent to the Syslog server. If you want to log all messages to the Syslog server you can use *.* or you can choose the type of logs defined in this config file itself. Read the below file and you will get to know different types. Defining *.* is not advisable since it will be flooding logs on the Syslog server and its storage might get full if you have many machines sending logs to the server at a time.

This should look like below. Check last line of file :

# cat /etc/syslog.conf

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 *

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log

user.*          @10.12.2.5

After editing conf file, restart syslog daemon to get this new config in action.

You can send test log to check if your setup is working using below command :

# logger -p user.info “Test log message”

This will send a user.info type messages to Syslog locally. It will be logged to local /var/log/messages and also gets forwarded to the Syslog server on the mentioned IP. You should see below entries :

On local i.e. client 
# tail -1 /var/log/messages
Dec  7 01:27:09 localhost root: “Test log message”

On syslog server 
# tail -1 /var/log/messages
Dec  7 01:27:09 server3 root: “Test log message”

This will confirm your Syslog server is accepting remote logs perfectly and the machine you configured as the client is sending logs to the server too!