Splunk – Centralized logs and log analysis

Splunk is one of the most popular  enterprise  log analyzer tools in the market. It support wide range of log management use cases . It understand the machine data , make it searchable and displays it on a nice web GUI. Machine data is useful for investigations , monitoring and making decisions regarding your apps ,servers etc.

Now the bad thing, It is costly ???? Small and medium businesses cant afford it .

They have a free offering for their enterprise edition where you get all the enterprise features, but can store only 500M data . That means you have to flush the indexes often. Though there are  opensource alternatives , what make splunk excel is its   excellent search language and beautiful GUI. If you are looking for a tool to analyze data for a small amount of time or log data volume is within 500M limits you should definitely go for splunk .

 

Installation on Centos

Splunk provides rpm for “splunk server” and “Universal log forwarder” . It is simple and very easy to setup

Register at www.splunk.com , go to Downloads section and download enterprise and splunk universal forwarder.  There will be  two files like this

1 2 rwrr. 1 root root 222237528 Sep 24 22:19 splunk6.5.059c8927def0flinux2.6x86_64.rpm rwrr. 1 root root 19651506 Sep 24 22:19 splunkforwarder6.5.059c8927def0flinux2.6x86_64.rpm

 

Install these RPMs

You can start splunk server using following command

 

1 /opt/splunkforwarder/bin/splunk start

 

It will prompt you to accept the Terms and start the service. Once its started you can access the web interface using http://192.168.2.21:8000/  (replace the IP with your IP address)

Use default login details

 

1 2 3 http://192.168.2.21:8000/ User : admin Password : changeme

Once you login you can start adding data. In most of the cases, data will come from a different machine/s which generate the logs/data.  In order to to receive data from other machines, we need to configure an input stream on a specific port . This can be done using, Settings >> “Forwarding and Receiving” >> Receive Data >> Configure Receiving link.   You can add any port here, but  prefer keeping  default port.

Install Universal forwarder

You need to install this on all machines which are sending data to Splunk Server

Download the “splunk forwarder”  RPM and Install it.

Once done you can start it using

1 /opt/splunkforwarder/bin/splunk start

Once service started, you can configure it .

Add forward server , here you will specify the address of splunk server

1 /opt/splunkforwarder/bin/splunk add forwardserver 192.168.2.21:9997


1 2 3 Username : admin   password : changeme

Add monitor, here you specify which all files need to be streamed to Splunk server

1 /opt/splunkforwarder/bin/splunk add monitor /var/log/messages

If everything goes well you can see your data in web UI , after login open the “search” app. For me it worked with

1 http://192.168.2.21:8000/en-US/app/search/

But they may change it future versions

You can configure searches and create alerts for search patterns. Also, you can create beautiful dashboards

On your data and share it across the team.

 

 

 
Author: , 0000-00-00