Monitoring Servers with Collectd, InfluxDB & Grafana

In this tutorial we are going to setup a complete monitoring  stack which has following components

Collect performance metrics from machines – Collectd
Store it in a datastore – Influxdb
Visualise timeseries data as graphs – Grafana

 

We will go through the setup, design and role of individual components and then jump into configuring individual components and linking them together

Collectd

CollectD is a performance monitoring daemon which can collect metrics and send it to various datastores, it support Graphite,Influxdb, statsd … It is built in “C” which makes it performant and portable across devices.

InfluxDB

InfluxDB (Docs here) is a time series database – the kind of databases which are made for storing and retrieving time series data. Imaging collecting data from a machine every 10 seconds and then using various mathematical functions to chart and determine trends over time and functions such as min, max, median over X hours et. There are other timeseries databases in the market, like Graphite, Prometheus ..what makes influxdb better is a superior query language

Grafana

Grafana is a dashboarding tool for building visualisations on top of data. It works well with time series data and support all major timeseries databases like graphite, prometheus , Influxdb ..

 

Setup

We have three machines with one of the components describer above installed on each , we can have all components on same machine also. But to simplify things we will setup three machines. Collectd application collect the system metrics and send it to Influxdb server. Grafana is running on third machine, which fetch data from influxdb and plot the graphs

 

How to install Collectd?

On Centos we can install collect application using yum

Once installed , we can start it using command

Or on Centos6

Configuring collect to send metrics to Influxdb

For measuring anything on system – we need to use plugins. For plugins there are two things you need to define – first load the needed plugin and secondly define appropriate settings for them. In following plugins there are default settings built in and simply loading them will start those measurements for us

Then we will Load and configure network Plugin which is used to send metrics to influxdb

In the configuration we mentioned the IP and Port number for the influx db host
we will restart collectd after making the changes

Installing and configuring influxDB

You can install Influxdb using yum

Start influxdb using

For InfluxDB there are quite a few configuration settings which we are using from template

The configuration file will look like this

Installing Grafana

You can install Grafana using Yum directly.

You can start Grafana by running:

This will start the grafana-server process as the grafana user, which is created during package installation. The default HTTP port is 3000, and default user and group is admin.

To configure the Grafana server to start at boot time:

Start the server (via systemd)

Enable the systemd service to start at boot

Once started you can access grafana by launching following url in browser

Linking all the components.

First we will verify if influxdb is getting data from collectd

On machine2, connect to influxdb server using the client

 

use the following commands inside influx console to see the time series data

 

 

Sample output is below

collectd-influxdb-grafana

 

If you can see the SERIES data from Machine1, we can confirm Influxdb is getting “timeseries” data from InfluxDB . Now we can go to Grafana and add influxdb as a new datasource. I have created a video on this explaining how to add Influxdb datasource in Grafana and  display graphs

 

Leave a Reply

Your email address will not be published. Required fields are marked *