More projects, more servers, more headaches identifying misbehaving apps. In this series, I will be documenting my process on how to setup centralized monitoring with 1 server collecting metrics on the rest of my network. Right now I am using Nagios for alerts, which does its job well but doesn’t provide good historical data.

Grafana makes really slick graphs of data and is a great platform for viewing lots of system metrics. InfluxDB is a database designed for logging time based data, and Telegraf feeds InfluxDB your system metrics.

To get started, I have a low power Ubuntu LTS machine that I will be using to host Grafana and the InfluxDB database. I have already configured SSH and iptables for security.

Step 1: Install Grafana and dependencies

Add the Grafana and InfluxDB Repos:

curl -sL https://repos.influxdata.com/influxdb.key | sudo apt-key add -
source /etc/lsb-release
echo "deb https://repos.influxdata.com/${DISTRIB_ID,,} ${DISTRIB_CODENAME} stable" | sudo tee /etc/apt/sources.list.d/influxdb.list
echo "https://packagecloud.io/grafana/stable/debian/ jessie main" | sudo tee /etc/apt/sources.list.d/grafana.list

sudo apt update
sudo apt install grafana influxdb

Now we need to modify the Grafana config.

I recommend using another database engine instead of the built in sqlite3 database. I chose to use MySQL. 

mysql -u root -p

CREATE USER 'grafana'@'localhost' IDENTIFIED BY 'pass';  
CREATE DATABASE grafana DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_general_ci;
GRANT ALL PRIVILEGES ON `grafana`. * TO 'grafana'@'localhost';
FLUSH PRIVILEGES;
exit

Open the /etc/grafana/grafana.ini file. You will want to change the database lines to these (replacing credentials):

# Either "mysql", "postgres" or "sqlite3", it's your choice
type = mysql
host = 127.0.0.1:3306
name = grafana
user = grafana
# If the password contains # or ; you have to wrap it with trippel quotes. Ex """#password;"""
password = grafana

Later down you may want to disable user registration

allow_sign_up = true

sudo systemctl start grafana
sudo systemctl start influxdb

Now is also a good time to secure our InfluxDB server.

influx

CREATE USER "myusernamehere" WITH PASSWORD 'mysecurepasswordhere' WITH ALL PRIVILEGES
quit

Open the /etc/influxdb/influxdb.conf file, and inside the [http] section, set auth to true.

[http]
  # Determines whether HTTP endpoint is enabled.
  # enabled = true

  # The bind address used by the HTTP service.
  # bind-address = ":8086"

  # Determines whether user authentication is enabled over HTTP/HTTPS.
  auth-enabled = true

sudo systemctl restart influxdb
sudo systemctl enable influxdb
sudo systemctl enable grafana-server</pre>
<pre>

InfluxDB and Grafana are now running! Login at http://localhost:3000 with admin / admin and change your password! Up next I will talk about how to configure these services to handle incoming Telegraf data.