Smokeping
Smokeping allows you to probe a list of servers, store that data using RRDtool, and generate statistical charts based on RRDtool's output. Smokeping consists of two parts. A daemon runs in the background pinging and collecting data at set intervals. A web interface displays that data in the form of graphs.
This wiki page covers a basic setup of the smokeping daemon and the CGI webinterface.
Installation
This section covers the installation of Smokeping using the smokeping package. FastCGI on Apache will be setup as described in Apache and FastCGI.
The smokeping package consists of two parts:
- The smokeping daemon and configs in
/etc/smokeping/. This daemon performs the monitoring. - The smokeping "htdocs" in
/srv/http/smokeping. These will be used by the web interface.
In addition to the smokeping package, you will need:
- A tool that smokeping can use for monitoring. fping is the simplest and default method for simple ping probes.
- apache and mod_fcgidAUR for the web interface if you are using Apache.
- fcgiwrap and start and enable fcgiwrap.socket if you are using Nginx
- An image cache directory that the FastCGI script can write to, e.g.
/srv/smokeping/imgcache - A data directory that the smokeping daemon can write to, and the FastCGI script can read, e.g.
/srv/smokeping/data - To ensure that the main configuration file is readable by the smokeping daemon.
Optional Prerequisites
If you want to use other probes such as the DNS or http probe you will need other packages as shown below. The configuration of these will is not covered by this wiki page.
| Probe | Package Needed |
|---|---|
| Curl | curl |
| DNS | bind (for the dig utility) |
| EchoPing | echoping |
| SSH | openssh |
| TelnetIOSPing | perl-net-telnet |
| AnotherDNS | perl-net-dns |
| LDAP | perl-ldap |
| LDAP (tls) | perl-io-socket-ssl |
| Authen | perl-authen-radius |
Configuration
Smokeping requires you to edit a few files. The unedited files end
with the .dist extension. Rename the .dist
files in /etc/smokeping to remove the suffix. The find command does this and
prints out each file that is being renamed and needs editing:
# cd /etc/smokeping
# find . -name '*.dist' -print -execdir sh -c 'mv {} $(basename {} .dist)' \;
# mv /srv/http/smokeping/smokeping.fcgi.dist /srv/http/smokeping/smokeping.fcgi
Editing the configuration file
Next, edit the /etc/smokeping/config file; this is smokeping's main configuration file. A brief description of the sections is followed by a complete example.
Note: most paths need changing, potentially from /usr/etc/ to /etc/smokeping, but confirm each one.
Paths for the various .dist files can be found with pacman -Ql smokeping
The General section of the /etc/smokeping/config file is the easiest to edit. Personalize the
top of the configuration file to match your information. The comments describe each field. Note that if you do not have the sendmail program installed (ie from postfix or sendmail) then use something else instead like /bin/false. The file you specify must exist or smokeping will error out.
The Alerts section does not require any changes for this example.
The Database section does not require any changes.
In the Presentation section the path to the template filename needs to be updated, (ie /etc/smokeping/basepage.html)
The Probes section specifies which probes are active. By default only the FPing probe is enabled. This section does not require any changes.
The Slaves section is not needed in this minimal example, so it can be either commented out or removed. Note that if you use the smokeping_secrets setting in the Slaves section, you will have to make that file unreadable to the rest of the world, or else smokeping will error: chmod 600 /etc/smokeping/smokeping_secrets.
The Targets section specifies which hosts will be probed (pinged in our example). Customize it so that it probes the host(s) you would like to collect statistics on, as shown in the example below.
You can learn more about the Smokeping configuration file with the examples at https://oss.oetiker.ch/smokeping/doc/smokeping_examples.en.html
/etc/smokeping/config
*** General ***
owner = Your Name Here # your name
contact = your.email@host.bla # your email
mailhost = your.smtp.server.bla # your mail server
sendmail = /bin/false # where the sendmail program is
imgcache = /srv/smokeping/imgcache # filesystem directory where we store files
imgurl = imgcache # URL directory to find them
datadir = /srv/smokeping/data # where we share data between the daemon & webapp
piddir = /var/run # filesystem directory to store PID file
cgiurl = http://localhost/smokeping/smokeping.fcgi # exterior URL
smokemail = /etc/smokeping/smokemail
tmail = /etc/smokeping/tmail
syslogfacility = local0
# each probe is now run in its own process
# disable this to revert to the old behaviour
# concurrentprobes = no
*** Alerts ***
to = alertee@address.somewhere
from = smokealert@company.xy
+someloss
type = loss
# in percent
pattern = >0%,*12*,>0%,*12*,>0%
comment = loss 3 times in a row
*** Database ***
step = 300
pings = 20
# consfn mrhb steps total
AVERAGE 0.5 1 1008
AVERAGE 0.5 12 4320
MIN 0.5 12 4320
MAX 0.5 12 4320
AVERAGE 0.5 144 720
MAX 0.5 144 720
MIN 0.5 144 720
*** Presentation ***
template = /etc/smokeping/basepage.html
+ charts
menu = Charts
title = The most interesting destinations
++ stddev
sorter = StdDev(entries=>4)
title = Top Standard Deviation
menu = Std Deviation
format = Standard Deviation %f
++ max
sorter = Max(entries=>5)
title = Top Max Roundtrip Time
menu = by Max
format = Max Roundtrip Time %f seconds
++ loss
sorter = Loss(entries=>5)
title = Top Packet Loss
menu = Loss
format = Packets Lost %f
++ median
sorter = Median(entries=>5)
title = Top Median Roundtrip Time
menu = by Median
format = Median RTT %f seconds
+ overview
width = 600
height = 50
range = 10h
+ detail
width = 600
height = 200
unison_tolerance = 2
nodata_color = ffb0b0
loss_background = yes
"Last 3 Hours" 3h
"Last 30 Hours" 30h
"Last 10 Days" 10d
"Last 400 Days" 400d
*** Probes ***
+ FPing
binary = /usr/bin/fping
*** Targets ***
probe = FPing
menu = Top
title = Network Latency Grapher
remark = Welcome to the SmokePing website of Arch User. \
Here you will learn all about the latency of our network.
+ targets
menu = Targets
title = Targets
++ CloudflareDNS
menu = Cloudflare DNS
title = Cloudflare DNS server
host = 1.1.1.1
++ GoogleDNS
menu = Google DNS
title = Google DNS server
host = 8.8.8.8
++ MultiHost
menu = Multihost example
title = CloudflareDNS and Google DNS
host = /targets/CloudflareDNS /targets/GoogleDNS
Notes on the smokeping configuration file syntax
Each + character defines a section in the hierarchy. Spaces are not allowed in the section names. Period and forward slashes should probably also be avoided in section names. This is probably because the RRD files are stored under the data directory with the same exact names as the sections.
In the Targets section, you can define host as either a real host name or the path to another section to generate a multiple host chart, as shown in the MultiHost example above.
Setup the rest of the system
Setup the extra directories referenced by the configuration file:
# mkdir -p /srv/smokeping/data # mkdir -p /srv/smokeping/imgcache # chown -R smokeping:smokeping /srv/smokeping # chown -R http:http /srv/smokeping/imgcache # chmod a+rx /srv/smokeping # chmod -R a+rx /srv/smokeping/data
Since the smokeping configuration is read by both the smokeping daemon and the FastCGI scripts, so it needs to be readable:
# chmod a+rx /etc/smokeping # chmod a+r /etc/smokeping/config
Notes on Smokeping in Master Slave configuration
A master-slave has one additional wrinkle, though the principles are the same.
For slaves, smokeping IS NOT writing the data to the RRD files - they're getting pushed there by the web-server. So, in master-slave mode you need to be sure that the user your web-server is running as [nginx running as user http] also has WRITE permissions to the RRD files (under /srv/smokeping/data).
When running Smokeping in Master / Slave mode, the master has effective permissions of the user the smokeping slave runs as. Do NOT run slaves as root. Read the security considerations first.
The Smokeping FAQ on their wiki details the complex permission requirements. If your master slave config is still not working, use setfacl to provide the minimum required permissions:
On the master, give both `http` and `smokeping` full access (rwx) to `/srv/smokeping/*` # setfacl -R -m u:http:rwx /srv/smokeping # setfacl -R -m u:smokeping:rwx /srv/smokeping Set specific read permissions for files inside `/etc/smokeping` # setfacl -m u:http:r /etc/smokeping/smokeping_secrets # setfacl -m u:smokeping:r /etc/smokeping/config /etc/smokeping/smokeping_secrets
Slaves require no config file, they pull their config from the master, and just need to read /etc/smokeping/smokeping_secrets:
# /usr/bin/smokeping --master-url=https://smokeping.site/smokeping.fcgi # setfacl -m u:http:r /etc/smokeping/smokeping_secrets
A sample slave's systemd unit:
[Unit] Description=Smokeping Slave: my.slave.net Documentation=man:smokeping_master_slave(7) [Service] ExecStart=/usr/sbin/smokeping \ --master-url=https://smokeping.site/smokeping.fcgi \ --cache-dir=/var/lib/smokeping \ --shared-secret=/etc/smokeping/smokeping_secrets \ --pid-dir=/run/smokeping \ --slave-name=my.slave.net # The --slave-name should be an exact match to both the master's config and smokeping_secrets ExecReload=/bin/kill -HUP $MAINPID [Service] Type=forking RuntimeDirectory=smokeping PIDFile=/run/smokeping/smokeping.pid User=smokeping Group=smokeping Restart=on-failure
Start and enable daemon
Start and enable smokeping.service. Then check that it is running.
Setup web frontend
Apache
Edit /etc/httpd/conf/httpd.conf so that it includes:
LoadModule fcgid_module modules/mod_fcgid.so <IfModule fcgid_module> AddHandler fcgid-script .fcgi </IfModule> Alias /smokeping/imgcache /srv/smokeping/imgcache Alias /smokeping /srv/http/smokeping <Directory "/srv/smokeping/imgcache"> AllowOverride all Require all granted </Directory> <Directory "/srv/http/smokeping"> Options FollowSymLinks ExecCGI AllowOverride all Require all granted </Directory>
Start Apache via the httpd.service.
Check that http://localhost/smokeping/smokeping.fcgi loads. The first data should appear after a couple of minutes.
If the fonts in the graphs are unreadable, you may need to install the ttf-dejavu package.
Caddy
Thanks to the Caddy community and with this configuration file /etc/smokeping/config and with enable fcgiwrap.socket
/etc/caddy/caddy.conf.d/smokeping.conf
smokeping.example.com {
log stdout
errors
tls john@example.com
root /srv/http/smokeping
fastcgi / unix:/var/run/fcgiwrap.sock {
env SCRIPT_FILENAME /srv/http/smokeping/smokeping.fcgi.dist
}
}
smokeping.example.com/js {
root /srv/http/smokeping/js
}
smokeping.example.com/css {
root /srv/http/smokeping/css
}
smokeping.example.com/cache {
root /var/cache/smokeping
}
An updated version of this Caddy configuration for smokeping - https://gist.github.com/Strykar/4df1eb8aebc4d5f7039f6045301352c7
Caddy 2
Thanks to the francislavoie and Caddy community and to the Caddy docs. Start and enable fcgiwrap.socket!
/etc/caddy/caddy.conf.d/smokeping.conf
smokeping.example.com {
handle /js/* {
root * /srv/http/smokeping/
file_server
}
handle /css/* {
root * /srv/http/smokeping/
file_server
}
handle /imgcache/* {
root * /srv/http/smokeping/
file_server
}
handle /images/* {
root * /srv/http/smokeping/
file_server
}
handle {
root * /srv/http/smokeping/
reverse_proxy unix//var/run/fcgiwrap.sock {
transport fastcgi {
env SCRIPT_FILENAME /srv/http/smokeping/smokeping.fcgi
split ""
}
}
}
}
Lighttpd
First install lighttpd and fcgi.
Edit the Lighttpd configuration file and ensure you have at least all of the following configuration directives:
/etc/lighttpd/lighttpd.conf
server.document-root = "/srv/http"
server.follow-symlink = "enable"
server.modules += ("mod_fastcgi")
fastcgi.server = (
".fcgi" =>
((
"bin-path" => "/srv/http/smokeping/smokeping.fcgi",
"host" => "localhost",
"port" => 8001,
)),
)
Start/enable the lighttpd.service
Systemd should show smokeping_cgi being managed by lighttpd.service.
$ systemctl status lighttpd
● lighttpd.service - Lighttpd Web Server
Loaded: loaded (/usr/lib/systemd/system/lighttpd.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2018-12-26 14:34:42 PST; 1 day 7h ago
Process: 17117 ExecReload=/bin/kill -HUP $MAINPID (code=exited, status=0/SUCCESS)
Main PID: 28321 (lighttpd-angel)
Tasks: 6 (limit: 4915)
Memory: 128.9M
CGroup: /system.slice/lighttpd.service
├─17119 /usr/bin/lighttpd -D -f /etc/lighttpd/lighttpd.conf
├─17126 /usr/bin/perl /usr/bin/smokeping_cgi /etc/smokeping/config
├─17127 /usr/bin/perl /usr/bin/smokeping_cgi /etc/smokeping/config
├─17128 /usr/bin/perl /usr/bin/smokeping_cgi /etc/smokeping/config
├─17129 /usr/bin/perl /usr/bin/smokeping_cgi /etc/smokeping/config
└─28321 /usr/bin/lighttpd-angel -D -f /etc/lighttpd/lighttpd.conf
Finally, add a symbolic link to allow images to load:
# ln -s -t /srv/http/smokeping /srv/smokeping/imgcache
Nginx
Ensure that fcgiwrap.socket and nginx.service are both running via systemctl.
Add a server block for smokeping to /etc/nginx/nginx.conf, following is an example for a TLS enabled block.
server {
server_name smokeping.example.com;
listen 443 ssl http2;
listen [::]:443 ssl http2;
root /srv/http/smokeping/;
index smokeping.fcgi;
gzip off;
access_log /var/log/nginx_smokeping.log combined;
error_log /var/log/nginx_smokeping.log;
location ~ \.fcgi$ {
fastcgi_intercept_errors on;
include /etc/nginx/fastcgi_params;
fastcgi_param SCRIPT_FILENAME /srv/http/smokeping/smokeping.fcgi;
fastcgi_pass unix:/var/run/fcgiwrap.sock;
}
location /imgcache {
alias /srv/smokeping/imgcache;
gzip off;
}
}
Verify that your configuration is fine via nginx -t as the root user and reload the configuration via nginx -s reload as the root user.
Advanced Configuration
Smokeping is a powerful tool that can be configured in many ways. You can setup many different types of probes. You can setup slave smokeping servers that can send their statistics and show you probes from other servers.
You can also create your custom probes in perl, see some third-party probes on their wiki. These options are currently not covered by this guide, please consult the documentation on the Smokeping website instead.
Also see - https://github.com/oetiker/SmokePing/wiki/FAQ
Notes
Smoketrace (Tr.cgi)
The SmokeTraceroute utility is gone since v2.5.0 according to the release notes.