Here is a Pager Duty alert for a persistent disk being greater than 80% full on an etcd node:
It turned out to be relatively simple to "turn on" disk monitoring on all jobs that contained persistent disks, across all deployments, across all stages of our client's pipeline.
We created a simple BOSH release that just adds some
monit monitoring; and BOSH health monitor does the rest. It was all built-in.
This BOSH release initially includes monitoring of ephemeral and persistent disks on a BOSH VM. Conceptually it could monitor for any global attribute of a VM that good ol' monit can monitor
[See Quick guide to using Monit in BOSH for an introduction to Monit in BOSH]
By default it alerts on either disks reaching 80% capacity.
See the monitor-server README for how to co-locate it with all your BOSH deployments for instant disk monitoring.
BOSH health alerts
BOSH already includes a Health Monitor that forwards any alerts from any VM's
monit monitor. All you have to do is turn it on.
Currently the following backends are supported:
- Pager Duty
- AWS CloudWatch
Read the BOSH docs for more information on monitoring: