How thresholds work
In this section you will learn how a metric value becomes a status, what each severity level means, and what happens when a threshold is left empty.
Severity levels
Each metric has three thresholds. They are evaluated from the most severe to the least severe, and the first one that is crossed wins.
| Level | Colour | Meaning |
|---|---|---|
| critical | red | the value crossed the critical threshold |
| major | orange | the value crossed the major threshold |
| minor | yellow | the value crossed the minor threshold |
| OK | green | the value crossed none of the configured thresholds |
For a metric to behave as expected, the three values must increase in order: minor < major < critical. A minor threshold higher than the major one means the major level can never be reported.
Empty thresholds mean “not monitored”
A threshold that is left empty is not configured, and Mugnsoft skips it entirely when evaluating a status. It is displayed as the infinity symbol.
This is evaluated level by level. You can configure only the critical threshold and leave minor and major empty: the value will be compared against the critical threshold alone and will never be reported as minor or major.
Note:
A metric with all three thresholds empty is always reported as OK, whatever its value. This is the intended behaviour: an unconfigured metric is still collected and charted, it simply never raises an alert.
If you want a metric to alert, you must give it at least one threshold, either by entering a fixed value or by enabling its auto-threshold.
Because an empty threshold is skipped, a value of 0 is also treated as “not configured”. You cannot express “alert as soon as the value is above zero” by entering 0 in the critical field. To alert on any non-zero value, enter a small positive number instead.
Know the unit before entering a value
Thresholds are compared in the metric’s own unit, which is not always a percentage. Entering a percentage where megabytes are expected is the most common cause of a healthy component being reported as critical.
| Metric | Scope | Unit |
|---|---|---|
| cpu, cpu user | system, process | percent |
| memory | system | percent |
| memory (RSS), swap | process | megabytes |
| iostat read, iostat write | process | milliseconds |
| num threads, num FDs | process | count |
| traffic in, traffic out | process | kbits/s |
| filesystem, filesystem inodes | system | percent |
| TCP sockets | system | count |
| swap-in rate, swap-out rate | system | bytes/s |
| disk queue | system | queue length |
| disk util | system | percent |
| disk read rate, disk write rate | system | MB/s |
| net bytes rate | system | bytes/s |
| net TX drops | system | drops/s |
Note:
Process memory is compared in megabytes, not as a percentage of system memory. A major threshold of 3 on process memory means “3 MB”, so every real process on the host will breach it immediately.
There is no sensible universal value for process memory or process swap. Leave them empty and let the auto-threshold learn each process’s own normal range.
Traffic thresholds are different
Process traffic thresholds do not use three levels. Instead you configure, per process and per direction:
- an operator,
>or< - a threshold value in kbits/s
- the severity to report when the threshold is crossed
- an averaging window, the number of consecutive samples averaged before the comparison
The < operator is useful to detect a process that has gone silent: traffic in < 0.5 kbits/s → major alerts when a service stops receiving traffic.
The averaging window smooths brief spikes. With a window of 4, the current sample is averaged with the previous 3 before being compared to the threshold, so a single spike does not raise an alert while a sustained breach still does. Raw values are always stored unchanged, only the comparison uses the average.
Where thresholds are configured
Thresholds for system and process metrics are set on the Sentinel Agent, in the component details page, under the system and process threshold tables. Each metric row offers the three fixed values and an auto-threshold toggle.
The values used at any moment are shown as pills at the top of the system and process report panels, so you can always compare what is being applied against the chart below it.
See also
- Auto-thresholds — let the Sentinel Agent learn each metric’s normal range
- Tuning thresholds — diagnose a metric reported with the wrong status
- System, process, log & folder monitoring — where the metrics these thresholds evaluate come from
- Sentinel Agent Configuration — factors and lookback windows that drive auto-thresholds