Monitor

The Monitor is a distributed monitoring probe that executes checks, collects performance data, and reports results. Deploy multiple instances across your infrastructure for distributed monitoring.

Overview

Service Management

CLI Commands

# Install as service and register with the Webserver
monitor install <webserver_ip>:<port>

# Re-register with the Webserver (e.g., after IP change)
monitor register <webserver_ip>:<port>

# Migrate to a new Webserver (regenerates the certificate, re-registers)
monitor migrate <webserver_ip>:<port> <jwt_token>

# Start / Stop / Restart the service
monitor start
monitor stop
monitor restart

# Run in foreground (Docker or debugging)
monitor run

# Remove the service
monitor uninstall

# Display help
monitor help

During install, the Monitor:

1. Creates the directory structure
2. Generates RSA key pair and TLS certificates
3. Sends a self-registration request to the Webserver
4. Uploads its TLS certificate for mTLS verification

Configuration

The monitor.json file is located in the same directory as the executable:

{
  "name": "probe1",
  "port": "8051",
  "webserver": "192.168.1.100:8050",
  "integratorEndPoint": "",
  "location": "Paris",
  "description": "Production monitoring probe",
  "logLevel": "info",
  "concurrencyLimit": "100",
  "dataRetention": "336",
  "backupInterval": "24",
  "nbDaysBackup": "336",
  "autoThreshold": "true"
}

Core Settings

Browser Configuration (for Web UI / EUM Monitors)

Auto-Thresholding

The Monitor can automatically calculate performance thresholds based on historical data using statistical analysis (standard deviation multipliers):

A nightly job (02:00) recomputes thresholds as mean + (multiplier * stddev) over successful executions of enabled monitors only. Ping monitors also get jitter thresholds (jitterThreshCri/Maj/Min). See Monitor Configuration → Auto-Thresholding for the full formula and tuning guidance.

Vault Integration

Log Rotation

Directory Structure

<install_dir>/
├── monitor(.exe)            # Executable
├── monitor.json             # Service configuration
├── config/
│   ├── sec/                 # RSA keys for JWT
│   └── ssl/                 # TLS certificates for mTLS
├── data/                    # Monitor definition files (JSON)
├── dbs/                     # embedded key-value databases (one per type)
│   ├── exec.db              # Web UI / EUM monitors
│   ├── url.db               # HTTP/HTTPS URL monitors
│   ├── api.db               # REST API monitors
│   ├── tcp.db               # TCP port monitors
│   ├── udp.db               # UDP port monitors
│   ├── ping.db              # ICMP ping monitors
│   ├── nslookup.db          # DNS lookup monitors
│   ├── db.db                # Database query monitors
│   ├── sys.db               # System metrics monitors
│   ├── snmp.db              # SNMP monitors
│   ├── wmi.db               # WMI (WQL) monitors
│   ├── websocket.db         # WebSocket (ws/wss) monitors
│   ├── workload.db          # Kubernetes / Docker workload monitors
│   ├── container_host.db    # Container hosts (clusters, Docker hosts)
│   ├── chart.db             # Chart/performance data
│   ├── monitor.db           # General monitor data
│   ├── metrics.db           # Probe self metrics (CPU, memory)
│   ├── host.db              # Host inventory data
│   ├── device.db            # Device inventory data
│   └── backup/              # Automated backups
├── log/                     # Rotated log files
├── scripts/                 # Custom scripts for monitors
├── actions/                 # Action scripts (notifications, remediation)
├── exec/                    # Platform-specific executables (webdrivers)
└── export/                  # Data exports

Monitor Types

1. Web UI / EUM (End User Monitoring)

Database: exec.db

Executes multi-step user journeys using Selenium WebDriver. Supports Chrome, Firefox, and Edge browsers.

Capabilities:

• Multi-step transaction recording (via MNS IDE or Selenium IDE)
• Screenshot capture on each step
• Video recording of the entire execution
• HAR (HTTP Archive) file capture
• Individual transaction timing with thresholds
• TOTP (Time-based One-Time Password) support for 2FA (RFC 6238 — 6 digits / 30 s / SHA-1)
• Proxy configuration
• Visual similarity checking

See the dedicated EUM Monitor page for the full specification (TOTP/Okta compatibility, key-vault integration, script functions).

MNS IDE Script Functions:

mnsStartTransaction("Login Page", 30, 5000, 3000); // name, timeout, critical_ms, major_ms
driver.findElement(By.id("username")).sendKeys("user");
driver.findElement(By.id("password")).sendKeys("pass");
driver.findElement(By.id("submit")).click();

2. HTTP/HTTPS URL

Database: url.db

Simple HTTP GET checks with detailed timing metrics.

Metrics collected:

• DNS lookup time
• TCP connection time
• TLS handshake time (with TLS version detection)
• Server processing time
• Content transfer time
• Total response time
• HTTP status code
• Content length
• SSL certificate expiry date

Additional features:

• Response body evaluation with named pattern groups (content match or value extraction with thresholds)
• Page-size check against an expected body size
• Status code validation
• Certificate expiry checking and alerting
• Basic auth, client certificates, and proxy support (HTTP/HTTPS, authenticated, or PAC URL)

3. REST API

Database: api.db

Full API endpoint monitoring with custom HTTP methods, headers, and body.

Supported methods: GET, POST, PUT, DELETE

Features:

• Custom request headers
• Request body configuration
• Response body evaluation with named pattern groups — each extracts one value and grades it against its own Minor / Major / Critical thresholds
• Basic auth and client certificates for mutual-TLS endpoints
• Response time measurement
• Status code validation

4. TCP / UDP Port

Databases: tcp.db, udp.db

Port connectivity checks. The two protocols are not symmetrical, because UDP has no handshake to succeed.

TCP metrics:

• Connection success/failure
• Connection time
• Destination IP resolution

UDP runs in one of two modes, decided by whether a payload is configured:

In reachability mode only an ICMP port unreachable proves a failure: silence is indistinguishable from a healthy silent service. The probe owns the behaviour (udp_check.go, pkg/udp); the Webserver validates the payload, the expectation and the thresholds at save time. See Monitor types, UDP.

5. ICMP Ping

Database: ping.db

ICMP Echo Request/Reply for network reachability testing.

Metrics:

• Minimum / Maximum / Average latency
• Jitter (standard deviation)
• Packets sent / received / lost
• Packet loss rate

6. DNS Lookup (Nslookup)

Database: nslookup.db

DNS resolution monitoring.

Metrics:

• DNS lookup time
• Response validation

7. SNMP

Database: snmp.db

SNMP polling for network device monitoring.

Features:

• SNMP v1/v2c/v3 support
• OID queries with MIB expressions
• Delta calculations between polls
• Expression evaluation on collected values

8. WMI

Database: wmi.db

WQL query polling for Windows hosts, in a namespace (default root\CIMV2).

Features:

• Value graded by operator (number >, number <) or regular expression
• Credential configurations and vault-backed secrets
• Live query verification, class browser and reusable query templates on WMI devices

9. Database Query

Database: db.db

Database connectivity, query execution and result-set monitoring.

Supported databases:

• MySQL
• Microsoft SQL Server (MSSQL)
• PostgreSQL
• Oracle

Metrics:

• Connection time
• Query execution time
• Engine-side timings (lock time, CPU / worker time, wait time — engine dependent)
• Result-set grading via column checks: pick a value by column name, by position (#2) or the row count (#rows), then grade it with an operator and Minor / Major / Critical thresholds

A query returning no rows, a NULL cell or a row past the end of the result is a CRITICAL verdict; an unresolvable or ambiguous column selector is an ERROR (the check is misconfigured). Result sets are drained into a capped projection — 64 columns, 512 characters per cell, 100 retained rows on a scheduled run — so a wide SELECT * cannot swamp the probe.

10. System Metrics

Database: sys.db

Self-monitoring of the probe host.

Metrics:

• CPU usage percentage
• Memory usage
• Disk usage

11. WebSocket

Database: websocket.db

Monitors ws:// and wss:// endpoints, from a plain connectivity check up to a full real-time session: authenticate, send messages, validate what comes back and measure ping/pong latency.

Each scheduled run is one complete session — the probe opens the connection, does its work, optionally holds it open for a configured window, then closes. Counters such as reconnects, connection duration and message rate therefore describe that single check window, not an uptime since boot.

Authentication: none, Basic, Bearer token, API key header, custom headers, cookies.

Connection options: TLS verification, client certificate, SNI override, proxy, subprotocols (Sec-WebSocket-Protocol), permessage-deflate compression, and redirect following during the HTTP upgrade.

Outgoing messages can be sent immediately on connect, after N seconds, or periodically, as plain text, JSON or binary — typically to authenticate, subscribe to a channel or request a heartbeat.

Validation rules are applied to incoming messages: exact text, contains, regular expression, valid JSON, or a JSONPath expression compared with eq / ne / contains / exists / gt / lt / regex. Every non-optional rule must be satisfied before the Receive within deadline below.

Timing rests on four independent fields. Confusing them is the usual cause of a monitor that either alerts too slowly or reports empty traffic counters:

The hold session window sits on top of the connect timeout rather than inside it, so lengthening it never delays a down alert — the handshake still has to finish within the connect timeout.

Without a hold session the run closes as soon as its criteria are met, which is right after the upgrade when there are no expected messages and no pings. That is enough for a pure connectivity check, but it leaves the Traffic panel near zero and makes the incoming and outgoing message rates meaningless, since they are divided by a session that lasted a few milliseconds. Set a window whenever those metrics matter, or whenever you alert on a message-rate floor or ceiling.

#Tries repeats the whole session, not just the connect: at the default of 2 tries, a 30-second window costs up to 60 seconds of wall clock on a failing run. The probe therefore clamps the window so that tries x (connect + window) still fits inside the schedule interval, leaving a 10-second margin — and the form warns when the value you typed is going to be clamped.

A run is healthy when the TCP connection succeeds, the TLS handshake succeeds (for wss://), the HTTP upgrade returns 101 Switching Protocols, authentication is accepted, the expected messages arrive and ping/pong latency stays within its threshold.

Metrics, grouped one panel per family in the report:

Alerting can be raised on connection failure, upgrade failure, authentication failure, an expected message not received in time, ping/pong latency above a threshold, reconnect count above a threshold, an unexpected close, a failed JSON validation, or an incoming message rate outside its floor/ceiling.

12. Workload (Kubernetes / Docker)

Databases: workload.db (the monitors), container_host.db (the devices)

Checks one Kubernetes workload, or one Docker container or compose service, through the platform API — the Kubernetes apiserver or the Docker Engine API — with no agent in the cluster. Each workload monitor is a child of a container host device and inherits its connection settings, so it runs without resolving its parent first.

Kubernetes: reads the workload, its pods (through the workload’s own selector) and metrics.k8s.io when metrics-server is installed. Authentication by ServiceAccount token, token file, client certificate, kubeconfig (token or certificate users only) or in-cluster credentials. Lists are paged, and responses are capped at 16 MiB.

Docker: finds containers by name or by the compose project / service labels, then inspects and samples them in parallel. Local socket on Linux probes; tcp:// or https:// endpoints (with a client certificate) on any probe. docker compose run one-off containers are ignored. A restart clears a container’s state, so OOM kills and failing exits since the previous check are read from the daemon’s /events log as well.

Grading: ready replicas against desired (zero ready is CRITICAL); hard faults — CrashLoopBackOff, ImagePullBackOff, OOMKilled, Evicted, unschedulable, unhealthy, failed job run are CRITICAL, a stalled rollout or a replica failure is MAJOR; and optional restart-delta, CPU, memory and throttling ladders. An API that cannot be reached or refuses the credentials grades ERROR. CronJobs and Jobs are graded on their runs, never on replicas.

Metrics: seven groups — Availability (A), Restarts & faults (R), CPU (C), Memory (M), Network & IO (N), Pod phases (P), Check errors (E). Every key is emitted on every check, zero-filled where the runtime has no equivalent.

Container host status (GET /containerHost/checkDevices) reports four signals computed on request: endpoint reachability, API health (/readyz or /_ping), the worst node condition (Kubernetes only) and the worst workload status. It is not scheduled and raises no alert of its own.

See Container monitoring for setup, grading and limits.

Scheduling

Each monitor type has its own independent cron scheduler (using robfig/cron/v3). This isolation ensures that a slow monitor type cannot block other types.

Predefined Schedules

Concurrency Control

A global semaphore limits the number of concurrent monitor executions (configurable via concurrencyLimit, default: 100). Each execution acquires a slot before starting and releases it when complete, preventing resource exhaustion on the probe host.

Alerting

The Monitor can send alerts directly without needing an Integrator.

Notification Channels

Alert Triggers

Custom Actions

Action scripts can be placed in the actions/ directory and configured per monitor. They execute automatically on failure or status change with a configurable timeout.

REST API Endpoints

Authentication

Monitor Management

Data Retrieval

Type-Specific Creation

Live Test

The Webserver calls this endpoint when a user clicks Test on a monitor form. :type is the monitor type (url, api, tcp, udp, ping, nslookup, db, snmp, wmi), the body carries the in-form monitor definition, and wantJSON: "true" selects the structured report (kind: "monitorTestResult") over the legacy one-line text reply. Nothing is stored: the run writes no history, changes no status and raises no alert. See Testing a monitor.

Configuration

Data Storage

Each monitor type stores results in its own embedded key-value database with multiple buckets per monitor:

All timestamps are stored as Unix milliseconds (epoch in ms).

Data is automatically purged based on the dataRetention setting (default: 336 hours / 14 days).

See also

• Monitor types — task-oriented catalogue of every type, with URL, API and Database in depth
• Testing a monitor — run any check live from its form before saving
• Monitor Configuration — complete settings reference
• Monitor HTTP API — full API documentation
• Monitor Operations — UI guide for managing monitors
• Platform Overview — how the Monitor fits into the architecture
• Exception Running a Monitor — troubleshooting monitor errors

Translations

• fr: Monitor