Data Flow

Understanding how data moves through the Mugnsoft platform is essential for troubleshooting, capacity planning, and integration design.

Overview

Data in Mugnsoft flows through three main paths:

Management flow – configuration and control commands from Webserver to components
Monitoring flow – check results from Monitors to Webserver and Integrator
Integration flow – transformed data from Integrator to third-party systems

1. Management Flow (Webserver to Components)

The Webserver is the central control plane. It manages component lifecycle and pushes configuration:

Key operations:

Push monitor definitions (create, update, delete monitors)
Push component settings (log level, retention, thresholds)
Trigger on-demand monitor execution
Kill running monitor processes
Reload TLS certificates
Backup/restore KV stores

All management commands are authenticated with JWT tokens and transported over mTLS.

2. Monitoring Flow (Data Collection)

Monitor to Webserver (Pull Model)

The Webserver pulls data from Monitor probes. Monitors do not push results to the Webserver on their own.

sequenceDiagram participant MON as Monitor Probe participant DB as embedded key-value store (local) participant WS as Webserver MON->>MON: Execute check (cron schedule) MON->>DB: Store result WS->>MON: GET /monitor/monitorsStatus (periodic poll) MON-->>WS: JSON status data WS->>MON: GET /monitor/timelineGraph (on-demand) MON-->>WS: Historical data

Data stored per monitor execution:

Bucket	Content
`STATUS`	Current status (OK, MINOR, MAJOR, CRITICAL, ERROR, DOWN)
`STATUS_PERF`	Performance metrics (response time, DNS lookup, TCP connect, TLS handshake, etc.)
`STATUS_TRANS`	Transaction timing data (for multi-step EUM scenarios)
`STATUS_CHANGE`	Status change history with timestamps
`STATUS_ERROR`	Error messages and details
`STATUS_HISTALL`	Complete historical data
`STATUS_PATTERN`	Pattern matching results

Monitor to Integrator (Push Model)

When a Monitor is linked to an Integrator, it pushes results directly after each execution:

graph LR MON["Monitor Probe"] -->|"POST /integrator/data2integrator"| INT["Integrator"] style MON fill:#5cb85c,stroke:#3d8b3d,color:#fff style INT fill:#d9984f,stroke:#b07830,color:#fff

The Monitor sends a JSON payload containing:

Monitor name, type, and probe identity
Current status and status integer (0=OK through 6=EXCEPTION)
Performance values (response time, DNS, TCP, TLS, etc.)
Transaction details (for EUM monitors)
Notification channel configuration (email, Slack, Teams, PagerDuty)
Alert state (status change indicator, alerting enabled/disabled)

Sentinel Agent to Integrator

The Sentinel Agent pushes system and process metrics to the Integrator:

graph LR DA["Sentinel Agent"] -->|"POST /integrator/data2integratorDisco"| INT["Integrator"] style DA fill:#9b59b6,stroke:#7d3c98,color:#fff style INT fill:#d9984f,stroke:#b07830,color:#fff

Metrics include per-process CPU, memory, I/O, network traffic, and system-wide metrics with threshold breach information.

3. Integration Flow (Outbound)

The Integrator receives data from Monitors and Sentinel Agents, then fans out to multiple destinations:

Push Integrations

Target	Protocol	Format
InfluxDB v1/v2/v3	HTTP Write API	Line protocol
Splunk	HTTP Event Collector (HEC)	JSON events
Elasticsearch	Bulk API (`_bulk`)	JSON documents
Kafka	TCP (with SASL/TLS)	Line protocol
Canopsis	HTTP API	JSON events

Pull Integrations

Target	Mechanism
Zabbix	Queries Integrator REST API (`/integrator/{bucket}/allV`) to read cached data

Data Export

Format	Description
CSV	Local file export per monitor
JSON	Local file export per monitor

4. Alerting Flow

Alerts can be triggered at two levels:

Component-Level Alerting (Monitor / Sentinel Agent)

Each Monitor and Sentinel Agent can send alerts directly, without needing an Integrator:

graph LR MON["Monitor /
Sentinel Agent"] EMAIL["📧 Email"] SLACK["💬 Slack"] TEAMS["💬 Teams"] PD["🔔 PagerDuty"] SCRIPT["⚙️ Custom Script"] MON --> EMAIL MON --> SLACK MON --> TEAMS MON --> PD MON --> SCRIPT style MON fill:#5cb85c,stroke:#3d8b3d,color:#fff

Integrator-Level Alerting

When monitors are linked to an Integrator, the Integrator handles alerting centrally:

graph LR MON["Monitor"] -->|"data"| INT["Integrator"] --> ALERT["Email / Slack / Teams /
PagerDuty / Script"] style INT fill:#d9984f,stroke:#b07830,color:#fff

Alert triggers:

On failure (emailOnF, slackOnF, etc.) – send alert when status is non-OK
On status change (emailOnSC, slackOnSC, etc.) – send alert only when status transitions
Notify after – wait for N consecutive failures before alerting (avoids flapping)
Notify for – continue alerting for N cycles after initial breach

5. Authentication Flow

Component Registration

sequenceDiagram participant COMP as New Component participant WS as Webserver COMP->>WS: POST /selfRegister (config + RSA public key) WS-->>COMP: 200 OK COMP->>WS: POST /selfRegisterCert (TLS certificate) WS-->>COMP: 200 OK

JWT Token Lifecycle

sequenceDiagram participant C as Component / User participant WS as Webserver C->>WS: POST /api/auth (user) or POST /loginComponent WS-->>C: JWT token (15min user / 60d component) Note over C,WS: Token included in all requests:
Authorization: Bearer <token> C->>WS: GET /refresh_token (before expiry) WS-->>C: New JWT token

Timestamp and Data Formats

Item	Format
Timestamps	Unix milliseconds (epoch in ms)
Status codes	0=OK, 1=MINOR, 2=MAJOR, 3=CRITICAL, 4=CONFIG, 5=TIMEOUT, 6=EXCEPTION
Status colors	Green (#5cb85c), Yellow (#D5D94F), Orange (#D9984F), Red (#d9534f), Blue (#428bca), Gray (#E1DFDF)
Data retention	Configurable per component (default: 336 hours / 14 days)
Backup interval	Configurable (default: 24 hours for Monitor, 8 hours for Webserver)

Translations

fr: Flux de donnees

Platform Overview

Security Model