Anomaly Detection

NetXMS provides AI-powered anomaly detection that automatically generates detection profiles from historical data and applies them in real time to identify abnormal metric behavior.

Overview

Traditional static thresholds require administrators to manually define acceptable ranges for every metric. AI-powered anomaly detection automates this by analyzing historical data patterns and generating a profile that captures normal behavior, including seasonal variations and expected rates of change.

When anomaly detection is enabled on a DCI, the system:

Collects 30 days of historical values (minimum 1,000 data points required).
Computes statistical baselines: min, max, mean, standard deviation, percentiles (p1, p5, p50, p95, p99), hourly baselines, and weekday patterns.
Sends the statistics to an AI/LLM provider which recommends detection parameters.
Stores the resulting anomaly profile as JSON in the DCI configuration.
Applies six detection methods against every new value collected.

Prerequisites

An AI/LLM provider must be configured in netxmsd.conf with at least the default slot. A provider with the analytical slot is preferred for profile generation. See AI Assistant Configuration for setup instructions.
The DCI must have sufficient historical data (a minimum of 1,000 data points; 30 days are analyzed).

Enabling Anomaly Detection

To enable AI anomaly detection on a DCI:

Open the DCI properties.
On the Thresholds page, enable Detect anomalies using AI.
Save the DCI configuration.

The profile is generated by the daily System.RegenerateAnomalyProfiles scheduled task (or on demand from NXSL via the DCI.generateAnomalyProfile() method). Profile generation runs asynchronously and does not block data collection.

Two additional anomaly mechanisms are available on the same page, independent of AI profiles:

Detect anomalies using isolation forest algorithm — a built-in statistical detector that needs no AI provider
The Anomaly threshold function — lets a regular threshold activate on detected anomalies

Detection Methods

The anomaly profile contains parameters for six independent detection methods. Each method can be individually enabled or disabled by the AI during profile generation:

Method	Description
Hard Bounds	Checks if the value falls outside absolute minimum and maximum physical limits. Catches values that are impossible regardless of time or season.
Seasonal Deviation	Compares the current value against the expected hourly baseline using a z-score with a sensitivity multiplier. Weekday factors scale the detection threshold for different days of the week.
Rate of Change	Checks the rate of value change per minute against a maximum threshold. Catches sudden spikes or drops that occur faster than normal.
Sustained High	Detects when a value remains above a threshold for a specified duration in minutes. Catches prolonged elevated states that may indicate a growing problem.
Sustained Low	Detects when a value remains below a threshold for a specified duration in minutes. Evaluated at runtime, but AI-generated profiles currently do not include parameters for this method — it is only active if added to the profile JSON manually.
Sudden Drop	Detects a percentage drop using an exponential moving average (EMA). Catches sharp declines relative to recent trending values.

Method

Description

Hard Bounds

Checks if the value falls outside absolute minimum and maximum physical limits. Catches values that are impossible regardless of time or season.

Seasonal Deviation

Compares the current value against the expected hourly baseline using a z-score with a sensitivity multiplier. Weekday factors scale the detection threshold for different days of the week.

Rate of Change

Checks the rate of value change per minute against a maximum threshold. Catches sudden spikes or drops that occur faster than normal.

Sustained High

Detects when a value remains above a threshold for a specified duration in minutes. Catches prolonged elevated states that may indicate a growing problem.

Sustained Low

Detects when a value remains below a threshold for a specified duration in minutes. Evaluated at runtime, but AI-generated profiles currently do not include parameters for this method — it is only active if added to the profile JSON manually.

Sudden Drop

Detects a percentage drop using an exponential moving average (EMA). Catches sharp declines relative to recent trending values.

Profile Regeneration

Anomaly profiles are automatically regenerated to adapt to changing system behavior:

The scheduled task System.RegenerateAnomalyProfiles runs daily at 12:00 to find DCIs with stale profiles.
A cooldown of 24 hours applies between regenerations for the same DCI.
If generation fails, up to 3 retries are attempted with 1-hour spacing.

Traditional Approaches

For metrics where AI anomaly detection is not suitable or an AI provider is not available, the following built-in approaches can be used:

Script thresholds — NXSL scripts that implement custom threshold logic with access to historical values.
DCI transformation scripts — normalize metric values before threshold evaluation.
Multiple thresholds per DCI — set warning and critical levels.

For information on configuring thresholds, see Thresholds.

Troubleshooting

Use the ai.anomaly debug tag to enable detailed logging of anomaly profile generation:

DebugTags = ai.anomaly:6

Common issues:

Profile not generated — verify the DCI has sufficient historical data (30 days, 1,000+ data points) and that an AI provider with the analytical or default slot is configured.
False positives — the profile may need regeneration after significant infrastructure changes. Wait for the automatic regeneration cycle or manually trigger profile update.
No AI provider available — use traditional script thresholds as a fallback. See Thresholds for details.