Data Collection Concepts

Data collection in NetXMS is built around Data Collection Items (DCIs). A DCI defines what metric to collect, how to collect it, where to store it, and what conditions should trigger alerts.

For step-by-step procedures on creating and configuring DCIs, see Managing DCIs. For DCI property details, data types, and options, see DCI Reference.

What is a DCI?

A DCI is the fundamental building block for monitoring in NetXMS. Each DCI represents a single metric or measurement collected from a managed node at regular intervals.

A DCI binds together several pieces of information: a data source (origin), the specific metric to collect, how often to poll, how long to retain the data, and what thresholds to evaluate. The server’s data collection engine uses this configuration to schedule and execute metric collection across all managed nodes.

Data Origins

NetXMS can collect data from many different sources. Each DCI specifies an origin that determines where and how the metric value is obtained.

The most common origins are:

  • NetXMS Agent — a native agent running on the monitored host provides access to hundreds of built-in metrics (CPU, memory, disk, network, processes)

  • SNMP — standard SNMP polling retrieves values from network devices by OID

  • Internal — metrics computed by the NetXMS server itself (object status, response times, server statistics)

  • Push — external scripts or applications send values to the server using nxpush or the API

  • Script — an NXSL script executed on the server computes the value

Additional origins include Web Service (HTTP/HTTPS endpoints), SSH (remote command execution), MQTT (message broker subscriptions), Modbus (industrial equipment registers), EtherNet/IP (CIP attributes), Windows Performance Counters, SM-CLP (hardware management), Network Device Driver, Cloud Connector, OTLP (OpenTelemetry), Traffic Observer, and NETCONF — 17 origins in total.

For a complete list with descriptions, see Data Origins Reference.

How Polling Works

The polling cycle for each DCI follows a consistent pattern:

  1. The server’s data collection thread picks up the DCI at its scheduled polling time

  2. The server contacts the data source (agent, SNMP device, script engine, etc.)

  3. If a transformation script is configured, it is executed

  4. The value is stored in the database

  5. Thresholds are evaluated against the new value

  6. If a threshold condition is met, the corresponding event is generated

Each DCI has its own polling interval (default 60 seconds); after server startup, each DCI’s polling starts with a random offset to avoid synchronized collection bursts.

Data Storage

Collected DCI values are stored in the NetXMS database. The retention time setting controls how long historical data is kept. The server’s housekeeping process periodically removes expired data.

For high-volume deployments, consider:

  • Using TimescaleDB for improved time-series performance

  • Adjusting polling intervals to reduce data volume

  • Configuring shorter retention for high-frequency metrics

  • Using fanout drivers to send data to external time-series databases (InfluxDB, ClickHouse)

Transformations and Delta Calculation

Before storing a collected value, NetXMS can apply two processing steps:

  1. Delta calculation derives the stored value from the current and previous raw readings. Four methods are available: None (keep original value) (no processing), Simple delta (plain difference between the current and previous reading), Average delta per second, and Average delta per minute. The per-second and per-minute methods are useful for monotonically increasing counters (like network traffic bytes) where the meaningful metric is the rate of change, not the absolute value.

  2. Transformation script — an NXSL script that can convert units, extract values from complex strings, compute derived metrics, or filter out invalid readings.

Both steps are optional and configured per DCI. See Configuring Transformation Scripts for practical setup instructions.

Instance Discovery

Some metrics exist in multiple instances on a single node — for example, disk volumes, network interfaces, or database tablespaces. Rather than manually creating a DCI for each instance, NetXMS can automatically discover available instances and create per-instance DCIs.

Instance discovery works by querying the data source for a list of instances (via agent list, SNMP walk, script, etc.) and creating a copy of the template DCI for each discovered instance. When instances appear or disappear, the server updates the DCI set accordingly.

See Configuring Instance Discovery for setup instructions.

Table DCIs

In addition to single-value DCIs, NetXMS supports table DCIs.

Table DCIs collect structured data with multiple columns and rows. They are useful for monitoring process lists, network interface statistics, file system information, and database table sizes. Table DCIs support a subset of data origins: NetXMS Agent, SNMP, Internal, Script, and Traffic Observer.

Agent lists (metrics returning one value per line) are not a DCI type — they are used as input for instance discovery and by object tools.

Agent Caching

Agent caching allows the NetXMS agent to store collected metric data locally when the connection to the server is lost. Once the connection is restored, the agent transmits all buffered data to the server, where transformation scripts and threshold evaluations are applied as if the data had been collected in real time.

A relevance time setting prevents a flood of outdated alerts when a long-disconnected agent reconnects — cached data older than the threshold is stored but does not generate threshold events.

See Configuring Agent Caching for setup instructions.

DCI Aggregation on Clusters

DCIs configured on cluster objects can aggregate values collected from cluster member nodes. Aggregation is configured on the DCI’s Cluster Options property page:

  • Associate with cluster resource — binds the DCI to a specific cluster resource, so data is collected from the node currently owning that resource

  • Aggregate values from cluster nodes — enables aggregation across all member nodes using one of the functions: Total, Average, Min, or Max

  • Run transformation script on aggregated data — applies the transformation script to the aggregated value instead of the individual node values

Threshold Processing and Event Flow

The threshold evaluation happens as part of the DCI polling cycle:

  1. Server collects the DCI value

  2. Transformation script runs (if configured)

  3. Value is stored in the database

  4. Thresholds are evaluated in order

  5. If a threshold condition is met and the threshold is not already active:

    • The threshold transitions to active state

    • The activation event is generated

  6. If the active threshold condition clears:

    • The threshold transitions to inactive state

    • The deactivation event is generated

Events generated by thresholds are processed by the Event Processing Policy (EPP), which determines actions such as alarm creation, notifications, and automated responses.

See Event Processing Policy for details on how events are processed. For threshold configuration details, see Thresholds.

Next Steps