Fanout Drivers

NetXMS can forward collected data to external time-series databases using fanout drivers. This allows you to maintain NetXMS as the primary monitoring and alerting system while sending a copy of collected metrics to external storage for long-term retention, advanced analytics, or integration with other tools.

Overview

Fanout drivers operate at the data collection level. When a DCI value is collected, it is stored in the NetXMS database and simultaneously forwarded to all configured external databases.

Supported Backends

Backend Description

InfluxDB

InfluxData InfluxDB time-series database

ClickHouse

ClickHouse columnar database (available since version 5.2)

InfluxDB Integration

Enabling the Driver

Add the following line to the server configuration file (netxmsd.conf):

PerfDataStorageDriver=influxdb

Driver parameters are specified in the [InfluxDB] section of the same file.

Configuration Parameters

Parameter Description

Hostname

InfluxDB server hostname. Default: localhost.

Port

InfluxDB port number. Default: 8086 for api-v1/api-v2, 8189 for udp.

Protocol

Connection protocol: udp, api-v1, or api-v2. Default: udp.

Database

Database name (v1). Default: netxms.

Bucket

Bucket name (v2).

Organization

Organization name (v2).

Token

Authentication token (v2).

User

User name (v1).

Password

Password (v1).

UseTemplateAttributes

Also read tag/ignore custom attributes from the DCI’s template. Default: false.

EnableUnsignedType

Enable unsigned data type. If disabled, unsigned values are sent as signed. Default: false.

MaxCacheWaitTime

Maximum time in ms before cache is flushed. Default: 30000.

QueueFlushThreshold

Cache will be flushed when reaching this size in bytes. Default: 32768.

Queues

Number of queues for parallel operation (1-32). Default: 1.

QueueSizeLimit

Upper limit on queue size in bytes; data is dropped if exceeded. Default: 4194304.

ValidateValues

Validate values according to DCI data type, drop invalid values. Default: false.

CorrectValues

If both ValidateValues and CorrectValues are true, correct invalid values instead of dropping them. Default: false.

Configuration Example

PerfDataStorageDriver=influxdb

[InfluxDB]
Protocol=api-v2
Organization=netxms
Bucket=netxms
Token=your-influxdb-token

Details of Operation

The measurement name is derived from the DCI metric name (except for SNMP, Modbus, and internal "Dummy" DCIs where description is used); the field key is always the literal value. The name is lowercased and trimmed, everything from the first ( is cut off, space characters are removed, :-.,# characters are replaced with _, and \\ is replaced with /.

Empty DCI values are not sent.

If a custom attribute named ignore_influxdb or pds:ignore (with any value) exists on the node or the DCI’s related object, the data is excluded from export. With UseTemplateAttributes enabled, the same attributes on the DCI’s template are honored as well.

Custom attributes with names starting with tag: or tag_ are used as InfluxDB tags (the prefix is stripped from the name).

ClickHouse Integration

Enabling the Driver

Add the following line to netxmsd.conf:

PerfDataStorageDriver=clickhouse

Driver parameters are specified in the [ClickHouse] section.

Connection Parameters

Parameter Description

Hostname

ClickHouse server hostname or IP. Default: localhost.

Port

ClickHouse HTTP API port. Default: 8123.

UseHTTPS

Use HTTPS instead of HTTP. Default: false.

Database

Target database name. Default: netxms.

Table

Target table name. Default: metrics.

User

ClickHouse username. Empty by default.

Password

ClickHouse password. Empty by default.

Queue Parameters

Parameter Description

Queues

Number of parallel sender threads (1-32). Default: 1.

QueueFlushThreshold

Number of messages to accumulate before flushing. Default: 1000.

MaxCacheWaitTime

Maximum time in ms before flushing partial queue. Default: 30000.

QueueSizeLimit

Maximum messages per queue; excess is dropped. Default: 10000.

Data Processing Parameters

Parameter Description

IgnoreStringMetics

Do not send string-type metrics. Default: true. (The parameter name really is spelled this way — IgnoreStringMetrics is not recognized.)

EnableUnsignedType

Enable unsigned integer types. Default: true.

ValidateValues

Validate values by DCI data type; drop invalid. Default: false.

CorrectValues

Correct invalid values instead of dropping. Default: false.

UseTemplateAttributes

Extract tags from template object custom attributes. Default: true.

Column Configuration

Column names in the target table can be customized. Setting a column name to none disables it (except mandatory columns: timestamp, host, name).

Default columns: timestamp (DateTime), host (String), name (String), ivalue (Int64), fvalue (Float64), instance (String), data_type (String), data_source (String), delta_type (String), tags (Map(String, String)).

User-defined columns can be added using Column.AdditionalData parameter (up to 16).

Configuration Example

PerfDataStorageDriver=clickhouse

[ClickHouse]
Hostname=clickhouse.example.com
Port=8123
Database=netxms
Table=metrics
User=netxms
Password=your-clickhouse-password
Queues=4
QueueFlushThreshold=500
Column.AdditionalData=environment:String
Column.AdditionalData=region:String

Example Table Schema

CREATE DATABASE IF NOT EXISTS netxms;

CREATE TABLE netxms.metrics
(
    timestamp DateTime,
    host String,
    name String,
    ivalue Int64,
    fvalue Float64,
    instance String,
    data_type String,
    data_source String,
    delta_type String,
    tags Map(String, String),
    environment String,
    region String
) ENGINE = MergeTree()
ORDER BY (host, name, timestamp);

Details of Operation

Metric name is taken from DCI metric name, except for SNMP, Modbus, and internal "Dummy" DCIs where description is used.

Tags and column values are extracted from custom attributes on: template objects (if UseTemplateAttributes is true), host (node) objects, and related objects. Custom attributes with names starting with tag: or tag_ are added as tags. Custom attributes starting with column: populate corresponding user-defined columns.

If a custom attribute named pds:ignore or ch:ignore exists on any of these objects, the metric is not sent.

Driver statistics are available on the management server node via the internal metric Server.PDS.DriverStat(ClickHouse, <metric>), where <metric> is one of senders, overflowQueues, messageDrops, queueSize.

Troubleshooting

No Data in External Database

  • Verify PerfDataStorageDriver is set in netxmsd.conf and the server was restarted

  • Check server logs for driver initialization and connection errors

  • Verify network connectivity from NetXMS server to the external database

  • Check authentication credentials

  • Verify the target database/table exists

Data Gaps

  • Check if the queue was full during the gap period (data is dropped when QueueSizeLimit is exceeded)

  • Verify the external database was accessible during the time period

  • Check server logs for write failures or timeouts

Next Steps