Diagnostics
This page describes how to collect diagnostic information, enable debug logging, and trace issues in NetXMS.
Server Log Files
Log File Location
The server log file location is configured by the LogFile parameter in netxmsd.conf:
LogFile = /var/log/netxmsd.log
The default is /var/log/netxmsd.log on UNIX-like systems and C:\netxmsd.log on Windows.
Special values are also accepted: {syslog} (system syslog on UNIX, Event Log on Windows), {systemd} (systemd journal), and {stdout} (standard output).
Log Rotation
Log rotation is controlled by these configuration parameters:
| Parameter | Default | Description |
|---|---|---|
|
2 |
0 = no rotation, 1 = daily rotation, 2 = rotate by size |
|
16777216 |
Maximum log file size in bytes before rotation (when mode = 2) |
|
4 |
Number of rotated log files to keep |
Debug Logging
Debug Levels
NetXMS uses debug levels from 0 (no debug output) to 9 (maximum verbosity). Each level includes all output from lower levels. The level is a verbosity threshold — there is no fixed meaning attached to individual levels, and it is independent of message severity (errors and warnings are always logged regardless of debug level).
| High debug levels generate very large amounts of log output and can affect server performance. Use them only for targeted troubleshooting and remember to restore level 0 when done. |
Enabling Debug Logging
There are three ways to enable debug output.
The debug console used below is available locally via nxadm -i or in the management client under Tools > Server debug console (see Debug Console).
Configuration file
Set the global debug level in netxmsd.conf:
DebugLevel = 4
Debug levels for specific tags can be set with the DebugTags parameter, a comma-separated list of tag:level entries:
DebugTags = poll.status:7,db.query:5
These settings require a server restart; DebugLevel affects all subsystems.
Debug console (runtime)
Change the debug level without restarting the server:
debug 4
This takes effect immediately but does not persist across restarts.
Debug tags (targeted diagnostics)
Debug tags allow you to set different debug levels for specific subsystems without flooding the log with unnecessary output:
debug poll.status 7
debug db.query 5
debug snmp.trap 6
This is the recommended approach for production troubleshooting as it limits the log volume to the relevant subsystem.
Common Debug Tags
| Tag | Description |
|---|---|
|
Status polling |
|
Configuration polling |
|
Network discovery |
|
ICMP polling |
|
Database queries (shows executed SQL) |
|
Database writer queues |
|
SNMP trap processing |
|
SNMP-based discovery |
|
Agent connections |
|
Agent tunnel management |
|
Event processing |
|
Event processing policy rule matching |
|
Event correlation |
|
Data collection configuration |
|
Data collection scheduling |
|
NXSL feature enablement and JSON parsing messages |
|
NXSL script compilation, execution, and errors |
|
HTTP requests from NXSL scripts ( |
|
Encryption and certificate operations |
|
Object synchronization |
|
Custom attributes (rejected structured values that are not a valid JSON object or array) |
|
Client session communication |
Agent Diagnostics
Agent Log Files
Agent log file location is configured in nxagentd.conf:
LogFile = /var/log/nxagentd
The agent supports the same debug levels and rotation settings as the server.
When the agent writes its log to a file, you can read the log from the management console without file system access to the host.
Right-click the node and select to open the last 1 MB of the log in the Agent log view, scrolled to the end, or to save the whole file as <node>-nxagentd.log.
The view shows the log as it was when it was opened; Refresh reads it again.
Only the current log file is read, not the rotated ones.
The menu items are shown for a node with an agent when you have the Download files access right on the node.
The file manager subagent is not needed, but the agent sends its log only to a server listed in its MasterServers (ControlServers is not enough); through a tunnel or a proxy, the address the agent sees must be listed.
If the request fails, the error message says why:
-
Agent log is not written to a file — the agent logs to syslog, the Windows Event Log, the systemd journal, or standard output, or it could not open its log file at startup.
-
Agent access denied — the server is not in the agent’s
MasterServers. -
Command not implemented — the agent is older than 7.1.
Enabling Agent Debug
Set debug level in nxagentd.conf:
DebugLevel = 4
DebugTags = tunnel.*:6
Restart the agent after changing the configuration for the new debug level to take effect.
For a one-off debug run, start the agent in the foreground with debug options on the command line:
nxagentd -D 6
nxagentd -t tunnel.*:6
Lowercase -d means "run as daemon", not debug.
|
Testing Agent Connectivity
Use nxget to test communication with an agent:
nxget <host> Agent.Version
nxget <host> System.CPU.Usage
nxget -I <host>
The -I flag lists all supported metrics from the agent.
Measuring Agent Connection Traffic
To see how much NXCP traffic the server exchanges with an agent:
-
On the node, create a DCI with origin Internal and metric
AgentTraffic.BytesReceivedorAgentTraffic.BytesSent. For the total of all agents, create the DCI on the management server node withServer.AgentTraffic.BytesReceivedorServer.AgentTraffic.BytesSent. -
Check that the data type is Counter 64-bit. Selecting the metric from the list sets it.
-
On the Transformation page, set delta calculation to Average delta per second for a rate, or Simple delta for the bytes per polling interval.
The counters start from zero when the server restarts. See Agent Traffic Metrics for what is counted.
Collecting Diagnostic Information
When reporting issues, collect the following information:
Server Diagnostics
-
Server version:
netxmsd -v -
Server log with debug level 4 or above (reproduce the issue with debug enabled)
-
Server configuration file (
netxmsd.conf) — redact passwords -
Thread pool statistics from the debug console:
show threads -
Database type and version
-
Operating system and version
-
Number of monitored nodes and active DCIs
Network Diagnostics
Verifying Port Connectivity
NetXMS uses the following default ports:
| Port | Protocol | Purpose |
|---|---|---|
4701 |
TCP |
Server to client communication |
4700 |
TCP |
Server to agent communication |
4703 |
TCP |
Agent tunnel connections |
4702 |
TCP |
Inter-server communication (ISC) listener, disabled by default |
4704 |
TCP |
High availability cluster channel between server nodes |
162 |
UDP |
SNMP trap receiver |
514 |
UDP |
Syslog receiver |
Verify connectivity using:
nc -zv <host> <port>
Database Diagnostics
Database Consistency Check
Use nxdbmgr to check and repair database consistency (the server must be stopped):
nxdbmgr check
The check is interactive — when problems are found, it explains them and asks before applying repairs.
| Always back up the database before applying repairs. |
Database Statistics
Monitor database health using the Server.DB.* internal metrics on the management server node, or query the database directly with native tools.
For additional troubleshooting, see Common Issues and Debug Console.