This is the documentation for the development version. There are no official binaries for it; build it from the source code in git. Download links on these pages work only after the release. For the current release, see version 7.0.

AI Assistant Concepts

Features marked with EE badge require Enterprise Edition.

This page explains the architecture and key concepts behind the NetXMS AI assistant.

For setup instructions, see Getting Started with the AI Assistant. For task-oriented guides, see AI Assistant How-to Guides. For configuration reference, see AI Assistant Reference.

Architecture Overview

The AI subsystem consists of the following components:

LLM Providers connect the NetXMS server to language models. Each provider is configured with a model, API endpoint, and authentication credentials. Multiple providers can be active simultaneously.

Provider Slots route AI requests to appropriate providers. Different slots (e.g., default, fast, analytical) can point to different providers, allowing you to balance cost, speed, and capability.

Tools are the bridge between the AI assistant and NetXMS data. When the AI assistant needs to access live information (alarms, objects, metrics, logs), it calls registered tools that execute on the server and return results. Well over a hundred tools are available covering alarms, incidents, objects, data collection, logs, SNMP, event processing, and more.

Skills are domain-specific instruction sets that the AI assistant uses on demand. Each skill includes a specialized prompt and a set of related tools. When the AI assistant determines that a skill is needed, it loads the skill into the chat or runs it in a worker.

Workers are child executions, each with its own context, that the AI assistant, an AI operator, or an AI task starts to carry out self-contained parts of a job in parallel.

Background Tasks allow the AI assistant to work autonomously. Tasks can run once or repeatedly with a configurable delay, preserving state between runs through a memento mechanism.

AI Messages are the output channel for background tasks. Tasks can post informational findings, alerts, or approval requests that users review in the management client.

Skills

Skills extend the AI assistant with domain-specific knowledge and capabilities. Each skill includes a detailed prompt with instructions and reference data, plus a set of tools for data access.

Skills and tools are provided by server modules. The aitools module, available in the open-source edition, provides the skills listed below that carry no EE badge. The badged skills come from the aiext and aissh Enterprise Edition modules (see Enterprise Skills); additional skills can be provided by third-party modules. Load a module by adding Module=<name> to netxmsd.conf (before any named section). Individual skills and tools can be disabled server-wide (see How to Disable AI Skills and Tools).
Skill Description

Incident Analysis

Root cause analysis, alarm correlation, incident history, topology context, and assignment recommendations.

Log Analysis

Search and correlate syslog, Windows events, SNMP traps, and NetXMS system events. Includes pattern detection and burst analysis.

Data Collection

Metric creation, threshold management, historical data analysis. Supports SNMP, agent, and script-based data origins.

Event Processing

Event template and processing policy management. Event flow analysis and action configuration.

Inventory

Hardware components, software packages, and network interface inventory. Integration with SNMP, WMI, and agent sources.

Maintenance

Maintenance mode scheduling and management. Alert suppression during maintenance windows.

Object Management

Changing the expected state of network interfaces and the management status of objects.

Alarm Management

Alarm browsing and lifecycle operations: acknowledge, resolve, and terminate alarms.

Device Backup

Access to network device configuration backups collected by the server.

Dashboard Building

Creation and modification of dashboards and dashboard elements.

Traffic Analysis

Network traffic data from traffic observers (ntopng and similar backends): hosts a node communicates with, per-application traffic, top talkers, throughput, drops, and retransmits.

Web Services

Inspection, testing, and creation of web service definitions: retrieve a live response and verify that a data extraction path returns the expected value.

NXSL Scripting EE

Compile and execute NXSL scripts, run and browse script library scripts.

Asset Management EE

Track physical assets, manage properties, link assets to monitored objects, and audit change history.

File Management EE

Remote file operations on nodes with NetXMS agent: read, write, delete, rename, and integrity verification.

Network Topology EE

IP/MAC address location, peer discovery, route tracing, and access to ARP caches, routing tables, forwarding databases, LLDP/CDP neighbors, OSPF/VRRP/STP state, VLANs, and physical device layout.

Scheduler Management EE

Create and manage scheduled tasks for automated maintenance, script execution, and discovery operations.

Agent Operations EE

Host and agent control: reboot or shut down hosts, restart agents, start/stop/restart system services, list and execute agent actions.

Package Deployment EE

List packages, deploy packages to nodes, and monitor or cancel deployment jobs.

SSH Command Execution EE

Secure remote command execution on servers and network devices with intelligent command classification and approval workflow.

The AI assistant loads skills automatically based on the conversation context. You can also request a specific skill explicitly:

Load the log analysis skill and search syslog for authentication
failures in the last 24 hours.

Loading Skills and Running Them in Workers

The AI assistant uses a skill in one of two ways:

  • Loaded into the chat with the load-skill tool: the skill’s prompt and tools stay available for the rest of the conversation. This suits work that needs the skill’s tools for follow-up questions.

  • Run in a worker with the run-workers tool: a separate context carries out one self-contained task with the skill and returns only a concise result, so the intermediate tool calls do not take up the chat’s context window.

Each skill declares a default mode, load or worker, which the AI assistant follows unless the situation clearly calls for the other one, and whether it can run in a worker at all; a skill that cannot is only loaded. The get-available-skills tool reports both as default_mode and supports_worker.

Workers

A worker is a child execution with its own context, started through the run-workers tool by an interactive chat, an AI operator, or an AI task. The tool’s workers argument is an array with one object per worker: task is a complete, self-contained task description (the worker does not see the conversation), and the optional skill names a skill to load into the worker. A worker without a skill has the same general tools as the requester.

  • The workers of one call run in parallel, and the call waits for all of them. A single call can start up to AI.Workers.MaxParallel workers (default: 4) and waits up to AI.Workers.Timeout seconds (default: 600); workers still running then are cancelled.

  • Workers run with the access rights of the requester’s user and inherit its origin, which is recorded with the memory entries and AI messages they create.

  • Workers cannot ask the user questions and cannot start other workers. When a task needs a decision, the worker stops and reports what is missing.

  • Worker requests use the worker provider slot (see Provider Slots).

  • The token usage of finished workers counts toward the requester — for example, toward the daily token budget of an AI operator.

The tool returns one entry per worker with its number (worker), the skill if one was given, the status (completed, failed, or timeout), and either result or error.

Node Health Snapshots

The get-node-health tool collects an operating-system health snapshot with findings about CPU, memory, storage, uptime, and interface errors. It tries the NetXMS agent first, then SNMP HOST-RESOURCES-MIB/IF-MIB, then the last collected DCI values when live data is unavailable. Agent data can also include services, processes, clock offset, and agent problems; those sections are unavailable with the SNMP fallback. The result identifies its data source and explains unavailable live data. The caller needs Read access to the node and the corresponding agent or SNMP read right for a live query.

Object Timeline

The get-object-timeline tool merges the records about a single object into one chronological stream, so the AI assistant can see what happened before and during a problem in a single call. The window is set by time_from and time_to, each an ISO 8601 timestamp, UNIX time, or relative time such as -6h; time_to defaults to now and time_from to 24 hours before time_to. Records belong to these categories:

Category Records

event

Other events from the event log, filtered by min_severity (default: normal).

threshold

DCI threshold activation and deactivation.

reachability

Agent, SNMP, ICMP, SSH, Modbus, and EtherNet/IP unreachable and OK events; node down, up, and unreachable.

maintenance

Maintenance mode entered and left; maintenance journal entries.

software

Packages installed, updated, or removed, as detected by configuration polls (the record time is when the change was detected, not when it happened).

alarm

Alarm creation and state changes.

audit

Configuration changes made to the object.

deployment

Package deployment jobs started from NetXMS.

Server modules can add categories; the result lists every available category. The categories parameter selects categories to include (default: all). Each category is limited separately by limit_per_category (1-200, default: 50); when a category is truncated, its most recent records are kept and its completeFrom value gives the time from which it is complete.

The caller needs Read access to the object. A category the caller has no access to is marked accessDenied: the categories built from the event log (event, threshold, reachability, maintenance, software) require the View event log system access right, alarm requires View alarms access to the object and lists only alarms in alarm categories the caller can view, audit requires View audit log, and deployment requires Manage packages. Alarm history is available only until housekeeping removes terminated alarms, and events configured not to be written to the event log never appear.

Connectivity Checks

The check-connectivity tool runs a live reachability test to a single target and reports DNS resolution, ICMP ping, and TCP connection or protocol check results in one call. Unlike topology-based path analysis, it sends real packets.

The target is an object name or ID, a host name, or an IP address; an object target is tested on its primary IP address. The test runs from the node given in from, which needs a NetXMS agent, or from the NetXMS server when from is omitted. A host name is resolved both at the server and at the source, and the address resolved at the source is tested.

Without port and protocol, only ICMP is tested. The protocol parameter selects tcp (a plain TCP connection, the default when only port is given), ssh, telnet, http, https, or tls; port defaults to the well-known port of the protocol and is mandatory for tcp. Protocol checks beyond a plain TCP connection run only from nodes, through the agent’s netsvc subagent; from the server, only the TCP connection is tested. ICMP checks from a node use the agent’s ping subagent.

The result carries one of these verdicts:

Verdict Meaning

reachable

The target responded.

host-unreachable

Neither ICMP nor the TCP connection got through. ICMP may be filtered, so this verdict is not conclusive when only ICMP was tested.

port-closed

The target answers ICMP, but the TCP connection was refused or timed out.

protocol-error

The port is open, but the TLS handshake or protocol response failed.

dns-failure

The host name could not be resolved.

source-cannot-test

The source agent is not reachable or does not have the ping or netsvc subagent loaded. This verdict says nothing about the target.

A check from a node requires Read agent data access to that node. A check from the server requires Read agent data access to the management server’s node object, and is denied when that object does not exist. An object target also requires Read access to the object.

Enterprise Skills

The following skills are available exclusively in the NetXMS Enterprise Edition. They are loaded from the aiext and aissh server modules and require a valid Enterprise Edition license.

Load the modules in netxmsd.conf:

Module = aiext
Module = aissh
Modules are mandatory by default — without a valid Enterprise Edition license, a server with these modules listed will refuse to start. The aissh module also requires a database schema upgrade (nxdbmgr upgrade) after being added.

In addition to the skills below, the aiext module registers always-available tools for reading agent metrics and tables and listing processes on managed nodes.

NXSL Scripting

The NXSL scripting skill allows the AI assistant to compile and execute NXSL (NetXMS Scripting Language) scripts. It supports both ad-hoc script execution and running scripts from the server script library.

The AI assistant can:

  • Compile NXSL scripts to check for syntax errors before execution.

  • Execute arbitrary NXSL scripts on the server.

  • Run existing scripts from the script library.

  • Browse and inspect scripts stored in the library.

The skill has no write access to the script library — it cannot create, modify, or delete library scripts.

Asset Management

The asset management skill provides the AI assistant with the ability to track physical assets, manage their properties, and link them to monitored objects.

The AI assistant can:

  • List and search assets by name, serial number, MAC address, or any custom property.

  • View complete asset details and individual property values.

  • Set, update, and delete asset properties with schema validation.

  • Link and unlink assets to monitored objects (nodes, sensors, access points, mobile devices, chassis, and racks), or link an asset to a replaceable component of a node (stack member, transceiver, disk, etc.) identified by its serial number from the node’s hardware inventory.

  • Inspect the asset attribute schema including data types, constraints, and enumeration values.

  • Review the full change log for compliance auditing.

File Management

The file management skill enables remote file operations on nodes that have the NetXMS agent installed.

The AI assistant can:

  • List directory contents on remote nodes.

  • Read text files (100 KB by default; larger reads up to 1 MB on request).

  • Write and overwrite text files (up to 1 MB).

  • Delete files and directories.

  • Create directories.

  • Get file metadata including MD5 and SHA256 checksums.

  • Rename files.

All operations are restricted to the RootFolder paths configured for the agent’s filemgr subagent and enforce user access rights.

Network Topology

The network topology skill provides access to the server’s network topology data.

The AI assistant can:

  • Find which device owns a specific IP or MAC address and where it connects to the network.

  • List all directly connected peers (neighbors) of a node.

  • Trace the full network path between any two nodes, showing every hop and intermediate device.

  • Read a node’s ARP cache, routing table, and MAC forwarding database (FDB).

  • Read LLDP/CDP neighbor tables, OSPF, VRRP, and STP state, and VLAN configuration.

  • Inspect the physical device layout (chassis, modules, port positions).

Most tools accept filters and result limits (500 entries by default, up to 5000).

Agent Operations

The agent operations skill gives the AI assistant control over hosts and agents — subject to the approval workflow:

  • Reboot or shut down monitored hosts.

  • Restart NetXMS agents.

  • Start, stop, and restart system services.

  • List and execute agent actions defined in agent configuration.

Package Deployment

The package deployment skill works with the server’s package manager:

  • List packages available on the server.

  • Deploy packages to nodes.

  • Monitor and cancel deployment jobs.

Scheduler Management

The scheduler management skill allows the AI assistant to create, modify, and monitor scheduled tasks.

Supported task types include:

  • Recurring tasks using cron-style schedule expressions (e.g., 0 2 * * * for daily at 2:00 AM).

  • One-time tasks with a specific execution time in ISO format, Unix timestamp, or relative notation (e.g., +30m).

SSH Command Execution

The SSH skill provides secure remote command execution on servers and network devices monitored by NetXMS. It features an intelligent command classification system that categorizes every command before execution.

Command classification:

  • Read-only — diagnostic and information-gathering commands that execute immediately without approval. This includes commands such as ps, df, show interfaces, display version, and most other read-only operations.

  • Write — commands that modify system configuration or state. These require explicit user approval before execution. Examples include service restarts, package management, file modifications, and configuration changes on network devices.

  • Dangerous — destructive commands that are always blocked, in any form. On Linux this includes every rm, rmdir, shred, dd, userdel, reboot, poweroff, and similar commands — not only catastrophic invocations; on network devices, commands such as erase startup-config or request system zeroize.

  • Unknown — commands that match no known pattern are treated as write commands and require approval.

Classification is platform-aware and applies device-specific patterns for Linux, Cisco IOS, Cisco NX-OS, Juniper JunOS, Huawei VRP, MikroTik RouterOS, and Extreme EXOS, with a generic fallback.

Execution modes:

  • Command channel mode — uses the SSH exec channel with full shell processing. Pipes, redirects, environment variables, and command chaining all work correctly.

  • Interactive mode — handles CLI interfaces with prompt detection, pagination control, and command echo removal.

The mode is selected by node capability, not by platform: the server probes SSH exec-channel support during configuration polls and uses interactive mode when the exec channel is unavailable. Interactive mode can also be forced with the skill’s prefer_interactive argument, which overrides the capability-based selection. Multi-line commands always use interactive mode (executed line by line) and fail if the node does not support an interactive channel.

Approval workflow:

When a write command requires approval, the workflow depends on the session type:

  • In an interactive chat session, the AI assistant asks the user for confirmation directly in the chat.

  • In a background task, the AI assistant creates an approval request that the user reviews in the AI Messages view of the AI perspective.

All SSH command executions are logged to an audit table with the timestamp, user, target node, command, classification, approval status, and result.

Tools Through MCP

The mcp server module exposes registered tools through the /mcp endpoint. The server’s built-in tools and script-defined tools remain available there even when no default LLM provider is configured. Module-provided tools, such as those registered by the aitools module, are also available without a provider. Interactive chat, AI memory, tasks, messages, operators, and workers still need a configured provider. For the endpoint’s transport, authentication, and access control, see MCP Endpoint; for the client setup, see How to Connect an MCP Client.

Tools that serve only the AI assistant’s own chats, AI tasks, and AI operators are not exposed: MCP clients do not see them in tools/list, and tools/call rejects them as unknown tools. These are:

  • Skills: get-available-skills.

  • AI tasks: register-ai-task, ai-task-list, delete-ai-task.

  • AI messages: create-ai-message, create-approval-request.

  • AI memory: remember-environment-fact, update-environment-fact, forget-environment-fact, remember-about-user, update-user-memory, forget-user-memory, remember-about-object, update-object-memory, forget-object-memory, recall-memory.

  • AI operator: record-observation, create-check, update-check, delete-check, test-check.

The chat tools load-skill and run-workers exist only inside a chat and are not exposed either. Script-defined nxsl-* tools are exposed.

In tools/list, a tool classified as read-only carries the annotation readOnlyHint: true, and a tool that may delete data or perform an action that cannot be undone carries readOnlyHint: false and destructiveHint: true. Other tools carry no annotations, so MCP clients apply the protocol defaults: the tool may modify its environment, and its changes may be destructive.

Tool Calling

The AI assistant uses tool calling to interact with the NetXMS server in real time. When it determines that it needs live data or wants to perform an action, it calls one or more registered tools. The results are fed back to the LLM for interpretation. One LLM round trip with tool calls is a step; the number of steps per request is limited by the AI.MaxInteractiveSteps and AI.MaxBackgroundSteps server configuration variables (see Server Configuration Variables). In the management client, tool calls appear as progress indicators in the chat, showing what data the AI assistant is accessing.

Script library scripts can be offered to the AI assistant as additional tools. A script with the metadata entry ai_tool = true (see Metadata) is registered as tool nxsl-<script name> (with : replaced by _); its description metadata entry becomes the tool description, and object_context = true adds a mandatory object argument that is bound as the script’s $object. The AI assistant passes arguments as a single string, and the script runs with the access rights of the user on whose behalf the AI assistant works.

Approval Workflow

For potentially impactful actions, the AI assistant can request user approval before proceeding. This applies to both interactive chat sessions and background tasks.

In interactive chat, the AI assistant presents confirmation dialogs with Approve/Reject, Yes/No, or Confirm/Cancel buttons depending on the context. It can also ask multiple-choice questions, which the user answers by selecting one of the offered options. Confirmation requests that are not answered time out after 300 seconds.

In non-interactive sessions (background tasks, AI operators, and event processing), the AI assistant cannot ask questions in the chat; it sends approval request messages to designated users instead. The approval request includes a description of the proposed action and spawns a follow-up task if approved. Workers cannot ask questions either; they report the missing decision back to the requester.

Background Task Execution Model

A background AI task can be executed several times; each execution is a run, shown in the Run column of the AI task execution log. State is preserved between runs, so the assistant can track progress and compare findings over time.

AI Memory

The AI assistant has a persistent memory that survives across chats, AI task runs, and AI operator cycles. Memory entries have one of three scopes:

  • Environment — facts about the monitored environment as a whole (naming conventions, site layout, ownership, known issues, operational policies), shared with all users and background runs. Writing environment entries requires the Manage AI memory system access right.

  • User — notes about a user (preferred level of detail, terminology, corrections), private to that user and used only in interactive chats.

  • Object — notes about a specific object (analysis results, baselines, context); writing requires modify access to the object.

Relevant entries are added to the AI assistant’s prompt up to the size limits set by the AI.Memory.* server configuration variables (see Server Configuration Variables); other entries are retrieved by the AI assistant on request. Entries written by the AI assistant are checked by the prompt injection guard before they are stored. Administrators review, create, edit, and delete entries in the AI Memory view of the Configuration perspective; a locked entry cannot be changed by the AI assistant. Setting or changing a memory entry’s locked state requires the Manage AI memory system access right. AI memory tools return only the current user’s user-scoped entries, even when that user has Manage AI memory; administrative access in the AI Memory view is separate.

AI Operators

AI operators are persistent, named AI assistant instances that watch part of the environment on their own, managed in the Configuration perspective under AI Operators. Each operator runs in cycles with the access rights of its owner account (by default the built-in ai-operator account), on its own model slot and the dedicated AI-OPERATOR thread pool. Between cycles, standing checks — NXSL scripts the server runs on a schedule without the LLM — can wake the operator or record observations. Scripts can read the state of operators with GetAIOperator() and GetAIOperators(). For configuration, standing instructions, standing checks, observations, and access rights, see AI Operators.

NXSL Integration

Four NXSL functions are available for integrating AI capabilities into scripts:

QueryAIAssistant(prompt, context)

Sends a prompt to the AI assistant synchronously and returns the response as a string. The optional context parameter accepts a NetXMS object (e.g., a Node) that provides additional context for the query. Returns NULL on failure.

RegisterAITask(description, prompt)

Registers a background AI task for asynchronous execution. Returns an integer task ID that can be used for tracking. Requires the Manage AI tasks system access right; returns null if the caller does not have it.

GetAIOperator(id)

Returns the AI operator with the given numeric ID, or null if not found. Requires the Manage AI operators system access right.

GetAIOperators()

Returns an array of all configured AI operators. Requires the Manage AI operators system access right; returns an empty array if the caller does not have it.