Server Log Integration

The platform provides SmartAgent-based log collection capabilities, supporting mainstream host devices such as Linux, Windows, and ARM, to meet the log collection requirements of different host types.

Prerequisites

If SmartAgent is not installed, you need to install it as prompted.

Getting Started

Navigate to the Log Configuration > Add Log page and switch to the Server-Side Logs tab.

Once the SmartAgent Log Collection toggle is enabled, the SmartAgent gains the ability to automatically discover logs. Paths not discovered automatically can be added via Custom Source Paths . Paths that do not require collection can be specified using the Blacklist .

In the Select Collection Paths section, select the host paths from which logs need to be collected. The system automatically updates the list of available paths every 60 seconds. You can set a collection strategy (e.g., "Collect upon selection") and choose the viewing perspective (by Host, Process Group, or Pod). Upon completion, logs will be collected based on the configuration for the selected hosts or process groups.

Viewing Modes

When viewing by Host , expanding a host entry reveals the names of its process groups. Expanding a process group then displays the list of associated log paths.

When viewing by Process Group , expanding a process group shows its log paths. Expanding a path further reveals the specific hosts associated with it.

When viewing by Pod , expanding a pod entry displays Kubernetes path information, along with data about the pod's cluster and namespace.

For log types that are disabled or for hosts where logging is disabled in the configuration, the recognized paths or hosts underneath them should be grayed out and unavailable for selection. Paths or hosts that have never been recognized are not displayed. The specific logic is as follows:

1. When logging for a specific type is turned off, the agent probes will no longer scan for directories of that type.
2. For directories that were previously detected and recorded, if their recorded time window expires, they will no longer be displayed.

Custom Source Paths (Optional)

The platform provides both pre-configured log monitoring types and custom log source paths, with support for enabling or disabling them. Once enabled, the agent will identify log directories of the corresponding type. Note that the platform does not immediately start collecting these logs; it only automatically discovers the paths for these log types. Custom source paths support search, deletion, and editing.

Automatic discovery requires that log files meet the following criteria:

1. The log file must belong to a critical process and be in an open state to be discoverable.
2. The log file must have existed for at least longer than 1 minute.
3. The log must have a supported character encoding. By default, the supported encoding is UTF-8. Other supported types include UTF-8 with BOM, and if the file contains a Byte Order Mark (BOM), UTF-16LE and UTF-16BE.
4. The log file must be at least larger than 0.5 KB in size.
5. The log file must have been updated within the last 7 days.
6. The log file must be located in an actual log folder or its subfolder.

Valid path examples:
c:\log\log_file.txt
c:\logs\NewFolder\log_file.txt

Invalid path example:
c:\log\NewFolder\NewFolder\log_file.txt

Or, the log filename must contain the string log, which must be immediately preceded or followed by a period (.) or underscore (_) character.

Valid filename examples:
c:\NewFolder\abc.log
c:\NewFolder\0865842.log.txt

Invalid filename example: (No _ or . immediately before or after log)
c:\NewFolder\logfile.txt

The custom log source configuration provides the capability to add log sources manually that were not auto-detected.

Source Definition (Custom log source configuration, supporting global configuration based on hosts or environments.)

• Allows defining log sources without a Process Group context (adding a Process Group is optional). A maximum of three Process Groups can be added to each log source. If no Process Group is needed, it must be placed in the virtual group: default process group name. If configured this way, this virtual group will exist on every host.
• Multiple path rules can be added within a single Process Group. Custom paths must be absolute paths; relative paths are not supported.
• Paths must be unique within the same Process Group but can be duplicated across different Process Groups.
• Paths support configurable tags, which are predefined attributes within the platform's definition.
• Wildcards are supported in directories. Custom log sources can include wildcards: # replaces a single digit character, and * replaces any string of characters except a forward slash (/) or backslash (). Note: # can only be used within filenames.

Security Restrictions

• The log path must not be located in any of the following: /etc, /boot, /dev, /bin, /sbin, /usr.
• The log path cannot be /usr, with the exception of /usr/local.
• The log path must not contain .ssh.
• The log path must not have a .pem extension.
• The log path must not be within a directory whose name begins with a . (e.g., /.hidden).
• The filename or path must meet one of the following requirements:
  • The log filename must have a log extension, separated by a ., -, or _ (which may be followed by another extension using the same set of separators).
  • The log path must reside at the first or second level within a log directory.
  • The log path must be at any level within the /var/log directory.
  • The log path must have the filename catalina.out.

Advanced Settings (Optional)

This interface provides advanced configuration for log collection and management, allowing fine-tuned control over the following core parameters:

• Data Reporting Policy : Configure the SmartAgent log reporting frequency and the file size threshold for reporting, balancing data real-time requirements with transfer efficiency.
• Time and Character Constraints : Set the default timezone (e.g., Beijing Time) and define limits for the number of bytes scanned for timestamps, and the number of characters/lines queried for log status, ensuring resource-controlled query operations.
• File Detection Rules : Define the minimum file size threshold for detection, ensuring only log files meeting the specification are effectively recognized.
• Path Aggregation Mechanism : Set rules for aggregating character and numeric fields in paths when they exceed a specified length, optimizing the storage and retrieval efficiency for path-based data.

All configuration changes take effect after clicking Save , enabling precise control over the entire log lifecycle (collection, transmission, querying, storage).