Threshold Detection
Threshold detection alert rules are used for continuous monitoring of metric data. When metric values exceed set thresholds, they automatically trigger alerts and notify relevant personnel, enabling proactive operation and maintenance.
Quick Start
Step 1: Enter the Creation Page
Enter Smart Alert → Alert Rules, click New Alert Rule, select Threshold Detection type and enter the configuration page.
Step 2: Configure Detection Rules
- Select Effective Scope (resource domain, default empty, required. Select the resource domain that the current user has permission for)
- In Metric Selection, select the metric to monitor, such as CPU usage, set the aggregation method (such as latest value) and add grouping dimension (such as host); to calculate across multiple metrics, click Add Query to add more metrics, then use Add Expression to combine them with basic arithmetic
- Set Detection Interval (such as 5 minutes, i.e., each detection will query 5 minutes of metric data)
- Configure Trigger Conditions: select the trigger mode (Continuous / Time Window), fill in the corresponding count, and fill in the corresponding thresholds for each alert level (Critical / Error / Warning)
- Configure Data Gap and Data Delay strategies as needed
Step 3: Fill in Alert Content and Save
- Fill in Alert Title (supports variables, such as
Host Name: ${host.customizedName}, IP Address: ${host.ipv4Address} ${metric} too high) - Fill in Notification Content as needed (supports rich text and variables)
- Select Notification Strategy
- Set Effective Time (All Time / Periodic Time / Custom Time)
- Click Save to complete creation
Feature Description
Detection Rules
Basic Configuration
| Field | Required | Description |
|---|---|---|
| Effective Scope | Yes | Select the resource domain to which the alert rule belongs, used to isolate alert configurations of different resource domains |
| Metric Selection | Yes | Supports two methods:Select Query and PQL Query (AI-assisted writing supported); Select Query also supports Metric Calculation, see below |
| Aggregation Method | Yes | Aggregate data within the detection interval, such as latest value, average value, etc. |
| Grouping Dimension | No | Query and calculate separately by specified dimensions (such as host) |
| Detection Interval | Yes | Time window length for each data query, default 5 minutes |
Detection interval supports 1-30min, provides quick options and supports manual input
Metric Calculation
Metric Selection supports adding multiple metrics to calculate together: click Add Query to add multiple metric queries (e.g., a, b), then click Add Expression to combine them into a new calculated metric using simple arithmetic operators +, -, *, / (e.g., b/a*100 to calculate an error rate). The combined result is then used as the detection target for the subsequent detection interval, aggregation, and trigger condition evaluation.
Metric Calculation is also available in the metric selection of Change Detection, AI Adaptive Detection, and AI Forecast Detection, configured the same way as here.
Trigger Conditions
| Field | Required | Description |
|---|---|---|
| Trigger Mode | Yes | SupportsContinuous and Time Window modes: Continuous — triggers only after N consecutive matches, suitable for sustained anomalies and avoiding false alarms from transient jitter; Time Window — triggers once M matches accumulate within a specified time window, suitable for intermittent jitter scenarios. Default is Continuous |
| Continuous Trigger Count / Window Trigger Count | Yes | When Trigger Mode isContinuous, fill in the number of consecutive matches N; when Trigger Mode is Time Window, set both the statistics window length (e.g., N minutes) and the number of matches M within that window, i.e. "M hits within N minutes." Default is 1 time |
| Comparison Method | Yes | Supports operators like>, >=, <, <=, = |
| Critical Threshold | No | Metric value exceeding this threshold triggers a critical level alert |
| Error Threshold | No | Metric value exceeding this threshold triggers an error level alert |
| Warning Threshold | No | Metric value exceeding this threshold triggers a warning level alert |
| Medium Threshold | No | Metric value exceeding this threshold triggers a medium level alert, this level is not displayed by default, can be added |
| Info Threshold | No | Metric value exceeding this threshold triggers an info level alert, this level is not displayed by default, can be added |
| Normal Recovery Count | Yes | When no events are generated for N consecutive detections, the alert status returns to normal, default 3 times |
Advanced Configuration
| Field | Required | Description |
|---|---|---|
| Data Gap | No | After enabling, when no data is reported within the specified time, the metric result is treated as 0 to participate in threshold judgment, preventing missed reports due to collection interruption. DefaultOff or enable data gap to trigger alerts of specified levels |
| Data Delay | No | After enabling, the query time window is moved forward by the specified duration to avoid missed reports due to long data links. DefaultEnabled, offset by 1 minute |
| Aggregation Rule | No | When there are multiple time series under the grouping dimension, define the aggregation granularity. Default aggregation by host, each host independently alerts; aggregation by network area, each network area generates an alert containing multiple hosts |
Alert Content
| Field | Required | Description |
|---|---|---|
| Alert Title | Yes | The title of the alert event, supports variables, such as${host.customizedName}, ${metric}, it is recommended to include object and metric name for quick identification |
| Notification Content | No | The body of the alert notification, supports rich text editing and variable interpolation. |
| Notification Strategy | No | Select the notification channel and recipient configuration after the alert is triggered, if none can clickCreate Notification Strategy to create a new one |
| Labels | No | Tag alert rules for easy filtering and classification management |
It is recommended to use variables instead of fixed text for alert titles to quickly locate problems in the alert list.
When notification content is left blank, the system default template is used, which includes basic information such as alert ID, time, status, and level, meeting the needs of most scenarios.
Status & Effective Time
| Field | Required | Description |
|---|---|---|
| Effective Time | Yes | All Time (7×24 hours) : Always effective; Periodic Time : Set by workday/weekend cycle; Custom Time : Specify specific time periods to be effective |
| Start/Stop Status | Yes | Control whether the alert rule is running. After being closed, the rule pauses detection and does not generate alert events, defaultEnabled |
Common Scenarios
Scenario: Monitoring production server CPU continuous high load Select CPU usage metric, set grouping dimension as host, set critical threshold to 90%, error to 80%, continuous trigger count to 3, to avoid false alarms from short-term peaks.
Scenario: Reduce alert sensitivity during non-working hours Select Custom Time for effective time, configure it to be effective from 9:00-18:00 on workdays, and do not send notifications at other times to reduce night disturbances.
Scenario: Still able to alert when host collection is interrupted Enable Data Gap and treat gap results as 0, cooperating with threshold detection to ensure that alerts can also be triggered when the host Agent is abnormal, rather than being silent.
Scenario: Fault-tolerant triggering for intermittent jitter A metric occasionally exceeds the threshold briefly and recovers, making it hard to match under Continuous mode. Set Trigger Mode to Time Window, configuring "3 hits within 10 minutes" to trigger, balancing sensitivity with noise tolerance.
Scenario: Calculate an error rate from multiple metrics before alerting
In Metric Selection, add request count (a) and error request count (b), then use Add Expression to configure b/a*100 to calculate the error rate, and set a threshold directly on the error rate — no need to pre-compute a dedicated error-rate metric.
Notes
Modifying Detection Interval or Continuous Trigger Count will affect the response delay of alerts: the longer the interval and the more times, the longer the time from the occurrence of an anomaly to receiving a notification. Please set it reasonably according to business tolerance. The same applies when Trigger Mode is set to Time Window — the longer the window and the higher the required hit count, the greater the response delay; set it reasonably according to business tolerance.