Skip to main content

AI Invocation Monitoring

Configure collection rules for AI application call chains. Only AI applications with probe version ≥ 10.0.0 are supported. Currently supports Python with frameworks: langGraph, langChain, and native OpenAI SDK.

Core Capabilities​

  • Global Toggle: Enable or disable AI invocation data collection globally. All rules stop working when turned off.
  • Custom Collection Rules: Configure collection strategies per service with priority ordering, and fine-grained control over Input/Output and session collection modes.
  • Default Rule Fallback: When no custom rule is matched, the default rule takes effect automatically to ensure no data is missed.
  • Priority Management: Drag to reorder rules. Lower numbers indicate higher priority; the first matched rule takes effect.

Use Cases​

  • Per-service Collection Strategy: Create separate rules for production services (full session) and test services (follow call chain config) to collect data differently per environment.
  • Blocking Sensitive Content: Set a specific service rule to "Do Not Collect" for privacy-sensitive Input/Output, while other services follow the default rule.
  • Instant Coverage for New Services: Newly onboarded services are automatically covered by the default rule, so you can view invocation data without any upfront configuration.

Feature Guide​

Custom Rules List​

The list displays the priority and description of all created rules. You can edit, reorder, enable/disable, or delete each rule.

Creating a Custom Rule​

  1. Click the Create button to open the creation dialog.
  2. Under Scope, select "Service" and choose the target service from the dropdown. Click + to add more conditions and set the logic to "AND" or "OR".
  3. Under Collection Settings, choose whether to collect Input/Output data.
  4. Select Collection Mode: Full Session Mode collects complete session content; Follow Call Chain Config defers to the existing call chain rules.
  5. Enter a Description to identify this rule in the list.
  6. Click Save to finish.

Default Collection Rule​

No configuration needed. The built-in default rule collects Input/Output data in Full Session Mode. It applies automatically when no custom rule is matched.

Notes​

Probe Version Requirement: Collection requires probe version ≥ 9.13.0. AI applications with lower versions will not trigger any collection.

Language & Framework Support: Currently only Python is supported, with frameworks langGraph, langChain, and native OpenAI SDK. Other frameworks are not supported.

Rule Priority: Custom rules are matched in list order; the first match wins. The default rule only applies when no custom rule is matched.

Session Attribute Collection: To collect session attribute mappings, configure the Server-side Request Parameters page first; otherwise those fields cannot be reported.

Enable State: Rules are disabled by default after creation. Manually toggle them on to activate.