Skip to main content
Version: 3.7.0

Introduction to Data Ingestion

Supports integrating various observability signals into the ONE platform. Based on data models, the platform integrates, maps, and correlates observability signals centered around the CMDB, enabling you to view your business systems comprehensively—both individually and how they impact the whole.

ingest

Prerequisites​

  • The corresponding account must be granted the Data Integration Management permission. Please refer to the Resource Domains Overview and Usage section.
  • Sufficient license entitlement for the intended observability data ingestion is required. Please refer to the License Management section.

Rich Data Access Methods​

The ONE platform provides three primary methods for data ingestion and comes pre-integrated with support for over 62 common data sources.

ingestCardWall

SmartAgent: Used for collecting host-based observability signals, including metrics, logs, traces, events, profiling data, and network stream data from hosts, services, and processes. [Deploy SmartAgent].

ONE SDK: Used for collecting observability signals from mobile and web applications, such as Application Not Responding (ANR) errors, user sessions, page load performance, and crashes. [Deploy ONE SDK].

External Integrations: Ingests observability signals from external systems or technology components using SmartGate, API, Kafka, databases, and other methods. Examples include Prometheus, Kubernetes, Zabbix, Databases, Message Queues, and cloud providers. [Deploy SmartGate].

Powerful Data Processing Capabilities​

The ONE platform manages the ingestion and processing of each data source through dedicated, isolated tasks. This task-level isolation minimizes cross-impact when ingesting data from multiple sources.

ingestTask

The processing pipeline for each task is highly configurable. The stream processing engine supports 5 categories encompassing 19 types of data processing components and over 9 categories with 60+ data processing functions. This allows for flexible parsing, enrichment, editing, and transformation of your data, ensuring all observability data seamlessly integrates into the ONE platform to achieve full-system observability.

ingestTopo

Intuitive Data Operations Scenarios​

The ONE platform records anomaly data from the last 7 days, enabling rapid awareness of recent abnormal data trends. You can quickly identify which components have the most anomalies, which data source ingestions are experiencing issues, along with the root causes and exception stack traces.

exceptionData