Skip to main content

Introduction to Data Framework

Data Framework is the entry point for managing a complete data governance framework for observability data in ONE, covering ingestion, processing, operations, and data modeling. It helps users quickly integrate metrics, alerts, logs, Trace data, and other data from external systems, components, cloud platforms, Agent, SDK, and other sources into the ONE platform. Through data processing, anomaly data monitoring, and data model management, it supports subsequent governance and enables comprehensive observability for your business systems, allowing you to view each part individually and understand how each part affects the whole.

dataFramework

Prerequisites​

  • The corresponding account must be granted the Data Integration Management permission. Please refer to the Resource Domains Overview and Usage section.
  • Sufficient license entitlement for the intended observability data ingestion is required. Please refer to the License Management section.

Rich Data Integration Capabilities​

The ONE platform provides Standard Integrations and Custom Integrations to meet data integration needs across different scenarios.

Standard Integrations: Integrations provided by the ONE platform with complete out-of-the-box capabilities, including data collection, governance, and monitoring templates.

ingestStandard

Custom Integrations: Used for custom data sources, external monitoring platforms, and similar scenarios. ONE provides best practices for these scenarios, and you can also edit data flows to implement data governance tailored to your own environment.

ingestCustom

Powerful Data Processing Capabilities​

The ONE platform manages the ingestion and processing of each data source through dedicated, isolated tasks. This task-level isolation minimizes cross-impact when ingesting data from multiple sources.

ingestTask

The processing pipeline for each task is highly configurable. The stream processing engine supports 5 categories encompassing 19 types of data processing components and over 9 categories with 60+ data processing functions. This allows for flexible parsing, enrichment, editing, and transformation of your data, ensuring all observability data seamlessly integrates into the ONE platform to achieve full-system observability.

ingestTopo

Intuitive Data Operations Scenarios​

The ONE platform records anomaly data from the last 7 days, enabling rapid awareness of recent abnormal data trends. You can quickly identify which components have the most anomalies, which data source ingestions are experiencing issues, along with the root causes and exception stack traces.

exceptionData

Complete Data Model Development​

Data models maintain how ONE understands observable objects. They define entities, entity relations, metrics, Meta Attributes, and dictionaries in the platform, allowing ingested data to be correctly identified, correlated, and displayed. Learn more in the Data Model section.

CMDB_topo