Skip to main content

Azure Metrics Integration

info

To view Azure service metrics in ONE, use SmartGate to ingest Azure Monitor data. Query metrics by resource and check how they change to confirm that the data can support performance analysis and alerts.

The Azure metrics integration uses Azure Resource Graph to discover target resources and the Azure Monitor metric definitions API to obtain metric names, aggregation methods, time grains, and dimensions before retrieving metric data. Select the integration for the target service, then configure the subscription, credentials, and metric scope. This guide uses Blob Storage to demonstrate the complete integration process.

Confirm the Integration Requirements​

Before configuring the integration, confirm that the platform permissions, SmartGate environment, and Azure source data are ready.

ItemRequirement
Platform permissionsYour account has the Data Integration Management permission and an available license entitlement for data ingestion.
SmartGate versionSmartGate 4.3.0 or later is deployed.
Supported system architectureLinux amd64
Platform connectivitySmartGate can communicate with Bonree ONE.
Azure network accessThe SmartGate runtime environment can access Microsoft Entra ID, Azure Resource Manager, and the Azure Monitor metrics endpoint for the target region.
Azure authenticationA Microsoft Entra ID application service principal with read permissions on the target subscription and a client secret are available.
Resource providerMicrosoft.Insights is registered in the target subscription.
Source dataThe target resource already provides the required metrics to Azure Monitor, with actual data points in the selected time range.

For deployment instructions, see Install SmartGate. The authentication and network settings in this guide apply to Azure public cloud. Required endpoints include login.microsoftonline.com, management.azure.com, and the target region's *.metrics.monitor.azure.com endpoint, accessed over HTTPS.

Collection reads platform metrics already provided by Azure Monitor. Shortening the task execution interval does not increase the resolution of source metrics or generate metrics that the resource does not yet provide.

Configure Azure Permissions and Obtain Credentials​

Prepare an identity that the collector can use to discover and read the target metrics. Grant Azure permissions to the application service principal actually used for collection.

Register an Application and Obtain Authentication Parameters​

  1. Sign in to the Azure portal and go to Microsoft Entra ID → App registrations → New registration. Register an application for collection in the tenant that owns the target subscription.
  2. Open the application's Overview page and record the Directory (tenant) ID and Application (client) ID. Enter them in tenant_id and client_id, respectively.
  3. Go to Certificates & secrets → Client secrets → New client secret, enter a description, and set an expiration. Save the client secret's Value immediately after creating it.
  4. Go to Subscriptions, select the subscription containing the target resources, and copy the Subscription ID from its overview page into subscription_id.

Use the client secret's Value for client_secret, not its secret ID. The value is displayed only when the secret is created. Update the secret used for collection before it expires. For instructions, see Register an application and Create a client secret.

ParameterRequiredDescription and how to obtain it
subscription_idYesThe ID of the subscription containing the target resources. Each collection task uses one subscription.
tenant_idYesThe Directory (tenant) ID on the application's overview page.
client_idYesThe Application (client) ID on the application's overview page, not the object ID.
client_secretEither this or the environment variable methodThe application's client secret value. Enter it as required by the platform's component form.
client_secret_envNoWhen client_secret is empty, reads the secret from the environment variable named by this parameter. The default variable name is AZURE_CLIENT_SECRET.

An explicitly configured client_secret takes precedence over the environment variable. When using the environment variable method, set client_secret_env to the variable's name and store the client secret in its value. The variable must be accessible to the SmartGate collector process. Environment variables on the operator's local machine are not automatically available to a remote SmartGate.

Configure Read Permissions at Subscription Scope​

  1. Ask an Azure administrator with role assignment permissions to go to the target Subscription → Access control (IAM).
  2. Select Add role assignment and choose the Monitoring Reader role.
  3. Select the application service principal used for collection as the member. Verify the subscription scope and application identity, then complete the assignment.
  4. Open Resource providers in the same subscription and search for Microsoft.Insights. Register it if it is not already registered.

This integration uses the Azure Monitor batch metrics API, metrics:getBatch, which requires metric read permissions at subscription scope. Assigning a role only to individual resources or resource groups does not satisfy the batch API's subscription-level authorization requirement, even if that assignment allows you to view metrics for those resources.

For role assignment instructions, see Assign Azure roles using the Azure portal. For resource provider operations, see Register resource providers. For the subscription authorization and Microsoft.Insights requirements, see the Azure Monitor batch metrics API guide.

Select Collection Targets and Prepare the Configuration​

Decide which resource types and metrics to collect and whether to split metrics by dimension. Azure services share the authentication and authorization steps above and the platform task workflow below. Each task covers one service type and one subscription.

Select a Service Integration​

Choose from the following integration targets based on your monitoring needs. Available metrics depend on the resource type, service configuration, and data already provided by Azure Monitor.

Integration targettypeResources collectedMain metrics and usage notes
Azure Blob StorageblobStorage accounts and Blob servicesTransactions, ingress and egress, availability, latency, and capacity. Accounts and Blob services are different resource levels.
Azure FilesfilesStorage accounts and File servicesFile service transactions, throughput, latency, capacity, and quotas. Metrics vary by storage configuration.
Azure Cosmos DBcosmosdbCosmos DB database accountsRequests, RU consumption, latency, and storage. Check supported dimensions before querying at database or container level.
Azure DNSdnsDNS zones and DNS Private ResolverDNS query and resolver metrics. Zones and resolvers have different metric time grains.
Azure Database for PostgreSQLpostgresqlPostgreSQL flexible serversCPU, memory, storage, connections, and I/O. Enhanced metrics require the corresponding features to be enabled at the source.
Azure Virtual NetworkvnetVirtual networks, network interfaces, public IP addresses, and NAT gatewaysNetwork traffic, connections, and related status. Some metrics are generated only when the corresponding network features are enabled.
Azure VPN GatewayvpngwVirtual network gatewaysVPN tunnel traffic, connections, and BGP status. Tunnel metrics depend on actual connections.

Blob Storage and Azure Files discover their service child resources from storage accounts. Their resource IDs end in /blobServices/default and /fileServices/default, respectively. The Azure DNS resolver resource type is Microsoft.Network/dnsResolvers; do not use DNS record sets as resolver resources. For available metrics, see the DNS Private Resolver metrics reference.

Obtain Resource and Metric Information​

  1. Open the target resource in the Azure portal and record its subscription, resource group, region, and full Resource ID. You can obtain the resource ID from the resource's properties or JSON view.
  2. Go to Monitoring → Metrics for the resource. Select the target resource scope, metric namespace, metric, and aggregation method, and confirm that recent data points exist.
  3. Use the Azure Monitor metric definitions API or the Azure CLI commands below to verify the metric's REST name, supported aggregation methods, time grains, and dimensions before configuring collection.

Metric display names in the portal may be localized. Use name.value from the metric definition for include_metrics[].name, preserving the actual name's capitalization, spaces, and other characters. Do not use a translated display name directly.

ParameterRequiredDescription and how to obtain it
typeYesThe service type to collect, matching the selected integration, such as blob or cosmosdb.
resource_groupsNoA list of resource group names. Defaults to [], which does not restrict collection by resource group.
resource_idsNoA list of full Azure resource IDs, not resource display names. Defaults to [], which automatically discovers resources for the target service.
include_metricsNoA list of metric configuration objects. When omitted or set to [], collects metrics discovered by the metric definitions API.
exclude_metricsNoA list of metric REST names to exclude. Defaults to [], which excludes no metrics.
static_labelsNoAdds fixed labels to metrics, such as environment or business ownership.

An example Azure resource ID is /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>. For initial integration, limit collection to one resource group and verify a few metrics before expanding the scope. Avoid placing the same metric in both the include and exclude lists.

Each entry in include_metrics supports the following fields:

FieldRequiredDescription
nameYesThe metric's REST name, such as Transactions.
aggregationNoThe aggregation method for this metric: average, minimum, maximum, total, or count. The metric must support the selected method.
split_byNoSplits time series by one specified dimension, such as ApiName. The dimension must be supported by the metric definition. When omitted, no split is applied through this field.
output_nameNoA custom output metric name. After changing it, also check the names used by the data flow and downstream queries.
unitNoAdds a metric unit identifier. Specifying a unit does not perform numeric conversion.
static_labelsNoAdds fixed labels to this metric. Entries with the same name override task-level static labels.

Use the Blob Storage Integration Example​

For initial integration, choose a Blob Storage resource that already has access data and collect the transaction count, Transactions, and end-to-end latency, SuccessE2ELatency. Use total for the former and average for the latter, rather than applying the same aggregation method to metrics with different meanings.

The following example limits collection to the rg-prod resource group and reads the client secret from an environment variable. Replace the subscription, tenant, application ID, and resource group with actual values, and provide AZURE_CLIENT_SECRET in the selected SmartGate runtime environment. If you enter the client secret in the platform form, configure client_secret as described above.

type: blob
subscription_id: your subscription_id
tenant_id: your tenant_id
client_id: your client_id
client_secret_env: AZURE_CLIENT_SECRET
resource_groups:
- rg-prod
resource_ids: []
include_metrics:
- name: Transactions
aggregation: total
- name: SuccessE2ELatency
aggregation: average
exclude_metrics: []
data_delay_seconds: 60
lookback_seconds: 300
interval_seconds: 60
static_labels:
environment: prod

To view transaction counts by API operation, add split_by: ApiName to the Transactions entry. Each split_by entry in this collection configuration specifies one dimension. Splitting produces more time series, so enable it only for metrics that require this analysis.

Request metrics such as Transactions can provide data at a one-minute time grain, while capacity metrics may be available hourly or less frequently. Before adding capacity metrics, check their time grains and increase the data delay and lookback window accordingly. Do not apply this example's query window without checking. For details, see the Blob service metrics reference.

The final metric names, labels, and units in the platform depend on the task's data flow output. When verifying data, retain resource information such as subscription, resource group, resource type, resource name, and region. Distinguish storage account metrics from Blob service metrics to avoid merging metrics with the same name from different resource levels.

Create and Start the Task in ONE​

Enter the prepared Azure credentials, collection targets, and metric scope in the platform. After saving, check the task and data flow to ensure that the configuration has been dispatched and is used for collection.

Create an Integration Task​

  1. Go to Data Framework → Integrations → Custom Integrations, select the target Azure service card, such as Azure Blob Storage, and click Create Integration.
  2. Open the Configuration tab, select a Data Center, enter a Task Name, and configure Data Tags as needed for data access control.
  3. Click Add Component, select the SmartGate that runs the collection task, and enter the subscription, authentication, and collection parameters required by the form. The YAML example above shows the collection configuration. When reading the secret from an environment variable, confirm that the selected SmartGate can read that variable.
  4. Check the component list and click Save. Go to Data Framework → Data Processing and find the task by name.
  5. Open the task details, check the data flow's input, processing, and output configuration, and confirm that the required version is deployed. Verify the metric names, labels, and units.
  6. Check whether the task is started or stopped. If it has not started, check the configuration before starting it. Verify the SmartGate configuration synchronization status and the time it was last updated.

For the complete platform workflow, see Create a Data Task with a Custom Integration. For data flow editing and publishing, see Task Management. If the integration card provides supporting resources, install and verify integration resources as needed.

Adjust Collection Parameters as Needed​

The task execution interval controls how often the task runs, the metric time grain defines the granularity of the source data, and the query window controls the time range read. The collector selects an available time grain from the Azure metric definition. Confirm the source data before adjusting task parameters.

ParameterRequiredDefault and purpose
default_aggregationNoWhen omitted, uses primaryAggregationType from the metric definition. A metric's individual aggregation setting takes precedence.
data_delay_secondsNoDefaults to 60 seconds. Offsets the query end time backward from the current time.
lookback_secondsNoDefaults to 300 seconds. Specifies how far back to query from the query end time.
interval_secondsNoDefaults to 60 seconds. Specifies the task execution interval.
timeout_secondsNoDefaults to 15 seconds. Specifies the timeout for a single Azure API request.
retry_countNoDefaults to 3. Specifies the number of additional retries after a request fails.
schedule.jitterNoDefaults to true. Spreads request times across multiple tasks.

For example, data_delay_seconds: 60 and lookback_seconds: 300 define a query window from approximately 6 minutes ago to 1 minute ago. For metrics with an hourly time grain or longer delays, increasing only interval_seconds does not ensure that data will be read. Also expand the query window to cover data points that have already been generated.

The aggregation method is resolved in this order: include_metrics[].aggregation, default_aggregation, then primaryAggregationType from the metric definition. Before setting an override, confirm that the metric supports the selected method.

Verify Metric Ingestion​

First check whether the platform contains actual data for the target resources. If the results are unexpected, use the Azure APIs to distinguish identity, permission, and source data issues.

Check Actual Data in the Platform​

  1. Confirm that the task has started, the SmartGate configuration is synchronized, and the target data flow version is deployed.
  2. Wait for at least one task execution interval, allowing for the source metric generation interval and the configured data delay.
  3. Check metric definitions in Data Framework → Data Model → Metrics, and query actual data in Insight → Data Explorer.
  4. Check the subscription, resource group, resource type, resource name, region, metric timestamps, and values, and observe whether subsequent data continues to update.
  5. When comparing with the Azure portal, use the same resource level, metric, time range, time grain, aggregation method, dimension split, and units.

Verify Each Layer with the Azure CLI​

For further troubleshooting, use an environment with the Azure CLI and the same service principal as the collection task. Run the following checks in the SmartGate runtime environment where possible. The CLI does not automatically read credentials from the platform's component configuration.

1. Verify the identity and subscription

First prepare AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, and AZURE_SUBSCRIPTION_ID in the current command environment, using the corresponding authentication parameters above. Run:

az login --service-principal \
--username "$AZURE_CLIENT_ID" \
--password="$AZURE_CLIENT_SECRET" \
--tenant "$AZURE_TENANT_ID" \
--output none

az account set --subscription "$AZURE_SUBSCRIPTION_ID"

az account show \
--query '{subscription:id,tenant:tenantId,identity:user.name}' \
--output json

Confirm that the returned subscription, tenant, and application identity match the collection configuration. A successful sign-in only proves that the identity can authenticate; it does not prove that resource discovery or metric read permissions have been granted. See Sign in to the Azure CLI with a service principal.

2. Verify metric definitions

The following example uses the Blob service. Replace the placeholders in RESOURCE_ID with actual values, preserving the /blobServices/default child resource level:

RESOURCE_ID='/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>/blobServices/default'

az monitor metrics list-definitions \
--resource "$RESOURCE_ID" \
--namespace Microsoft.Storage/storageAccounts/blobServices \
--output json

Confirm that the result includes a metric whose name.value is Transactions. Check supportedAggregationTypes, primaryAggregationType, metricAvailabilities[].timeGrain, and dimensions. Finding a metric definition only means that the resource supports the metric; it does not mean that data exists in the current time range.

3. Verify actual data points

az monitor metrics list \
--resource "$RESOURCE_ID" \
--namespace Microsoft.Storage/storageAccounts/blobServices \
--metrics Transactions \
--aggregation Total \
--interval 1m \
--offset 1h \
--output json

This command queries data from the last hour. Check whether value[].timeseries[].data contains data points with timestamps and numeric total values. Empty time series or empty values do not indicate successful ingestion. When comparing with collection results, use --start-time and --end-time to specify the same query window as the task. If dimension splitting is configured, use the same dimensions. For command parameters, see the Azure CLI metrics query reference.

These CLI metrics commands verify the query path for a single resource. Collection also uses the regional metrics:getBatch API. Even after a single-resource query succeeds, verify that the service principal has Monitoring Reader at subscription scope, Microsoft.Insights is registered, and SmartGate can access the corresponding regional metrics endpoint.

Troubleshoot Integration Issues​

SymptomCheck first
Authentication fails or the client secret is invalid.Check that tenant_id and client_id refer to the same application, that an object ID or secret ID was not used by mistake, that the client secret has not expired, and that the SmartGate collector process can actually read the environment variable.
Insufficient permissions to query resources or metrics.Check that the role is assigned to the service principal actually used for collection, that its scope covers the target subscription, and that recent changes have taken effect.
Single-resource queries succeed, but batch metrics queries fail.Check whether permissions were granted only at resource or resource group scope, whether Monitoring Reader is assigned at subscription scope, whether Microsoft.Insights is registered, and whether the regional metrics endpoint is accessible.
The task runs, but the target resource is not discovered.Check subscription_id, type, resource groups, and resource IDs. Confirm that the target resource type is supported by the selected integration and that the Blob/File service child resource level is correct.
Metric definitions are available, but no data points are returned.Check whether the metric has data in the Azure portal, whether the resource has relevant activity or the required feature enabled, and whether the query window covers existing data points.
A metric, aggregation method, or dimension is reported as unsupported.Check that the REST metric name is used with the correct capitalization and spaces, and that aggregation and split_by are supported by the metric definition.
Metrics with an hourly time grain are missing or intermittent.Check whether the delay and lookback window cover the source data period, and whether the execution interval was increased while leaving the query window too short.
Azure has data, but no data can be queried in the platform.Check that the task has started, the SmartGate configuration is synchronized, and the target data flow is deployed. Confirm that the data flow outputs standard metrics and that the query name, time range, and resource labels match.
Values differ between the platform and the Azure portal.Check that both queries use the same resource level, time range, time grain, aggregation method, dimension split, and units, and that account and child service data have not been merged.

For task and SmartGate status checks, see Operations and Maintenance. For further investigation, download SmartGate logs and examine the authentication, resource discovery, metric definition, and metric query errors for the corresponding task.