Skip to main content
Version: 6.2

AI Observability: Health Metrics

The AI Observability module provides ready global metrics for the Service Monitor Toolkit module.
They are calculated from data in gen_ai_* indexes and allow the AI infrastructure to be included in the organization's overall health model. Metrics are used as indicators of the AI environment's status in the resource-service model. They enable tracking AI component degradation, setting criticality thresholds, and linking the technical state of AI infrastructure to service impact.

Purpose​

Health metrics allow:

  • displaying the status of AI components alongside infrastructure and application services
  • accounting for GPUs, AI services, traces, and requests in a unified health model
  • setting degradation thresholds for key AI infrastructure indicators
  • using AI infrastructure as a separate layer in end-to-end service models

Metric Descriptions​

1. GPU Temperature, °C​

Data source: gen_ai_gpu_metrics*

The metric shows GPU core temperature in degrees Celsius. It helps identify signs of hardware degradation: critical GPU heat can affect AI service performance and request processing stability.

GPU Temperature,°C

2. GPU Utilization, %​

Data source: gen_ai_gpu_metrics*

The metric shows current GPU utilization as a percentage. It helps determine whether GPU resources are sufficient to process AI requests or whether the infrastructure needs to be scaled.

GPU Utilization, %

3. GPU Memory Utilization, %​

Data source: gen_ai_gpu_metrics*

The metric shows the percentage of GPU video memory in use. It is particularly important for inference scenarios with large LLMs, where available video memory directly affects service stability.

GPU Memory Utilization, %

4. Request Duration, ms​

Data source: gen_ai_traces*

The metric shows the average duration of AI requests and operations in milliseconds. It helps track AI service response time and detect degradation before users notice it.

Request Duration, ms

5. Trace Errors​

Data source: gen_ai_traces*

The metric shows the number of errors recorded in AI service traces. It helps determine whether degradation is related to operation errors, not only load or latency.

Trace Errors

6. Service Availability​

Data source: gen_ai_metrics*

The metric shows AI service availability using the up metric. It allows service availability to be included in the overall health model and helps identify components that have stopped sending data.

Service Availability

Operations Application​

Health metrics allow:

  • detect early signs of degradation — increases in GPU temperature, memory utilization, errors, or request duration are visible before the issue affects users
  • monitor AI infrastructure in the overall health model — GPUs, AI services, and traces become part of a unified resource-service model
  • set criticality thresholds — metric values can be adapted to the environment, load, and models in use
  • link technical failures to service impact — AI component degradation can be taken into account when calculating the status of related services

Health metrics are installed by the installation utility. Running it again overwrites the supplied metrics, so changes to the metrics and their permissions must be applied again.