Skip to main content
Version: 6.1

AI Observability

Purpose​

The AI Observability module is designed for observability of AI infrastructure in Search Anywhere Framework.
It collects AI environment telemetry, normalizes it into a unified set of gen_ai_* indexes, and provides ready boxed content for operations.

The module covers the following observability directions:

  • LLM gateway — requests, model routing, errors, latency
  • Inference runtime — performance and load of inference service
  • GPU environment — graphics card status: temperature, memory, load, errors
  • AI agents — questions, answers, agent steps, tool calls, traces
  • Local AI clients — monitoring Claude Code and Codex as telemetry sources
  • LLM costs — tokens, spend data by models and providers
  • Service availability — general operational slice of AI components

AI Monitoring Architecture

Module Components​

  1. Dashboards — 8 dashboards for all AI observability directions
  2. Inventory — automatically maintained GPU asset
  3. Service Monitor Toolkit global metrics — 6 key operational metrics for end-to-end monitoring
  4. Data sources — unified set of gen_ai_* indexes with normalized fields

Documentation Sections​