Production-ready metrics, monitoring, and dashboards

Kinference exports a complete set of metrics in a Prometheus-compatible form. These metrics include both operational metrics that cover system health and operation, as well as inference-specific “golden metrics” such as:

• TTFT: Time-to-first-token
• ITL: Inter-token latency
• E2E: End-to-end latency

Inference metrics can be recorded on a per-tenant basis, allowing you to capture tenant behavior and build tenant-specific SLOs. See Tracking tenants for more.

Kinference also ships with a fully-featured Grafana dashboard that provides an out-of-the-box operational view of the state of your Kinference deployment. See Installing the Kinference Grafana dashboards for more.