Guides

Here are some common tasks that you may need to perform when using Kinference.

• Basic inference serving Configure Kinference to route inference requests to your existing vLLM workers and explore basic observability.
• Installing the Kinference Grafana dashboards Install the Kinference Grafana dashboards and populate them with metrics from Prometheus.
• Tracking tenants Identify tenants from a request header and track inference metrics on a per-tenant basis.
• Writing inference policies Write a basic inference policy that routes, forwards, or denies inference requests.