Docs
Getting started
Install the gateway, Kinference and the kinfctl CLI in five steps.
Features
Cache-aware load balancing, token rate limits, model aliases, metrics and semantic classification.
Guides
Step-by-step: inference serving, monitoring, tenant tracking and writing policies.
Reference
Configuration, CRDs, kinfctl, metrics and policy syntax.
Quick links
Writing inference policies
KV cache-aware load balancing
Token-based rate limiting
InferencePolicy CRD
Grafana dashboards
Blog
Enterprise
Go to GitHub ↗
Reference
Getting started
Features
KV cache-aware load balancing
Model aliases
Policy-driven inference serving
Production-ready metrics, monitoring, and dashboards
Semantic classification
Token-based rate limiting
Guides
Basic inference serving
Installing the Kinference Grafana dashboards
Tracking tenants
Writing inference policies
Reference
Configuration
CRDs
GatewayBackend
InferencePolicy
InferencePool
InferenceProvider
InferenceWorkerSet
kinfctl
Metrics
Policies
Reference
Configuration
Kinference configuration file reference
CRDs
Kinference Custom Resource Definition (CRD) reference
kinfctl
Kinference CLI with policy checking
Metrics
Kinference metrics and telemetry
Policies
Kinference policies
