Tracking tenants

Kinference policies can take into account the tenant (aka client) that issued the request. It can also optionally track metrics on a per-tenant basis.

Setting the tenant identification header #

Kinference determines the tenant id simply by reading it from a header. To enable this behavior, you must configure the header to use.

Kinference trusts this header unconditionally. To avoid spoofing, if this header is enabled, ensure that it cannot be arbitrarily set by users.

To configure the header that determines Kinference’s client id, set the extension.tenantHeader configuration in the Kinference Helm chart. For example, to set it to l5d-client-id, run this command:

TENANT_HEADER=l5d-client-id # Linkerd client identity
helm upgrade -i kinference oci://cr.kinference.ai/kinference-ai/charts/kinference --devel --reuse-values --set extension.tenantHeader=$TENANT_HEADER --wait

You can now apply policies to specific tenants or groups thereof.

If this configuration is not supplied, or if it is supplied, but the header for a request is unset, the tenant value "" will be used. You may write policies that use this header value directly.

Disabling per-tenant metrics #

By default, Kinference tracks metrics for all tenants and creates timeseries for each.

Per-tenant metrics have potentially very high cardinality: each combination of tenant, model, and QoS class will create almost 100 time series. For example, a single tenant that uses five models across three QoS classes will create 1,500 time series. Across many tenants the cardinality of metrics may place undue strain on your metrics stack.

If you expect to have an unbounded number of tenants, there are two alternatives:

Use an allowlist
Disable per-tenant metrics tracking alltogether.

In either case, tenants for whom metrics are not tracked (as well as tenants for whom no value is provided in the corresponding header) are bucketed into the default tenant for metrics purposes.

To use the allowlist approach, set the config.metrics.tenant_labels value in the Kinference Helm chart to a list of tenant ids. For example, to only track metrics for the tenants alice and bob, run this command:

helm upgrade kinference oci://cr.kinference.ai/kinference-ai/charts/kinference --devel --reuse-values --set "config.metrics.tenant_labels={alice,bob}" --wait

To disable per-tenant metrics tracking altogether, set it to none instead:

helm upgrade kinference oci://cr.kinference.ai/kinference-ai/charts/kinference --devel --reuse-values --set config.metrics.tenant_labels=none --wait

The Kinference pods will restart, and metrics collection going forward will use the tenant tracking you configured.