Skip to content

Knative Serving Metrics

Administrators can monitor Serving control plane based on the metrics exposed by each Serving component.

Note

These metrics may change as we flush out our migration from OpenCensus to OpenTelemetry

Queue Proxy

The queue proxy is the per-pod sidecar that enforces container concurrency and provides metrics to the autoscaler. The following metrics provide you insights into queued requests and user-container behavior.

kn.queueproxy.depth

Instrument Type: Int64Gauge

Unit (UCUM): {request}

Description: Number of current items in the queue proxy queue

kn.queueproxy.app.duration

Instrument Type: Float64Histogram

Unit (UCUM): s

Description: The duration of the task execution

Activator

The following metrics can help you to understand how an application responds when traffic passes through the activator. For example, when scaling from zero, high request latency might mean that requests are taking too much time to be fulfilled.

kn.revision.request.concurrency

Instrument Type: Float64Gauge

Unit (UCUM): {request}

Description: Concurrent requests that are routed to the Activator

The following attributes are included with the metric

Name Type Description
k8s.namespace.name string Namespace of the Revision
kn.service.name string Knative Service name associated with this Revision
kn.configuration.name string Knative Configuration name associated with this Revision
kn.revision.name string The name of the Revision

Autoscaler

Autoscaler component exposes a number of metrics related to its decisions per revision. For example, at any given time, you can monitor the desired pods the Autoscaler wants to allocate for a Service, the average number of requests per second during the stable window, or whether autoscaler is in panic mode (KPA).

The following attributes are included with the autoscaling metrics below

Name Type Description
k8s.namespace.name string Namespace of the Revision
kn.service.name string Knative Service name associated with this Revision
kn.configuration.name string Knative Configuration name associated with this Revision
kn.revision.name string The name of the Revision

kn.autoscaler.scrape.duration

Instrument Type: Float64Histogram

Unit (UCUM): s

Description: The duration of scraping the revision

kn.revision.pods.desired

Instrument Type: Int64Gauge

Unit (UCUM): {pod}

Description: Number of pods the autoscaler wants to allocate

kn.revision.capacity.excess

Instrument Type: Float64Gauge

Unit (UCUM): {concurrency}

Description: Excess burst capacity observed over the stable window

kn.revision.concurrency.stable

Instrument Type: Float64Gauge

Unit (UCUM): {concurrency}

Description: Average of request count per observed pod over the stable window

kn.revision.concurrency.panic

Instrument Type: Float64Gauge

Unit (UCUM): {concurrency}

Description: Average of request count per observed pod over the panic window

kn.revision.concurrency.target

Instrument Type: Float64Gauge

Unit (UCUM): {concurrency}

Description: The desired concurrent requests for each pod

kn.revision.rps.stable

Instrument Type: Float64Gauge

Unit (UCUM): {request}/s

Description: Average of requests-per-second per observed pod over the stable window

kn.revision.rps.panic

Instrument Type: Float64Gauge

Unit (UCUM): {request}/s

Description: Average of requests-per-second per observed pod over the panic window

kn.revision.pods.requested

Instrument Type: Int64Gauge

Unit (UCUM): {pod}

Description: Number of pods autoscaler requested from Kubernetes

kn.revision.pods.count

Instrument Type: Int64Gauge

Unit (UCUM): {pod}

Description: Number of pods that are allocated currently

kn.revision.pods.not_ready.count

Instrument Type: Int64Gauge

Unit (UCUM): {pod}

Description: Number of pods that are not ready currently

kn.revision.pods.pending.count

Instrument Type: Int64Gauge

Unit (UCUM): {pod}

Description: Number of pods that are pending currently

kn.revision.pods.terminating.count

Instrument Type: Int64Gauge

Unit (UCUM): {pod}

Description: Number of pods that are terminating currently

Webhook Metrics

Webhook metrics report useful info about operations. For example, if a large number of operations fail, this could indicate an issue with a user-created resource.

http.server.request.duration

Knative implements the semantic conventions for HTTP Servers using the OpenTelemetry otel-go/otelhttp package.

Please refer to the OpenTelemetry docs for details about the HTTP Server metrics it exports.

The following attributes are included with the metric

Name Type Description Examples
kn.webhook.type string Specifies the type of webhook invoked admission, defaulting, validation, conversion
kn.webhook.resource.group string Specifies the resource Kubernetes group name
kn.webhook.resource.version string Specifies the resource Kubernetes group version
kn.webhook.resource.kind string Specifies the resource Kubernetes group kind
kn.webhook.subresource string Specifies the subresource "" (empty), status, scale
kn.webhook.operation.type string Specifies the operation that invoked the webhook CREATE, UPDATE, DELETE
kn.webhook.operation.status string Specifies whether the operation was successful success, failed

kn.webhook.handler.duration

Instrument Type: Histogram

Unit ([UCUM): s

Description: The duration of task execution.

The following attributes are included with the metric

Name Type Description Examples
kn.webhook.type string Specifies the type of webhook invoked admission, defaulting, validation, conversion
kn.webhook.resource.group string Specifies the resource Kubernetes group name
kn.webhook.resource.version string Specifies the resource Kubernetes group version
kn.webhook.resource.kind string Specifies the resource Kubernetes group kind
kn.webhook.subresource string Specifies the subresource "" (empty), status, scale
kn.webhook.operation.type string Specifies the operation that invoked the webhook CREATE, UPDATE, DELETE
kn.webhook.operation.status string Specifies whether the operation was successful success, failed

Workqueue Metrics

Knative controllers expose client-go workqueue metrics

The following attributes are included with the metrics below

Name Type Description
name string Name of the work queue

kn.workqueue.depth

Instrument Type: Int64UpDownCounter

Unit (UCUM): {item}

Description: Number of current items in the queue

kn.workqueue.adds

Instrument Type: Int64Counter

Unit (UCUM): {item}

Description: Number of items added to the queue

kn.workqueue.queue.duration

Instrument Type:

Unit (UCUM): s

Description: How long an item stays in workqueue

kn.workqueue.process.duration

Instrument Type: Float64Histogram

Unit (UCUM): s

Description: How long in seconds processing an item from workqueue takes

kn.workqueue.unfinished_work

Instrument Type: Float64Gauge

Unit (UCUM): s

Description: How many seconds of work the reconciler has done that is in progress and hasn't been observed by duration. Large values indicate stuck threads. One can deduce the number of stuck threads by observing the rate at which this increases.

kn.workqueue.longest_running_processor

Instrument Type: Float64Gauge

Unit (UCUM): s

Description: How long the longest worker thread has been running

kn.workqueue.retries

Instrument Type: Int64Counter

Unit (UCUM): {item}

Description: Number of items re-added to the queue

Go Runtime

Knative implements the semantic conventions for Go runtime metrics using the OpenTelemetry otel-go/instrumentation/runtime package.

Please refer to the OpenTelemetry docs for details about the go runtime metrics it exports.

We use analytics and cookies to understand site traffic. Information about your use of our site is shared with Google for that purpose. Learn more.

× OK