Knative Serving Metrics¶
Administrators can monitor Serving control plane based on the metrics exposed by each Serving component.
Note
These metrics may change as we flush out our migration from OpenCensus to OpenTelemetry
Queue Proxy¶
The queue proxy is the per-pod sidecar that enforces container concurrency and provides metrics to the autoscaler. The following metrics provide you insights into queued requests and user-container behavior.
kn.queueproxy.depth
¶
Instrument Type: Int64Gauge
Unit (UCUM): {request}
Description: Number of current items in the queue proxy queue
kn.queueproxy.app.duration
¶
Instrument Type: Float64Histogram
Unit (UCUM): s
Description: The duration of the task execution
Activator¶
The following metrics can help you to understand how an application responds when traffic passes through the activator. For example, when scaling from zero, high request latency might mean that requests are taking too much time to be fulfilled.
kn.revision.request.concurrency
¶
Instrument Type: Float64Gauge
Unit (UCUM): {request}
Description: Concurrent requests that are routed to the Activator
The following attributes are included with the metric
Name | Type | Description |
---|---|---|
k8s.namespace.name |
string | Namespace of the Revision |
kn.service.name |
string | Knative Service name associated with this Revision |
kn.configuration.name |
string | Knative Configuration name associated with this Revision |
kn.revision.name |
string | The name of the Revision |
Autoscaler¶
Autoscaler component exposes a number of metrics related to its decisions per revision. For example, at any given time, you can monitor the desired pods the Autoscaler wants to allocate for a Service, the average number of requests per second during the stable window, or whether autoscaler is in panic mode (KPA).
The following attributes are included with the autoscaling metrics below
Name | Type | Description |
---|---|---|
k8s.namespace.name |
string | Namespace of the Revision |
kn.service.name |
string | Knative Service name associated with this Revision |
kn.configuration.name |
string | Knative Configuration name associated with this Revision |
kn.revision.name |
string | The name of the Revision |
kn.autoscaler.scrape.duration
¶
Instrument Type: Float64Histogram
Unit (UCUM): s
Description: The duration of scraping the revision
kn.revision.pods.desired
¶
Instrument Type: Int64Gauge
Unit (UCUM): {pod}
Description: Number of pods the autoscaler wants to allocate
kn.revision.capacity.excess
¶
Instrument Type: Float64Gauge
Unit (UCUM): {concurrency}
Description: Excess burst capacity observed over the stable window
kn.revision.concurrency.stable
¶
Instrument Type: Float64Gauge
Unit (UCUM): {concurrency}
Description: Average of request count per observed pod over the stable window
kn.revision.concurrency.panic
¶
Instrument Type: Float64Gauge
Unit (UCUM): {concurrency}
Description: Average of request count per observed pod over the panic window
kn.revision.concurrency.target
¶
Instrument Type: Float64Gauge
Unit (UCUM): {concurrency}
Description: The desired concurrent requests for each pod
kn.revision.rps.stable
¶
Instrument Type: Float64Gauge
Unit (UCUM): {request}/s
Description: Average of requests-per-second per observed pod over the stable window
kn.revision.rps.panic
¶
Instrument Type: Float64Gauge
Unit (UCUM): {request}/s
Description: Average of requests-per-second per observed pod over the panic window
kn.revision.pods.requested
¶
Instrument Type: Int64Gauge
Unit (UCUM): {pod}
Description: Number of pods autoscaler requested from Kubernetes
kn.revision.pods.count
¶
Instrument Type: Int64Gauge
Unit (UCUM): {pod}
Description: Number of pods that are allocated currently
kn.revision.pods.not_ready.count
¶
Instrument Type: Int64Gauge
Unit (UCUM): {pod}
Description: Number of pods that are not ready currently
kn.revision.pods.pending.count
¶
Instrument Type: Int64Gauge
Unit (UCUM): {pod}
Description: Number of pods that are pending currently
kn.revision.pods.terminating.count
¶
Instrument Type: Int64Gauge
Unit (UCUM): {pod}
Description: Number of pods that are terminating currently
Webhook Metrics¶
Webhook metrics report useful info about operations. For example, if a large number of operations fail, this could indicate an issue with a user-created resource.
http.server.request.duration
¶
Knative implements the semantic conventions for HTTP Servers using the OpenTelemetry otel-go/otelhttp package.
Please refer to the OpenTelemetry docs for details about the HTTP Server metrics it exports.
The following attributes are included with the metric
Name | Type | Description | Examples |
---|---|---|---|
kn.webhook.type |
string | Specifies the type of webhook invoked | admission , defaulting , validation , conversion |
kn.webhook.resource.group |
string | Specifies the resource Kubernetes group name | |
kn.webhook.resource.version |
string | Specifies the resource Kubernetes group version | |
kn.webhook.resource.kind |
string | Specifies the resource Kubernetes group kind | |
kn.webhook.subresource |
string | Specifies the subresource | "" (empty), status , scale |
kn.webhook.operation.type |
string | Specifies the operation that invoked the webhook | CREATE , UPDATE , DELETE |
kn.webhook.operation.status |
string | Specifies whether the operation was successful | success , failed |
kn.webhook.handler.duration
¶
Instrument Type: Histogram
Unit ([UCUM): s
Description: The duration of task execution.
The following attributes are included with the metric
Name | Type | Description | Examples |
---|---|---|---|
kn.webhook.type |
string | Specifies the type of webhook invoked | admission , defaulting , validation , conversion |
kn.webhook.resource.group |
string | Specifies the resource Kubernetes group name | |
kn.webhook.resource.version |
string | Specifies the resource Kubernetes group version | |
kn.webhook.resource.kind |
string | Specifies the resource Kubernetes group kind | |
kn.webhook.subresource |
string | Specifies the subresource | "" (empty), status , scale |
kn.webhook.operation.type |
string | Specifies the operation that invoked the webhook | CREATE , UPDATE , DELETE |
kn.webhook.operation.status |
string | Specifies whether the operation was successful | success , failed |
Workqueue Metrics¶
Knative controllers expose client-go workqueue metrics
The following attributes are included with the metrics below
Name | Type | Description |
---|---|---|
name |
string | Name of the work queue |
kn.workqueue.depth
¶
Instrument Type: Int64UpDownCounter
Unit (UCUM): {item}
Description: Number of current items in the queue
kn.workqueue.adds
¶
Instrument Type: Int64Counter
Unit (UCUM): {item}
Description: Number of items added to the queue
kn.workqueue.queue.duration
¶
Instrument Type:
Unit (UCUM): s
Description: How long an item stays in workqueue
kn.workqueue.process.duration
¶
Instrument Type: Float64Histogram
Unit (UCUM): s
Description: How long in seconds processing an item from workqueue takes
kn.workqueue.unfinished_work
¶
Instrument Type: Float64Gauge
Unit (UCUM): s
Description: How many seconds of work the reconciler has done that is in progress and hasn't been observed by duration. Large values indicate stuck threads. One can deduce the number of stuck threads by observing the rate at which this increases.
kn.workqueue.longest_running_processor
¶
Instrument Type: Float64Gauge
Unit (UCUM): s
Description: How long the longest worker thread has been running
kn.workqueue.retries
¶
Instrument Type: Int64Counter
Unit (UCUM): {item}
Description: Number of items re-added to the queue
Go Runtime¶
Knative implements the semantic conventions for Go runtime metrics using the OpenTelemetry otel-go/instrumentation/runtime package.
Please refer to the OpenTelemetry docs for details about the go runtime metrics it exports.