Google Cloud Monitoring Metrics

Key
Metric Type
Display Name
Kind, Type, Unit Description
Labels

🚧

Metric prefixes

All metrics listed below are prefixed with external.googleapis.com/bluemedora/generic_node/kafka except for the metrics that are also listed in the associated Stackdriver Agent Migration page. If the collector is installed on the Kafka host system, the metrics listed there will be prefixed with agent.googleapis.com/kafka

Google Cloud Monitoring Metrics
/controller/kafka/active
Active controllers
GAUGE, DOUBLE, n/aThe number of active controllers in the cluster.
/broker/cpu/process_load
CPU Process Load
GAUGE, DOUBLE, %CPU load from the Kafka processes.
/broker/cpu/system_load
CPU System Load
GAUGE, DOUBLE, %CPU load of the server.
/broker/error_count
Error Count
CUMULATIVE, INT64, ErrorsNumber of errors in responses.
/broker/incoming_bytes
Incoming Bytes
CUMULATIVE, INT64, ByTotal number of incoming bytes.
source_type: Outgoing data destination ('broker', 'client').
/broker/topics/incoming_message_count
Incoming messages
CUMULATIVE, INT64, n/aThe number of incoming messages in all topics.
/replica_manager/isr/expand_count
Gaining replicas
CUMULATIVE, INT64, n/aThe number of replicas catching up (ISR = in-sync replica).
/replica_manager/isr/shrink_count
Lagging replicas
CUMULATIVE, INT64, n/aThe number of replicas lagging behind (ISR = in-sync replica).
/controller/leader_elections/election_count
Leader elections
CUMULATIVE, INT64, n/aThe total number of leader elections.
/controller/leader_elections/unclean_count
Stale leader elections
CUMULATIVE, INT64, n/aThe total number of leader elections where the leader is out-of-sync.
/replica_manager/leaders
Leaders
GAUGE, DOUBLE, n/aThe current number of leaders on this broker.
/log/flush_count
Flushes
CUMULATIVE, INT64, n/aThe number of log flushes.
/broker/memory/free_bytes
Memory Free Bytes
GAUGE, INT64, ByFree memory on the server.
/broker/memory/size_bytes
Memory Size Bytes
GAUGE, INT64, ByTotal memory of the server.
/broker/memory/temporary_bytes
Memory Temporary Bytes
GAUGE, INT64, ByTemporary memory used for message format conversions and decompression
/broker/memory/utilization
Memory Utilization
GAUGE, DOUBLE, %Memory utilization of the server.
/broker/message_conversion_count
Message Conversion Count
CUMULATIVE, INT64, n/aNumber of records which required message format conversion.
/broker/message_conversion_time
Message Conversation Time
GAUGE, DOUBLE, msTime in milliseconds spent on message format conversions.
/controller/kafka/offline_partitions
Offline partitions
GAUGE, DOUBLE, n/aThe number of partitions that don’t have an active leader and are hence not writable or readable.
/broker/outgoing_bytes
Outgoing Bytes
CUMULATIVE, INT64, ByTotal number of bytes outgoing to other systems.
source_type: Outgoing data destination ('broker', 'client').
/replica_manager/partitions
Partitions
GAUGE, DOUBLE, n/aThe current number of partitions on this broker.
/purgatory/num_delayed_requests
Delayed purgatory requests
GAUGE, DOUBLE, RequestsThe current number of delayed requests in purgatory.
request_type: The type of request (fetch or produce).
/purgatory/size
Purgatory requests
GAUGE, DOUBLE, RequestsThe current number of requests in purgatory.
request_type: The type of the request, fetch or produce.
/replica_fetcher/max_lag
Maximum lag
GAUGE, DOUBLE, n/aThe maximum lag in messages between the follower and leader replicas.
/replica_fetcher/min_fetch_rate
Minimum fetch rate
GAUGE, DOUBLE, 1/sThe minimum rate at which the follower replicas send fetch requests to the leaders.
/network/request_count
Requests
CUMULATIVE, INT64, RequestsThe number of requests.
request_type: The type of request (fetch or produce).
/broker/topics/failed_request_count
Failed requests
CUMULATIVE, INT64, RequestsThe number of failed requests.
request_type: The type of request (fetch or produce).
/broker/topics/traffic
Traffic
CUMULATIVE, INT64, ByThe number of bytes sent and received in all topics.
direction: Data direction, in or out.
/replica_manager/under_replicated_partitions
Unreliable partitions
GAUGE, DOUBLE, n/aThe current number of under-replicated partitions.
/cluster/broker_count
Broker Count
GAUGE, INT64, n/aNumber of brokers.
/cluster/consumer_count
Consumer Count
GAUGE, INT64, n/aNumber of consumers.
/cluster/producer_count
Producer Count
GAUGE, INT64, n/aNumber of producers.
/consumer/assigned_partition_count
Assigned Partition Count
GAUGE, INT64, n/aNumber of assigned partitions.
/consumer/average_commit_latency
Average Commit Latency
GAUGE, DOUBLE, nsAverage time taken for a commit request.
/consumer/average_join_time
Average Join Time
GAUGE, DOUBLE, nsAverage time taken for a group rejoin.
/consumer/average_sync_time
Average Sync Time
GAUGE, DOUBLE, nsAverage time taken for a group sync.
/consumer/commit_count
Commit Count
CUMULATIVE, INT64, n/aTotal number of commit calls.
/consumer/connection_close_count
Connection Close Count
CUMULATIVE, INT64, n/aTotal number of connections closed.
/consumer/connection_count
Connection Count
GAUGE, INT64, n/aNumber of connections.
/consumer/connection_create_count
Connection Create Count
CUMULATIVE, INT64, n/aTotal number of new connections established.
/consumer/incoming_bytes
Incoming Bytes
CUMULATIVE, INT64, ByTotal number of bytes read off all sockets.
/consumer/io/time
I/O Time
CUMULATIVE, INT64, nsTotal time the I/O thread spent performing I/O.
/consumer/io/wait_time
I/O Wait Time
CUMULATIVE, INT64, nsTotal time the I/O thread spent waiting.
/consumer/io/average_wait_time
Average IO Wait Time
GAUGE, DOUBLE, nsAverage length of time the I/O thread spent waiting for a socket ready for reads or writes.
/consumer/io/time_ratio
IO Time Ratio
GAUGE, DOUBLE, %Fraction of time the I/O thread spent doing I/O.
/consumer/io/wait_time_ratio
IO Wait Time Ratio
GAUGE, DOUBLE, %Fraction of time the I/O thread spent waiting.
/consumer/join_count
Join Count
CUMULATIVE, INT64, n/aTotal number of group joins.
/consumer/maximum_commit_latency
Maximum Commit Latency
GAUGE, DOUBLE, nsMaximum latency experienced during a commit.
/consumer/maximum_join_time
Maximum Join Time
GAUGE, DOUBLE, nsMaximum time spent in a single join.
/consumer/maximum_lag
Maximum Lag
GAUGE, DOUBLE, RecordsThe maximum lag in terms of number of records for any partition in this window.
/consumer/maximum_sync_time
Maximum Sync Time
GAUGE, DOUBLE, nsMaximum time spent in a single sync.
/consumer/outgoing_bytes
Outgoing Bytes
CUMULATIVE, INT64, ByTotal number of outgoing bytes sent to all servers.
/consumer/request_count
Request Count
CUMULATIVE, INT64, RequestsTotal number of requests sent.
/consumer/response_count
Response Count
CUMULATIVE, INT64, n/aTotal number of responses received.
/consumer/sync_count
Sync Count
CUMULATIVE, INT64, n/aTotal number of group syncs.
/producer/average_request_latency
Average Request Latency
GAUGE, DOUBLE, nsAverage latency of requests.
/producer/buffer_memory/available_bytes
Buffer Memory Available Bytes
GAUGE, INT64, ByTotal amount of buffer memory that is not being used (either unallocated or in the free list).
/producer/buffer_memory/total_bytes
Buffer Memory Total Bytes
GAUGE, INT64, ByMaximum amount of buffer memory the client can use (whether or not it is currently used).
/producer/connection_closed_count
Connection Closed Count
CUMULATIVE, INT64, ConnectionsTotal number of connections closed.
/producer/connection_count
Connection Count
GAUGE, INT64, ConnectionsNumber of connections.
/producer/connection_created_count
Connection Created Count
CUMULATIVE, INT64, ConnectionsTotal number of new connections established.
/producer/incoming_bytes
Incoming Bytes
CUMULATIVE, INT64, ByTotal number of bytes read off all sockets.
/producer/io/time
I/O Time
CUMULATIVE, INT64, nsTotal time the I/O thread spent doing I/O.
/producer/io/wait_time
I/O Wait Time
CUMULATIVE, INT64, nsTotal time the I/O thread spent waiting.
/producer/io/average_wait_time
Average IO Wait Time
GAUGE, DOUBLE, nsAverage length of time the I/O thread spent waiting for a socket ready for reads or writes.
/producer/io/time_ratio
IO Time Ratio
GAUGE, DOUBLE, %Fraction of time the I/O thread spent doing I/O.
/producer/io/wait_time_ratio
IO Wait Time Ratio
GAUGE, DOUBLE, %Fraction of time the I/O thread spent waiting.
/producer/maximum_request_latency
Maximum Request Latency
GAUGE, DOUBLE, nsMaximum latency experienced by a single request.
/producer/outgoing_bytes
Outgoing Bytes
CUMULATIVE, INT64, ByTotal number of outgoing bytes sent to all servers.
/producer/record/error_count
Record Error Count
CUMULATIVE, INT64, ErrorsTotal number of record sends that resulted in errors.
/producer/record/retry_count
Record Retry Count
CUMULATIVE, INT64, n/aTotal number of retried record sends.
/producer/record/send_count
Record Send Count
CUMULATIVE, INT64, RecordsTotal number of records sent.
/producer/request_count
Request Count
CUMULATIVE, INT64, RequestsTotal number of requests sent.
/producer/response_count
Response Count
CUMULATIVE, INT64, n/aTotal number of responses received.
/producer/waiting_thread_count
Waiting Thread Count
GAUGE, INT64, ThreadsNumber of user threads blocked waiting for buffer memory to enqueue their records.