Google Cloud Monitoring Metrics

Metric Type
Display Name

Kind, Type, Unit

Description

Labels

/controller/kafka/active
Active controllers

GAUGE, DOUBLE, n/a

The number of active controllers in the cluster.

/broker/cpu/process_load
CPU Process Load

GAUGE, DOUBLE, %

CPU load from the Kafka processes.

/broker/cpu/system_load
CPU System Load

GAUGE, DOUBLE, %

CPU load of the server.

/broker/error_count
Error Count

CUMULATIVE, INT64, Errors

Number of errors in responses.

/broker/incoming_bytes
Incoming Bytes

CUMULATIVE, INT64, By

Total number of incoming bytes.

source_type: Outgoing data destination ('broker', 'client').

/broker/topics/incoming_message_count
Incoming messages

CUMULATIVE, INT64, n/a

The number of incoming messages in all topics.

/replica_manager/isr/expand_count
Gaining replicas

CUMULATIVE, INT64, n/a

The number of replicas catching up (ISR = in-sync replica).

/replica_manager/isr/shrink_count
Lagging replicas

CUMULATIVE, INT64, n/a

The number of replicas lagging behind (ISR = in-sync replica).

/controller/leader_elections/election_count
Leader elections

CUMULATIVE, INT64, n/a

The total number of leader elections.

/controller/leader_elections/unclean_count
Stale leader elections

CUMULATIVE, INT64, n/a

The total number of leader elections where the leader is out-of-sync.

/replica_manager/leaders
Leaders

GAUGE, DOUBLE, n/a

The current number of leaders on this broker.

/log/flush_count
Flushes

CUMULATIVE, INT64, n/a

The number of log flushes.

/broker/memory/free_bytes
Memory Free Bytes

GAUGE, INT64, By

Free memory on the server.

/broker/memory/size_bytes
Memory Size Bytes

GAUGE, INT64, By

Total memory of the server.

/broker/memory/temporary_bytes
Memory Temporary Bytes

GAUGE, INT64, By

Temporary memory used for message format conversions and decompression

/broker/memory/utilization
Memory Utilization

GAUGE, DOUBLE, %

Memory utilization of the server.

/broker/message_conversion_count
Message Conversion Count

CUMULATIVE, INT64, n/a

Number of records which required message format conversion.

/broker/message_conversion_time
Message Conversation Time

GAUGE, DOUBLE, ms

Time in milliseconds spent on message format conversions.

/controller/kafka/offline_partitions
Offline partitions

GAUGE, DOUBLE, n/a

The number of partitions that don’t have an active leader and are hence not writable or readable.

/broker/outgoing_bytes
Outgoing Bytes

CUMULATIVE, INT64, By

Total number of bytes outgoing to other systems.

source_type: Outgoing data destination ('broker', 'client').

/replica_manager/partitions
Partitions

GAUGE, DOUBLE, n/a

The current number of partitions on this broker.

/purgatory/num_delayed_requests
Delayed purgatory requests

GAUGE, DOUBLE, Requests

The current number of delayed requests in purgatory.

request_type: The type of request (fetch or produce).

/purgatory/size
Purgatory requests

GAUGE, DOUBLE, Requests

The current number of requests in purgatory.

request_type: The type of the request, fetch or produce.

/replica_fetcher/max_lag
Maximum lag

GAUGE, DOUBLE, n/a

The maximum lag in messages between the follower and leader replicas.

/replica_fetcher/min_fetch_rate
Minimum fetch rate

GAUGE, DOUBLE, 1/s

The minimum rate at which the follower replicas send fetch requests to the leaders.

/network/request_count
Requests

CUMULATIVE, INT64, Requests

The number of requests.

request_type: The type of request (fetch or produce).

/broker/topics/failed_request_count
Failed requests

CUMULATIVE, INT64, Requests

The number of failed requests.

request_type: The type of request (fetch or produce).

/broker/topics/traffic
Traffic

CUMULATIVE, INT64, By

The number of bytes sent and received in all topics.

direction: Data direction, in or out.

/replica_manager/under_replicated_partitions
Unreliable partitions

GAUGE, DOUBLE, n/a

The current number of under-replicated partitions.

/cluster/broker_count
Broker Count

GAUGE, INT64, n/a

Number of brokers.

/cluster/consumer_count
Consumer Count

GAUGE, INT64, n/a

Number of consumers.

/cluster/producer_count
Producer Count

GAUGE, INT64, n/a

Number of producers.

/consumer/assigned_partition_count
Assigned Partition Count

GAUGE, INT64, n/a

Number of assigned partitions.

/consumer/average_commit_latency
Average Commit Latency

GAUGE, DOUBLE, ns

Average time taken for a commit request.

/consumer/average_join_time
Average Join Time

GAUGE, DOUBLE, ns

Average time taken for a group rejoin.

/consumer/average_sync_time
Average Sync Time

GAUGE, DOUBLE, ns

Average time taken for a group sync.

/consumer/commit_count
Commit Count

CUMULATIVE, INT64, n/a

Total number of commit calls.

/consumer/connection_close_count
Connection Close Count

CUMULATIVE, INT64, n/a

Total number of connections closed.

/consumer/connection_count
Connection Count

GAUGE, INT64, n/a

Number of connections.

/consumer/connection_create_count
Connection Create Count

CUMULATIVE, INT64, n/a

Total number of new connections established.

/consumer/incoming_bytes
Incoming Bytes

CUMULATIVE, INT64, By

Total number of bytes read off all sockets.

/consumer/io/time
I/O Time

CUMULATIVE, INT64, ns

Total time the I/O thread spent performing I/O.

/consumer/io/wait_time
I/O Wait Time

CUMULATIVE, INT64, ns

Total time the I/O thread spent waiting.

/consumer/io/average_wait_time
Average IO Wait Time

GAUGE, DOUBLE, ns

Average length of time the I/O thread spent waiting for a socket ready for reads or writes.

/consumer/io/time_ratio
IO Time Ratio

GAUGE, DOUBLE, %

Fraction of time the I/O thread spent doing I/O.

/consumer/io/wait_time_ratio
IO Wait Time Ratio

GAUGE, DOUBLE, %

Fraction of time the I/O thread spent waiting.

/consumer/join_count
Join Count

CUMULATIVE, INT64, n/a

Total number of group joins.

/consumer/maximum_commit_latency
Maximum Commit Latency

GAUGE, DOUBLE, ns

Maximum latency experienced during a commit.

/consumer/maximum_join_time
Maximum Join Time

GAUGE, DOUBLE, ns

Maximum time spent in a single join.

/consumer/maximum_lag
Maximum Lag

GAUGE, DOUBLE, Records

The maximum lag in terms of number of records for any partition in this window.

/consumer/maximum_sync_time
Maximum Sync Time

GAUGE, DOUBLE, ns

Maximum time spent in a single sync.

/consumer/outgoing_bytes
Outgoing Bytes

CUMULATIVE, INT64, By

Total number of outgoing bytes sent to all servers.

/consumer/request_count
Request Count

CUMULATIVE, INT64, Requests

Total number of requests sent.

/consumer/response_count
Response Count

CUMULATIVE, INT64, n/a

Total number of responses received.

/consumer/sync_count
Sync Count

CUMULATIVE, INT64, n/a

Total number of group syncs.

/producer/average_request_latency
Average Request Latency

GAUGE, DOUBLE, ns

Average latency of requests.

/producer/buffer_memory/available_bytes
Buffer Memory Available Bytes

GAUGE, INT64, By

Total amount of buffer memory that is not being used (either unallocated or in the free list).

/producer/buffer_memory/total_bytes
Buffer Memory Total Bytes

GAUGE, INT64, By

Maximum amount of buffer memory the client can use (whether or not it is currently used).

/producer/connection_closed_count
Connection Closed Count

CUMULATIVE, INT64, Connections

Total number of connections closed.

/producer/connection_count
Connection Count

GAUGE, INT64, Connections

Number of connections.

/producer/connection_created_count
Connection Created Count

CUMULATIVE, INT64, Connections

Total number of new connections established.

/producer/incoming_bytes
Incoming Bytes

CUMULATIVE, INT64, By

Total number of bytes read off all sockets.

/producer/io/time
I/O Time

CUMULATIVE, INT64, ns

Total time the I/O thread spent doing I/O.

/producer/io/wait_time
I/O Wait Time

CUMULATIVE, INT64, ns

Total time the I/O thread spent waiting.

/producer/io/average_wait_time
Average IO Wait Time

GAUGE, DOUBLE, ns

Average length of time the I/O thread spent waiting for a socket ready for reads or writes.

/producer/io/time_ratio
IO Time Ratio

GAUGE, DOUBLE, %

Fraction of time the I/O thread spent doing I/O.

/producer/io/wait_time_ratio
IO Wait Time Ratio

GAUGE, DOUBLE, %

Fraction of time the I/O thread spent waiting.

/producer/maximum_request_latency
Maximum Request Latency

GAUGE, DOUBLE, ns

Maximum latency experienced by a single request.

/producer/outgoing_bytes
Outgoing Bytes

CUMULATIVE, INT64, By

Total number of outgoing bytes sent to all servers.

/producer/record/error_count
Record Error Count

CUMULATIVE, INT64, Errors

Total number of record sends that resulted in errors.

/producer/record/retry_count
Record Retry Count

CUMULATIVE, INT64, n/a

Total number of retried record sends.

/producer/record/send_count
Record Send Count

CUMULATIVE, INT64, Records

Total number of records sent.

/producer/request_count
Request Count

CUMULATIVE, INT64, Requests

Total number of requests sent.

/producer/response_count
Response Count

CUMULATIVE, INT64, n/a

Total number of responses received.

/producer/waiting_thread_count
Waiting Thread Count

GAUGE, INT64, Threads

Number of user threads blocked waiting for buffer memory to enqueue their records.