Aerospike

Data Collection Setup

Aerospike metrics are collected via interaction with the Aerospike Java Client. Detail and statistical metrics for the Cluster, Namespaces, the Nodes, and the Secondary Indexes and Sets on the Nodes are collected.

Network Requirements

Port: 3000

The service port used by Aerospike DB and XDR

There are several default ports related to Aerospike DB. For more information about these, please refer to the Aerospike Network Configuration documentation.

Least Privileged User

The requirement for an LPU is a user that only has read permissions. For more information, please read the Aerospike documentation around Configuring Access Control

🚧

Access Control for Aerospike Databases

Access Control is only available with the Enterprise edition of Aerospike

Supported Versions

Aerospike DB: 3.9+

Connection Parameters

Name

Required?

Description

Host

Required

Aerospike node to connect to.

Port

The port for communication to the Aerospike Server.

SSL Configuration

The SSL mode to use when connecting to the target. Can be configured to not use SSL (No SSL), use SSL but do not verify the target's certificate (No Verify), or use SSL and verify the target's certificate (Verify).

Username

Password

Collect Sets

Whether to collect set resources from Aerospike nodes.

Maximum Parallel Requests

Maximum number of requests to make to the Aerospike nodes at one time.

Connection Parameters

Name

Required?

Description

Host

Required

Aerospike node to connect to.

Port

The port for communication to the Aerospike Server.

SSL Configuration

The SSL mode to use when connecting to the target. Can be configured to not use SSL (No SSL), use SSL but do not verify the target's certificate (No Verify), or use SSL and verify the target's certificate (Verify).

Username

Password

Collect Sets

Whether to collect set resources from Aerospike nodes.

Maximum Parallel Requests

Maximum number of requests to make to the Aerospike nodes at one time.

Metrics

Cluster

Name

Description

All Nodes Healthy

Whether none of the nodes in the cluster are faulty or dead.

Key

Random value generated by Aerospike to identify the cluster.

Maximum Clock Skew (Milliseconds)

The maximum clock skew between nodes in the cluster.

Nodes Quiesced

Number of nodes in the cluster which are quiesced.

Size

Number of nodes in the cluster.

Namespace

Name

Description

Available Bin Names

Remaining number of unique bins that can be created.

Average Query Aggregation Response Size (Records)

Average number of records returned by the underlying query of query aggregations.

Average Secondary Index Query Response Size (Records)

Average number of records returned by secondary index query lookups.

Basic Scan Errors

Number of failed basic scans.

Basic Scans Aborted

Number of basic scans which were aborted.

Basic Scans Completed

Number of basic scans which were completed.

Batch Index Proxied Sub Transaction Errors (Errors)

Number of proxied batch index sub transactions which failed with an error.

Batch Index Proxied Sub Transaction Timeouts

Number of proxied batch index sub transactions which timed out.

Batch Index Proxied Sub Transactions Completed

Number of proxied batch index sub transactions completed.

Batch Index Read Errors

Number of batch index read transactions which failed with an error.

Batch Index Read Not Found Errors (Errors)

Number of batch index read sub transactions which failed with a 'record not found' error.

Batch Index Read Successes (Records)

Number of records successfully read by batch index sub transactions.

Batch Index Read Timeouts

Number of batch index read sub transactions which timed out.

Batch Index Transaction Errors (Errors)

Number of batch index sub transactions which failed in the transaction service.

Batch Index Transaction Timeouts

Number of batch index sub transactions which timed out in the transaction service.

Cache Read Ratio (%)

Ratio of read transactions which are hit the post-write queue.

Client Delete Errors (Errors)

Number of client delete transactions which failed with an error.

Client Delete Not Found Errors (Errors)

Number of client delete transactions which failed with a 'record not found' error.

Client Delete Successes (Transactions)

Number of successful client delete transactions.

Client Delete Timeouts (Transactions)

Number of client delete transactions which resulted in a timeout.

Client Proxy Timeouts (Transactions)

Number of successful client-initiated proxy transactions which completed.

Client Proxy Transaction Errors (Errors)

Number of successful client-initiated proxy transactions which failed with an error.

Client Proxy Transactions Completed (Transactions)

Number of successful client-initiated proxy transactions which completed.

Client Read Errors (Errors)

Number of client-initiated read transactions which failed with an error.

Client Read Successes (Transactions)

Number of successful client-initiated read transactions.

Client Read Timeouts (Transactions)

Number of client-initiated read transactions which timed out.

Client Transaction Errors (Errors)

Number of client-initiated transactions which failed with an error.

Client Transaction Timeouts (Transactions)

Number of client-initiated transactions which timed out.

Client UDF Delete Successes (Transactions)

Number of client-initiated UDF transactions which successfully deleted a record.

Client UDF Errors (Errors)

Number of client-initiated UDF transactions which failed due to a Lua error in execution.

Client UDF Read Successes (Transactions)

Number of successful client-initiated UDF read transactions.

Client UDF Transaction Errors (Errors)

Number of client-initiated UDF transactions which failed with an error, not including timeouts.

Client UDF Transaction Timeouts (Transactions)

Number of client-initiated UDF transactions which timed out.

Client UDF Transactions Completed (Transactions)

Number of completed client-initiated UDF transactions.

Client UDF Write Successes (Transactions)

Number of successful client-initiated UDF write transactions.

Client Write Errors (Errors)

Number of client-initiated write transactions which failed with an error.

Client Write Successes (Transactions)

Number of successful client-initiated write transactions.

Client Write Timeouts (Transactions)

Number of client-initiated write transactions which timed out.

Contiguous Free Device Space Ratio (%)

Amount of contiguous free blocks across all devices in the namespace, in proportion to all blocks across all devices in the namespace.

Data Memory Usage (Bytes)

Amount of memory used data of the namespace on its node.

Dead Partitions

Number of dead partitions in the namespace.

Deletes (Transactions)

Delete operations performed in the namespace.

Device Compression Ratio

Moving average of the ratio of compressed size to uncompressed size.

Disk Space Available (Bytes)

Disk space allocated to the namespace on its node.

Disk Space Used (Bytes)

Disk space used by the namespace on its node.

Effective Replication Factor

Effective replication factor of the namespace.

Emigration Instance Objects

Number of instance objects managing emigrations.

Emigration Partitions

Number of partitions currently emigrating from the namespace on its node.

Evictions

Number of objects that have been evicted from the namespace on its node since the server started.

Expirations

Number of objects that have expired from the namespace on its node since the server started.

Failed UDF Queries

Number of UDF background queries which failed.

Free Device Space Ratio (%)

Amount of free space across all devices in the namespace, in proportion to all space across all devices in the namespace.

Generation Check Failures

Number of read/write transactions which failed on generation check.

High Watermark Exceeded

Whether the memory or disk high watermark is exceeded. When true, nodes will begin to evict data.

Hot Key Failures

Number of read/write transactions which failed due to too many other transactions waiting to operate on the same record.

Immigration Instance Objects

Number of instance objects managing immigrations.

Immigration Partitions

Number of partitions currently immigrating to the namespace on its node.

Index Memory Usage (Bytes)

Amount of memory used by the index of the namespace on its node.

Initial Emigration Partitions

Number of migrations the node will send during the current migration cycle for the namespace.

Initial Immigration Partitions

Number of migrations the node will receive during the current migration cycle for the namespace.

Lead Migrations Remaining

Number of initially scheduled emigrations which are not delayed by the 'migrate-fill-delay' configuration.

Long Running Queries

Number of long running queries in progress.

Long Running Query Queue Full Errors (Errors)

Number of long running queries which encountered queue full errors.

Master Objects (Records)

Number of records in the namespace on its node which are active masters.

Master Sub-Objects

Number of LDT sub-records in the namespace on its node which are active masters.

Master Tombstones

Number of tombstones in the namespace on its node which are active masters.

Memory Free (%)

Amount of free memory for the namespace in proportion to its memory capacity.

Memory Usage (Bytes)

Amount of memory used by the namespace on its node.

Migration Imbalance Failures

Number of migration failures which may cause partitions to be imbalanced.

Migration Records Received (Records)

Record insert requests received for the namespace on its node by immigration.

Migration Records Retransmitted (Records)

Records retransmitted by emigration.

Migration Records Sent (Records)

Records read and sent by emigration.

Migration Records Skipped (Records)

Records that were not shipped due to the remote node already being up to date.

Name

Name of the namespace.

Namespace Supervisor Cycle Duration (Milliseconds)

Duration of the most recent namespace supervisor cycle.

Node

Name of the node containing the namespace.

Non-Expirable Objects (Records)

Number of records with non-expirable TTLs.

Non-Replica Objects (Records)

Number of records which are neither masters nor replicas.

Non-Replica Sub-Objects (Records)

Number of LDT sub-records which are neither masters nor replicas.

Non-Replica Tombstones

Number of tombstones which are neither masters nor replicas.

Objects (Records)

Number of records in the namespace, not including tombstones.

Query Aggregation Errors (Errors)

Number of query aggregations attempted on the node which resulted in an error due to an internal error.

Query Aggregation Successes

Number of successful query aggregations completed on the node.

Query Aggregations

Number of query aggregations attempted on the node.

Query Aggregations Aborted

Number of query aggregations aborted by the user and seen by the node.

Query Errors (Errors)

Number of queries attempted on the node which resulted in an error due to an internal error.

Query Requests (Requests)

Number of query requests attempted on the node.

Quiesce Pending

Whether the node has received the 'quiesce' command but has not yet quiesced.

Quiesced Nodes

Number of Nodes which are quiesced.

Reads (Transactions)

Read operations performed in the namespace.

Record Too Big Failures

Number of write transactions which failed due to the record size being above the 'write-block-size' limit.

Records Deleted by Set (Records)

Number of records deleted by a set.

Remaining Emigration Partitions

Number of migrations not yet sent by the node for the namespace's current migration cycle.

Remaining Immigration Partitions

Number of migrations not yet received by the node for the namespace's current migration cycle.

Replica Objects (Records)

Number of records which are replicas on the node.

Replica Sub-Objects (Records)

Number of LDT sub-records which are replicas on the node.

Replica Tombstones

Number of tombstones which are replicas on the node.

Scan Aggregation Errors

Number of failed scan aggregations.

Scan Aggregations Aborted

Number of scan aggregations which were aborted.

Scan Aggregations Completed

Number of scan aggregations which were completed.

Secondary Index Memory Usage (Bytes)

Amount of memory used by the secondary index of the namespace on its node.

Secondary Index Queries

Number of secondary index query lookups attempted on the node.

Secondary Index Queries Aborted

Number of secondary index queries aborted by the user.

Secondary Index Query Errors (Errors)

Number of secondary index query lookup errors.

Secondary Index Query Successes

Number of successful secondary index query lookups.

Short Running Queries

Number of short running queries in progress.

Short Running Query Queue Full Errors (Errors)

Number of short running queries which encountered queue full errors.

Sub-Objects

Number of LDT sub-objects.

Successful UDF Queries

Number of successful UDF background queries.

Tombstones

Number of tombstones in the namespace on its node.

Truncated Records

Number of records which were deleted by truncation in the namespace.

UDF Background Scan Errors

Number of failed UDF background scans.

UDF Background Scans Aborted

Number of UDF background scans which were aborted.

UDF Background Scans Completed

Number of UDF background scans which were completed.

UDF Delete Sub-Transaction Successes

Number of successful UDF delete sub-transactions.

UDF Sub-Transaction Errors (Errors)

Number of UDF sub-transaction errors.

UDF Sub-Transaction Failures (Errors)

Number of failed UDF sub-transactions, excluding timeouts.

UDF Read Sub-Transaction Successes

Number of successful UDF read sub-transactions.

UDF Sub-Transaction Timeouts

Number of UDF sub-transactions which timed out.

UDF Sub-Transaction Transaction Errors (Errors)

Number of UDF sub-transactions which failed with an error in the transaction service.

UDF Sub-Transaction Transaction Timeouts

Number of UDF sub-transactions which timed out in the transaction service.

UDF Write Sub-Transaction Successes

Number of successful UDF write sub-transactions.

UDF Sub-Transactions Completed

Number of completed UDF sub-transactions.

Writes (Transactions)

Write operations performed in the namespace.

Writes Stopped

Whether writes are currently disallowed for the namespace.

Node

Name

Description

Address

Address of the node.

Batch Index Buffers Available

Number of batch index response buffers currently in the buffer pool and not in use.

Batch Index Buffers Created

Number of 128KB response buffers in response to batch index transactions.

Batch Index Buffers Destroyed

Number of batch index response buffers that have been destroyed due to the buffer pool size being above its limit.

Batch Index Delays

Number of times a batch index response has been delayed.

Batch Index Errors (Errors)

Number of batch index requests which completed with an error.

Batch Index Huge Buffers Created

Number of created batch index response buffers that were larger than 128KB.

Batch Index Requests (Requests)

Number of batch index requests received.

Batch Index Requests Completed (Requests)

Number of batch index requests completed by the node.

Batch Index Timeouts

Number of batch index requests which timed out on the node before being processed.

Client Connections (Connections)

Number of clients currently connected to the node.

Closed Idle Connections (Connections)

Number of idle connections closed by the node.

Deletes (Transactions)

Delete operations performed on the node.

Fabric Bulk Channel Received Throughput (Bytes per Second)

Average data receive rate by the fabric bulk channel (migrations), over the configured ticker interval.

Fabric Bulk Channel Sent Throughput (Bytes per Second)

Average data send rate by the fabric bulk channel (migrations), over the configured ticker interval.

Fabric Connections (Connections)

Number of open fabric connections to the node.

Fabric Ctrl Channel Received Throughput (Bytes per Second)

Average data receive rate by the fabric ctrl channel (paxos and migration control), over the configured ticker interval.

Fabric Ctrl Channel Sent Throughput (Bytes per Second)

Average data send rate by the fabric ctrl channel (paxos and migration control), over the configured ticker interval.

Fabric Meta Channel Received Throughput (Bytes per Second)

Average data receive rate by the fabric meta channel (system metadata), over the configured ticker interval.

Fabric Meta Channel Sent Throughput (Bytes per Second)

Average data send rate by the fabric meta channel (system metadata), over the configured ticker interval.

Fabric RW Channel Received Throughput (Bytes per Second)

Average data receive rate by the fabric rw channel (read/write transactions), over the configured ticker interval.

Fabric RW Channel Sent Throughput (Bytes per Second)

Average data send rate by the fabric rw channel (read/write transactions), over the configured ticker interval.

Heap Active (Kibibytes)

Memory allocated to in-use pages.

Heap Allocated (Kibibytes)

Memory allocated to the asd daemon.

Heap Efficiency (%)

Ratio of the size of allocated heap space to the process's resident set size.

Heap RSS (Kibibytes)

Amount of memory allocated to the process which is in RAM.

Heartbeat Connections (Connections)

Number of active heartbeat connections to the node.

Heartbeats Received

Number of heartbeats received by the node.

Heartbeats Sent

Number of heartbeats multicast by the node.

Info Requests Completed

Number of info requests completed.

Info Requests Queued

Number of info requests pending in the queue.

Long-Running Queries

Number of long-running queries that have been attempted on the node.

Memory Free (%)

Amount of free memory in proportion to the total memory available.

Migration Partitions Remaining

Number of partitions remaining to migrate.

Migrations Allowed

Whether migrations are allowed to run on the node. Will be false if a rebalance is in progress.

Name

Server node name.

Objects

Number of replicated objects on the node.

Pending Transactions (Transactions)

Number of requests pending in the transaction queue on the node.

Proxies

Number of proxies in progress on the node.

Read/Write Transactions In Progress (Transactions)

Number of read/write transactions in progress on the node.

Reads (Transactions)

Read operations performed on the node.

Scans (Transactions)

Number of scans in progress on the node.

Secondary Index Entries Processed

Number of entries processed by the secondary index garbage collector.

Secondary Index Garbage Cleaned

Number of secondary index garbage entries cleaned by its garbage collector.

Secondary Index Garbage Deletion Time (Milliseconds)

Time spent cleaning secondary index garbage entries.

Secondary Index Garbage Found

Number of secondary index garbage entries found by its garbage collector.

Short-Running Queries

Number of short-running queries that have been attempted on the node.

Time Since Rebalance (Seconds)

Time since the last reclustering event.

Tombstones

Number of tombstones on the node.

Version

Version of Aerospike running on the node.

Writes (Transactions)

Write operations performed on the node.

Secondary Index

Name

Description

Aggregation Queries

Aggregation queries attempted for the secondary index on its node.

Average Lookup Query Record Size

Average size of records returned by lookup queries against the secondary index.

Average Lookup Query Response Size (Records)

Average number of records returned by lookup queries against the secondary index.

Average Query Aggregation Record Size

Average size of records returned by the underlying query of query aggregations.

Average Query Aggregation Response Size (Records)

Average number of records returned by the underlying query of query aggregations.

Average Query Record Size

Average size of records returned by queries against the secondary index.

Average Query Response Size (Records)

Average number of records returned by queries against the secondary index.

Collection Type

Collection type of the secondary index.

Creation Duration (Milliseconds)

How long it took to fully create the secondary index.

Creation Progress (%)

How much progress creation of the secondary index has made.

Delete Errors (Errors)

Errors encountered while processing delete transactions.

Delete Successes (Transactions)

Number of successful delete transactions.

Entries (Records)

Number of records indexed by the secondary index.

Entry Size (Bytes)

Memory consumed by the entries of the secondary index.

Garbage Collected Records (Records)

Number of records which have been removed from memory by garbage collection.

Garbage Collection Duration

Time spent in garbage collection for the secondary index.

Key Size (Bytes)

Memory consumed by the keys of the secondary index.

Keys

Number of secondary keys in the secondary index.

Lookup Queries

Number of lookup queries attempted against the secondary index.

Name

Name of the secondary index.

Namespace

Namespace of the secondary index.

Node

Node of the secondary index.

Queries

Number of queries attempted against the secondary index.

Set

Set that the secondary index is for.

Size (Bytes)

Amount of memory consumed by the secondary index on its node.

State

State of the secondary index.

Successful Writes (Transactions)

Number of successful write transactions for the secondary index.

Type

Data type of the secondary index.

Write Errors (Errors)

Number of errors encountered during write transactions for the secondary index.

Set

Name

Description

Data Size (Bytes)

Memory used by the data part of the set. Will be 0 if data is not configured for storage in memory.

Name

Name of the set.

Namespace

Namespace containing the set.

Node

Name of the node containing the set's namespace.

Objects

Number of objects in the set on its node, including master and all replicas.

Tombstones

Number of tombstones in the set on its node, including master and all replicas.

Truncate Last Update Time

Broadest truncation 'last update time' for the set.