Bindplane

The Bindplane Developer Hub

Welcome to the Bindplane developer hub. You'll find comprehensive guides and documentation to help you start working with Bindplane as quickly as possible, as well as support if you get stuck. Let's jump right in!

Apache Hadoop

Monitor Apache Hadoop Database Platform

Data Collection Setup

Metrics are collected via the JMX interface on the NameNode and ResourceManager servers for the following Hadoop components:

  • Nodes
  • HDFS (Hadoop Distributed File System)
  • YARN (Yet Another Resource Negotiator)

Network Requirements

Port: 50070 (TCP) Default to the NameNode’s JMX endpoint
Port: 8088 (TCP) Default to the ResourceManager’s JMX endpoint

Least Privileged User

N/A (JMX is read-only.)

Supported Versions

Apache Hadoop: 2.7.x +, with the following modules:

  • HDFS
  • YARN

Connection Parameters

NameRequired?Description
Name Node HostRequiredThe Name Node to connect to.
Name Node PortThe port for communication to the Name Node.
Resource Manager HostRequiredThe Resource Manager to connect to.
Resource Manager PortThe port for communication to the Resource Manager.
Collection ModeStandard collection mode will collect all data. Light collection mode will collect only the main services without their sub-services.
Request Timeout (seconds)The number of seconds to allow for the API to return a response.
Maximum ThreadsMaximum number of threads to spawn in addition to the required 2 primary collection threads. You have at least one additional thread.

Metrics

Cluster

Name
Node Managers Active
Node Managers Decommissioned
Node Managers Lost
Node Managers Rebooted
Node Managers Unhealthy
Sum of Containers Completed from Node Manager Descendants
Sum of Containers Failed from Node Manager Descendants
Sum of Containers Initializing from Node Manager Descendants
Sum of Containers Killed from Node Manager Descendants
Sum of Containers Launched from Node Manager Descendants
Sum of Containers Running from Node Manager Descendants
Sum of Users Active from Resource Manager Descendants
Unique Child Hostnames

Data Node

NameDescription
Blocks Read (Blocks)Attribute BlocksRead from Bean Hadoop:service=DataNode,name=DataNodeActivity-.*
Blocks Removed (Blocks)Attribute BlocksRemoved from Bean Hadoop:service=DataNode,name=DataNodeActivity-.*
Blocks Replicated (Blocks)Attribute BlocksReplicated from Bean Hadoop:service=DataNode,name=DataNodeActivity-.*
Blocks Written (Blocks)Attribute BlocksWritten from Bean Hadoop:service=DataNode,name=DataNodeActivity-.*
Cache Capacity (Gigabytes)Attribute CacheCapacity from Bean Hadoop:service=DataNode,name=FSDatasetState-.*
Cache Used (Gigabytes)Attribute CacheUsed from Bean Hadoop:service=DataNode,name=FSDatasetState-.*
Cached (Blocks)Attribute NumBlocksCached from Bean Hadoop:service=DataNode,name=FSDatasetState-.*
Capacity Lost Due To Volume Failure (Gigabytes)Attribute EstimatedCapacityLostTotal from Bean Hadoop:service=DataNode,name=FSDatasetState-.*
Capacity Used (Gigabytes)Capacity Used
Capacity Used Ratio (%)Capacity Used Ratio
Current Heap Memory Committed (Mebibytes)Attribute MemHeapCommittedM from Bean Hadoop:service=\w+,name=JvmMetrics
Current Heap Memory Used (Mebibytes)Attribute MemHeapUsedM from Bean Hadoop:service=\w+,name=JvmMetrics
Current Non-heap Memory Committed (Mebibytes)Attribute MemNonHeapCommittedM from Bean Hadoop:service=\w+,name=JvmMetrics
Current Non-heap Memory Used (Mebibytes)Attribute MemNonHeapUsedM from Bean Hadoop:service=\w+,name=JvmMetrics
Data Read (Gigabytes)Attribute BytesRead from Bean Hadoop:service=DataNode,name=DataNodeActivity-.*
Data Written (Gigabytes)Attribute BytesWritten from Bean Hadoop:service=DataNode,name=DataNodeActivity-.*
Disk Capacity (Gigabytes)Attribute Capacity from Bean Hadoop:service=DataNode,name=FSDatasetState-.*
Disk Remaining (Gigabytes)Attribute Remaining from Bean Hadoop:service=DataNode,name=FSDatasetState-.*
Distributed File System Used (Gigabytes)Attribute DfsUsed from Bean Hadoop:service=DataNode,name=FSDatasetState-.*
Failed To Cache (Blocks)Attribute NumBlocksFailedToCache from Bean Hadoop:service=DataNode,name=FSDatasetState-.*
Failed To Uncache (Blocks)Attribute NumBlocksFailedToUncache from Bean Hadoop:service=DataNode,name=FSDatasetState-.*
Failed VolumesAttribute NumFailedVolumes from Bean Hadoop:service=DataNode,name=FSDatasetState-.*
Free Physical Memory (Gibibytes)Attribute FreePhysicalMemorySize from Bean java.lang:type=OperatingSystem
Garbage Collection CountAttribute GcCount from Bean Hadoop:service=\w+,name=JvmMetrics
Garbage Collection Count Mark Sweep CompactAttribute GcCountMarkSweepCompact from Bean Hadoop:service=\w+,name=JvmMetrics
Garbage Collection Time (Milliseconds)Attribute GcTimeMillis from Bean Hadoop:service=\w+,name=JvmMetrics
Garbage Collection Time Mark Sweep Compact (Milliseconds)Attribute GcTimeMillisMarkSweepCompact from Bean Hadoop:service=\w+,name=JvmMetrics
Group Resolution Average Time (Milliseconds)Attribute GetGroupsAvgTime from Bean Hadoop:service=\w+,name=UgiMetrics
Group ResolutionsAttribute GetGroupsNumOps from Bean Hadoop:service=\w+,name=UgiMetrics
HostnameHostname
JVM VersionAttribute SystemProperties from Bean java.lang:type=Runtime
Kerberos Login Failure Average Time (Milliseconds)Attribute LoginFailureAvgTime from Bean Hadoop:service=\w+,name=UgiMetrics
Kerberos Login FailuresAttribute LoginFailureNumOps from Bean Hadoop:service=\w+,name=UgiMetrics
Kerberos Login Success Average Time (Milliseconds)Attribute LoginSuccessAvgTime from Bean Hadoop:service=\w+,name=UgiMetrics
Kerberos Login SuccessesAttribute LoginSuccessNumOps from Bean Hadoop:service=\w+,name=UgiMetrics
Last Volume Failure Date (from Unix epoch) (Milliseconds)Attribute LastVolumeFailureDate from Bean Hadoop:service=DataNode,name=FSDatasetState-.*
Maximum Heap Memory Size (Mebibytes)Attribute MemHeapMaxM from Bean Hadoop:service=\w+,name=JvmMetrics
Maximum Memory Size (Mebibytes)Attribute MemMaxM from Bean Hadoop:service=\w+,name=JvmMetrics
Maximum Non-heap Memory Size (Mebibytes)Attribute MemNonHeapMaxM from Bean Hadoop:service=\w+,name=JvmMetrics
Metric EndpointMetric Endpoint
OS NameAttribute Name from Bean java.lang:type=OperatingSystem
OS VersionAttribute Version from Bean java.lang:type=OperatingSystem
Physical Memory Utilization (%)Physical Memory Utilization
Process CPU Load (%)Attribute ProcessCpuLoad from Bean java.lang:type=OperatingSystem
Publish Average Time (Milliseconds)Attribute PublishAvgTime from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Publish Operations (Operations)Attribute PublishNumOps from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Publishes DroppedAttribute DroppedPubAll from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Sinks TotalAttribute NumAllSinks from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Snapshot Average Time (Milliseconds)Attribute SnapshotAvgTime from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Snapshot Operations (Operations)Attribute SnapshotNumOps from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Sources ActiveAttribute NumActiveSources from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Sources TotalAttribute NumAllSources from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Storage InfoAttribute StorageInfo from Bean Hadoop:service=DataNode,name=FSDatasetState-.*
System CPU Load (%)Attribute SystemCpuLoad from Bean java.lang:type=OperatingSystem
Tag HostnameAttribute tag.Hostname from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Threads In BLOCKED StateAttribute ThreadsBlocked from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In NEW StateAttribute ThreadsNew from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In RUNNABLE StateAttribute ThreadsRunnable from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In TERMINATED StateAttribute ThreadsTerminated from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In TIMED_WAITING StateAttribute ThreadsTimedWaiting from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In WAITING StateAttribute ThreadsWaiting from Bean Hadoop:service=\w+,name=JvmMetrics
Total ERROR Log MessagesAttribute LogError from Bean Hadoop:service=\w+,name=JvmMetrics
Total FATAL Log MessagesAttribute LogFatal from Bean Hadoop:service=\w+,name=JvmMetrics
Total INFO Log MessagesAttribute LogInfo from Bean Hadoop:service=\w+,name=JvmMetrics
Total Physical Memory (Gibibytes)Attribute TotalPhysicalMemorySize from Bean java.lang:type=OperatingSystem
Total WARN Log MessagesAttribute LogWarn from Bean Hadoop:service=\w+,name=JvmMetrics
Used Physical Memory (Gibibytes)Used Physical Memory
VersionAttribute Version from Bean Hadoop:service=DataNode,name=DataNodeInfo

Name Node

NameDescription
Active NodesIsolated hostnames in LiveNodes entry from Bean Hadoop:service=NameNode,name=NameNodeInfo
Add Block Operations (Operations)Attribute AddBlockOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Allow Snapshot Operations (Operations)Attribute AllowSnapshotOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Block Capacity (Blocks)Attribute BlockCapacity from Bean Hadoop:service=NameNode,name=FSNamesystem
Block Deletion Start Date (from Unix epoch) (Milliseconds)Attribute BlockDeletionStartTime from Bean Hadoop:service=NameNode,name=FSNamesystemState
Block Operations Batched (Operations)Attribute BlockOpsBatched from Bean Hadoop:service=NameNode,name=NameNodeActivity
Block Operations Queued (Operations)Attribute BlockOpsQueued from Bean Hadoop:service=NameNode,name=NameNodeActivity
Block Pool Used Space (Gigabytes)Attribute BlockPoolUsedSpace from Bean Hadoop:service=NameNode,name=NameNodeInfo
Block Pool Utilization (%)Attribute PercentBlockPoolUsed from Bean Hadoop:service=NameNode,name=NameNodeInfo
Block Received And Deleted Operations (Operations)Attribute BlockReceivedAndDeletedOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Block Report Average TimeAttribute BlockReportAvgTime from Bean Hadoop:service=NameNode,name=NameNodeActivity
Block Report Operations (Operations)Attribute BlockReportNumOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Blocks MissingAttribute NumberOfMissingBlocks from Bean Hadoop:service=NameNode,name=NameNodeInfo
Blocks Missing With Replication Factor OneAttribute NumberOfMissingBlocksWithReplicationFactorOne from Bean Hadoop:service=NameNode,name=NameNodeInfo
Cache Capacity (Gigabytes)Attribute CacheCapacity from Bean Hadoop:service=NameNode,name=NameNodeInfo
Cache Report Average Time (Milliseconds)Attribute CacheReportAvgTime from Bean Hadoop:service=NameNode,name=NameNodeActivity
Cache Report Operations (Operations)Attribute CacheReportNumOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Cache Used (Gigabytes)Attribute CacheUsed from Bean Hadoop:service=NameNode,name=NameNodeInfo
Capacity Lost Due To Volume Failure (Gigabytes)Attribute EstimatedCapacityLostTotal from Bean Hadoop:service=NameNode,name=FSNamesystemState
Capacity Remaining (Gigabytes)Attribute CapacityRemaining from Bean Hadoop:service=NameNode,name=FSNamesystem
Capacity Total (Gigabytes)Attribute CapacityTotal from Bean Hadoop:service=NameNode,name=FSNamesystem
Capacity Used (Gigabytes)Attribute CapacityUsed from Bean Hadoop:service=NameNode,name=FSNamesystem
Capacity Used Non-distributed File System (Gigabytes)Attribute CapacityUsedNonDFS from Bean Hadoop:service=NameNode,name=FSNamesystem
Capacity Used Ratio (%)Capacity Used Ratio
Connections (Connections)Attribute TotalLoad from Bean Hadoop:service=NameNode,name=FSNamesystem
Corrupt (Blocks)Attribute CorruptBlocks from Bean Hadoop:service=NameNode,name=FSNamesystem
Create File Operations (Operations)Attribute CreateFileOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Create Snapshot Operations (Operations)Attribute CreateSnapshotOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Create Symlink Operations (Operations)Attribute CreateSymlinkOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Current Heap Memory Committed (Mebibytes)Attribute MemHeapCommittedM from Bean Hadoop:service=\w+,name=JvmMetrics
Current Heap Memory Used (Mebibytes)Attribute MemHeapUsedM from Bean Hadoop:service=\w+,name=JvmMetrics
Current Non-heap Memory Committed (Mebibytes)Attribute MemNonHeapCommittedM from Bean Hadoop:service=\w+,name=JvmMetrics
Current Non-heap Memory Used (Mebibytes)Attribute MemNonHeapUsedM from Bean Hadoop:service=\w+,name=JvmMetrics
DataNodes DeadAttribute NumDeadDataNodes from Bean Hadoop:service=NameNode,name=FSNamesystemState
DataNodes Decommissioned DeadAttribute NumDecomDeadDataNodes from Bean Hadoop:service=NameNode,name=FSNamesystemState
DataNodes Decommissioned LiveAttribute NumDecomLiveDataNodes from Bean Hadoop:service=NameNode,name=FSNamesystemState
DataNodes DecommissioningAttribute NumDecommissioningDataNodes from Bean Hadoop:service=NameNode,name=FSNamesystemState
DataNodes LiveAttribute NumLiveDataNodes from Bean Hadoop:service=NameNode,name=FSNamesystemState
DataNodes StaleAttribute StaleDataNodes from Bean Hadoop:service=NameNode,name=FSNamesystem
Dead NodesAttribute DeadNodes from Bean Hadoop:service=NameNode,name=NameNodeInfo
Decommissioned NodesAttribute DecomNodes from Bean Hadoop:service=NameNode,name=NameNodeInfo
Delete File Operations (Operations)Attribute DeleteFileOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Delete Snapshot Operations (Operations)Attribute DeleteSnapshotOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Disallow Snapshot Operations (Operations)Attribute DisallowSnapshotOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Distinct Version CountAttribute DistinctVersionCount from Bean Hadoop:service=NameNode,name=NameNodeInfo
Excess (Blocks)Attribute ExcessBlocks from Bean Hadoop:service=NameNode,name=FSNamesystem
Expired HeartbeatsAttribute ExpiredHeartbeats from Bean Hadoop:service=NameNode,name=FSNamesystem
File Info Operations (Operations)Attribute FileInfoOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
File System Image Load Time (Milliseconds)Attribute FsImageLoadTime from Bean Hadoop:service=NameNode,name=NameNodeActivity
File System StateAttribute FSState from Bean Hadoop:service=NameNode,name=FSNamesystemState
Files AppendedAttribute FilesAppended from Bean Hadoop:service=NameNode,name=NameNodeActivity
Files CreatedAttribute FilesCreated from Bean Hadoop:service=NameNode,name=NameNodeActivity
Files DeletedAttribute FilesDeleted from Bean Hadoop:service=NameNode,name=NameNodeActivity
Files In Get Listing Operations (Operations)Attribute FilesInGetListingOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Files RenamedAttribute FilesRenamed from Bean Hadoop:service=NameNode,name=NameNodeActivity
Files TotalAttribute FilesTotal from Bean Hadoop:service=NameNode,name=FSNamesystem
Files TruncatedAttribute FilesTruncated from Bean Hadoop:service=NameNode,name=NameNodeActivity
Free Physical Memory (Gibibytes)Attribute FreePhysicalMemorySize from Bean java.lang:type=OperatingSystem
Garbage Collection CountAttribute GcCount from Bean Hadoop:service=\w+,name=JvmMetrics
Garbage Collection Count Mark Sweep CompactAttribute GcCountMarkSweepCompact from Bean Hadoop:service=\w+,name=JvmMetrics
Garbage Collection Time (Milliseconds)Attribute GcTimeMillis from Bean Hadoop:service=\w+,name=JvmMetrics
Garbage Collection Time Mark Sweep Compact (Milliseconds)Attribute GcTimeMillisMarkSweepCompact from Bean Hadoop:service=\w+,name=JvmMetrics
Get Additional DataNode Operations (Operations)Attribute GetAdditionalDatanodeOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Get Block LocationsAttribute GetBlockLocations from Bean Hadoop:service=NameNode,name=NameNodeActivity
Get Edit Average Time (Milliseconds)Attribute GetEditAvgTime from Bean Hadoop:service=NameNode,name=NameNodeActivity
Get Edit Operations (Operations)Attribute GetEditNumOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Get Image Average Time (Milliseconds)Attribute GetImageAvgTime from Bean Hadoop:service=NameNode,name=NameNodeActivity
Get Image Operations (Operations)Attribute GetImageNumOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Get Link Target Operations (Operations)Attribute GetLinkTargetOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Get Listing Operations (Operations)Attribute GetListingOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Group Resolution Average Time (Milliseconds)Attribute GetGroupsAvgTime from Bean Hadoop:service=\w+,name=UgiMetrics
Group ResolutionsAttribute GetGroupsNumOps from Bean Hadoop:service=\w+,name=UgiMetrics
Hadoop Detailed VersionAttribute Version from Bean Hadoop:service=NameNode,name=NameNodeInfo
Hadoop VersionAttribute SoftwareVersion from Bean Hadoop:service=NameNode,name=NameNodeInfo
High Availability StateAttribute tag.HAState from Bean Hadoop:service=NameNode,name=FSNamesystem
HostnameHostname
JVM VersionAttribute SystemProperties from Bean java.lang:type=Runtime
Kerberos Login Failure Average Time (Milliseconds)Attribute LoginFailureAvgTime from Bean Hadoop:service=\w+,name=UgiMetrics
Kerberos Login FailuresAttribute LoginFailureNumOps from Bean Hadoop:service=\w+,name=UgiMetrics
Kerberos Login Success Average Time (Milliseconds)Attribute LoginSuccessAvgTime from Bean Hadoop:service=\w+,name=UgiMetrics
Kerberos Login SuccessesAttribute LoginSuccessNumOps from Bean Hadoop:service=\w+,name=UgiMetrics
Last Checkpoint Date (from Unix epoch) (Milliseconds)Attribute LastCheckpointTime from Bean Hadoop:service=NameNode,name=FSNamesystem
Last High Availability Transition Time (Milliseconds)Attribute LastHATransitionTime from Bean Hadoop:service=NameNode,name=NameNodeStatus
Last Written Transaction IDAttribute LastWrittenTransactionId from Bean Hadoop:service=NameNode,name=FSNamesystem
List Snapshottable Directory Operations (Operations)Attribute ListSnapshottableDirOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Maximum Heap Memory Size (Mebibytes)Attribute MemHeapMaxM from Bean Hadoop:service=\w+,name=JvmMetrics
Maximum Memory Size (Mebibytes)Attribute MemMaxM from Bean Hadoop:service=\w+,name=JvmMetrics
Maximum Non-heap Memory Size (Mebibytes)Attribute MemNonHeapMaxM from Bean Hadoop:service=\w+,name=JvmMetrics
Maximum ObjectsAttribute MaxObjects from Bean Hadoop:service=NameNode,name=FSNamesystemState
Metric EndpointMetric Endpoint
Missing (Blocks)Attribute MissingBlocks from Bean Hadoop:service=NameNode,name=FSNamesystem
Missing With Replication Factor One (Blocks)Attribute MissingReplOneBlocks from Bean Hadoop:service=NameNode,name=FSNamesystem
Non-distributed File System Used Space (Gigabytes)Attribute NonDfsUsedSpace from Bean Hadoop:service=NameNode,name=NameNodeInfo
OS NameAttribute Name from Bean java.lang:type=OperatingSystem
OS VersionAttribute Version from Bean java.lang:type=OperatingSystem
Pending DataNode MessagesAttribute PendingDataNodeMessageCount from Bean Hadoop:service=NameNode,name=FSNamesystem
Pending Deletion (Blocks)Attribute PendingDeletionBlocks from Bean Hadoop:service=NameNode,name=FSNamesystem
Pending Replication (Blocks)Attribute PendingReplicationBlocks from Bean Hadoop:service=NameNode,name=FSNamesystem
Physical Memory Utilization (%)Physical Memory Utilization
Postponed Misreplicated Blocks (Blocks)Attribute PostponedMisreplicatedBlocks from Bean Hadoop:service=NameNode,name=FSNamesystem
Process CPU Load (%)Attribute ProcessCpuLoad from Bean java.lang:type=OperatingSystem
Publish Average Time (Milliseconds)Attribute PublishAvgTime from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Publish Operations (Operations)Attribute PublishNumOps from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Publishes DroppedAttribute DroppedPubAll from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Put Image Average Time (Milliseconds)Attribute PutImageAvgTime from Bean Hadoop:service=NameNode,name=NameNodeActivity
Put Image Operations (Operations)Attribute PutImageNumOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Rename Snapshot Operations (Operations)Attribute RenameSnapshotOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Retry Cache ClearsAttribute CacheCleared from Bean Hadoop:service=NameNode,name=RetryCache.NameNodeRetryCache
Retry Cache HitsAttribute CacheHit from Bean Hadoop:service=NameNode,name=RetryCache.NameNodeRetryCache
Retry Cache UpdatesAttribute CacheUpdated from Bean Hadoop:service=NameNode,name=RetryCache.NameNodeRetryCache
Safe ModeAttribute Safemode from Bean Hadoop:service=NameNode,name=NameNodeInfo
Safe Mode Time (Milliseconds)Attribute SafeModeTime from Bean Hadoop:service=NameNode,name=NameNodeActivity
Scheduled Replication (Blocks)Attribute ScheduledReplicationBlocks from Bean Hadoop:service=NameNode,name=FSNamesystem
Security EnabledAttribute SecurityEnabled from Bean Hadoop:service=NameNode,name=NameNodeStatus
Sinks TotalAttribute NumAllSinks from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Snapshot Average Time (Milliseconds)Attribute SnapshotAvgTime from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Snapshot Diff Report Operations (Operations)Attribute SnapshotDiffReportOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Snapshot Operations (Operations)Attribute SnapshotNumOps from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
SnapshotsAttribute Snapshots from Bean Hadoop:service=NameNode,name=FSNamesystem
Snapshottable DirectoriesAttribute SnapshottableDirectories from Bean Hadoop:service=NameNode,name=FSNamesystem
Sources ActiveAttribute NumActiveSources from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Sources TotalAttribute NumAllSources from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Space Free (Gigabytes)Attribute Free from Bean Hadoop:service=NameNode,name=NameNodeInfo
Space Remaining (%)Attribute PercentRemaining from Bean Hadoop:service=NameNode,name=NameNodeInfo
Space Total (Gigabytes)Attribute Total from Bean Hadoop:service=NameNode,name=NameNodeInfo
Space Used (Gigabytes)Attribute Used from Bean Hadoop:service=NameNode,name=NameNodeInfo
Space Utilization (%)Attribute PercentUsed from Bean Hadoop:service=NameNode,name=NameNodeInfo
Stale StoragesAttribute NumStaleStorages from Bean Hadoop:service=NameNode,name=FSNamesystemState
Startup Completion (%)Attribute PercentComplete from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Elapsed Time (Milliseconds)Attribute ElapsedTime from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Loading Edits Completion (%)Attribute LoadingEditsPercentComplete from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Loading Edits CountAttribute LoadingEditsCount from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Loading Edits Elapsed Time (Milliseconds)Attribute LoadingEditsElapsedTime from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Loading Edits TotalAttribute LoadingEditsTotal from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Loading File System Image Completion (%)Attribute LoadingFsImagePercentComplete from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Loading File System Image CountAttribute LoadingFsImageCount from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Loading File System Image Elapsed Time (Milliseconds)Attribute LoadingFsImageElapsedTime from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Loading File System Image TotalAttribute LoadingFsImageTotal from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Safe Mode Completion (%)Attribute SafeModePercentComplete from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Safe Mode CountAttribute SafeModeCount from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Safe Mode Elapsed Time (Milliseconds)Attribute SafeModeElapsedTime from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Safe Mode TotalAttribute SafeModeTotal from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Saving Checkpoint Completion (%)Attribute SavingCheckpointPercentComplete from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Saving Checkpoint CountAttribute SavingCheckpointCount from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Saving Checkpoint Elapsed Time (Milliseconds)Attribute SavingCheckpointElapsedTime from Bean Hadoop:service=NameNode,name=StartupProgress
Startup Saving Checkpoint TotalAttribute SavingCheckpointTotal from Bean Hadoop:service=NameNode,name=StartupProgress
StateAttribute State from Bean Hadoop:service=NameNode,name=NameNodeStatus
Storage Block Report Operations (Operations)Attribute StorageBlockReportOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Syncs Average Time (Milliseconds)Attribute SyncsAvgTime from Bean Hadoop:service=NameNode,name=NameNodeActivity
Syncs Operations (Operations)Attribute SyncsNumOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
System CPU Load (%)Attribute SystemCpuLoad from Bean java.lang:type=OperatingSystem
Tag HostnameAttribute tag.Hostname from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Threads In BLOCKED StateAttribute ThreadsBlocked from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In NEW StateAttribute ThreadsNew from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In RUNNABLE StateAttribute ThreadsRunnable from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In TERMINATED StateAttribute ThreadsTerminated from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In TIMED_WAITING StateAttribute ThreadsTimedWaiting from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In WAITING StateAttribute ThreadsWaiting from Bean Hadoop:service=\w+,name=JvmMetrics
Time Since Last Loaded Edits (Milliseconds)Attribute MillisSinceLastLoadedEdits from Bean Hadoop:service=NameNode,name=FSNamesystem
Total Blocks (Blocks)Attribute TotalBlocks from Bean Hadoop:service=NameNode,name=NameNodeInfo
Total ERROR Log MessagesAttribute LogError from Bean Hadoop:service=\w+,name=JvmMetrics
Total FATAL Log MessagesAttribute LogFatal from Bean Hadoop:service=\w+,name=JvmMetrics
Total File Operations (Operations)Attribute TotalFileOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Total FilesAttribute TotalFiles from Bean Hadoop:service=NameNode,name=NameNodeInfo
Total INFO Log MessagesAttribute LogInfo from Bean Hadoop:service=\w+,name=JvmMetrics
Total Physical Memory (Gibibytes)Attribute TotalPhysicalMemorySize from Bean java.lang:type=OperatingSystem
Total WARN Log MessagesAttribute LogWarn from Bean Hadoop:service=\w+,name=JvmMetrics
Transactions Average Time (Milliseconds)Attribute TransactionsAvgTime from Bean Hadoop:service=NameNode,name=NameNodeActivity
Transactions Batched In SyncAttribute TransactionsBatchedInSync from Bean Hadoop:service=NameNode,name=NameNodeActivity
Transactions Operations (Operations)Attribute TransactionsNumOps from Bean Hadoop:service=NameNode,name=NameNodeActivity
Transactions Since Last CheckpointAttribute TransactionsSinceLastCheckpoint from Bean Hadoop:service=NameNode,name=FSNamesystem
Transactions Since Last Log RollAttribute TransactionsSinceLastLogRoll from Bean Hadoop:service=NameNode,name=FSNamesystem
Under-replicated (Blocks)Attribute UnderReplicatedBlocks from Bean Hadoop:service=NameNode,name=FSNamesystem
Upgrade FinalizedAttribute UpgradeFinalized from Bean Hadoop:service=NameNode,name=NameNodeInfo
Used Physical Memory (Gibibytes)Used Physical Memory
Volume Failures TotalAttribute VolumeFailuresTotal from Bean Hadoop:service=NameNode,name=FSNamesystemState

Node Manager

NameDescription
Container Launch Duration Average Time (Seconds)Attribute ContainerLaunchDurationAvgTime from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Container Launch Duration Operations (Operations)Attribute ContainerLaunchDurationNumOps from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Containers AllocatedAttribute AllocatedContainers from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Containers CompletedAttribute ContainersCompleted from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Containers FailedAttribute ContainersFailed from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Containers InitializingAttribute ContainersIniting from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Containers KilledAttribute ContainersKilled from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Containers LaunchedAttribute ContainersLaunched from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Containers RunningAttribute ContainersRunning from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Current Heap Memory Committed (Mebibytes)Attribute MemHeapCommittedM from Bean Hadoop:service=\w+,name=JvmMetrics
Current Heap Memory Used (Mebibytes)Attribute MemHeapUsedM from Bean Hadoop:service=\w+,name=JvmMetrics
Current Non-heap Memory Committed (Mebibytes)Attribute MemNonHeapCommittedM from Bean Hadoop:service=\w+,name=JvmMetrics
Current Non-heap Memory Used (Mebibytes)Attribute MemNonHeapUsedM from Bean Hadoop:service=\w+,name=JvmMetrics
Free Physical Memory (Gibibytes)Attribute FreePhysicalMemorySize from Bean java.lang:type=OperatingSystem
Garbage Collection CountAttribute GcCount from Bean Hadoop:service=\w+,name=JvmMetrics
Garbage Collection Count Mark Sweep CompactAttribute GcCountMarkSweepCompact from Bean Hadoop:service=\w+,name=JvmMetrics
Garbage Collection Time (Milliseconds)Attribute GcTimeMillis from Bean Hadoop:service=\w+,name=JvmMetrics
Garbage Collection Time Mark Sweep Compact (Milliseconds)Attribute GcTimeMillisMarkSweepCompact from Bean Hadoop:service=\w+,name=JvmMetrics
Group Resolution Average Time (Milliseconds)Attribute GetGroupsAvgTime from Bean Hadoop:service=\w+,name=UgiMetrics
Group ResolutionsAttribute GetGroupsNumOps from Bean Hadoop:service=\w+,name=UgiMetrics
HostnameHostname
JVM VersionAttribute SystemProperties from Bean java.lang:type=Runtime
Kerberos Login Failure Average Time (Milliseconds)Attribute LoginFailureAvgTime from Bean Hadoop:service=\w+,name=UgiMetrics
Kerberos Login FailuresAttribute LoginFailureNumOps from Bean Hadoop:service=\w+,name=UgiMetrics
Kerberos Login Success Average Time (Milliseconds)Attribute LoginSuccessAvgTime from Bean Hadoop:service=\w+,name=UgiMetrics
Kerberos Login SuccessesAttribute LoginSuccessNumOps from Bean Hadoop:service=\w+,name=UgiMetrics
Maximum Heap Memory Size (Mebibytes)Attribute MemHeapMaxM from Bean Hadoop:service=\w+,name=JvmMetrics
Maximum Memory Size (Mebibytes)Attribute MemMaxM from Bean Hadoop:service=\w+,name=JvmMetrics
Maximum Non-heap Memory Size (Mebibytes)Attribute MemNonHeapMaxM from Bean Hadoop:service=\w+,name=JvmMetrics
Memory Allocated (Gibibytes)Attribute AllocatedGB from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Memory Allocated Ratio (Gibibytes)Memory Allocated Ratio
Memory Available (Gibibytes)Attribute AvailableGB from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Metric EndpointMetric Endpoint
OS NameAttribute Name from Bean java.lang:type=OperatingSystem
OS VersionAttribute Version from Bean java.lang:type=OperatingSystem
Physical Memory Utilization (%)Physical Memory Utilization
Process CPU Load (%)Attribute ProcessCpuLoad from Bean java.lang:type=OperatingSystem
Publish Average Time (Milliseconds)Attribute PublishAvgTime from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Publish Operations (Operations)Attribute PublishNumOps from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Publishes DroppedAttribute DroppedPubAll from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Sinks TotalAttribute NumAllSinks from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Snapshot Average Time (Milliseconds)Attribute SnapshotAvgTime from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Snapshot Operations (Operations)Attribute SnapshotNumOps from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Sources ActiveAttribute NumActiveSources from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Sources TotalAttribute NumAllSources from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
System CPU Load (%)Attribute SystemCpuLoad from Bean java.lang:type=OperatingSystem
Tag HostnameAttribute tag.Hostname from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Threads In BLOCKED StateAttribute ThreadsBlocked from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In NEW StateAttribute ThreadsNew from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In RUNNABLE StateAttribute ThreadsRunnable from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In TERMINATED StateAttribute ThreadsTerminated from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In TIMED_WAITING StateAttribute ThreadsTimedWaiting from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In WAITING StateAttribute ThreadsWaiting from Bean Hadoop:service=\w+,name=JvmMetrics
Total ERROR Log MessagesAttribute LogError from Bean Hadoop:service=\w+,name=JvmMetrics
Total FATAL Log MessagesAttribute LogFatal from Bean Hadoop:service=\w+,name=JvmMetrics
Total INFO Log MessagesAttribute LogInfo from Bean Hadoop:service=\w+,name=JvmMetrics
Total Physical Memory (Gibibytes)Attribute TotalPhysicalMemorySize from Bean java.lang:type=OperatingSystem
Total WARN Log MessagesAttribute LogWarn from Bean Hadoop:service=\w+,name=JvmMetrics
Used Physical Memory (Gibibytes)Used Physical Memory
Virtual Cores AllocatedAttribute AllocatedVCores from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Virtual Cores Allocated RatioVirtual Cores Allocated Ratio
Virtual Cores AvailableAttribute AvailableVCores from Bean Hadoop:service=NodeManager,name=NodeManagerMetrics
Virtual Cores TotalVirtual Cores Total

Resource Manager

NameDescription
Active NodesIsolated hostnames in LiveNodeManagers entry from Bean Hadoop:service=ResourceManager,name=RMNMInfo
Application Master Launch Delay Average Time (Milliseconds)Attribute AMLaunchDelayAvgTime from Bean Hadoop:service=ResourceManager,name=ClusterMetrics
Application Master Launch Delay Operations (Operations)Attribute AMLaunchDelayNumOps from Bean Hadoop:service=ResourceManager,name=ClusterMetrics
Application Master Register Delay Average Time (Milliseconds)Attribute AMRegisterDelayAvgTime from Bean Hadoop:service=ResourceManager,name=ClusterMetrics
Application Master Register Delay Operations (Operations)Attribute AMRegisterDelayNumOps from Bean Hadoop:service=ResourceManager,name=ClusterMetrics
Applications ActiveAttribute ActiveApplications from Bean Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Applications CompletedAttribute AppsCompleted from Bean Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Applications FailedAttribute AppsFailed from Bean Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Applications KilledAttribute AppsKilled from Bean Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Applications PendingAttribute AppsPending from Bean Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Applications RunningAttribute AppsRunning from Bean Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Applications SubmittedAttribute AppsSubmitted from Bean Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Current Heap Memory Committed (Mebibytes)Attribute MemHeapCommittedM from Bean Hadoop:service=\w+,name=JvmMetrics
Current Heap Memory Used (Mebibytes)Attribute MemHeapUsedM from Bean Hadoop:service=\w+,name=JvmMetrics
Current Non-heap Memory Committed (Mebibytes)Attribute MemNonHeapCommittedM from Bean Hadoop:service=\w+,name=JvmMetrics
Current Non-heap Memory Used (Mebibytes)Attribute MemNonHeapUsedM from Bean Hadoop:service=\w+,name=JvmMetrics
Free Physical Memory (Gibibytes)Attribute FreePhysicalMemorySize from Bean java.lang:type=OperatingSystem
Garbage Collection CountAttribute GcCount from Bean Hadoop:service=\w+,name=JvmMetrics
Garbage Collection Count Mark Sweep CompactAttribute GcCountMarkSweepCompact from Bean Hadoop:service=\w+,name=JvmMetrics
Garbage Collection Time (Milliseconds)Attribute GcTimeMillis from Bean Hadoop:service=\w+,name=JvmMetrics
Garbage Collection Time Mark Sweep Compact (Milliseconds)Attribute GcTimeMillisMarkSweepCompact from Bean Hadoop:service=\w+,name=JvmMetrics
Group Resolution Average Time (Milliseconds)Attribute GetGroupsAvgTime from Bean Hadoop:service=\w+,name=UgiMetrics
Group ResolutionsAttribute GetGroupsNumOps from Bean Hadoop:service=\w+,name=UgiMetrics
HostnameHostname
JVM VersionAttribute SystemProperties from Bean java.lang:type=Runtime
Kerberos Login Failure Average Time (Milliseconds)Attribute LoginFailureAvgTime from Bean Hadoop:service=\w+,name=UgiMetrics
Kerberos Login FailuresAttribute LoginFailureNumOps from Bean Hadoop:service=\w+,name=UgiMetrics
Kerberos Login Success Average Time (Milliseconds)Attribute LoginSuccessAvgTime from Bean Hadoop:service=\w+,name=UgiMetrics
Kerberos Login SuccessesAttribute LoginSuccessNumOps from Bean Hadoop:service=\w+,name=UgiMetrics
Maximum Heap Memory Size (Mebibytes)Attribute MemHeapMaxM from Bean Hadoop:service=\w+,name=JvmMetrics
Maximum Memory Size (Mebibytes)Attribute MemMaxM from Bean Hadoop:service=\w+,name=JvmMetrics
Maximum Non-heap Memory Size (Mebibytes)Attribute MemNonHeapMaxM from Bean Hadoop:service=\w+,name=JvmMetrics
Metric EndpointMetric Endpoint
Node Managers ActiveAttribute NumActiveNMs from Bean Hadoop:service=ResourceManager,name=ClusterMetrics
Node Managers DecommissionedAttribute NumDecommissionedNMs from Bean Hadoop:service=ResourceManager,name=ClusterMetrics
Node Managers LostAttribute NumLostNMs from Bean Hadoop:service=ResourceManager,name=ClusterMetrics
Node Managers RebootedAttribute NumRebootedNMs from Bean Hadoop:service=ResourceManager,name=ClusterMetrics
Node Managers UnhealthyAttribute NumUnhealthyNMs from Bean Hadoop:service=ResourceManager,name=ClusterMetrics
OS NameAttribute Name from Bean java.lang:type=OperatingSystem
OS VersionAttribute Version from Bean java.lang:type=OperatingSystem
Physical Memory Utilization (%)Physical Memory Utilization
Process CPU Load (%)Attribute ProcessCpuLoad from Bean java.lang:type=OperatingSystem
Publish Average Time (Milliseconds)Attribute PublishAvgTime from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Publish Operations (Operations)Attribute PublishNumOps from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Publishes DroppedAttribute DroppedPubAll from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Queue Memory Allocated (Mebibytes)Attribute AllocatedMB from Bean Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Queue Memory Available (Mebibytes)Attribute AvailableMB from Bean Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Sinks TotalAttribute NumAllSinks from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Snapshot Average Time (Milliseconds)Attribute SnapshotAvgTime from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Snapshot Operations (Operations)Attribute SnapshotNumOps from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Sources ActiveAttribute NumActiveSources from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Sources TotalAttribute NumAllSources from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
System CPU Load (%)Attribute SystemCpuLoad from Bean java.lang:type=OperatingSystem
Tag HostnameAttribute tag.Hostname from Bean Hadoop:service=\w+,name=MetricsSystem,sub=Stats
Threads In BLOCKED StateAttribute ThreadsBlocked from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In NEW StateAttribute ThreadsNew from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In RUNNABLE StateAttribute ThreadsRunnable from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In TERMINATED StateAttribute ThreadsTerminated from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In TIMED_WAITING StateAttribute ThreadsTimedWaiting from Bean Hadoop:service=\w+,name=JvmMetrics
Threads In WAITING StateAttribute ThreadsWaiting from Bean Hadoop:service=\w+,name=JvmMetrics
Total ERROR Log MessagesAttribute LogError from Bean Hadoop:service=\w+,name=JvmMetrics
Total FATAL Log MessagesAttribute LogFatal from Bean Hadoop:service=\w+,name=JvmMetrics
Total INFO Log MessagesAttribute LogInfo from Bean Hadoop:service=\w+,name=JvmMetrics
Total Physical Memory (Gibibytes)Attribute TotalPhysicalMemorySize from Bean java.lang:type=OperatingSystem
Total WARN Log MessagesAttribute LogWarn from Bean Hadoop:service=\w+,name=JvmMetrics
Used Physical Memory (Gibibytes)Used Physical Memory
Users ActiveAttribute ActiveUsers from Bean Hadoop:service=ResourceManager,name=QueueMetrics,q0=root

Updated 8 months ago

Apache Hadoop


Monitor Apache Hadoop Database Platform

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.