Cloud Foundry
Pivotal Cloud Foundry (PCF) Monitoring
This source has been deprecated
observIQ is in the process of transitioning a subset of BindPlane's monitoring capabilities to the observIQ OpenTelemetry Collector. As a result, this Source is no longer publicly available in BindPlane. If you need access to this Source, please reach out to our support via chat or via [email protected].
Data Collection Setup
Metrics are collected via 3 distinct APIs within the PCF Foundation: the Blue Medora Nozzle, Pivotal Cloud Controller, and Pivotal BOSH Director API.
Network Requirements
Nozzle API (Tile Deployment): 443 (TCP) Default
- Must match your nozzle configuration. (e.g., BOSH install default port is 8081).
Cloud Controller API Port: 443 (TCP) Default
- The source also accesses the UAA at the address specified in the authorization_endpoint in /info on the Cloud Controller API.
BOSH Director API Port: 25555 (TCP) Default
- The source also accesses the UAA at the address specified in the user_authentication endpoint in /info on the BOSH Director API.
Obtaining Nozzle, Cloud Controller, and BOSH Director Credentials.
The Data Provider for PCF requires the following credential types:
- Blue Medora Nozzle for PCF Credentials (UAA Client)
- Cloud Controller API Credentials (UAA User)
- BOSH Director Credentials (BOSH Director UAA Client)
Blue Medora Nozzle for PCF Credentials (UAA Client)
For the Blue Medora Nozzle for PCF, you will need a UAA Client and password
The UAA Client used for your nozzle credentials must be the same UAA Client used when configuring your Nozzle.
To create a UAA Client for the nozzle:
- Complete steps 1-4 in Cloud Foundry's Create an Admin User topic. These steps walk you through installing the UAA CLI, targeting your UAA server, and obtaining an access token for the admin client from the UAA server.
- Next, execute the following command:
Be sure to replace the{clientname}
and{foundation}
parameters with the name you want to assign your UAA client and your PCF foundation, respectively.
uaac client add {clientname} \
--authorities "oauth.login,doppler.firehose" \
--scope "openid,oauth.approvals,doppler.firehose" \
--authorized_grant_types "authorization_code,client_credentials,refresh_token" \
--access_token_validity 1209600 \
--autoapprove true \
--redirect_uri https://apps.{foundation}
Cloud Controller API Credentials (UAA User)
For the Cloud Controller API, you will need a UAA User and password in our data provider configuration.
Complete steps 1-4 in Cloud Foundry's Create an Admin Read-Only User topic, but adding the new user to the cloud_controller.admin group instead of cloud_controller.admin_read_only, as indicated in step 4. (Make sure to add the new user to the uaa.admin and scim.read groups as well.)
User Privileges with the Cloud Controller API
NOTE: You may choose to assign the new user lesser privileges by adding it to the cloud_controller.admin_read_only group instead of cloud_controller.admin; however, CPU, Memory, and Disk Usage for Applications and Application Instances will not be returned with that lesser permission.
BOSH Director Credentials (BOSH Director UAA Client)
For BOSH Director, you will need a BOSH Director UAA Client and password
To create a BOSH Director UAA Client:
Complete steps 1-5 in Pivotal Cloud Foundry's Creating UAA Clients for BOSH Director topic, but adding the new user to the bosh.read group instead of bosh.admin, as indicated in step 5.
Setting up the Blue Medora Nozzle for PCF
The Blue Medora Nozzle for PCF is a Cloud Foundry component that is required by the management pack in order to connect to the Cloud Foundry Loggregator Firehose. It exposes PCF metrics via a RESTful API.
PCF Nozzle Documentation
We recommend upgrading to the latest version of the nozzle for optimized performance. See our nozzle documentation on the Pivotal Partners site for details.
Prerequisites
You will need Blue Medora Nozzle for PCF credentials prior to deploying and configuring the Nozzle. See: Obtaining Nozzle, Cloud Controller, and BOSH Director Credentials above.
Recommended Deployment Method: Blue Medora Nozzle for PCF Tile
Download the Blue Medora Nozzle for PCF Tile from the Pivotal Network, then follow the instructions for installing, configuring, and using the nozzle as documented on our Pivotal Partners documentation site.
Least Privileged User
See:
Cloud Controller API Credentials (UAA User) section above
Supported Versions
Pivotal Cloud Foundry Central:
1.8+, 2.x
Connection Parameters
Name | Required? | Description |
---|---|---|
BOSH Director Auth URL | ||
BOSH Director API Host | Required | The BOSH Director to connect to. |
BOSH Director API Port | The port for communication to the BOSH Director. | |
BOSH Director API Username | Required | |
BOSH Director API Password | Required | |
SSL Configuration | The SSL mode to use when connecting to the target. Can be configured to not use SSL (No SSL), use SSL but do not verify the target's certificate (No Verify), and use SSL and verify the target's certificate (Verify). | |
Use Nozzle? | Whether to expect Nozzle connection information. If set to false, events, resources, and metrics depending on the Loggregator Firehose will be missing. | |
Use Cloud Controller API? | Whether to expect Cloud Controller connection information. If false, Applications, Services, Spaces, and Routes metrics and relationships may be missing. | |
Nozzle API Host | Required | The Nozzle API to connect to. |
Nozzle API Port | The port for communication to the Nozzle API. | |
Nozzle API Username | Required | |
Nozzle API Password | Required | |
Cloud Controller API Host | Required | The Cloud Controller API to connect to. |
Cloud Controller API Port | The port for communication to the Cloud Controller API. | |
Cloud Controller API Username | Required | |
Cloud Controller API Password | Required | |
Foundation Name | Specifies the name for the resource representing the Foundation, which is the top-level resource describing a Pivotal Cloud deployment. Enter a Foundation Name so you can distinguish data from other Pivotal Cloud deployments in your environment. | |
Connection Timeout (seconds) | The number of seconds to allow for connecting to the target. | |
Collect Events | Controls whether events are collected and returned. |
Metrics
Access
Name | Description |
---|---|
File Server Active GoRoutines | Instantaneous number of active GoRoutines in the process |
File Server CPUs | Number of CPUs on the machine |
File Server Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
File Server Memory Allocations | Lifetime number of memory allocations |
File Server Memory Deallocations | Lifetime number of memory deallocations |
File Server Used Heap Memory (Bytes) | Instantaneous count memory allocated on the main heap and still in use |
File Server Used Memory (Bytes) | Instantaneous count of memory allocated and still in use |
File Server Used Stack Memory (Bytes) | Instantaneous count of memory allocated on the main stack and still in use |
SSH Proxy Active GoRoutines | Instantaneous number of active GoRoutines in the process |
SSH Proxy CPUs | Number of CPUs on the machine |
SSH Proxy Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
SSH Proxy Memory Allocations | Lifetime number of memory allocations |
SSH Proxy Memory Deallocations | Lifetime number of memory deallocations |
SSH Proxy Used Heap Memory (Bytes) | Instantaneous count memory allocated on the main heap and still in use |
SSH Proxy Used Memory (Bytes) | Instantaneous count of memory allocated and still in use |
SSH Proxy Used Stack Memory (Bytes) | Instantaneous count of memory allocated on the main stack and still in use |
Application
Name |
---|
Average Instance Disk Usage (Bytes) |
Average Instance Memory Usage (Bytes) |
Instance Count |
Instance CPU Utilization (%) |
Total Instance Disk Quota (Bytes) |
Total Instance Disk Usage (Bytes) |
Total Instance Memory Quota (Bytes) |
Total Instance Memory Usage (Bytes) |
Application Instance
Name |
---|
CPU Utilization (%) |
Disk Quota (Bytes) |
Disk Usage (Bytes) |
File Descriptors Quota |
Memory Quota (Bytes) |
Memory Usage (Bytes) |
BBS
Name | Description |
---|---|
Active GoRoutines | Instantaneous number of active GoRoutines in the Doppler process |
Average Request Latency (Nanoseconds) | Average time the BBS took to handle requests to its API endpoints in the last 15 minutes. |
CF-Apps Domain Up-To-Date | Whether the 'cf-apps' domain is Up-To-Date, so that CF apps from CC have been synchronized with DesiredLRPs for Diego to run. |
CF-Tasks Domain Up-To-Date | Whether the 'cf-tasks' domain is Up-To-Date, so that CF tasks from CC have been synchronized with tasks for Diego to run. |
CPU Count | Number of CPUs on the machine |
Cumulative Convergence Tasks Kicked | Cumulative number of times the BBS has updated a Task during its Task convergence pass. |
Cumulative Convergence Tasks Pruned | Cumulative number of times the BBS has deleted a malformed Task during its Task convergence pass. |
Cumulative LRP Convergence Pass Runs | Cumulative number of times BBS has run its LRP convergence pass. |
Cumulative Malformed ActualLRP Instances Deleted | Cumulative number of times the BBS has detected and deleted a malformed ActualLRP in its LRP convergence pass. |
Cumulative Malformed DesiredLRP RunInfo Detected | Cumulative number of times the BBS has detected a malformed DesiredLRP RunInfo in its LRP convergence pass. |
Cumulative Malformed DesiredLRP SchedulingInfo Detected | Cumulative number of times the BBS has detected a malformed DesiredLRP SchedulingInfo in its LRP convergence pass. |
Cumulative Request Count | Cumulative number of requests the BBS has handled through its API. |
Cumulative Task Convergence Pass Runs | Cumulative number of times the BBS has run its Task convergence pass. |
etcd Leader Received Data Rate (Bytes per Second) | Number of data per second this node is receiving (leader only). This value is undefined on single member clusters. |
etcd Leader Received Request Rate (per Second) | Number of requests per second this node is receiving (leader only). This value is undefined on single member clusters. |
etcd Leader Send Data Rate (Bytes per Second) | Number of data per second this node is sending (leader only). This value is undefined on single member clusters. |
etcd Leader Send Request Rate (per Second) | Number of requests per second this node is sending (leader only). This value is undefined on single member clusters. |
etcd Raft Term | Raft term of the etcd cluster. |
etcd Watchers | Number of watches set against the etcd cluster. |
Extra LRP Instance Average | Average number of LRP instances that are no longer desired but still have a BBS record over the last 5 minutes. |
Extra LRP Instances | Total number of LRP instances that are no longer desired but still have a BBS record. |
Is Master | Signals if BBS is on master |
Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
Lock Held Duration (Nanoseconds) | Time the active BBS has held the BBS lock. |
Lock Is Held | Whether a BBS holds the BBS lock. |
LRP Convergence Pass Duration (Nanoseconds) | Time the BBS took to run its LRP convergence pass. |
LRP Convergence Pass Duration Maximum (Nanoseconds) | Maximum Time the BBS took to run its LRP convergence pass in the last 15 minutes. |
LRP Instances Desired | Total number of LRP instances desired across all LRPs. |
LRP Instances Running | Total number of LRP instances that are running on cells. |
Maximum Locket Active Locks | The maximum count of how many locks the system components are holding in the past 5 minutes. |
Maximum Locket Active Presences | Maximum count of active presences in the past 15 minutes. Presences are defined as the registration records that the cells maintain to advertise themselves to the platform. |
Memory Allocations | Lifetime number of memory allocations |
Memory Deallocations | Lifetime number of memory deallocations |
Migration Duration (Nanoseconds) | Time the BBS took to run migrations against its persistence store. |
Reporting Metrics Duration (Nanoseconds) | Time the BBS took to emit metrics about etcd. |
Request Latency (Nanoseconds) | Time the BBS took to handle requests to its API endpoints. |
Task Convergence Pass Duration | Time the BBS took to run its Task convergence pass. |
Unclaimed LRP Instances | Total number of LRP instances that have not yet been claimed by a cell. |
Used Heap Memory (Bytes) | Instantaneous count of memory allocated on the main heap and still in use |
Used Memory (Bytes) | Instantaneous count of memory allocated and still in use |
Used Stack Memory (Bytes) | Instantaneous count of memory used by the stack allocator |
VM Active Memory Usage (Kibibytes) | VM Active Memory Usage |
VM Active Memory Utilization (%) | VM Active Memory Utilization |
VM CPU Utilization (%) | Combined amount of CPU spent in system and user processes |
VM Ephemeral Disk Utilization (%) | VM Ephemeral Disk Utilization |
VM Load Average (15m) | Unix-style load average for the past 15 minutes |
VM Persistent Disk Utilization (%) | VM Persistent Disk Utilization |
VM Swap Usage (Kibibytes) | VM Swap Usage |
VM Swap Utilization (%) | VM Swap Utilization |
VM System Disk Utilization (%) | VM System Disk Utilization |
VM User CPU Utilization (%) | Time CPU spent in user mode |
Bosh Job
Name |
---|
VM Active Memory Usage (Kibibytes) |
VM Active Memory Utilization (%) |
VM CPU Utilization (%) |
VM Ephemeral Disk Utilization (%) |
VM Load Average (15m) |
VM Persistent Disk Utilization (%) |
VM Swap Usage (Kibibytes) |
VM Swap Utilization (%) |
VM System Disk Utilization (%) |
VM User CPU Utilization (%) |
CC-Bridge
Name | Description |
---|---|
CC-Uploader Active GoRoutines | Instantaneous number of active GoRoutines in the process |
CC-Uploader CPUs | Number of CPUs on the machine |
CC-Uploader Heap Memory Usage (Bytes) | Instantaneous count memory allocated on the main heap and still in use |
CC-Uploader Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
CC-Uploader Memory Allocations | Lifetime number of memory allocations |
CC-Uploader Memory Deallocations | Lifetime number of memory deallocations |
CC-Uploader Memory Usage (Bytes) | Instantaneous count of memory allocated and still in use |
CC-Uploader Stack Memory Usage (Bytes) | Instantaneous count of memory allocated on the main stack and still in use |
Nsync Bulker Active GoRoutines | Instantaneous number of active GoRoutines in the process |
Nsync Bulker Convergence Lock Held | Whether an Nsync Bulker holds the Nsync Bulker lock. |
Nsync Bulker Convergence Lock Held Duration (Nanoseconds) | Time the active Nsync Bulker has held the convergence lock. |
Nsync Bulker Invalid DesiredLRP Instances | Number of invalid DesiredLRPs found during Nsync Bulker periodic synchronization |
Nsync Bulker Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
Nsync Bulker LRP Instances Desired | Cumulative number of LRPs desired through the Nsync API. |
Nsync Bulker LRP Synchronization Duration (Nanoseconds) | Time the Nsync Bulker took to synchronize CF apps and Diego DesiredLRPs. |
Stager Staging Tasks Failed | Cumulative number of failed staging tasks handled by each stager. |
Stager Staging Tasks Failed Duration (Nanoseconds) | Time the failed staging task took to run. |
Stager Staging Tasks Requests | Cumulative number of requests to start a staging task. |
Stager Staging Tasks Succeeded | Cumulative number of successful staging tasks handled by each stager. |
Stager Staging Tasks Succeeded Duration (Nanoseconds) | Time the successful staging task took to run. |
TPS Watcher Lock Held | Whether a tps-watcher holds the tps-watcher lock. |
TPS Watcher Lock Held Duration (Nanoseconds) | Time the active tps-watcher has held the convergence lock |
Cloud Controller
Name | Description |
---|---|
All Log Messages | Total number of log messages, sum of messages of all severity levels. |
cc-generic Failed Jobs | Number of failed jobs in the cc-generic queue. |
cc-generic Delayed Jobs | Number of delayed jobs in the cc-generic queue. |
Completed Requests | Number of requests that have been processed. |
CPU Cores | The number of CPU cores of a host machine. |
CPU Utilization (%) | Amount of CPU spent in the Cloud Controller process. |
Free Memory (Bytes) | Total memory available according to the OS. |
HTTP Responses | Number of HTTP responses. |
Memory Usage (Bytes) | Total memory used (active + wired) according to the OS. |
Outstanding Requests | Number of requests currently being processed. |
Real Memory (Bytes) | The RSS data (resident set size) or real memory of the Cloud Controller process. |
Running Tasks | Number of currently running tasks. |
Running Tasks Memory Usage (Megabytes) | Memory being consumed by all currently running tasks. |
Thread Connections | Number of open connections to event machine |
Thread Result Scheduled Tasks | Number of scheduled tasks in the result |
Thread Result Unscheduled Tasks | Number of unscheduled tasks in the result |
Thread Scheduled Tasks | Number of scheduled tasks in the threadqueue |
Thread Unscheduled Tasks | Number of unscheduled tasks in the threadqueue |
Threads | Total number of threads that are either runnable or stopped |
Uptime (Seconds) | The uptime of the Cloud Controller process. |
VM Active Memory Usage (Kibibytes) | VM Active Memory Usage |
VM Active Memory Utilization (%) | VM Active Memory Utilization |
VM CPU Utilization (%) | Combined amount of CPU spent in system and user processes |
VM Ephemeral Disk Utilization (%) | VM Ephemeral Disk Utilization |
VM Load Average (15m) | Unix-style load average for the past 15 minutes |
VM Persistent Disk Utilization (%) | VM Persistent Disk Utilization |
VM Swap Usage (Kibibytes) | VM Swap Usage |
VM Swap Utilization (%) | VM Swap Utilization |
VM System Disk Utilization (%) | VM System Disk Utilization |
VM User CPU Utilization (%) | Time CPU spent in user mode |
Consul
Name | Description |
---|---|
Active GoRoutines | Instantaneous number of active GoRoutines in the process. |
CID | Virtual Machine Cloud ID |
Counter Events | Number of CounterEvents aggregated in Metron. |
CPUs | Number of CPUs on the machine. |
Deployment | Deployment |
Doppler Sent Messages | Number of messages sent to Doppler regardless of protocol. |
Heap Memory Usage (Bytes) | Instantaneous count of memory allocated on the main heap and still in use. |
Index | Index |
IP | IP |
Job | Job |
Job State | Job State |
Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause. |
Memory Allocations | Lifetime number of memory allocations. |
Memory Deallocations | Lifetime number of memory deallocations. |
Memory Usage (Bytes) | Instantaneous count of memory allocated and still in use. |
Processed Uncategorized Events | Number of non-(CounterEvent HTTPStart HTTPStop) events processed by aggregator. |
Stack Memory Usage (Bytes) | Instantaneous count of memory used by the stack allocator. |
TCP Send Errors (Errors) | Number of errors if writing to Doppler over TCP fails |
TCP Sent Data (Bytes) | Lifetime data sent to Doppler over TCP |
TCP Sent Messages | Number of sent messages to Doppler over TCP |
TLS Send Errors (Errors) | Number of errors if writing to Doppler over TLS fails |
TLS Sent Data (Bytes) | Lifetime data sent to Doppler over TLS |
TLS Sent Messages | Number of sent messages to Doppler over TLS |
UDP Send Errors (Errors) | Number of errors if writing to Doppler over UDP fails |
UDP Sent Data (Bytes) | Lifetime data sent to Doppler over UDP |
UDP Sent Messages | Number of sent messages to Doppler over UDP |
VM Active Memory Usage (Kibibytes) | VM Active Memory Usage |
VM Active Memory Utilization (%) | VM Active Memory Utilization |
VM CPU Utilization (%) | Combined amount of CPU spent in system and user processes |
VM Ephemeral Disk Utilization (%) | VM Ephemeral Disk Utilization |
VM Load Average (15m) | Unix-style load average for the past 15 minutes |
VM Persistent Disk Utilization (%) | VM Persistent Disk Utilization |
VM Swap Usage (Kibibytes) | VM Swap Usage |
VM Swap Utilization (%) | VM Swap Utilization |
VM System Disk Utilization (%) | VM System Disk Utilization |
VM User CPU Utilization (%) | Time CPU spent in user mode |
Diego Brain
Name | Description |
---|---|
Auctioneer Active GoRoutines | Instantaneous number of active GoRoutines in the process |
Auctioneer CPUs | Number of CPUs on the machine |
Auctioneer Fetch State Duration (Nanoseconds) | Time the auctioneer took to fetch state from all the cells when running its auction. |
Auctioneer Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
Auctioneer Lock Held Duration (Nanoseconds) | Time the active auctioneer has held the auctioneer lock. |
Auctioneer Lock Is Held | Whether an auctioneer holds the auctioneer lock. |
Auctioneer Memory Allocations | Lifetime number of memory allocations |
Auctioneer Memory Deallocations | Lifetime number of memory deallocations |
Auctioneer Requests | Cumulative number of requests the auctioneer has handled through its API. |
Auctioneer Used Heap Memory (Bytes) | Instantaneous count memory allocated on the main heap and still in use |
Auctioneer Used Memory (Bytes) | Instantaneous count of memory allocated and still in use |
Auctioneer Used Stack Memory (Bytes) | Instantaneous count of memory allocated on the main stack and still in use |
CID | Virtual Machine Cloud ID |
Convergence Tasks Pruned | Cumulative number of times the BBS has deleted a malformed Task during its Task convergence pass. |
Converger Active GoRoutines | Instantaneous number of active GoRoutines in the process |
Converger CPUs | Number of CPUs on the machine |
Converger Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
Converger Lock Held Duration (Nanoseconds) | Time the active converger has held the converger lock. |
Converger Lock Is Held | Whether an converger holds the auctioneer lock. |
Converger Memory Allocations | Lifetime number of memory allocations |
Converger Memory Deallocations | Lifetime number of memory deallocations |
Converger Task Pass Duration (Nanoseconds) | Time the BBS took to run its Task convergence pass. |
Converger Task Pass Runs | Cumulative number of times the BBS has run its Task convergence pass. |
Converger Tasks Kicked | Cumulative number of times the BBS has updated a Task during its Task convergence pass. |
Converger Used Heap Memory (Bytes) | Instantaneous count memory allocated on the main heap and still in use |
Converger Used Memory (Bytes) | Instantaneous count of memory allocated and still in use |
Converger Used Stack Memory (Bytes) | Instantaneous count of memory allocated on the main stack and still in use |
Cumulative Auctioneer LRP Instances Failed | Cumulative number of LRP instances that the auctioneer failed to place on Diego cells. |
Cumulative Auctioneer LRP Instances Started | Cumulative number of LRP instances that the auctioneer successfully placed on Diego cells. |
Cumulative Auctioneer Task Auctions Started | Cumulative number of Tasks that the auctioneer successfully placed on Diego cells. |
Cumulative Auctioneer Task Placements Failed | Cumulative number of Tasks that the auctioneer failed to place on Diego cells. |
Delta Average Cumulative Auctioneer LRP Instances Failed | The delta average over the past 5 minutes for each minute of the cumulative number of LRP instances that the auctioneer failed to place on Diego cells. |
Delta Average Cumulative Auctioneer LRP Instances Started | The delta average over the past 5 minutes for each minute of the cumulative number of LRP instances that the auctioneer successfully placed on Diego cells. |
Delta Average Cumulative Auctioneer Task Placements Failed | The delta average over the past 5 minutes for each minute of the cumulative number of Tasks that the auctioneer failed to place on Diego cells. |
Deployment | Deployment |
Index | Index |
IP | IP |
Job | Job |
Job State | Job State |
Maximum Auctioneer Fetch State Duration (Nanoseconds) | The maximum duration over the past 5 minutes the auctioneer took to fetch state from all the cells when running its auction. |
Maximum Auctioneer Locks Held | Whether an Auctioneer instance holds the expected Auctioneer lock (in Locket). 1 means the active Auctioneer holds the lock, and 0 means the lock was lost. The metric may occasionally be 0 when the Auctioneer instances are performing a leader transition, but a prolonged value of 0 indicates an issue with Auctioneer. |
VM Active Memory Usage (Kibibytes) | VM Active Memory Usage |
VM Active Memory Utilization (%) | VM Active Memory Utilization |
VM CPU Utilization (%) | Combined amount of CPU spent in system and user processes |
VM Ephemeral Disk Utilization (%) | VM Ephemeral Disk Utilization |
VM Load Average (15m) | Unix-style load average for the past 15 minutes |
VM Persistent Disk Utilization (%) | VM Persistent Disk Utilization |
VM Swap Usage (Kibibytes) | VM Swap Usage |
VM Swap Utilization (%) | VM Swap Utilization |
VM System Disk Utilization (%) | VM System Disk Utilization |
VM User CPU Utilization (%) | Time CPU spent in user mode |
Diego Cell
Name | Description |
---|---|
Active GoRoutines | Instantaneous number of active GoRoutines in the process |
CID | Virtual Machine Cloud ID |
Consul Down Mode | Health status of the consul associated with this route-rmitter. |
Counter Events | Number of Counter Events aggregated in Metron |
CPUs | Number of CPUs on the machine |
Current Buffer Count (Bytes) | Number of Dropsonde messages read by UDP socket but not yet unmarshalled |
Deployment | Deployment |
Dropped Messages | Number of intentionally dropped messages from Metron's batch writer buffer. Batch writing is performed over TCP/TLS only. |
Garden Active GoRoutines | Instantaneous number of active GoRoutines in the process |
Garden Backing Stores | Number of container backing store files |
Garden CPUs | Number of CPUs on the machine |
Garden Depot Directories | Number of directories in the Garden depot |
Garden Heap Memory Usage (Bytes) | Instantaneous count memory allocated on the main heap and still in use |
Garden Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
Garden Loop Devices | Number of attached loop devices |
Garden Memory Allocations | Lifetime number of memory allocations |
Garden Memory Deallocations | Lifetime number of memory deallocations |
Garden Memory Usage (Bytes) | Instantaneous count of memory allocated and still in use |
Garden Metrics Reporting | How long it took to emit the BackingStores, DepotDirs, and LoopDevices metrics |
Garden Stack Memory Usage (Bytes) | Instantaneous count of memory allocated on the main stack and still in use |
Heap Memory Usage (Bytes) | Instantaneous count memory allocated on the main heap and still in use |
HTTPStart Count | Number of HTTPStart aggregated in Metron |
HTTPStartStop Count | Number of HTTPStartStop events emitted by Metron (created by combining HTTPStart and HTTPStop events) |
HTTPStop Count | Number of HTTPStop aggregated in Metron |
Index | Index |
IP | IP |
Job | Job |
Job State | Job State |
Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
Log Messages | Count of application log messages. |
Marshalled ContainerMetric Messages | Number of ContainerMetric messages marshalled |
Marshalled CounterEvent Messages | Number of CounterEvent messages marshalled |
Marshalled Error Messages | Number of Error messages marshalled |
Marshalled Heartbeat Messages | Number of Heartbeat messages marshalled |
Marshalled HTTPStart Messages | Number of HttpStart messages marshalled |
Marshalled HTTPStartStop Messages | Number of HttpStartStop messages marshalled |
Marshalled HTTPStop Messages | Number of HttpStop messages marshalled |
Marshalled LogMessage Messages | Number of LogMessage messages marshalled |
Marshalled ValueMetric Messages | Number of ValueMetric messages marshalled |
Marshalling Errors (Errors) | Number of errors when marshalling messages |
Maximum Route Emitter Synchronization Pass Duration (Nanoseconds) | The max time for the last 15 minutes the active route-emitter took to perform its synchronization pass. |
Memory Allocations | Lifetime number of memory allocations |
Memory Deallocations | Lifetime number of memory deallocations |
Memory Usage (Bytes) | Instantaneous count of memory allocated and still in use |
Received Data (Bytes) | Number of bytes of Dropsonde messages read by UDP socket |
Received Legacy Data (Bytes) | Number of bytes of Legacy messages read by UDP socket |
Received Legacy Messages | Number of Legacy messages read by UDP socket |
Received Messages | Number of Dropsonde messages read by UDP socket |
REP Active GoRoutines | Instantaneous number of active GoRoutines in the process |
REP ActualLRP Synchronization Duration (Nanoseconds) | Time the cell rep took to synchronize the ActualLRPs it has claimed with its actual garden containers. |
REP ActualLRP Synchronization Maximum Duration (Nanoseconds) | The maximum time for the last 15 minutes the cell rep took to synchronize the ActualLRPs it has claimed with its actual garden containers. |
REP Containers | Number of containers hosted on the cell. |
REP Containers Remaining Space | Remaining number of containers this cell can host. |
REP CPUs | Number of CPUs on the machine |
REP Garden Creation Time (Nanoseconds) | Time the rep Garden backend took to create a container. |
REP Heap Memory Usage (Bytes) | Instantaneous count memory allocated on the main heap and still in use |
REP is Unhealthy | Whether the cell has failed to pass its health check against the garden backend. |
REP Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
REP Memory Allocations | Lifetime number of memory allocations |
REP Memory Deallocations | Lifetime number of memory deallocations |
REP Memory Usage (Bytes) | Instantaneous count of memory allocated and still in use |
REP Remaining Disk Available (Mebibytes) | Remaining amount of disk available for this cell to allocate to containers. |
REP Remaining Memory Available (Mebibytes) | Remaining amount of memory available for this cell to allocate to containers. |
REP Stack Memory Usage (Bytes) | Instantaneous count of memory allocated on the main stack and still in use |
REP Total Available Containers | Total number of containers this cell can host. |
REP Total Available Disk (Mebibytes) | Total amount of disk available for this cell to allocate to containers. |
REP Total Available Memory (Mebibytes) | Total amount of memory available for this cell to allocate to containers. |
Route Emitter Active Goroutines | Instantaneous number of active GoRoutines in the process |
Route Emitter CPUs | Number of CPUs on the machine |
Route Emitter Heap Memory Usage (Bytes) | Instantaneous count of memory allocated on the main heap and still in use |
Route Emitter Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
Route Emitter Lock Held | Whether a Route Emitter holds the Route Emitter lock. |
Route Emitter Lock Held Duration (Nanoseconds) | Time the active Route Emitter has held the convergence lock. |
Route Emitter Memory Allocations | Lifetime number of memory allocations |
Route Emitter Memory Deallocations | Lifetime number of memory deallocations |
Route Emitter Memory Usage (Bytes) | Instantaneous count of memory allocated and still in use |
Route Emitter Messages Emitted | The cumulative number of registration messages that this process has sent |
Route Emitter Registered Routes | Cumulative number of route registrations emitted from the route-emitter as it reacts to changes to LRPs |
Route Emitter Routes | Number of routes in the route-emitter's routing table. |
Route Emitter Stack Memory Usage (Bytes) | Instantaneous count of memory used by the stack allocator |
Route Emitter Synchronization Pass Duration (Nanoseconds) | Time the active route-emitter took to perform its synchronization pass. |
Route Emitter Synchronized Routes | Cumulative number of route registrations emitted from the route-emitter during its periodic route-table synchronization |
Route Emitter Unregistered Routes | Cumulative number of route unregistrations emitted from the route-emitter as it reacts to changes to LRPs |
Sent Messages | Number of messages sent to Doppler regardless of protocol |
Stack Memory Usage (Bytes) | Instantaneous count of memory allocated on the main stack and still in use |
TCP Send Errors (Errors) | Number of errors if writing to Doppler over TCP fails |
TCP Sent Data (Bytes) | Lifetime data sent to Doppler over TCP |
TCP Sent Messages | Number of sent messages to Doppler over TCP |
TLS Send Errors (Errors) | Number of errors if writing to Doppler over TLS fails |
TLS Sent Data (Bytes) | Lifetime data sent to Doppler over TLS |
TLS Sent Messages | Number of sent messages to Doppler over TLS |
UDP Send Errors (Errors) | Number of errors if writing to Doppler over UDP fails |
UDP Sent Data (Bytes) | Lifetime data sent to Doppler over UDP |
UDP Sent Messages | Number of sent messages to Doppler over UDP |
Uncategorized Events | Number of non-(CounterEvent HTTPStart HTTPStop) events processed by aggregator |
Unmarshalled ContainerMetric Messages | Number of ContainerMetric messages unmarshalled |
Unmarshalled CounterEvent Messages | Number of CounterEvent messages unmarshalled |
Unmarshalled Error Messages | Number of Error messages unmarshalled |
Unmarshalled HTTPStart Messages | Number of HttpStart messages unmarshalled |
Unmarshalled HTTPStartStop Messages | Number of HttpStartStop messages unmarshalled |
Unmarshalled HTTPStop Messages | Number of HttpStop messages unmarshalled |
Unmarshalled LogMessage Messages | Number of LogMessage messages unmarshalled |
Unmarshalled ValueMetric Messages | Number of ValueMetric messages unmarshalled |
Unmarshalling Errors (Errors) | Number of errors when unmarshalling messages |
Unmatched HTTPStart Count | Number of HTTPStart events for which no HTTPStop was received |
Unmatched HTTPStop Count | Number of HTTPStop events for which no HTTPStart was received |
VM Active Memory Usage (Kibibytes) | VM Active Memory Usage |
VM Active Memory Utilization (%) | VM Active Memory Utilization |
VM CPU Utilization (%) | Combined amount of CPU spent in system and user processes |
VM Ephemeral Disk Utilization (%) | VM Ephemeral Disk Utilization |
VM Load Average (15m) | Unix-style load average for the past 15 minutes |
VM Persistent Disk Utilization (%) | VM Persistent Disk Utilization |
VM Swap Usage (Kibibytes) | VM Swap Usage |
VM Swap Utilization (%) | VM Swap Utilization |
VM System Disk Utilization (%) | VM System Disk Utilization |
VM User CPU Utilization (%) | Time CPU spent in user mode |
Doppler Server
Name | Description |
---|---|
Active GoRoutines | Instantaneous number of active Goroutines in the Doppler process |
CID | Virtual Machine Cloud ID |
Container Metric Sinks | Instantaneous number of container metric sinks known to the SinkManager |
CPUs | Number of CPUs on the machine |
Deployment | Deployment |
Dropped Messages | Number of messages dropped inside Doppler for various reasons (downstream consumer can't keep up internal object wasn't ready for message, etc.) |
Dump Sinks | Instantaneous number of dump sinks known to the SinkManager |
Errors When Unmarshalling (Errors) | Number of errors when unmarshalling messages |
File Handles | Number of file handles for the Doppler's process |
Firehose Dropped Messages | Number of messages dropped by the firehose |
Firehose Dropped Messages Delta Maximum | The maximum delta between each minute over the past 5 minutes of the cumulative number of messages dropped by the firehose |
Firehose Received Messages Delta Maximum | The maximum delta between each minute over the past 5 minutes of the cumulative number of messages received across all Doppler listeners (UDP, TCP, TLS) |
Firehose Sent Messages | Number of sent messages through the firehose per subscription id |
Firehose Sinks | Instantaneous number of firehose sinks known to the SinkManager |
Firehose Total Received Messages | Total number of messages received across all of Doppler's listeners (UDP, TCP, TLS) |
Heap Memory Usage (Bytes) | Instantaneous count of memory allocated on the main heap and still in use |
Index | Index |
Invalid Signature Errors (Errors) | Number of messages received with an invalid signature |
IP | IP |
Job | Job |
Job State | Job State |
Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause in nanoseconds |
Memory Allocations | Lifetime number of memory allocations |
Memory Deallocations | Lifetime number of memory deallocations |
Memory Usage (Bytes) | Instantaneous count of memory allocated and still in use |
Messages Received | Number of messages received by Doppler's internal MessageRouter |
Missing Signature Errors (Errors) | Number of messages received that are too small to contain a signature |
Stack Memory Usage (Bytes) | Instantaneous count of memory used by the stack allocator |
Syslog Sinks | Instantaneous number of syslog sinks known to the SinkManager |
TCP Errors (Errors) | Number of errors encountered by Doppler's TCP Listener while handshaking, decoding or reading from the connection |
TCP Messages Received | Number of messages received by Doppler's TCP Listener |
TCP Received (Bytes) | Lifetime data received by Doppler's TCP Listener |
TLS Errors (Errors) | Number of errors encountered by Doppler's TLS Listener while handshaking, decoding or reading from the connection |
TLS Messages Received | Number of messages received by Doppler's TLS Listener |
TLS Received (Bytes) | Lifetime data received by Doppler's TLS Listener |
Truncating Buffer Dropped Messages | Number of messages intentionally dropped by Doppler from the sink for the specific sink. This counter event will correspond with log messages "Log message output is too high." |
UDP Errors (Errors) | Number of errors encountered by Doppler's UDP Listener while reading from the connection |
UDP Messages Received | Number of messages received by Doppler's UDP Listener |
UDP Received (Bytes) | Lifetime data received by Doppler's UDP Listener |
Unmarshalled ContainerMetric Messages | Number of ContainerMetric messages unmarshalled |
Unmarshalled CounterEvent Messages | Number of CounterEvent messages unmarshalled |
Unmarshalled Error Messages | Number of Error messages unmarshalled |
Unmarshalled HttpStart Messages | Lifetime number of HttpStart messages unmarshalled |
Unmarshalled HttpStartStop Messages | Number of HttpStartStop messages unmarshalled |
Unmarshalled HttpStop Messages | Number of HttpStop messages unmarshalled |
Unmarshalled LogMessage Messages | Number of LogMessage messages unmarshalled |
Unmarshalled ValueMetric Messages | Number of ValueMetric messages unmarshalled |
Uptime (Seconds) | Uptime for the Doppler's process |
Valid Signatures | Number of messages received with valid signatures |
VM Active Memory Usage (Kibibytes) | VM Active Memory Usage |
VM Active Memory Utilization (%) | VM Active Memory Utilization |
VM CPU Utilization (%) | Combined amount of CPU spent in system and user processes |
VM Ephemeral Disk Utilization (%) | VM Ephemeral Disk Utilization |
VM Load Average (15m) | Unix-style load average for the past 15 minutes |
VM Persistent Disk Utilization (%) | VM Persistent Disk Utilization |
VM Swap Usage (Kibibytes) | VM Swap Usage |
VM Swap Utilization (%) | VM Swap Utilization |
VM System Disk Utilization (%) | VM System Disk Utilization |
VM User CPU Utilization (%) | Time CPU spent in user mode |
Websocket Sinks | Instantaneous number of websocket sinks known to the SinkManager |
etcd Server
Name | Description |
---|---|
Appends Received | Number of append requests this node has processed |
Appends Sent | Number of append requests this node has sent |
CID | Virtual Machine Cloud ID |
Compare and Delete Fail Operations (Operations) | Compare And Delete Fail operation count |
Compare and Delete Success Operations (Operations) | Compare And DeleteSuccess operation count |
Compare and Swap Fail Operations (Operations) | Compare And SwapFail operation count |
Compare and Swap Success Operations (Operations) | Compare And SwapSuccess operation count |
Create Fail Operations (Operations) | Create Fail operation count |
Create Success Operations (Operations) | Create Success operation count |
Delete Fail Operations (Operations) | Delete Fail operation count |
Delete Success Operations (Operations) | Delete Success operation count |
Deployment | Deployment |
etcd Followers | Number of etcd followers |
Expire Count Operations (Operations) | Expire Count operation count |
Gets Fail Operations (Operations) | Gets Fail operation count |
Gets Success Operations (Operations) | Gets Success operation count |
Index | Index |
IP | IP |
Is Leader | Signals if etcd is leader |
Job | Job |
Job State | Job State |
Leader Latency (Seconds) | Current latency to a specific follower |
Raft Term | X-Raft-Term value from the /stats/store endpoint |
Received Data Rate (Bytes per Second) | Number of data received |
Received Request Rate (per Second) | Number of requests received. |
Send Data Rate (Bytes per Second) | Number of data sent. |
Send Request Rate (per Second) | Number of requests sent. |
Sets Fail Operations (Operations) | Sets Fail operation count |
Sets Success Operations (Operations) | Sets Success operation count |
Update Fail Operations (Operations) | Update Fail operation count |
Update Success Operations (Operations) | Update Success operation count |
VM Active Memory Usage (Kibibytes) | VM Active Memory Usage |
VM Active Memory Utilization (%) | VM Active Memory Utilization |
VM CPU Utilization (%) | Combined amount of CPU spent in system and user processes |
VM Ephemeral Disk Utilization (%) | VM Ephemeral Disk Utilization |
VM Load Average (15m) | Unix-style load average for the past 15 minutes |
VM Persistent Disk Utilization (%) | VM Persistent Disk Utilization |
VM Swap Usage (Kibibytes) | VM Swap Usage |
VM Swap Utilization (%) | VM Swap Utilization |
VM System Disk Utilization (%) | VM System Disk Utilization |
VM User CPU Utilization (%) | Time CPU spent in user mode |
Watchers Operations (Operations) | Watchers operation count |
GoRouter
Name | Description |
---|---|
Active GoRoutines | Instantaneous number of active GoRoutines in the process. |
Average Route-Emitter Received Messages Rate (per Second) | Average number of router register messages received for Route-Emitter rate. |
Average Total Requests (Requests per Second) | Average rate of Requests. |
Bad Gateway Count | The number of bad gateways associated with this GoRouter. |
CID | Virtual Machine Cloud ID |
CPUs | Number of CPUs on the machine. |
Delta Average Total Routes Registered | The average change in the number of routes associated with this GoRouter. |
Deployment | Deployment |
Exhausted Connections Maximum Rate (per Minute) | The maximum number of file descriptors currently used by the Gorouter job over the last 5 minutes |
Heap Memory Usage (Bytes) | Instantaneous count of memory allocated on the main heap and still in use. |
Index | Index |
IP | IP |
Job | Job |
Job State | Job State |
Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
Maximum Bad Gateway Count | The maximum number of bad gateways associated with this GoRouter in the last 5 minutes. |
Maximum File Descriptors | The maximum number of file descriptors currently used by the Gorouter job over the last 5 minutes |
Maximum Server Errors Rate | The maximum number of server errors associated with this GoRouter in the last 5 minutes. |
Maximum Time Since Last Registry Update (Milliseconds) | Maximum number of milliseconds since the last registry update in the last 5 minutes. |
Memory Allocations | Lifetime number of memory allocations. |
Memory Deallocations | Lifetime number of memory deallocations. |
Memory Usage (Bytes) | Instantaneous count of memory allocated and still in use. |
Router Handling Average Latency (Milliseconds) | The router handling latency. |
Router Handling Latency (Milliseconds) | The router handling latency. |
Server Errors | The number of server errors associated with this GoRouter. |
Stack Memory Usage (Bytes) | Instantaneous count of memory used by the stack allocator. |
Time Since Last Registry Update (Milliseconds) | Number of milliseconds since the last registry update. |
Total Cloud Controller Received Messages | Total number of router register messages received for Cloud Controller. |
Total Cloud Controller Received Requests | Total number of requests received for Cloud Controller. |
Total HTTP Responses | Total number of HTTP responses. |
Total Received Messages | Total number of router register messages received. |
total requests | Total number of request received. |
Total Route-Emitter Received Messages | Total number of router register messages received for Route-Emitter. |
Total Routes Registered | The number of routes associated with this GoRouter. |
VM Active Memory Usage (Kibibytes) | VM Active Memory Usage |
VM Active Memory Utilization (%) | VM Active Memory Utilization |
VM CPU Utilization (%) | Combined amount of CPU spent in system and user processes |
VM Ephemeral Disk Utilization (%) | VM Ephemeral Disk Utilization |
VM Load Average (15m) | Unix-style load average for the past 15 minutes |
VM Persistent Disk Utilization (%) | VM Persistent Disk Utilization |
VM Swap Usage (Kibibytes) | VM Swap Usage |
VM Swap Utilization (%) | VM Swap Utilization |
VM System Disk Utilization (%) | VM System Disk Utilization |
VM User CPU Utilization (%) | Time CPU spent in user mode |
Organization
Name | Description |
---|---|
Application Count | Application Count |
Billing Enabled | Billing Enabled |
Creation Date | Creation Date |
GUID | GUID |
Service Count | Service Count |
Space Count | Space Count |
Status | Status |
Service
Name | Description |
---|---|
Creation Date | Creation Date |
Description | Description |
GUID | GUID |
Is Active | Is Active |
Is Bindable | Is Bindable |
Service Plan
Name | Description |
---|---|
Description | Description |
GUID | GUID |
Space
Name | Description |
---|---|
Application Count | Application Count |
Creation Date | Creation Date |
GUID | GUID |
Organization GUID | Organization GUID |
Service Count | Service Count |
SSH Allowed | SSH Allowed |
Syslog Drain Binder
Name | Description |
---|---|
Active GoRoutines | Instantaneous number of active GoRoutines in the Doppler process |
CID | Virtual Machine Cloud ID |
Cloud Controller Polled Count | Number of times the syslog drain binder has polled the cloud controller for syslog drain bindings |
CPUs | Number of CPUs on the machine |
Deployment | Deployment |
Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause |
Heap Memory Usage (Bytes) | Instantaneous count of memory allocated on the main heap and still in use |
Index | Index |
IP | IP |
Job | Job |
Memory Allocations | Lifetime number of memory allocations |
Memory Deallocations | Lifetime number of memory deallocations |
Memory Usage (Bytes) | Instantaneous count of memory allocated and still in use |
Stack Memory Usage (Bytes) | Instantaneous count of memory used by the stack allocator |
Syslog Drains Returned by Cloud Controller | Number of syslog drains returned by cloud controller |
Traffic Controller
Name | Description |
---|---|
Active GoRoutines | Instantaneous number of active GoRoutines in the Doppler process |
CID | Virtual Machine Cloud ID |
CPUs | Number of CPUs on the machine |
Deployment | Deployment |
File Handles | Number of file handles for the TrafficController's process |
Heap Memory Usage (Bytes) | Instantaneous count of memory allocated on the main heap and still in use |
Index | Index |
IP | IP |
Job | Job |
Job State | Job State |
Last Garbage Collector Pause (Nanoseconds) | Duration of the last Garbage Collector pause in nanoseconds |
Memory Allocations | Lifetime number of memory allocations |
Memory Deallocations | Lifetime number of memory deallocations |
Memory Usage (Bytes) | Instantaneous count of memory allocated and still in use |
Serving Container Metrics Duration (Milliseconds) | Duration for serving container metrics via the containermetrics endpoint |
Serving Recent Logs Duration (Milliseconds) | Duration for serving recent logs via the recentLogs endpoint |
Stack Memory Usage (Bytes) | Instantaneous count of memory used by the stack allocator |
Uptime (Seconds) | Uptime for the Traffic Controller's process |
VM Active Memory Usage (Kibibytes) | VM Active Memory Usage |
VM Active Memory Utilization (%) | VM Active Memory Utilization |
VM CPU Utilization (%) | Combined amount of CPU spent in system and user processes |
VM Ephemeral Disk Utilization (%) | VM Ephemeral Disk Utilization |
VM Load Average (15m) | Unix-style load average for the past 15 minutes |
VM Persistent Disk Utilization (%) | VM Persistent Disk Utilization |
VM Swap Usage (Kibibytes) | VM Swap Usage |
VM Swap Utilization (%) | VM Swap Utilization |
VM System Disk Utilization (%) | VM System Disk Utilization |
VM User CPU Utilization (%) | Time CPU spent in user mode |
Updated almost 2 years ago