Please refer to the
Microsoft Azure Sources topic for additional information on how to configure the LPU, and general Azure Data Collection setup details.
Using the Azure CLI Client, find the
Subscription ID and Tenant ID from your account list Create a custom RBAC role using the JSON provided. Include your
Subscription ID and rename the file to azure.json Create an Active Directory Service Principal and assign the custom RBAC role t it.
"Name": "LPU HDInsight",
"Description": "LPU for HDInsight",
Name Required? Description Subscription ID Required GUID Subscription ID Tenant ID Required GUID Tenant ID (also known as Directory ID) Client ID Required GUID Client ID (also known as Application ID) Client Secret Required The Secret (also known as Key) corresponding to the Client ID. Maximum HTTP Retry Time (seconds) The maximum amount of time in seconds to retry each API request when the API is throttling. HTTP Request Timeout (seconds) The maximum amount of time in seconds before a single HTTP request will fail.
Name Description Average Pages The average amount of pages needed for a paged resource type. Average Request Retries The average number of retry requests per unique requests made. Average Retry Attempts The average number of retry requests made per unique request that was retried. Average Retry Wait (Milliseconds) The average amount of time retried requests spent waiting. Client ID The client ID used to make API calls. Failed Requests The total number of requests that returned a failure response. Maximum Pages The most amount of pages needed for a paged resource type. Maximum Retries The highest number of retries made for a single request. Maximum Retry Wait (Milliseconds) The most amount of time a retried request spent waiting. Minimum Pages The least amount of pages needed for a paged resource type. Minimum Retry Wait (Milliseconds) The least amount of time a retried request spent waiting. Other Status Responses The total number of successful requests that responded with some other accepted status. Request Timeouts The total number of requests that timed out waiting for a response. Requests Retried The number of unique requests that were retried. Retry Status Responses The total number of successful requests that responded with the status TOO MANY REQUESTS (429). Retry Timeouts The total number of requests that needed to be retried, but the request retry time exceeded the maximum retry time. Status OK Responses The total number of successful requests that responded with the status OK (200). Subscription ID The subscription ID used to make API calls. Successful Requests The total number of requests that returned a successful response. Tenant ID The tenant ID used to make API calls. Total Monitor Requests The total number of requests made to get monitor metrics. Total Paged Requests The total amount of resource types that required paging. Total Requests The total number of requests made during collection. Total Retries The total number of retry requests that were made. Unique Monitor Requests The number of unique requests made to get monitor metrics. Unique Requests The number of requests made with unique endpoints.
Name Description ID Unique resource identifier. Name Specifies the name of the application. The name in the form 'clusterName/applicationName'. Provisioning State Indicates the current provisioning state. State Indicates the more detailed application state while provisioning is in progress. Type Microsoft Azure resource type.
Name Description Blueprint Specifies the cluster blueprint url. Categorized Gateway Requests (Requests) Number of gateway requests by categories (1xx/2xx/3xx/4xx/5xx). CPU Used (Cores) Amount of CPU allocation used by cluster. Created Date Datetime when the cluster create request was received. Gateway Requests (Requests) Number of gateway requests. ID Specifies the resource identifier of the cluster. Kind Specifies the cluster type. Valid values are hadoop, hbase, storm & spark. Location Specifies the supported Azure location where the cluster is located. Name Specifies the name of the cluster. OS Type Specifies the Operating system for the cluster. Valid values are Linux and Windows. Provisioning State Indicates the current provisioning state. State Indicates the more detailed HDInsight cluster state while provisioning is in progress. Tags Specifies the tags that are assigned to the cluster. Tier Microsoft Azure tier type for the cluster. Type Microsoft Azure resource type. Version Specifies the cluster version.
Name Description Location Specifies the URL to connect. Name Friendly name for the connectivity endpoint. Parent ID The Cluster ID that the endpoint belongs to. Port Specifies the port to connect. Protocol Specifies the Protocol to use (example: HTTPS, SSH).
Name Description Instance Count Specifies the target instance count for the role. Name Specifies the role name. Parent ID The Cluster ID that the role belongs to. Virtual Machine Size Specifies the size (tier) of the Virtual Machine.