Google Cloud BigQuery

Monitor Google BigQuery the Large Dataset Interaction Tool

🚧

Not Included in the BindPlane with Google Cloud Monitoring offering

All of the Google Cloud Platform sources listed within this documentation are not included with the BindPlane with Google Cloud Monitoring offering.

For more information on how to use the below LPU and other Google Cloud Data Collection setup. See the Google Cloud Platform Sources

Least Privileged User

A user role with the following permissions is required:

📘

Deploying the Least Privileged User

To learn more about how to deploy a role with these permissions to a GCP Organization, or a GCP Project, please refer to this documentation:
Deploy an Individual LPU role to a GCP Project, or GCP Organization

- cloudnotifications.activities.list
- monitoring.alertPolicies.get
- monitoring.alertPolicies.list
- monitoring.dashboards.get
- monitoring.dashboards.list
- monitoring.groups.get
- monitoring.groups.list
- monitoring.metricDescriptors.get
- monitoring.metricDescriptors.list
- monitoring.monitoredResourceDescriptors.get
- monitoring.monitoredResourceDescriptors.list
- monitoring.notificationChannelDescriptors.get
- monitoring.notificationChannelDescriptors.list
- monitoring.notificationChannels.get
- monitoring.notificationChannels.list
- monitoring.publicWidgets.get
- monitoring.publicWidgets.list
- monitoring.timeSeries.list
- monitoring.uptimeCheckConfigs.get
- monitoring.uptimeCheckConfigs.list
- resourcemanager.projects.get
- resourcemanager.projects.list
- stackdriver.projects.get
- bigquery.datasets.get
- bigquery.datasets.getIamPolicy
- bigquery.tables.export
- bigquery.tables.get
- bigquery.tables.getData
- bigquery.tables.list

Connection Parameters

NameRequired?Description
Private Key JSONRequiredThe contents of the private key JSON file created when setting up a service account.
Metric CollectionControls which metrics get requested from GCP's Stackdriver API.
ProjectsRequiredA comma separated whitelist of project IDs. If the wildcard "*" is used, resources will be collected from all available projects.
Connection TimeoutThe number of seconds to allow for connecting to the target.

**

Metrics

Dataset

NameDescription
Friendly NameA descriptive name for the dataset.
IDA unique ID for this dataset, without the project name.
KindThe resource type.
LabelsThe labels associated with this dataset. You can use these to organize and group your datasets.
LocationThe geographic location where the data resides.
Project IDThe project ID in which this resource was created.
Stored Data (Bytes)Amount of data stored.
Table Count (Tables)Number of tables.
Uploaded Data (Bytes per Second)Rate of data upload.
Uploaded Data Billed (Bytes per Second)Rate of data upload that incurred a billing charge.
Uploaded Rows (Rows per Second)Rate of uploaded rows.

Project

NameDescription
Average Query CountAverage number of in-flight queries.
Average Query Execution Time (Seconds)Average query execution time.
Average Slots AllocatedAverage number of BigQuery slots currently allocated for the project.
Average Slots AvailableAverage number of BigQuery slots available for the project.
IDThe id of the project.
Scanned Data (Bytes per Second)Scanned data rate for queries overall.
Scanned Data Billed (Bytes per Second)Scanned data billed rate.

Table

NameDescription
Creation TimeThe time when this table was created, in milliseconds since the epoch.
Dataset IDThe id of the dataset that holds the table.
Expiration TimeThe time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.
Friendly NameThe user-friendly name for this table.
IDThe id of the table.
KindThe resource type.
LabelsThe labels associated with this table. You can use these to organize and group your tables.
Project IDThe id of the project that holds the table.
Stored Data (Bytes)Amount of data stored.
Time Partitioning Expiration (Milliseconds)Amount of time for which to keep the storage for a partition.
Time Partitioning FieldIf not set, the table is partitioned by pseudo column '_PARTITIONTIME'; if set, the table is partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED.
Time Partitioning FilterIf set to true, queries over this table require a partition filter that can be used for partition elimination to be specified.
Time Partitioning TypeThe amount of time for which one partition will be generated.
Total ItemsThe total number of tables in the dataset.
TypeThe type of table. Possible values are: TABLE, VIEW.
Uploaded Data (Bytes per Second)Rate of data upload.
Uploaded Data Billed (Bytes per Second)Rate of data upload that incurred a billing charge.
Uploaded Rows (Rows per Second)Rate of uploaded rows.
View Use Legacy SQLTrue if view is defined in legacy SQL dialect, false if in standard SQL.