Bindplane

The Bindplane Developer Hub

Welcome to the Bindplane developer hub. You'll find comprehensive guides and documentation to help you start working with Bindplane as quickly as possible, as well as support if you get stuck. Let's jump right in!

Amazon Kinesis

Least Privileged User

Navigate to the AWS console and create an IAM user with programmatic access. The user will need the following permissions. You can create a policy specifically for these permissions and apply the permissions to the user.

For more information, see: High Level AWS Source Configuration

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "kinesisvideo:ListFragments",
        "cloudwatch:GetMetricData",
        "kinesisvideo:ListStreams",
        "kinesis:GetShardIterator",
        "kinesisvideo:GetDataEndpoint",
        "kinesis:DescribeStream",
        "kinesisvideo:GetMedia",
        "kinesisanalytics:DescribeApplication",
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics",
        "kinesis:ListTagsForStream",
        "firehose:DescribeDeliveryStream",
        "kinesis:ListStreams",
        "kinesisvideo:GetMediaForFragmentList",
        "kinesisanalytics:DiscoverInputSchema",
        "kinesisvideo:ListTagsForStream",
        "cloudwatch:DescribeAlarmHistory",
        "cloudwatch:DescribeAlarmsForMetric",
        "cloudwatch:DescribeAlarms",
        "firehose:ListDeliveryStreams",
        "kinesis:GetRecords",
        "kinesisanalytics:ListApplications",
        "kinesisvideo:DescribeStream",
        "kinesis:DescribeLimits"
      ],
      "Resource": "*"
    }
  ]
}

Connection Parameters

NameRequired?Description
Region
Access Key IDRequired
Secret Access KeyRequired
Additional ThreadsThe number of additional threads allowed to be utilized during collection.
Request Timeout (seconds)The number of seconds to allow for the API to return a response.
Collect CloudWatch Metrics
CloudWatch Historic ModeIf enabled, retrieves a history of data points from CloudWatch. Otherwise, collects only the most recent data point for each metric.

Metrics

Analytic

NameDescription
ApplicationThe name of the analytic application.
Data Throughput (Bytes)The amount of data flowing through the application.
Firehose ARNThe Amazon Resource Name (ARN) of the Amazon Kinesis Firehose delivery stream.
Firehose Role ARNThe ARN of the IAM role that Amazon Kinesis Analytics assumes to access the Firehose delivery stream.
FlowThe direction of data movement, either input or output.
IDThe ID of the analytic.
In Application Stream NamesA CSV list of in-application stream names that are mapped to the stream source.
Input Parallelism CountThe number of in-application streams mapped to the streaming source.
Lambda ARNThe ARN of the AWS Lambda function that is used to preprocess the records in the stream.
Lambda Role ARNThe ARN of the IAM role that is used to access the AWS Lambda function.
Records (Records)The number of records being processed by the application.
RegionThe AWS Region this object belongs to.
Starting PositionThe starting position on the stream.
Stream ARNThe Amazon Resource Name (ARN) of the Amazon Kinesis stream.
Stream Role ARNThe ARN of the IAM role that Amazon Kinesis Analytics assumes to access the Kinesis stream.
Successes (Operations)The number of successful operations performed by the application.
Time Behind Latest (Milliseconds)Indicates how far behind an application is reading from the streaming source.

Application

NameDescription
ARNThe Amazon Resource Name (ARN) of the application.
CodeReturns the application code that was provided to perform data analysis on any of the in-application streams in the application.
DescriptionDescription of the application.
NameName of the application.
RegionThe AWS Region this object belongs to.
StatusStatus of the application.
Version IDThe current application version.

Firehose Delivery Stream

NameDescription
ARNThe Amazon Resource Name (ARN) of the delivery stream.
Data Read From Kinesis Stream (Bytes)The amount of data read from the stream.
Data Read From Kinesis Stream Records (Records)The number of records read from the stream.
Delivery To Elasticsearch Data (Bytes)The amount of data indexed to Amazon ES over the specified time period.
Delivery To Elasticsearch Records (Records)The number of records indexed to Amazon ES over the specified time period.
Delivery To Elasticsearch Successes (%)Sum of the successfully indexed records over the sum of records that were attempted.
Delivery To Redshift Data (Bytes)The amount of data copied to Amazon Redshift over the specified time period.
Delivery To Redshift Records (Records)The number of records copied to Amazon Redshift over the specified time period.
Delivery To Redshift Successes (%)Sum of successful Amazon Redshift COPY commands over the sum of all Amazon Redshift COPY commands.
Delivery To S3 Data (Bytes)The amount of data delivered to Amazon S3 over the specified time period.
Delivery To S3 Data Freshness (Seconds)Age (from getting into Kinesis Firehose to now) of the oldest record in Kinesis Firehose. Any record older than this age has been delivered to the S3 bucket.
Delivery To S3 Records (Records)The number of records delivered to Amazon S3 over the specified time period.
Delivery To S3 Successes (%)Sum of successful Amazon S3 put commands over sum of all Amazon S3 put commands.
Delivery To Splunk Data (Bytes)The amount of data delivered to Splunk over the specified time period.
Delivery To Splunk Data Freshness (Seconds)Age (from getting into Kinesis Firehose to now) of the oldest record in Kinesis Firehose. Any record older than this age has been delivered to Splunk.
Delivery To Splunk Records (Records)The number of records delivered to Splunk over the specified time period.
Delivery To Splunk Successes (%)The sum of the successfully indexed records over the sum of records that were attempted.
DescribeDeliveryStream Latency (Milliseconds)The time taken per DescribeDeliveryStream operation, measured over the specified time period.
DescribeDeliveryStream Requests (Requests)Total number of DescribeDeliveryStream requests.
Incoming Data (Bytes)The amount of data ingested into the Kinesis Firehose delivery stream over the specified time period.
Incoming Records (Records)The number of records ingested into the Kinesis Firehose delivery stream over the specified time period.
Kinesis Time Behind Latest (Milliseconds)Time difference between the current server time stamp and the server time stamp of the last fragment sent.
ListDeliveryStreams Latency (Milliseconds)The time taken per ListDeliveryStream operation, measured over the specified time period.
ListDeliveryStreams Requests (Requests)Total number of ListFirehose requests.
NameThe name of the Firehose Delivery Stream.
PutRecord Batch Data (Bytes)The amount of data put to the Kinesis Firehose delivery stream using PutRecordBatch over the specified time period.
PutRecord Batch Latency (Milliseconds)The time taken per PutRecordBatch operation, measured over the specified time period.
PutRecord Batch Records (Records)Total number of records from PutRecordBatch operations.
PutRecord Batch Requests (Requests)Total number of PutRecordBatch requests.
PutRecord Data (Bytes)The amount of data put to the Kinesis Firehose delivery stream using PutRecord over the specified time period.
PutRecord Latency (Milliseconds)The time taken per PutRecord operation, measured over the specified time period.
PutRecord Requests (Requests)Total number of PutRecord requests, which is equal to total number of records from PutRecord operations.
RegionThe AWS Region this object belongs to.
Source Stream ARNThe ARN of the source Kinesis stream.
Source Stream RoleThe ARN of the role used by the source Kinesis stream.
StatusThe status of the delivery stream.
Stream TypeThe delivery stream type.
Throttled DescribeStream (Requests)The total number of throttled DescribeStream requests.
Throttled GetRecords (Requests)The total number of throttled GetRecords requests.
Throttled GetShardIterator (Requests)The total number of throttled GetShardIterator requests.
UpdateDeliveryStream Latency (Milliseconds)The time taken per UpdateDeliveryStream operation, measured over the specified time period.
UpdateDeliveryStream Requests (Requests)Total number of UpdateDeliveryStream requests.
Version IDEach time the destination is updated for a delivery stream, the version ID is changed. This value is the latest version.

Shard

NameDescription
Adjacent Parent Shard IDThe shard ID of the shard adjacent to the shard's parent.
IDThe shard ID within the Kinesis stream.
Incoming Data (Bytes)The amount of data successfully put to the shard over the specified time period. This metric includes data from PutRecord and PutRecords operations.
Incoming Records (Records)The number of records successfully put to the shard over the specified time period. This metric includes record counts from PutRecord and PutRecords operations.
Iterator Age (Milliseconds)The age of the last record in all GetRecords calls made against a shard, measured over the specified time period. Age is the difference between the current time and when the last record of the GetRecords call was written to the stream.
Outgoing Data (Bytes)The amount of data retrieved from the shard, measured over the specified time period.
Outgoing Records (Records)The number of records retrieved from the shard, measured over the specified time period.
Parent Shard IDThe shard ID of the shard's parent.
Read Provisioned Throughput Exceeded (Requests)The number of GetRecords calls throttled for the shard over the specified time period.
RegionThe AWS Region this object belongs to.
Stream NameThe name of the shard's parent Kinesis stream.
Write Provisioned Throughput Exceeded (Records)The number of records rejected due to throttling for the shard over the specified time period. This metric includes throttling from PutRecord and PutRecords operations.

Stream

NameDescription
ARNThe Amazon Resource Name (ARN) for the stream.
Creation TimestampThe approximate time that the stream was created.
Encryption TypeThe server-side encryption type used on the stream.
GetRecords Data (Bytes)The amount of data retrieved from the Kinesis stream, measured over the specified time period.
GetRecords Iterator Age (Milliseconds)The age of the last record in all GetRecords calls made against an Kinesis stream, measured over the specified time period. Age is the difference between the current time and when the last record of the GetRecords call was written to the stream.
GetRecords Latency (Milliseconds)The time taken per GetRecords operation, measured over the specified time period.
GetRecords Record Count (Records)The number of records retrieved from the shard, measured over the specified time period.
GetRecords Successes (%)The number of successful GetRecords operations per stream, measured over the specified time period.
Incoming Data (Bytes)The amount of data successfully put to the Kinesis stream over the specified time period. This metric includes bytes from PutRecord and PutRecords operations.
Incoming Records (Records)The number of records successfully put to the Kinesis stream over the specified time period. This metric includes record counts from PutRecord and PutRecords operations.
KMS Key IDThe GUID for the customer-managed KMS key to use for encryption.
NameThe name of the Kinesis stream.
PutRecord Data (Bytes)The amount of data put to the Kinesis stream using the PutRecord operation over the specified time period.
PutRecord Latency (Milliseconds)The time taken per PutRecord operation, measured over the specified time period.
PutRecord Successes (%)Reflects the percentage of successful writes to a stream.
PutRecords Data (Bytes)The amount of data put to the Kinesis stream using the PutRecords operation over the specified time period.
PutRecords Latency (Milliseconds)The time taken per PutRecords operation, measured over the specified time period.
PutRecords Record Count (Records)The number of successful records in a PutRecords operation per Kinesis stream, measured over the specified time period.
PutRecords Successes (Requests)The number of PutRecords operations where at least one record succeeded, per Kinesis stream, measured over the specified time period.
Read Provisioned Throughput Exceeded (Requests)The number of GetRecords calls throttled for the stream over the specified time period.
RegionThe AWS Region this object belongs to.
Retention Period (Hours)The current retention period.
StatusThe current status of the stream.
Write Provisioned Throughput Exceeded (Records)The number of records rejected due to throttling for the stream over the specified time period. This metric includes throttling from PutRecord and PutRecords operations.

Video Stream

NameDescription
ARNThe Amazon Resource Name (ARN) of the video stream.
Data Retention Period (Hours)How long the stream retains data.
Device NameThe name of the device associated with the stream.
GetMediaForFragmentList Outgoing Data (Bytes)Total amount of data sent out from the service as part of the GetMediaForFragmentList API for a given stream.
GetMediaForFragmentList Outgoing FragmentsTotal number of fragments sent out from the service as part of the GetMediaForFragmentList API for a given stream.
GetMediaForFragmentList Outgoing Frames (Frames)Total number of frames sent out from the service as part of the GetMediaForFragmentList API for a given stream.
GetMediaForFragmentList Requests (Requests)Total number of GetMediaForFragmentList API requests for a given stream.
GetMedia Outgoing Data (Bytes)Total amount of data sent out from the service as part of the GetMedia API for a given stream.
GetMedia Outgoing FragmentsTotal number of fragments sent while doing GetMedia for the stream.
GetMedia Outgoing Frames (Frames)Total number of frames sent during GetMedia on the given stream.
GetMedia Requests (Requests)Total number of GetMedia API requests for a given stream.
GetMedia Successes (%)Indicates the rate of success.
GetMedia Time Behind Latest (Milliseconds)Time difference between the current server time stamp and the server time stamp of the last fragment sent.
KMS Key IDThe ID of the AWS Key Management Service (AWS KMS) key that Kinesis Video Streams uses to encrypt data on the stream.
ListFragments Latency (Milliseconds)Latency of the ListFragments API calls for the given stream name.
Media TypeThe MediaType of the stream.
NameThe name of the Kinesis video stream.
PutMedia Buffering ACK Latency (Milliseconds)Time difference between when the first byte of a new fragment is received by Kinesis Video Streams and when the Buffering Ack is sent for the fragment.
PutMedia Connection Errors (Errors)Total number of errors while establishing PutMedia connection for the stream.
PutMedia Error ACK CountTotal number of Error ACKs sent while doing PutMedia for the stream.
PutMedia Fragment Ingestion Latency (Milliseconds)Time difference between when the first and last bytes of a fragment are received by Kinesis Video Streams.
PutMedia Fragment Persist Latency (Milliseconds)Time taken from when the complete fragment data is received and archived.
PutMedia Incoming Data (Bytes)Amount of data received as part of PutMedia for the stream.
PutMedia Incoming FragmentsTotal number of complete fragments received as part of PutMedia for the stream.
PutMedia Incoming FramesTotal number of complete frames received as part of PutMedia for the stream.
PutMedia Persisted ACK Latency (Milliseconds)Time difference between when the last byte of a new fragment is received by and when the Persisted ACK is sent for the fragment.
PutMedia Received ACK Latency (Milliseconds)Time difference between when the last byte of a new fragment is received by and when the Buffering ACK is sent for the fragment.
PutMedia Requests (Requests)Total number of PutMedia API requests for a given stream.
PutMedia Successes (%)Indicates how many complete, valid fragments are sent.
RegionThe AWS Region this object belongs to.
StatusThe status of the stream.
VersionThe version of the stream.

Updated 8 months ago

Amazon Kinesis


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.