AWS Integration - Kinesis Data Firehose

Mackerel supports obtaining and monitoring the metric of Amazon Kinesis Data Firehose in AWS Integration. When integrating with AWS Integration, billable targets are determined using the conversion 1 delivery stream = 1 Micro Host. In addition to this, depending on the number of metrics retrieved, you may be charged for exceeding the maximum number of metrics per micro host.

Please refer to the following page for AWS Integration configuration methods and a list of supported AWS services.
AWS Integration

Obtaining metrics

The metrics obtainable with AWS Integration's support for Kinesis Data Firehose are as follows. For Metric explanations, refer to the AWS help page.

The maximum number of metrics obtainable is detemined by destination services:

  • In S3 cases:49
  • In Elasticsearch Service cases:38
  • In OpenSearch Service cases:38
  • In Redshift cases:42
  • In Splunk cases:39
  • In HTTP Endpoint cases:40
Graph name Metric Metric name in Mackerel Unit Statistics
Service Bytes BackupToS3.Bytes
DeliveryToElasticsearch.Bytes
DeliveryToAmazonOpenSearchService.Bytes
DeliveryToRedshift.Bytes
DeliveryToS3.Bytes
DeliveryToSplunk.Bytes
IncomingBytes
DataReadFromKinesisStream.Bytes
DeliveryToHttpEndpoint.Bytes
DeliveryToHttpEndpoint.ProcessedBytes
firehose.service_bytes.backup_to_s3
firehose.service_bytes.delivery_to_elasticsearch
firehose.service_bytes.delivery_to_opensearch
firehose.service_bytes.delivery_to_redshift
firehose.service_bytes.delivery_to_s3
firehose.service_bytes.delivery_to_splunk
firehose.service_bytes.incoming
firehose.service_bytes.read_from_kinesis_streams
firehose.service_bytes.delivery_to_http_endpoint
firehose.service_bytes.delivery_to_http_endpoint_processed
bytes Sum
Service Records BackupToS3.Records
DeliveryToElasticsearch.Records
DeliveryToAmazonOpenSearchService.Records
DeliveryToRedshift.Records
DeliveryToS3.Records
DeliveryToSplunk.Records
IncomingRecords
DataReadFromKinesisStream.Records
DeliveryToHttpEndpoint.Records
DeliveryToHttpEndpoint.ProcessedRecords
firehose.service_records.backup_to_s3
firehose.service_records.delivery_to_elasticsearch
firehose.service_records.delivery_to_opensearch
firehose.service_records.delivery_to_redshift
firehose.service_records.delivery_to_s3
firehose.service_records.delivery_to_splunk
firehose.service_records.incoming
firehose.service_records.read_from_kinesis_streams
firehose.service_records.delivery_to_http_endpoint
firehose.service_records.delivery_to_http_endpoint_processed
integer Sum
Service Success BackupToS3.Success
DeliveryToElasticsearch.Success
DeliveryToAmazonOpenSearchService.Success
DeliveryToRedshift.Success
DeliveryToS3.Success
DeliveryToSplunk.Success
DeliveryToHttpEndpoint
firehose.service_success.backup_to_s3
firehose.service_success.delivery_to_elasticsearch
firehose.service_success.delivery_to_opensearch
firehose.service_success.delivery_to_redshift
firehose.service_success.delivery_to_s3
firehose.service_success.delivery_to_splunk
firehose.service_success.delivery_to_http_endpoint
float Average
Service Data Freshness BackupToS3.DataFreshness
DeliveryToElasticsearch.DataFreshness
DeliveryToAmazonOpenSearchService.DataFreshness
DeliveryToRedshift.DataFreshness
DeliveryToS3.DataFreshness
DeliveryToSplunk.DataFreshness
DeliveryToHttpEndpoint.DataFreshness
firehose.service_data_freshness.backup_to_s3
firehose.service_data_freshness.delivery_to_elasticsearch
firehose.service_data_freshness.delivery_to_opensearch
firehose.service_data_freshness.delivery_to_redshift
firehose.service_data_freshness.delivery_to_s3
firehose.service_data_freshness.delivery_to_splunk
firehose.service_data_freshness.delivery_to_http_endpoint
float Maximum
Service Data Ack Latency DeliveryToSplunk.DataAckLatency firehose.service_data_ack_latency.delivery_to_splunk float Maximum
Millis Behind Latest KinesisMillisBehindLatest firehose.millis_behind_latest.kinesis integer Sum
API Latency DescribeDeliveryStream.Latency
PutRecord.Latency
PutRecordBatch.Latency
UpdateDeliveryStream.Latency
firehose.api_latency.describe_delivery_stream_average
firehose.api_latency.describe_delivery_stream_maximum
firehose.api_latency.put_record_average
firehose.api_latency.put_record_maximum
firehose.api_latency.put_record_batch_average
firehose.api_latency.put_record_batch_maximum
firehose.api_latency.update_delivery_stream_average
firehose.api_latency.update_delivery_stream_maximum
float Average
Maximum
API Bytes PutRecord.Bytes
PutRecordBatch.Bytes
firehose.api_bytes.put_record
firehose.api_bytes.put_record_batch
bytes Sum
API Requests DescribeDeliveryStream.Requests
PutRecord.Requests
PutRecordBatch.Requests
UpdateDeliveryStream.Requests
firehose.api_requests.describe_delivery_stream
firehose.api_requests.put_record
firehose.api_requests.put_record_batch
firehose.api_requests.update_delivery_stream
integer Sum
API Records PutRecordBatch.Records firehose.api_records.put_record_batch integer Sum
API Throttled ThrottledDescribeStream
ThrottledGetRecords
ThrottledGetShardIterator
firehose.api_throttled.describe_stream
firehose.api_throttled.get_records
firehose.api_throttled.get_shard_iterator
integer Sum
Data Transformation Duration ExecuteProcessing.Duration firehose.data_transformation_duration.execute float Average
Data Transformation Success ExecuteProcessing.Success firehose.data_transformation_success.execute float Average
Data Transformation Records SucceedProcessing.Records firehose.data_transformation_records.succeed integer Sum
Data Transformation Bytes SucceedProcessing.Bytes firehose.data_transformation_bytes.succeed bytes Sum
Format Conversion Records SucceedConversion.Records
FailedConversion.Records
firehose.format_conversion_records.succeed
firehose.format_conversion_records.failed
integer Sum
Format Conversion Bytes SucceedConversion.Bytes
FailedConversion.Bytes
firehose.format_conversion_bytes.succeed
firehose.format_conversion_bytes.failed
bytes Sum
Incoming Put Requests IncomingPutRequests firehose.incoming_put_requests.count integer Sum
Throttled Records ThrottledRecords firehose.throttled_records.records integer Sum
Bytes Per Second Limit BytesPerSecondLimit firehose.bytes_per_second_limit.limit bytes Average
Records Per Second Limit RecordsPerSecondLimit firehose.records_per_second_limit.limit float Average
KMS Exceptions KMSKeyAccessDenied
KMSKeyDisabled
KMSKeyInvalidState
KMSKeyNotFound
firehose.kms_exceptions.key_access_denied
firehose.kms_exceptions.key_disabled
firehose.kms_exceptions.key_invalid_state
firehose.kms_exceptions.key_not_found
integer Sum
Put Requests Per Second Limit PutRequestsPerSecondLimit firehose.put_requests_per_second_limit.limit float Average
Active Partitions PartitionCount firehose.active_partitions.count integer Maximum
Partition Processing Duration [ms] JQProcessing.Duration firehose.partition_processing_duration.jq float Average
Partition Limit PartitionCountExceeded firehose.partition_limit.exceeded integer Maximum
Per Partition Throughput PerPartitionThroughput firehose.per_partition_throughput.maximum
firehose.per_partition_throughput.average
firehose.per_partition_throughput.minimum
bytes/sec Maximum
Average
Minimum
Delivered S3 Objects DeliveryToS3.ObjectCount firehose.delivered_s3_objects.count integer Sum