This function is only offered in the Trial plan and Standard plan.
AWS Integration allows you to manage your AWS cloud products as Mackerel hosts and monitor metrics.
Of the AWS cloud products, EC2 is billed as a standard host and the other products are billed as micro hosts. Additionally, the API of AWS will be called every 5 minutes for each targeted metric to be obtained. Please take note, for this reason, an Amazon CloudWatch API usage fee may occur.
Table of Contents (click to open/close)
- Supported AWS cloud products
- Integration method
- Configure automatic new metrics addition
- Configure automatic retirement
- Limit metrics retrieved
- Filter by tag
- Assigning roles by tags
- IAM policies used in AWS Integration
- FAQ
Supported AWS cloud products
Currently, the following AWS cloud products are supported. For information on obtaining metrics, please refer to each individual document.
EC2・ELB (CLB)・ALB・NLB・RDS・ElastiCache・Redshift・Lambda・SQS・DynamoDB・CloudFront・API Gateway・Kinesis Data Streams・S3・OpenSearch Service・ECS・SES・Step Functions・EFS・Kinesis Data Firehose・Batch・WAF・Billing・Route 53・Connect・DocumentDB・CodeBuild・Athena
Integration method
There are two ways to integrate AWS integration: by IAM role with authentication by AssumeRole, or by Access Key ID and Secret Access Key.
You are required to set up one of them, but from a security standpoint, we strongly recommend that you set up a set using IAM roles.
- 1. Choose one of the following methods to configure integration.
- 2. Grant policies
- 3. Confirm the host
1. Create an IAM role or IAM user for integration
1-a. Configure the IAM role
Creating a role with the IAM Management Console
Create a new role with the IAM Management Console.
Select Another AWS account from the role types.
Click the 'Create' button in the Mackerel AWS Integration Settings page to get an External ID. Enter 217452466226 as the authorized Account ID. Select the Require external ID option and specify the External ID obtained from Mackerel's configuration page. The Mackerel system uses the account to access the user’s role. With this configuration, only the Mackerel account can access the created role. Create the role without checking Require MFA.
The next step is to grant policies.
1-b. Configure the Access Key ID and Secret Access Key
The following method is NOT recommended for security protection reasons.
Creating a user with the IAM Management Console
Create a new user with the IAM Management Console. We recommend assigning an easy-to-understand name like MackerelAWSIntegrationUser for use in Mackerel’s AWS integration.
Registering the Access Key in Mackerel
Register the Access Key ID and Secret Access Key (displayed on the screen when creating the account) in Mackerel. Be careful not to mistake the organization to be registered.
2. Grant policies
Grant the policies listed below for the role. Be sure to use the policies and actions listed below and be careful not to grant FullAccess permission.
Also, the maximum number of policies that can be attached to one IAM role is limited to 10. This is a specification of AWS. If necessary, you can apply to AWS for an upper limit extension.
If you want to configure all the permissions used in AWS Integration, please refer this list of IAM policies used in AWS Integration.
| AWS Cloud Products | Policy / Action | Note |
|---|---|---|
| EC2 | AmazonEC2ReadOnlyAccess | |
| ELB (CLB) | AmazonEC2ReadOnlyAccess | |
| ALB | AmazonEC2ReadOnlyAccess | |
| NLB | AmazonEC2ReadOnlyAccess | |
| RDS | AmazonRDSReadOnlyAccess | |
| ElastiCache | AmazonElastiCacheReadOnlyAccess | |
| Redshift | AmazonRedshiftReadOnlyAccess | |
| Lambda *1 | AWSLambda_ReadOnlyAccess | |
| SQS | AmazonSQSReadOnlyAccess | |
| DynamoDB | AmazonDynamoDBReadOnlyAccess | |
| CloudFront *1 | CloudFrontReadOnlyAccess | |
| API Gateway *1 | apigateway:GET |
Specify a resource policy like so arn:aws:apigateway:ap-northeast-1::/*.You can not limit integration targets by resource policy. |
| Kinesis Data Streams *1 | AmazonKinesisReadOnlyAccess | |
| S3 *1 | AmazonS3ReadOnlyAccess | Request metrics must be enabled in the S3 configuration. See Creating a CloudWatch metrics configuration for all the objects in your bucket and set the filter name as EntireBucket. |
| OpenSearch Service *1 *2 | AmazonOpenSearchServiceReadOnlyAccess | |
| ECS *1 | ecs:Describe* ecs:List* |
|
| SES *1 | AmazonSESReadOnlyAccess ses:Describe* |
|
| Step Functions *1 | AWSStepFunctionsReadOnlyAccess | |
| EFS *1 | AmazonElasticFileSystemReadOnlyAccess | |
| Kinesis Data Firehose *1 | AmazonKinesisFirehoseReadOnlyAccess | |
| Batch *1 | batch:Describe* batch:List* |
|
| WAF *1 | AWSWAFReadOnlyAccess | |
| Billing *1 | AWSBudgetsReadOnlyAccess | |
| Route 53 *1 | AmazonRoute53ReadOnlyAccess | |
| Connect *1 | AmazonConnectReadOnlyAccess | |
| DocumentDB | AmazonRDSReadOnlyAccess | |
| CodeBuild *1 | codebuild:BatchGetProjects codebuild:ListProjects |
|
| Athena *1 | athena:ListWorkGroups athena:ListTagsForResource |
*1 CloudWatchReadOnlyAccess is required in addition to the required policies/actions for a single linkage of the relevant AWS products.
*2 You can continue to use AmazonESReadOnlyAccess from the former Elasticsearch Service.
Furthmore, AWS Integration lets you filter using tags. However, additional policies need to be added to use this function with ElastiCache, SQS or Step Functions. For more details, refer to Filter by tag.
If you exclude a metric, you will still need to grant an additional policy as well as the ability to filter by tag, as it refers to AWS tags. See the Restrict which metrics to retrieve section for more information.
3. Confirm the host
After a short while, your AWS cloud product will be registered as a host in Mackerel and begin posting metrics. By creating monitoring rules, you can also be notified of alerts. For more information, see Setting up monitoring and alerts.
Configure automatic new metrics addition
Mackerel may add new metrics as each AWS service is updated.
By turning on "Add new metrics automatically", you can avoid having to select metrics for retrieval on the AWS integration settings page each time they are added.
The usage fee for AWS integration is determined by the number of hosts calculated from the number of metrics acquired (Reference: Handling of host conversion when plan limits are exceeded). Therefore, when a large number of metrics are added, the number of hosts will increase due to exceeding the upper limit, which may also affect usage charges.
If you want to avoid unexpected increases in usage charges, turn off "Add new metrics automatically". Metrics added thereafter will not be automatically acquired, so please select the required metrics individually.
Configure automatic retirement
Some services offer the option to retire hosts from Mackerel upon removal of AWS resources. To enable automatic retirement, check the Enable automatic retirement checkbox in the configuration page.
Automatic retirement of AWS integration provides automatic retirement of the associated host when it is determined that a resource linked to Mackerel has been deleted. Therefore, automatic retirement will not occur in the following cases:
- Resources that had already been excluded by the tag filter deleted
- Exception: if the target service is EC2, it will be retired even if it has already been excluded by the tag
- The service was linked in the past, but is now disabled in the integration settings
- The service was linked in the past, but now the entire integration setup has been deleted
In addition, AWS integration check only once when it detects the deletion of an AWS resource, so those that have enabled link after the detection of a deletion are not automatically retired. If the same host is linked to multiple integration settings, it will be automatically retired when any one of the settings is determined to be deleted.
Limit metrics retrieved
You can reduce the number of hosts and cost of the CloudWatch API by limiting the metrics to be retrieved.
1. Grant permissions to limit the metrics to be retrieved
In order to restrict the metrics to be retrieved, additional permissions for the following actions are required in addition to the policies granted for the AWS Integration setup.
elasticache:ListTagsForResourcesqs:ListQueueTagsstates:ListTagsForResource
To add policies, use the Inline Policies process.
2. Configure settings to limit the metrics to be retrieved
The number of hosts is calculated using a moving average of the past month. For more information about that, refer to the FAQ page Calculating the number of hosts.
Select the metrics you wish to retrieve in the Mackerel configuration page. Uncheck any unwanted metrics and save them.
For example, you can specify to not retrieve a metric like kinesis.latency.#.minimum by simply unchecking the box as shown in the image below. This configuration limits the minimum for GetRecords.Latency,PutRecord.Latency, and PutRecords.Latency and reduces a maximum of 3 metrics.
Filter by tag
AWS cloud products to be registered as hosts and retrieve metrics can be filtered based on the tags appended by AWS.
1. Grant permissions for tag retrieval
In order to filter using an AWS tag, permission for the following actions is required in addition to the policies that were granted to configure AWS integration.
elasticache:ListTagsForResourcesqs:ListQueueTagsstates:ListTagsForResource
To add policies, use the Inline Policies process.
2. Tag filtering settings
You can specify tags from the Mackerel settings screen. Confirm the number of linked hosts and save your changes.
By specifying the tag as service:foo, service:bar, instances tagged with a key of service and value of foo or a key of service and value of bar will be targeted. If the key or value contains characters such as a colon : or comma ,, enclose it with quotation marks (" or '). For example, if the key is service:role and the value is foo,bar, specify it as "service:role":"foo,bar".
Assigning roles by tags
There is a feature that is about assigning roles by tags.
This feature enables tags with the label mackerel-integration on AWS resources to be automatically imported as Mackerel services and roles.
The tag format is described as follows:
- Key:
mackerel-integration - Value:
<Service>:<Role> [ / <Service>:<Role> ...]- Service:
/^[a-zA-Z0-9_-]+$/ - Role:
/^[a-zA-Z0-9_-]+$/
- Service:
To assign multiple roles to a host, specify them by separating with /. If a specified Mackerel service and role doesn’t exist, it will be automatically created for you.
Additional policies need to be added to use this feature with ElastiCache, SQS or Step Functions. For more details, refer to Grant permissions for tag retrieval section of Filter by tag.
Please note that hosts registered with the mackerel-integration tag will not be assigned the default roles specified in the AWS Integration settings of the web console.
When the tag settings are not reflected
Upon noticing that the contact of the tag hasn’t been reflected in the role, please check if all of the following items are applicably:
- The tag value you assigned must be valid according to Mackerel’s service and role naming rules
- AWS Integration supports the
mackerel-integrationtag for the AWS service- Please refer to the relevant documentation for the latest support status
Please note that it may take several hours for the role to be reflected on the host.
IAM policies used in AWS Integration
The following is a list of all permissions used in AWS Integration. Create and attach your own policies or specify them in Inline Policies.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"apigateway:Get*",
"application-autoscaling:DescribeScalableTargets",
"athena:ListWorkGroups",
"athena:ListTagsForResource",
"batch:Describe*",
"batch:ListJobs",
"budgets:ViewBudget",
"cloudfront:Get*",
"cloudfront:List*",
"cloudwatch:Get*",
"cloudwatch:List*",
"codebuild:BatchGetProjects",
"codebuild:ListProjects",
"connect:ListInstances",
"dynamodb:Describe*",
"dynamodb:List*",
"ds:DescribeDirectories",
"ec2:DescribeInstances",
"ecs:DescribeClusters",
"ecs:List*",
"elasticache:Describe*",
"elasticache:ListTagsForResource",
"elasticfilesystem:Describe*",
"elasticloadbalancing:Describe*",
"es:DescribeDomain",
"es:List*",
"firehose:DescribeDeliveryStream",
"firehose:List*",
"kinesis:Describe*",
"kinesis:List*",
"lambda:GetFunctionConfiguration",
"lambda:List*",
"rds:Describe*",
"rds:ListTagsForResource",
"redshift:Describe*",
"route53:List*",
"s3:ListAllMyBuckets",
"s3:GetBucketLocation",
"s3:GetBucketLogging",
"s3:GetBucketTagging",
"s3:GetEncryptionConfiguration",
"s3:GetMetricsConfiguration",
"ses:DescribeActiveReceiptRuleSet",
"ses:GetSendQuota",
"ses:ListIdentities",
"sqs:GetQueueAttributes",
"sqs:List*",
"states:DescribeStateMachine",
"states:List*",
"waf-regional:Get*",
"waf-regional:List*",
"waf:Get*",
"waf:List*",
"wafv2:GetWebACL",
"wafv2:List*"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
FAQ
Regarding the access key’s authority check and CreateInternetGateway
In order to check whether or not the access key registered by the user has an unnecessarily strong authority, AWS Integration periodically calls the CreateInternetGateway API in dry-run. Metrics will not be obtained or posted if the access key has an authority more than necessary so be careful. The reason why checks still periodically occur after registration is because there is a possibility that policies will get added to the access key, resulting in a key with stronger authority.
Regarding retiring hosts linked with AWS Integration
By configuring the integration above, AWS cloud products complying with the target service and tag conditions are automatically integrated with Mackerel and registered as a host. On the other hand, if the service does not support automatic retirement or automatic retirement feature is disabled, hosts in Mackerel will not be deleted (retired) simply by deleting instances etc. in AWS. In order to remove hosts integrated with AWS from Mackerel's managed targets, you need to retire them separately.
Even if a host is not retired, host information will remain and hosts without metric posts will not be subject to billing.
Regarding the aggregation of monitoring content with plugins in integrated hosts
custom_identifier can be specified in the settings of mackerel-agent's custom metrics and check monitoring plugins. custom_identifier is a mechanism that allows the user to assign a distinct identifier to a host. Using this, it is possible to aggregate metrics and check monitors that have been posted from mackerel-agent installed on another machine as a part of the AWS integration host. custom_identifier is specified in the configuration of compatible plugins.
For example, the endpoint in the case of Amazon RDS, and the DNS Name in the case of ELB, will be the custom_identifierstring.
Example uses
Here are two example uses. In either case, the agent needs to be restarted after adding to the mackerel-agent configuration file.
The first example is of MySQL monitoring using the mackerel-plugin-mysql plugin for Amazon RDS. Metrics retrieved by the plugin can be aggregated as custom metrics for the RDS host by adding the plugin configuration which includes the custom_identifier as shown below to the mackerel-agent.conf.
[plugin.metrics.mysql] command = ["mackerel-plugin-mysql", "-host", "<RDS endpoint>", "-username", "user", "-password", "pass"] custom_identifier = "<RDS endpoint>"
The second example is of OpenSearch monitoring using the Amazon OpenSearch Service and the check-elasticsearch plugin. OpenSearch Service Cluster Health Checks can be aggregated as check monitoring for OpenSearch Service hosts by adding the plugin configuration which includes the custom_identifier as shown below to the mackerel-agent.conf.
[plugin.checks.elasticsearch] command = ["check-elasticsearch", "-s", "https", "-H", "<OpenSearch Service endpoint>", "-p", "443"] custom_identifier = "<OpenSearch Service domain ARN>"
