AWS Integration

Using AWS integration, you can manage AWS cloud products as a host of Mackerel and monitor its metrics. This function is only offered in the Trial plan and Standard plan.

Each AWS cloud product will be registered as one host in Mackerel and therefore be counted as a billable host. Additionally, the API of AWS will be called every 5 minutes for each targeted metric to be obtained. Please take note, for this reason, an Amazon CloudWatch API usage fee may occur.

Currently, the following AWS cloud products are supported. For information on obtaining metrics, please refer to each individual document.

EC2ELB (CLB)ALBRDSElastiCacheRedshiftLambda

Integration method

There are two ways to integrate AWS Integration.

  • Configure the IAM role to allow access only from the AWS account of the Mackerel system and authenticate with AssumeRole
  • Configure the Access Key ID and Secret Access Key

From a security standpoint, we strongly recommend configuring with the IAM role.

How to configure an IAM role

1. Creating a role with the IAM Management Console

Create a new role with the IAM Management Console. We recommend assigning an easy-to-understand name like MackerelAWSIntegrationRole for use in Mackerel’s AWS integration.

Allow access from the Mackerel AWS account. Select Another AWS account from the role types.

Enter 217452466226 for the Account ID, choose Require external ID and enter Mackerel-AWS-Integration for the External ID. Mackerel system uses the account to access the user’s role. With this configuration, only the Mackerel account can access the created role. Create the role without checking Require MFA.

2. Granting policies

Grant the policies listed below for the newly created user. Be careful not to grant FullAccess permission.

  • AmazonEC2ReadOnlyAccess
  • AmazonElastiCacheReadOnlyAccess
  • AmazonRDSReadOnlyAccess
  • AmazonRedshiftReadOnlyAccess
  • AWSLambdaReadOnlyAccess

3. Register an ARN role in Mackerel

Register an ARN role in Mackerel. Be careful not to mistake the organization to be registered.

4. Confirm the host

After a short while, your AWS cloud product will be registered as a host in Mackerel and begin posting metrics. By creating monitoring rules, you can also be notified of alerts. For more information, see Setting up monitoring and alerts.

Configure the Access Key ID and Secret Access Key

From a security standpoint, the following method is not recommended.

1. Creating a user with the IAM Management Console

Create a new user with the IAM Management Console. We recommend assigning an easy-to-understand name like MackerelAWSIntegrationUser for use in Mackerel’s AWS integration.

2. Registering the Access Key in Mackerel

Register the Access Key ID and Secret Access Key (displayed on the screen when creating the account) in Mackerel. Be careful not to mistake the organization to be registered.

3. Granting policies

Grant the policies listed below for the newly created user. Be careful not to grant FullAccess permission.

  • AmazonEC2ReadOnlyAccess
  • AmazonElastiCacheReadOnlyAccess
  • AmazonRDSReadOnlyAccess
  • AmazonRedshiftReadOnlyAccess
  • AWSLambdaReadOnlyAccess

4. Verify the host

After a short while, your AWS cloud product will be registered as a host in Mackerel and begin posting metrics. By creating monitoring rules, you can also be notified of alerts. For more information, see Setting up monitoring and alerts.

Filter by tag

AWS cloud products to be registered as hosts and retrieve metrics can be filtered based on the tags appended by AWS.

1. Assign permissions for tag retrieval

Filtering by AWS tags requires permissions for the API to retrieve tags for each service. Check the respective policy and determine whether the actions below can be performed:

  • ec2:DescribeTags
  • elasticloadbalancing:DescribeTags
  • rds:ListTagsForResource
  • elasticache:ListTagsForResource
  • redshift:DescribeTags
  • lambda:ListTags

Authority regarding the following action is necessary when configuring using the Access Key and Secret Access Key. It is not necessary for configuration via the IAM role.

  • iam:GetUser for linked users

In particular, the AmazonElastiCacheReadOnlyAccess management policy on AWS cannot perform the elasticache:ListTagsForResource action, so when filtering ElastiCache by tag, the policy must be added.

To add policies, use the Inline Policies process.

2. Tag filtering settings

You can specify tags on the Mackerel settings screen. Confirm the number of linked hosts and save your changes.

By specifying the tag as service:foo, service:bar, instances tagged with a key of service and value of foo or a key of service and value of bar will be targeted. If the key or value contains characters such as a colon : or comma ,, enclose it with quotation marks (" or '). For example, if the key is service:role and the value is foo,bar, specify it as "service:role":"foo,bar".

FAQ

Regarding the access key’s authority check and CreateInternetGateway

In order to check whether or not the access key registered by the user has an unnecessarily strong authority, AWS Integration periodically calls the CreateInternetGateway API in dry-run. Metrics will not be obtained or posted if the access key has an authority more than necessary so be careful. The reason why checks still periodically occur after registration is because there is a possibility that policies will get added to the access key, resulting in a key with stronger authority.

Regarding retiring hosts linked with AWS Integration

By configuring the integration above, AWS cloud products complying with the target service and tag conditions are automatically integrated with Mackerel and registered as a host. On the other hand, hosts in Mackerel will not be deleted (retired) simply by deleting instances etc. in AWS. In order to remove hosts integrated with AWS from Mackerel’s managed targets, you need to retire them separately.

mackerel.io

Even if a host is not retired, host information will remain and hosts without metric posts will not be subject to billing.

Regarding the aggregation of custom metrics obtained with the plugin in integrated hosts

In mackerel-agent’s plugin configuration, a custom_identifier can be specified. custom_identifier is a mechanism to grant an identifier that is unique to the user to be used as a host identifier. By using this, metrics that have been posted from mackerel-agent installed on another machine can be aggregated as metrics of a host integrated with AWS Integration. Specify the custom_identifier in the plugin configuration of the plugin to be sending the custom metrics.

For example, if using Amazon RDS and the mackerel-plugin-mysql plugin, by adding the custom_identifier description as shown below in the plugin configuration of mackerel-agent.conf, metrics obtained by the plugin can be aggregated as custom metrics of an RDS host.

[plugin.metrics.mysql]
command = "mackerel-plugin-mysql -host=<RDS endpoint> -username=user -password=pass"
custom_identifier = "<RDS endpoint>"

The endpoint (for Amazon RDS) and the DNS Name (for ELB) each become a custom_identifier string.

After making the addition to the conf file, you’ll need to restart the agent.