Mackerel blog #mackerelio

The Official Blog of Mackerel

Legacy Custom Dashboards will gradually be repealed in preparation for it’s full discontinuation at the end of March 2021

Thank you for choosing Mackerel. Today’s announcement concerns the discontinuation of the Legacy Custom Dashboard function. Please note that because migration will not be automatic, we recommend taking necessary measures in advance. See below for more details.

Target functions

Schedule

The schedule for the discontinuation of these functions is as follows.

  • As of the end of January 2021, creating new Legacy Custom Dashboards will no longer be available.
  • As of the end of March 2021, viewing / updating existing Legacy Custom Dashboards will no longer be available.

Migrating from Legacy Custom Dashboards to the current Custom Dashboards

Migration from an existing legacy custom dashboard to the current custom dashboard will not be automatic.

  • We apologize for the inconvenience, but ask that you migrate to the current Custom Dashboards yourself.
  • Definitions for Legacy Custom Dashboards are in Markdown format and can be used with the Markdown widget supported for the current Custom Dashboards.
  • The Web API for current Custom Dashboards can also be used.

Other points of note

  • As stated in the ‘Target Functions’ section of this announcement, the Web API and command line tool will soon become unavailable.
    • An update to make the current custom dashboards compatible with mkr is currently under consideration (mkr dashboards is currently only supported for Legacy Custom Dashboards).

Again, we apologize for any inconvenience that this may cause those of you who are currently using this function. On the bright side, compared to the Legacy Custom Dashboards, the current Custom Dashboards supports a handful of useful widgets like the Markdown widget mentioned above and lets you intuitively rearrange the size, layout, and display range of each widget, making this function handy all-around.

mackerel.io

We hope you will take this opportunity to utilize the current Custom Dashboards and all of its functions. Thank you for your understanding and cooperation.

Some changes have been made to our Help page regarding IAM policies required for AWS・Lambda Integration to go along with the announcement from AWS

Hello. Mackerel team CRE Inoue (id:a-know) here again wishing you a happy new year! Here’s to hoping 2021 is a great one!

Without further ado, let’s get to today’s announcement.

Some changes have been made to our Help page regarding IAM policies required for AWS・Lambda Integration to go along with the announcement from AWS

Have you seen the recent announcement from AWS?

docs.aws.amazon.com

To go along with this announcement, we are re-examining the Mackerel Help page regarding AWS Integration.

  • Before
    • AWSLambdaReadOnlyAccess
  • After
    • AWSLambda_ReadOnlyAccess

The old policy AWSLambdaReadOnlyAccess will become deprecated after January 25th, 2021 and will no longer be attachable. Instead please use the new policy AWSLambda_ReadOnlyAccess (Note that policies that have already been attached should still be available).

Metrics added / replaced for Kinesis Data Streams / Step Functions AWS Integration and more

Hello. Mackerel team CRE Inoue (id:a-know) here with the last update announcement of 2020. Thank you for choosing Mackerel this year.

Metrics added / replaced for Kinesis Data Streams AWS Integration

The following metrics have been added / replaced for Kinesis Data Streams.

Before

  • Records
    • GetRecords.Records (kinesis.records.get_records)
    • IncomingRecords (kinesis.records.incoming)
    • PutRecords.Records (kinesis.records.put_records)

Now

  • Records
    • GetRecords.Records (kinesis.records.get_records)
    • IncomingRecords (kinesis.records.incoming)
    • PutRecords.SuccessfulRecords (kinesis.records.put_records_successful)
    • PutRecords.TotalRecords (kinesis.records.put_records_total)
    • PutRecords.FailedRecords (kinesis.records.put_records_failed)
    • PutRecords.ThrottledRecords (kinesis.records.put_records_throttled)

The metric previously recorded as PutRecords.Records (kinesis.records.put_records) is now PutRecords.SuccessfulRecords (kinesis.records.put_records_successful). Three new types of records data are now integrated as well. The name change from kinesis.records.put_records to kinesis.records.put_records_successful should be noted in particular because it may affect things such as monitors.

Also, the maximum number of metrics per host has been increased from 31 to 34.

Metrics added for AWS Integration with Step Functions

  • Service Integrations
    • ServiceIntegrationsFailed (states.service_integrations.SERVICE_INTEGRATION_NAME.failed)
    • ServiceIntegrationsScheduled (states.service_integrations.SERVICE_INTEGRATION_NAME.scheduled)
    • ServiceIntegrationsStarted (states.service_integrations.SERVICE_INTEGRATION_NAME.started)
    • ServiceIntegrationsSucceeded (states.service_integrations.SERVICE_INTEGRATION_NAME.succeeded)
    • ServiceIntegrationsTimedOut (states.service_integrations.SERVICE_INTEGRATION_NAME.timedout)
  • Service Integration Time
    • ServiceIntegrationRunTime (states.service_integration_time.SERVICE_INTEGRATION_NAME.run)
    • ServiceIntegrationScheduleTime (states.service_integration_time.SERVICE_INTEGRATION_NAME.schedule)
    • ServiceIntegrationTime (states.service_integration_time.SERVICE_INTEGRATION_NAME.all)

The service integration name replaces SERVICE_INTEGRATION_NAME.

With this addition, the maximum number of metrics per host is now calculated using the formula 7 + 9 × (number of activities) + 8 × (number of Lambda functions) + 8 × (number of service integrations). Please be sure to take note of the number of metrics as this may affect your micro hosts.

The same applies to the above-mentioned Kinesis Data Streams integration. If necessary, you can select integration targets using the optional selection function.

Network interface metrics can now be excluded with mackerel-agent

You can now exclude network interface metrics by writing the following in the mackerel-agent.conf.

[interfaces]
ignore = "bond.*"

Using a regular expression, metrics from the specified interface will not be collected. For Windows Server, specify the interface name that corresponds to that in Device Manager.

See the help page below for more information.

mackerel.io

Along with this update, the mkr command line tool has been updated from 0.42.0 to 0.43.0.

github.com

Official support for mackerel-agent in CentOS6 environments has ended

With CentOS6 ending their support on November 30th of this year, Mackerel too has officially ended support for CentOS6 environments.

Even if an environment is kept the same, surrounding environments are always changing (for example, managing old versions of TLS communication and host CA certificate renewal etc.), and so we believe that the best course of action is to update and switch environments. We apologize for the inconvenience and thank you for your cooperation.

Email notifications now available for events such as host registration and monitor changes and more

Hello. Mackerel team CRE Inoue (id:a-know) here with another update announcement.

Email notifications now available for events such as host registration and monitor changes

Up until now, event notifications for things like registering a host in Mackerel and changes to monitors were only supported for some notification channels such as Slack and Webhook.

Now, these notifications are also available for email as well.

We hope that email notifications will help you stay on top of the unexpected.

New metric targets added for AWS・S3 Integration

Two types of metrics have been added, Bucket Size and Number Of Objects. See the help page linked below.

mackerel.io

Please note that both are daily metrics.

OSS Updates

Today’s update covers a wide range of OSS products.

mackerel-agent-plugins(v0.63.2)

mackerel-check-plugins(v0.38.0)

  • [check-postgresql] Added support for the sslrootcert option used to encrypt plugin communication

mackerel-container-agent(v0.5.0)

  • CPU / memory metrics for container-agent now available with Kubernetes 1.18 and later.
    • Additionally, a get operation (verb) is no longer required for the nodes/spec resource in the RBAC configuration of Kubernetes (EKS on Fargate).

mkr(v0.42.0)

  • [monitors subcommand] Support added for alertStatusOnGone (specifying the alert status when an alert occurs) with push pull.

Thank you to everyone who contributed!

Support availability during the year-end and New Year holidays

The end of the year is fast approaching with less than three weeks left in 2020. And thank you for choosing Mackerel this year.

Mackerel’s official support window will be closed from December 28, 2020 (Thursday) until January 3, 2021 (Sunday).

For those of you planning to look over your Mackerel settings etc. before the end of the year and New Year holidays, we recommend you do so as soon as possible. And if you have any questions or concerns, please do not hesitate to contact us.

Mackerel’s Official Support Request Form

mackerel-container-agent is now available on Amazon ECR public

Amazon ECR public was just released at AWS’s annual event re:Invent 2020.

To go along with this, we’ve released mackerel-container-agent on Amazon ECR public.

gallery.ecr.aws

The container image for mackerel-container-agent was previously only available from Docker Hub. Now, the mackerel-container-agent container image is also available from Amazon ECR public. Now you have multiple options to choose from.

Billing data can now be monitored with AWS Integration and more

Hello. Mackerel team CRE Inoue (id:a-know) here with another update announcement.

Billing data can now be monitored with AWS Integration

2020-11-20 Postscript

We’ve recently confirmed the occurrence of hosts being registered in Mackerel, but not posting graphs if the budgets name is longer than a certain length on the AWS side.

We apologize for the inconvenience and appreciate your cooperation as we proceed with further investigation to resolve this event.

End of Postscript

f:id:mackerelio:20201118143501p:plain

It is now possible to monitor Billing data with AWS Integration. With this, you can now monitor/visualize billing and budget metrics for your AWS account. If using AWS Organizations for consolidated billing, you can also retrieve metrics for billing amounts per member account. Prices are calculated using the conversion 1 management account = 1 micro host.

Account IDs are used for host names. And Budget names are incorporated as part of metric names, but because Japanese characters cannot be used for metric names in Mackerel, if such a character string is included in a Budget name, those characters are encoded as shown below.

f:id:mackerelio:20201119110122p:plain

Check out the help page below for more details.

mackerel.io

For information regarding how to set up AWS integration, you can refer to the following help page (If using an existing IAM role, setting an additional AWSBudgetsReadOnlyAccess policy is required).

mackerel.io

For a while now, we’ve received a lot of interest in being able to use Mackerel to aggregate and monitor not only the statuses of various infrastructure resources, but also billing data such as this. This release is in response to those requests. Be sure to give it a try!

Support added for 1000+ registered queues in AWS / SQS Integration.

mackerel.io

Up until now, only up to 1000 items could be integrated with AWS / SQS Integration, but with this update, that number is now unlimited.

This will probably only be relevant to a few special use cases, but if applicable to you, be sure to have a look.

Azure Application Gateway now supported with Azure Integration and more!

Hello. Mackerel team CRE Inoue (id:a-know) here with another update announcement.

Azure Application Gateway now supported with Azure Integration

f:id:mackerelio:20201106145512p:plain

Azure Application Gateway can now be monitored with Azure Integration! Application Gateway is Azure's managed load balancer service that enables you to do application layer (OSI layer 7) load balancing.

Check out the help page linked below for configuration methods.

mackerel.io

And the following link for obtainable metrics with Application Gateway.

mackerel.io

Notification content expanded for alerts to PagerDuty

The content that can be included in alert notifications sent from Mackerel to PagerDuty has been expanded. The following items have been added.

  • Monitor notes
  • A thumbnail image of the target metric
  • custom_identifier

Now when an alert occurs, more information can be obtained when looking at PagerDuty. You may want to take this opportunity to consider using PagerDuty or at least review the troubleshooting flow.

Also, please note that this expansion is only available with the Event API v2. For more information regarding alert notifications from Mackerel to PagerDuty, be sure to take a look at the following help page.

mackerel.io

Monitor memos now included in Webhook notification payload

Similar to the previous update regarding Mackerel supported notification channels, Webhook notifications have added monitor memos to its payload.

A variety of information can be included in monitor memos. For example:

  • Information regarding the initial response to an alert (commands or emergency measures)
  • An explanation for or the reason behind the monitor
  • URLs of related documents, etc.

This information is also included in Webhook requests, potentially enabling you to respond more smoothly and reduce recovery time. So be sure to give it a try!