Mackerel blog #mackerelio

The Official Blog of Mackerel

mkr dashboards command now supported with current Custom Dashboards and more

Hello. Mackerel team CRE Miura (id:missasan) here with another update announcement.

Today’s updates include the addition of the mkr dashboards command mentioned in the title, and 3 bug fixes for all OSS. mkr dashboards command now supported with current Custom Dashboards

mkr dashboards command now supported with current Custom Dashboards

Legacy Custom Dashboards will be completely discontinued at the end of March.

The following announcement was made on January 12th.

mackerel.io

In preparation for the full discontinuation at the end of March, we respectfully request that those still using Legacy Custom Dashboards begin the process of migrating to the current Custom Dashboards. As you can see in an excerpt from the announcement linked above, the official command line tool mkr dashboards was not supported for the current Custom Dashboards at that time.

(Here’s an excerpt from the above announcement)

An update to make the current custom dashboards compatible with mkr is currently under consideration (mkr dashboards is currently only supported for Legacy Custom Dashboards).

Migrating from Legacy Custom Dashboards made easy with the mkr dashboards migrate command

With this update, you can now operate current Custom Dashboards with mkr dashboard commands (mkr v0.44.1).

The following 4 commands are supported.

  • mkr dashboards : Obtain List of Custom Dashboards
  • mkr dashboards pull : Download advanced Custom Dashboard settings as a local file
  • mkr dashboards push : Update Custom Dashboards based on the specified local file
  • mkr dashboards migrate : Migrate Legacy Custom Dashboards with Markdown widgets to current Custom Dashboards

Please note that legacy dashboards cannot be updated with mkr dashboard push.

This function was developed by one of our Mackerel Ambassadors. @fujiwara

He noticed that mkr commands weren’t supported when the announcement was made, and half a month later, he sent us a PR!

All of us on the Mackerel team were super impressed by how quickly this was implemented. We, and I’m sure all Mackerel users, are very grateful. This just goes to show that Mackerel truly is a service supported by its users.

Thank you so much! @fujiwara

If you haven't migrated your Legacy Custom Dashboard yet, you’ll want to take advantage of this.

mackerel-plugin-aws-ses to use AWS Signature Version v4

With the release of mackerel-agent-plugins v0.63.5, mackerel-plugin-aws-ses will now use AWS Signature Version v4 instead of using Signature Version v3, which will be discontinued on February 28, 2021.

Points of note

  • Older versions of mackerel-plugin-aws-ses will no longer be available for use after the discontinuation of Signature Version v3 on February 28, 2021.
  • Please update for this release or migrate to AWS Integration SES.
  • Due to this change, the --endpoint option is no longer recommended.

The issue of commands ending in error when check-load becomes CRITICAL has been fixed

Prior to this release, if the CRITICAL threshold for check-load was exceeded, commands would end in an error without issuing alerts. This issue has been fixed in go-check-plugins v0.39.2.

The issue of settings being deleted with mkr monitors push/diff has been fixed

With the release of mkr v0.44.1, the issue of settings being deleted unintentionally when executing commands mkr monitors push and mkr monitors diff has been fixed.

Thank you to everyone who contributed!

Amazon Route 53 for AWS Integration and Azure Blob Storage for Azure Integration now supported and more

Hello. Mackerel team CRE Inoue (id:a-know) here with another update announcement.

Amazon Route 53 can now be monitored with AWS Integration

Monitoring Amazon Route 53 is now available with AWS Integration. Amazon Route 53 is a scalable cloud Domain Name System service.

aws.amazon.com

This service lets you visualize / monitor the metrics listed in the help page below.

mackerel.io

Check out this help page for more on how to set up AWS Integration.

mackerel.io

Azure Blob Storage can now be monitored with Azure Integration

Monitoring Azure Blob Storage is now available with Azure Integration. Azure Blob Storage is a scalable cloud object storage service.

azure.microsoft.com

This service lets you visualize / monitor the metrics listed in the help page below.

mackerel.io

Check out this help page for more on how to set up Azure Integration.

mackerel.io

Legacy Custom Dashboards can no longer be created

As announced in a previous blog post, the Legacy Custom Dashboard function will gradually be discontinued by the end of March 2020.

mackerel.io

As stated in the article above, creating new Legacy Custom Dashboards is no longer available as of the end of January 2021. In addition to this, the following alert message will be displayed if you are currently using a Legacy Custom Dashboard.

The image displayed here is an example of the Legacy Custom Dashboard function that will be discontinued. Please be sure to check the above article and migrate or delete as necessary (Legacy Custom Dashboards that have not been deleted will automatically become not viewable after the end of March).

Azure Files can now be monitored with Azure Integration and more

Hello. Mackerel team CRE Inoue (id:a-know) here with another update announcement.

Azure Files can now be monitored with Azure Integration

It is now possible to monitor Azure Files with Azure integration. Azure Files is a fully managed service for file sharing that can be accessed via SMB and NFS protocols.

azure.microsoft.com

With this you can visualize and monitor the metrics listed in the help page linked below.

mackerel.io

See the following help page for more on how to set up Azure Integration.

mackerel.io

New metric targets added for Azure Integration・Application Gateway

Newly obtainable metric targets have been added for Azure / Application Gateway Integration. The new metrics are Listener related and listed below.

  • Application Gateway Total
    • azure.application_gateway.application_gateway_total_time.#.milliseconds
  • Bytes Sent/Received
    • azure.application_gateway.bytes_sent_received.#.sent
    • azure.application_gateway.bytes_sent_received.#.received
  • Client RTT
    • azure.application_gateway.client_rtt.#.milliseconds
  • Client TLS Protocol
    • azure.application_gateway.tls_protocol.#.nontls
    • azure.application_gateway.tls_protocol.#.tlsv1
    • azure.application_gateway.tls_protocol.#.tlsv1_1
    • azure.application_gateway.tls_protocol.#.tlsv1_2

The "#" in the metric name is a placeholder for the Listener information of the target application gateway. Check out the help page linked below for more details.

mackerel.io

On top of these additions, the number of metrics changes from 25 to 33 for both Standard V2 and WAF V2, and will be converted into 2 micro hosts.

OSS Updates

The following updates have been made to Mackerel related OSS. A big thank you to everyone who contributed!

mackerel-agent-plugins(v0.63.3)

  • [mysql] An issue was fixed where the labels innodb_tables_in_use and innodb_locked_tables in the MySQL innodb Tables In Use graph were reversed.

mackerel-check-plugins(v0.39.1)

  • [check-disk] The output result is now sorted by usage rate.

Legacy Custom Dashboards will gradually be repealed in preparation for it’s full discontinuation at the end of March 2021

Thank you for choosing Mackerel. Today’s announcement concerns the discontinuation of the Legacy Custom Dashboard function. Please note that because migration will not be automatic, we recommend taking necessary measures in advance. See below for more details.

Target functions

Schedule

The schedule for the discontinuation of these functions is as follows.

  • As of the end of January 2021, creating new Legacy Custom Dashboards will no longer be available.
  • As of the end of March 2021, viewing / updating existing Legacy Custom Dashboards will no longer be available.

Migrating from Legacy Custom Dashboards to the current Custom Dashboards

Migration from an existing legacy custom dashboard to the current custom dashboard will not be automatic.

  • We apologize for the inconvenience, but ask that you migrate to the current Custom Dashboards yourself.
  • Definitions for Legacy Custom Dashboards are in Markdown format and can be used with the Markdown widget supported for the current Custom Dashboards.
  • The Web API for current Custom Dashboards can also be used.

Other points of note

  • As stated in the ‘Target Functions’ section of this announcement, the Web API and command line tool will soon become unavailable.
    • An update to make the current custom dashboards compatible with mkr is currently under consideration (mkr dashboards is currently only supported for Legacy Custom Dashboards).

Again, we apologize for any inconvenience that this may cause those of you who are currently using this function. On the bright side, compared to the Legacy Custom Dashboards, the current Custom Dashboards supports a handful of useful widgets like the Markdown widget mentioned above and lets you intuitively rearrange the size, layout, and display range of each widget, making this function handy all-around.

mackerel.io

We hope you will take this opportunity to utilize the current Custom Dashboards and all of its functions. Thank you for your understanding and cooperation.

Some changes have been made to our Help page regarding IAM policies required for AWS・Lambda Integration to go along with the announcement from AWS

Hello. Mackerel team CRE Inoue (id:a-know) here again wishing you a happy new year! Here’s to hoping 2021 is a great one!

Without further ado, let’s get to today’s announcement.

Some changes have been made to our Help page regarding IAM policies required for AWS・Lambda Integration to go along with the announcement from AWS

Have you seen the recent announcement from AWS?

docs.aws.amazon.com

To go along with this announcement, we are re-examining the Mackerel Help page regarding AWS Integration.

  • Before
    • AWSLambdaReadOnlyAccess
  • After
    • AWSLambda_ReadOnlyAccess

The old policy AWSLambdaReadOnlyAccess will become deprecated after January 25th, 2021 and will no longer be attachable. Instead please use the new policy AWSLambda_ReadOnlyAccess (Note that policies that have already been attached should still be available).

Metrics added / replaced for Kinesis Data Streams / Step Functions AWS Integration and more

Hello. Mackerel team CRE Inoue (id:a-know) here with the last update announcement of 2020. Thank you for choosing Mackerel this year.

Metrics added / replaced for Kinesis Data Streams AWS Integration

The following metrics have been added / replaced for Kinesis Data Streams.

Before

  • Records
    • GetRecords.Records (kinesis.records.get_records)
    • IncomingRecords (kinesis.records.incoming)
    • PutRecords.Records (kinesis.records.put_records)

Now

  • Records
    • GetRecords.Records (kinesis.records.get_records)
    • IncomingRecords (kinesis.records.incoming)
    • PutRecords.SuccessfulRecords (kinesis.records.put_records_successful)
    • PutRecords.TotalRecords (kinesis.records.put_records_total)
    • PutRecords.FailedRecords (kinesis.records.put_records_failed)
    • PutRecords.ThrottledRecords (kinesis.records.put_records_throttled)

The metric previously recorded as PutRecords.Records (kinesis.records.put_records) is now PutRecords.SuccessfulRecords (kinesis.records.put_records_successful). Three new types of records data are now integrated as well. The name change from kinesis.records.put_records to kinesis.records.put_records_successful should be noted in particular because it may affect things such as monitors.

Also, the maximum number of metrics per host has been increased from 31 to 34.

Metrics added for AWS Integration with Step Functions

  • Service Integrations
    • ServiceIntegrationsFailed (states.service_integrations.SERVICE_INTEGRATION_NAME.failed)
    • ServiceIntegrationsScheduled (states.service_integrations.SERVICE_INTEGRATION_NAME.scheduled)
    • ServiceIntegrationsStarted (states.service_integrations.SERVICE_INTEGRATION_NAME.started)
    • ServiceIntegrationsSucceeded (states.service_integrations.SERVICE_INTEGRATION_NAME.succeeded)
    • ServiceIntegrationsTimedOut (states.service_integrations.SERVICE_INTEGRATION_NAME.timedout)
  • Service Integration Time
    • ServiceIntegrationRunTime (states.service_integration_time.SERVICE_INTEGRATION_NAME.run)
    • ServiceIntegrationScheduleTime (states.service_integration_time.SERVICE_INTEGRATION_NAME.schedule)
    • ServiceIntegrationTime (states.service_integration_time.SERVICE_INTEGRATION_NAME.all)

The service integration name replaces SERVICE_INTEGRATION_NAME.

With this addition, the maximum number of metrics per host is now calculated using the formula 7 + 9 × (number of activities) + 8 × (number of Lambda functions) + 8 × (number of service integrations). Please be sure to take note of the number of metrics as this may affect your micro hosts.

The same applies to the above-mentioned Kinesis Data Streams integration. If necessary, you can select integration targets using the optional selection function.

Network interface metrics can now be excluded with mackerel-agent

You can now exclude network interface metrics by writing the following in the mackerel-agent.conf.

[interfaces]
ignore = "bond.*"

Using a regular expression, metrics from the specified interface will not be collected. For Windows Server, specify the interface name that corresponds to that in Device Manager.

See the help page below for more information.

mackerel.io

Along with this update, the mkr command line tool has been updated from 0.42.0 to 0.43.0.

github.com

Official support for mackerel-agent in CentOS6 environments has ended

With CentOS6 ending their support on November 30th of this year, Mackerel too has officially ended support for CentOS6 environments.

Even if an environment is kept the same, surrounding environments are always changing (for example, managing old versions of TLS communication and host CA certificate renewal etc.), and so we believe that the best course of action is to update and switch environments. We apologize for the inconvenience and thank you for your cooperation.

Email notifications now available for events such as host registration and monitor changes and more

Hello. Mackerel team CRE Inoue (id:a-know) here with another update announcement.

Email notifications now available for events such as host registration and monitor changes

Up until now, event notifications for things like registering a host in Mackerel and changes to monitors were only supported for some notification channels such as Slack and Webhook.

Now, these notifications are also available for email as well.

We hope that email notifications will help you stay on top of the unexpected.

New metric targets added for AWS・S3 Integration

Two types of metrics have been added, Bucket Size and Number Of Objects. See the help page linked below.

mackerel.io

Please note that both are daily metrics.

OSS Updates

Today’s update covers a wide range of OSS products.

mackerel-agent-plugins(v0.63.2)

mackerel-check-plugins(v0.38.0)

  • [check-postgresql] Added support for the sslrootcert option used to encrypt plugin communication

mackerel-container-agent(v0.5.0)

  • CPU / memory metrics for container-agent now available with Kubernetes 1.18 and later.
    • Additionally, a get operation (verb) is no longer required for the nodes/spec resource in the RBAC configuration of Kubernetes (EKS on Fargate).

mkr(v0.42.0)

  • [monitors subcommand] Support added for alertStatusOnGone (specifying the alert status when an alert occurs) with push pull.

Thank you to everyone who contributed!

Support availability during the year-end and New Year holidays

The end of the year is fast approaching with less than three weeks left in 2020. And thank you for choosing Mackerel this year.

Mackerel’s official support window will be closed from December 28, 2020 (Thursday) until January 3, 2021 (Sunday).

For those of you planning to look over your Mackerel settings etc. before the end of the year and New Year holidays, we recommend you do so as soon as possible. And if you have any questions or concerns, please do not hesitate to contact us.

Mackerel’s Official Support Request Form