Mackerel blog #mackerelio

The Official Blog of Mackerel

Anomaly Detection for roles released as experimental feature for Windows hosts and more

Hello. Mackerel team CRE Miura (id:missasan) here with another update announcement.

Anomaly Detection for roles released as experimental feature for Windows hosts

The Anomaly Detection for roles function which uses machine learning is now supported for Windows hosts (as an experimental feature).

While offered as an experimental feature, Anomaly Detection for roles can be used free of charge.

This function is currently being offered free of charge during its period as an experimental feature. If you’re using a Windows server, try enabling Anomaly Detection for roles and see what is detected as an anomaly. We also welcome your feedback.

You can select Anomaly Detection for roles from the Create a New Monitor screen, then select a role composed of Windows hosts for the Target role.

mackerel.io

What is Anomaly Detection for roles?

The Anomaly Detection for roles function uses machine learning to detect and notify users via alerts when load tendencies deviate from the norm. The function uses about 2 weeks worth of metrics as learning data and recognizes the difference between daytime and nighttime load trends and weekday and holiday load trends. For example, the following conditions would be determined as anomalies.

  • When there’s a sudden load increase at night when the load is usually low.
  • When only one host in a role has a low load during the daytime when the load should be high

As described above, Anomaly Detection for roles can catch abnormalities that are not easily detected with general fixed threshold settings.

Check out the following article for a more detailed use case.

mackerel.io

Legacy Custom Dashboards officially discontinued with April 15, 2021 release

As previously announced in the article linked below, Legacy Custom Dashboards was gradually being repealed, but is now completely discontinued as of the release on April 15.

mackerel.io

All of the features for Legacy Custom Dashboards listed below are now unavailable. From now on, please use the current Custom Dashboards.

To all of the users who made the effort and migrated over to the current Custom Dashboards, we really appreciate your cooperation.

New metrics added for AWS Integration - EC2

New metrics related to Amazon EBS metrics for Nitro-based instances have been added for AWS Integration with EC2.

The new metrics are as follows.

  • EBSReadOps
  • EBSWriteOps
  • EBSReadBytes
  • EBSWriteBytes
  • EBSIOBalance%
  • EBSByteBalance%

Check out the help page linked below for metric details.

mackerel.io

Support window closed during Golden Week

Mackerel’s official support window (inquiries made using the "Feedback" option displayed in the upper right while logged in to Mackerel) will be closed during the following period.

Period of Closure: April 29, 2021 (Thursday) - May 5, 2021 (Wednesday) JST

We will respond to all inquiries received during this period starting from May 6, 2021 (Thursday). We greatly appreciate your understanding.

New metrics added for AWS・EFS Integration

Hello. Mackerel team CRE Nishiyama (id:tukaelu) here with another update announcement.

New metrics added for AWS・EFS Integration

In addition to all of the items available up until now, the following metrics can now be monitored as well.

  • Data IO Count
    • MeteredIOBytes(efs.data_io_count.metered)
  • Data IO Sum Bytes
    • MeteredIOBytes(efs.data_io_sum_bytes.metered)
  • Data IO Bytes
    • MeteredIOBytes.Minimum(efs.data_io_bytes.Metered.minimum)
    • MeteredIOBytes.Average(efs.data_io_bytes.Metered.average)
    • MeteredIOBytes.Maximum(efs.data_io_bytes.Metered.maximum)
  • Storage
    • StorageBytes.Standard(efs.storage.standard)
    • StorageBytes.IA(efs.storage.ia)
    • StorageBytes.Total(efs.storage.total)

With this new addition, the maximum number of metrics per host has increased from 28 to 36. Please be sure to pay attention to the number of metrics per micro host.

Be sure to check out the following document as well.

mackerel.io

Osaka Region now supported with AWS Integration

Hello. Mackerel team CRE Nishiyama (id:tukaelu) here with another update announcement.

Osaka Region now supported with AWS Integration

Just the other day, AWS announced Osaka as its secondary domestic Region!

aws.amazon.com

From a DR (Disaster Recovery) perspective, I’m sure a lot of people have been anticipating a multi-region configuration for Japan. To go along with this, Mackerel now supports the Osaka region with AWS Integration as well!

This can be applied by selecting ap-northeast-3 for Region in the AWS Integration settings.

You can also send alerts and other events to EventBridge via the notification channel settings. This isn’t currently available for the Osaka region, but will be addressed in future updates.

mackerel.io

If you have any questions or concerns regarding this update, feel free to contact our support team.

mkr dashboards command now supported with current Custom Dashboards and more

Hello. Mackerel team CRE Miura (id:missasan) here with another update announcement.

Today’s updates include the addition of the mkr dashboards command mentioned in the title, and 3 bug fixes for all OSS. mkr dashboards command now supported with current Custom Dashboards

mkr dashboards command now supported with current Custom Dashboards

Legacy Custom Dashboards will be completely discontinued at the end of March.

The following announcement was made on January 12th.

mackerel.io

In preparation for the full discontinuation at the end of March, we respectfully request that those still using Legacy Custom Dashboards begin the process of migrating to the current Custom Dashboards. As you can see in an excerpt from the announcement linked above, the official command line tool mkr dashboards was not supported for the current Custom Dashboards at that time.

(Here’s an excerpt from the above announcement)

An update to make the current custom dashboards compatible with mkr is currently under consideration (mkr dashboards is currently only supported for Legacy Custom Dashboards).

Migrating from Legacy Custom Dashboards made easy with the mkr dashboards migrate command

With this update, you can now operate current Custom Dashboards with mkr dashboard commands (mkr v0.44.1).

The following 4 commands are supported.

  • mkr dashboards : Obtain List of Custom Dashboards
  • mkr dashboards pull : Download advanced Custom Dashboard settings as a local file
  • mkr dashboards push : Update Custom Dashboards based on the specified local file
  • mkr dashboards migrate : Migrate Legacy Custom Dashboards with Markdown widgets to current Custom Dashboards

Please note that legacy dashboards cannot be updated with mkr dashboard push.

This function was developed by one of our Mackerel Ambassadors. @fujiwara

He noticed that mkr commands weren’t supported when the announcement was made, and half a month later, he sent us a PR!

All of us on the Mackerel team were super impressed by how quickly this was implemented. We, and I’m sure all Mackerel users, are very grateful. This just goes to show that Mackerel truly is a service supported by its users.

Thank you so much! @fujiwara

If you haven't migrated your Legacy Custom Dashboard yet, you’ll want to take advantage of this.

mackerel-plugin-aws-ses to use AWS Signature Version v4

With the release of mackerel-agent-plugins v0.63.5, mackerel-plugin-aws-ses will now use AWS Signature Version v4 instead of using Signature Version v3, which will be discontinued on February 28, 2021.

Points of note

  • Older versions of mackerel-plugin-aws-ses will no longer be available for use after the discontinuation of Signature Version v3 on February 28, 2021.
  • Please update for this release or migrate to AWS Integration SES.
  • Due to this change, the --endpoint option is no longer recommended.

The issue of commands ending in error when check-load becomes CRITICAL has been fixed

Prior to this release, if the CRITICAL threshold for check-load was exceeded, commands would end in an error without issuing alerts. This issue has been fixed in go-check-plugins v0.39.2.

The issue of settings being deleted with mkr monitors push/diff has been fixed

With the release of mkr v0.44.1, the issue of settings being deleted unintentionally when executing commands mkr monitors push and mkr monitors diff has been fixed.

Thank you to everyone who contributed!

Amazon Route 53 for AWS Integration and Azure Blob Storage for Azure Integration now supported and more

Hello. Mackerel team CRE Inoue (id:a-know) here with another update announcement.

Amazon Route 53 can now be monitored with AWS Integration

Monitoring Amazon Route 53 is now available with AWS Integration. Amazon Route 53 is a scalable cloud Domain Name System service.

aws.amazon.com

This service lets you visualize / monitor the metrics listed in the help page below.

mackerel.io

Check out this help page for more on how to set up AWS Integration.

mackerel.io

Azure Blob Storage can now be monitored with Azure Integration

Monitoring Azure Blob Storage is now available with Azure Integration. Azure Blob Storage is a scalable cloud object storage service.

azure.microsoft.com

This service lets you visualize / monitor the metrics listed in the help page below.

mackerel.io

Check out this help page for more on how to set up Azure Integration.

mackerel.io

Legacy Custom Dashboards can no longer be created

As announced in a previous blog post, the Legacy Custom Dashboard function will gradually be discontinued by the end of March 2020.

mackerel.io

As stated in the article above, creating new Legacy Custom Dashboards is no longer available as of the end of January 2021. In addition to this, the following alert message will be displayed if you are currently using a Legacy Custom Dashboard.

The image displayed here is an example of the Legacy Custom Dashboard function that will be discontinued. Please be sure to check the above article and migrate or delete as necessary (Legacy Custom Dashboards that have not been deleted will automatically become not viewable after the end of March).

Azure Files can now be monitored with Azure Integration and more

Hello. Mackerel team CRE Inoue (id:a-know) here with another update announcement.

Azure Files can now be monitored with Azure Integration

It is now possible to monitor Azure Files with Azure integration. Azure Files is a fully managed service for file sharing that can be accessed via SMB and NFS protocols.

azure.microsoft.com

With this you can visualize and monitor the metrics listed in the help page linked below.

mackerel.io

See the following help page for more on how to set up Azure Integration.

mackerel.io

New metric targets added for Azure Integration・Application Gateway

Newly obtainable metric targets have been added for Azure / Application Gateway Integration. The new metrics are Listener related and listed below.

  • Application Gateway Total
    • azure.application_gateway.application_gateway_total_time.#.milliseconds
  • Bytes Sent/Received
    • azure.application_gateway.bytes_sent_received.#.sent
    • azure.application_gateway.bytes_sent_received.#.received
  • Client RTT
    • azure.application_gateway.client_rtt.#.milliseconds
  • Client TLS Protocol
    • azure.application_gateway.tls_protocol.#.nontls
    • azure.application_gateway.tls_protocol.#.tlsv1
    • azure.application_gateway.tls_protocol.#.tlsv1_1
    • azure.application_gateway.tls_protocol.#.tlsv1_2

The "#" in the metric name is a placeholder for the Listener information of the target application gateway. Check out the help page linked below for more details.

mackerel.io

On top of these additions, the number of metrics changes from 25 to 33 for both Standard V2 and WAF V2, and will be converted into 2 micro hosts.

OSS Updates

The following updates have been made to Mackerel related OSS. A big thank you to everyone who contributed!

mackerel-agent-plugins(v0.63.3)

  • [mysql] An issue was fixed where the labels innodb_tables_in_use and innodb_locked_tables in the MySQL innodb Tables In Use graph were reversed.

mackerel-check-plugins(v0.39.1)

  • [check-disk] The output result is now sorted by usage rate.

Legacy Custom Dashboards will gradually be repealed in preparation for it’s full discontinuation at the end of March 2021

Thank you for choosing Mackerel. Today’s announcement concerns the discontinuation of the Legacy Custom Dashboard function. Please note that because migration will not be automatic, we recommend taking necessary measures in advance. See below for more details.

Target functions

Schedule

The schedule for the discontinuation of these functions is as follows.

  • As of the end of January 2021, creating new Legacy Custom Dashboards will no longer be available.
  • As of the end of March 2021, viewing / updating existing Legacy Custom Dashboards will no longer be available.

Migrating from Legacy Custom Dashboards to the current Custom Dashboards

Migration from an existing legacy custom dashboard to the current custom dashboard will not be automatic.

  • We apologize for the inconvenience, but ask that you migrate to the current Custom Dashboards yourself.
  • Definitions for Legacy Custom Dashboards are in Markdown format and can be used with the Markdown widget supported for the current Custom Dashboards.
  • The Web API for current Custom Dashboards can also be used.

Other points of note

  • As stated in the ‘Target Functions’ section of this announcement, the Web API and command line tool will soon become unavailable.
    • An update to make the current custom dashboards compatible with mkr is currently under consideration (mkr dashboards is currently only supported for Legacy Custom Dashboards).

Again, we apologize for any inconvenience that this may cause those of you who are currently using this function. On the bright side, compared to the Legacy Custom Dashboards, the current Custom Dashboards supports a handful of useful widgets like the Markdown widget mentioned above and lets you intuitively rearrange the size, layout, and display range of each widget, making this function handy all-around.

mackerel.io

We hope you will take this opportunity to utilize the current Custom Dashboards and all of its functions. Thank you for your understanding and cooperation.