Mackerel blog #mackerelio

The Official Blog of Mackerel

mkr third party plugin installer feature

In last week's release, we announced the addition of our plugin installer feature to the mkr command. This feature makes it easier for users to install their own third party plugins in the system.

Up until now, in order to install your own plugin on the system, you either had to arrange a method of installation by yourself or submit source code to the following central repository.

Using this new feature, it is now possible to install plugins created in your own GitHub repository using the mkr command. This allows the creator of the plugin the ability to continually perform maintenance.

Additionally, when installing a plugin by submitting code to the central repository, developing with Golang was required. However, with this feature, it’s now possible to develop using the language of your choice.

How to use

Using the mkr plugin install command, you can install the contents of the specified Mackerel plugin package on your system.

% sudo mkr plugin install <owner>/<repo>[@<release_tag>]

By specifying the Github owner name, repository name, as well as the tag name for Github Releases, the Mackerel plugin package is obtained from GitHub Releases and installed.

Mackerel plugin package format

The Mackerel plugin package is a simple zip file. There should be an executable file starting with mackerel-plugin- or check- stored in the archive. As long as it’s an executable file, any language can be used.

The mkr plugin install command regards these executable files as Mackerel plugins and places them in the specific directory. By default, they will be located under /opt/mackerel-agent/plugins/bin.

Uploading the Mackerel plugin package in GitHub Releases

% sudo mkr plugin install <owner>/<repo>[@<release_tag>]

When the above command is executed, mkr attempts to acquire the Mackerel plugin package of the following URL.<owner>/<repo>/releases/download/<release_tag>/<repo>_<GOOS>_<GOARCH>.zip

If the Mackerel plugin package is placed in GitHub Releases’ <release_tag> according to the naming convention <repo>_<GOOS>_<GOARCH>.zip, you’ll be able to download the Mackerel plugin package from the above URL. Take a look at the following with mackerel-plugin-sample used as an example.

The official plugin registry

Registering useful plugins in the official plugin registry makes it easier for other users to find plugins. Plugins registered in the plugin registry can be installed using the format mkr plugin install <plugin_name>.

Registration is done by sending a Pull Request to the plugin registry’s Github repository. For the contents of the Pull Request, simply create a file called <plugin_name>.json under the plugins/ directory. The JSON file format is as follows.

You can see a list of the plugins that are currently registered at

Understand that contribution authority and licensing for the plugins registered in the official plugin registry correspond with the referent repository.

Please do not hesitate to send us a Pull Request.

On a separate note, since package management registries like this often have cool names like CPAN, RubyGems, homebrew, we considered doing the same for our official plugin registry. Hatena CTO id:motemen even came up with the catchy name tsukiji, but it was ultimately dismissed for various reasons, and we settled with the safe name of plugin-registry.

Pull Requests to the existing central repository

Up until now, it was acceptable to register a new plugin with the above-mentioned central repository, but as a general rule from now on, adding new plugins this way will no longer be accepted. Adding a feature to an existing plugin or fixing a bug is of course welcome.

If you create a new plugin, you should create your own plugin repository on GitHub and register it in the official plugin registry.

If you would like to incorporate a registered plugin into Mackerel’s official plugin package distributed by rpm or deb, please contact us through a separate issue etc. Conversely, the Mackerel development team might reach out to plugin creators to ask for permission to incorporate registered packages into the official plugin package. In such a case, we would appreciate your cooperation.

More detailed documentation

For more details on how to install the third party plugin installer described in this entry or how to create a Mackerel plugin package, please refer to the following help pages.

Future expansion

We are considering the following expansions in the future.

  • A flexible template-like naming configuration, since naming for the Mackerel plugin package is currently defined strictly as <repo>_<GOOS>_<GOARCH>.zip
  • An update feature for installed plugins
  • Metadata plugin support

Of course, we welcome additional proposals and Pull Requests for this feature.

First, create your own plugin, then send a Pull Request to the official plugin registry.

You can now have notifications sent to notification channels for events such as host status changes etc. 

Hello! Mackerel team CRE Inoue (id:a-know) here.

It certainly seems as though winter has arrived as temperatures drop all around the country today. This morning I pulled out my winter gloves without evening thinking about it. The air seems really dry as well. I hope everyone out there is staying healthy.

Anyways, now for this week’s updates.

You can now have notifications sent to notification channels for events such as host status changes and more

A new field labeled "Notification events" is now provided as a configuration item for certain notification channels. With this, in addition to the alert notifications available up to now, it is now possible to have notifications sent the channel for the following events.

  • Host status changes
  • Hosts registered to the organization
  • Hosts retired

Not only is this feature useful for letting you know when events like those mentioned above occur, but combined with Webhook notifications and Amazon API Gateway + Lambda etc., it also gives the user the ability to build a mechanism that will execute arbitrary actions in accordance with the occurrence of these events.

We plan to expand the types of events that can be selected for notification in the future. We are also open to requests / opinions!

OpsGenie V2 REST API now supported

We previously made an announcement regarding the OpsGenie V2 REST API in the entry below, but with this week’s release, support has been completed.

We apologize for the inconvenience.

Plugins are being added one after another to Mackerel’s official plugin registry

As mentioned in last week’s release and yesterday’s explanation article, a third party plugin installer feature has been added to the CLI tool · mkr.

In order to make it easier for users to find third party plugins, we’ve made an official plugin registry.

Plugins are being registered one after another. You can register by sending a Pull Request to the plugin registry’s Github repository.

Even plugins such as mackerel-plugin-aws-batch are being added, so be sure to check it out!

【Only a few spots left】Monitoring Seminar in mercari

On Wednesday, November 29th, we are renting out the venue of Mercari (a heavy user of Mackerel) to hold a seminar titled "Monitoring Seminar in mercari"! (Japanese only)

This will be a great chance to exchange knowledge on not only Mackerel but also server monitoring in general. This will also be an opportunity to think about and discuss "server monitoring" in a more familiar way. By all means, join us (Only a few remaining seats)!

Plugin installer feature added to the mkr・CLI tool etc.

Yesterday, we issued a press release as a company announcing the release of a new feature for cloud providers, "Cloud Provider Integration". (Japanese only)

As the first step in providing this "Cloud Provider integration" feature, we also announced our collaboration with “BIGLOBE Cloud Hosting”, the public cloud of BIGLOBE Co., Ltd.

By developing such features, we hope to be a valuable resource to even more server administrators.

Now onto this week’s update content.

Plugin installer feature added to the mkr・CLI tool

A plugin installer feature has been added to Mackerel’s mkr CLI tool.

Simply put, this "feature makes it easier to install plugins other than official plugin repositories".

All official plugins are developed with Golang, however plugins supported in the plugin installer can use any development language. Mackerel has an official plugin registry, so be sure register your created plugin here!

Sometime next week, we plan on releasing a separate entry about this plugin installer and how to use, so stay tuned!

Releases for mackerel-agent and more

Releases for various OSS including mackerel-agent have been made. The details follow below. Thank you to all of our contributors!

mackerel-agent v0.47.2

  • Can now be built using Go 1.9.2
  • ERROR (open /sys/block: no such file or directory) level logs that appear in Heroku and other environments have been changed to DEBUG level messages.

mackerel-agent-plugins v0.38.0

  • [postgres] Processing improved
  • [docker] The graph that displays the CPU usage rate was changed
    • This is an incompatible change
    • Up until now it was a raw counter value, but with this version upgrade it will be displayed in CPU percentage
    • By specifying the -cpu-format option, it is also possible to return to the previous behavior
  • [cloudfront] -metric-key-prefix option now available

The Mackerel advent calendar!

It’s already November and about that time of a year for Mackerel’s technical advent calendar. We made one this year as well. (Japanese only)

As of right now, there are still lots of open spots! We hope that you can join us!

Introducing the Keyboard shortcut feature etc.

On Tuesday, October 31st, Mackerel ran a booth at the AWS CloudRoadShow in Fukuoka.

We received so many visitors! Thank you to everyone who stopped by!

Now for this week’s update information.

Keyboard shortcuts have been implemented

Much like the idea behind the mkr CLI tool, we believe that improving work efficiency for engineers who primarily use Mackerel is a valuable component of our service.

And we believe that most users would prefer to be able to use the keyboard to carry out as many operations as possible when working from the Mackerel web terminal. For this reason, a keyboard shortcut feature was implemented with this week’s release!

The shortcut cheat sheet can be displayed with ?. If you have a chance, give it a try. And if you are one of the many who requested this feature, please give us feedback!

A limit on the number of check monitor posts per request has been set

With this week’s Mackerel API update, a limit has been set for the number of check monitor results that can be posted per request.

As a result, if you are using a version of mackerel-agent before v0.46.0 and you have more than 101 items configured for check monitoring, you will no longer be able to post check results. This will not affect versions v.0.47.0 and later. Please update your agent if this applies to you.

Commands can now be invoked based on check monitoring results etc.

Last week, we made an announcement regarding the termination of the “im.kayac” service (Termination schedule for the notification channel "im.kayac" - Mackerel blog #mackerelio). And as scheduled, the feature became unavailable as a notification destination in Mackerel on October 26th.

The following operations are no longer available.

  • Creating a new im.kayac channel in the “Channel Settings”
  • Display of existing registered im.kayac channels is channel list
  • Sending of notifications to existing registered im.kayac channels

For those of you who used im.kayac as a notification destination, please review the notification settings once again.

Anyways, here is this week’s update information.

Arbitrary commands can now be invoked based on check monitoring results

Check monitoring in Mackerel allows the user to monitor from various points of view, such monitoring text logs with check-log and monitoring processes with check-procs. With this week's update, it is now possible to execute commands based on those monitoring results.

In order to use this feature, you’ll need to add the action option to mackerel-agent.conf as shown below.

command = "check-log -f /path/to/file -p PATTERN"
action = { command = "bash -c '[ \"$MACKEREL_STATUS\" != \"OK\" ]' && ruby /path/to/something.rb", user = "someone" }

In the environment variable MACKEREL_STATUS, the execution result (either OK, WARNING, CRITICAL, or UNKNOWN) of the immediately preceding command (not action.command) is set. In the above example, if the execution result of the previous check monitor is not OK, the command is invoked as the user specified in the user (configuration) field.

There’s also an environment variable called MACKEREL_PREVIOUS_STATUS which sets the “execution result of the command before the previous command”, and this will allow you to flexibly automate operation, to some extent, based on check monitoring results. Check out the help page below for more details about this feature.

Tags can now be filtered in AWS Integration - SQS

In order for sqs: ListQueueTags actions can be performed, policies need to be granted. For details, refer to the "Filter by tag" section in the following help page.

Expressions no longer included in sentences of expression monitoring alert notifications

This is an Experimental Feature for Expression Monitoring in Mackerel, and with a monitor rule which uses this, long expressions will no longer be included in the contents of alert notifications.

The text of mail will be displayed as always.

Other releases

  • Support for Amazon Linux version 2017.09
  • Command line tool mkr updated, support added for new metrics command
    • For details, refer to the help page Using the mkr CLI tool - Mackerel help
  • mackerel-check-plugins updated
    • check-disk plugin corrections

Thank you to everyone who contributed Pull Requests!!

Various events

Mackerel team CRE id:Soudai will be presenting at Geeks Who Drink in Fukuoka - Monitoring Workshop Edition- held at the Fukuoka branch of Nulab Inc. this coming Monday, October 30th! (Japanese only)

The day after that, Mackerel will be running a booth at the AWS CloudRoadShow Fukuoka held on Tuesday, October 31st!

AWS Cloud Roadshow 2017 福岡 powered by Intel® 広島、大阪、名古屋、福岡の 4 都市を巡る無料クラウドカンファレンス開催! | アマゾン ウェブ サービス (Japanese only)

To all of the engineers in the Fukuoka area, don’t miss out!

AWS Integration now supports NLB etc.

As announced on this blog the other day, the service, one of Mackerel’s integratable notification partners, is ending on October 31st, 2017. Accompanying this, the im.kayac feature will no longer be available as a Mackerel notification destination.

If this affects you, check out the following blog entry for more details.

Anyways, here is this week’s update information.

Countermeasures taken against .io domain malfunction have been restored

As previously announced, Mackerel extended the judgment interval for connectivity monitoring as a countermeasure in response to the .io domain malfunction and has now been restored to its’ original interval on Tuesday, October 17th as scheduled.

Again, please update your agent to minimize the impact of the possible future .io domain failure. Check out last week’s entry for the content of the agent updates.

AWS Integration now supports NLB

The recently released AWS Network Load Balancer (NLB) is now also supported in AWS Integration.

Check out the following help page for more information on how to use it.

Graph guide now uniformly displayed

The guide which is displayed when hovering over a graph in the host detail screen, service details screen, etc. is now displayed in all other graphs within the same screen.

We think this will be handy for tasks such as investigating server resources.

Updates for Mackerel related OSS

mackerel-agent v0.47.0

  • The method for calculating the CPU usage rate in Linux was corrected
    • The guest usage rate is now subtracted from cpu.user
  • In AzureVM, an issue in which the program would abnormally end under certain circumstances was fixed

mackerel-agent-plugins v0.37.0

  • mackerel-plugin-mcrouter was added
    • However, it has not been added to the package, so you’ll need to build it yourself

Thank you to everyone who reported issues and contributed!

A Mackerel workshop for student/young working engineers

The theme of the next “Supporterz Workshop” organized by Supporterz, Inc. is "New to Mackerel, from a Hatena engineer" and we are now currently accepting applications. (Japanese only)

id:papix, the presenter for this event is an Hatena Web application engineer and also the founder of Mackerel UG.

This workshop is geared toward students wanting to become engineers (any grade level is okay) and young professionals who have recently entered the engineering field (2010 graduates and up). Right now we still have a lot of spots left, but applicants will be accepted on a first-come-first-served basis. If this event sounds interesting to you, by all means, come and join us!

Termination schedule for the notification channel "im.kayac"

Service for, one of Mackerel’s notification partners, will end on October 31st, 2017.

Additionally, integration with Mackerel will also be ending according to the following schedule.

  • The im.kayac feature will no longer be available as a notification destination for Mackerel following our regular release on Thursday, October 26th.
    • Creating a new im.kayac channel in Channel Settings will no longer be available
    • Existing registered im.kayac channels will no longer be displayed in the channel list
    • Notifications will no longer be sent for existing registered im.kayac channels
  • After stoppage of the feature has been completed, data for existing registered im.kayac channels will be deleted as soon as preparations are complete
    • Date and time are still pending (As of now, there is no announcement schedule regarding the date and time)

This concludes the termination schedule.

im.kayac, thank you for providing a great service for such a long time.

Thank you for choosing Mackerel.