Mackerel blog #mackerelio

The Official Blog of Mackerel

You can now configure the maximum number of attempts for host/service metric monitors etc.

Thanks to all of you, the Mackerel Advent Calendar that we talked about last week has been completely filled!

qiita.com (Japanese only)

Thank you!

Also, be sure to check out the advent calendar that Mackerel team CRE id:Soudai is working so hard on, not missing a single day and continuing to release blogs full of information and useful knowledge.

qiita.com (Japanese only)

I hope you enjoy these advent calendars leading up to Christmas.

Now onto this week’s update information.

Customized (expression) graphs can now be added to GraphBoard

"GraphBoard" is an interactive feature that lets the user freely arrange arbitrary role and service metrics graphs.

mackerel.io

With this week's update, we’ve increased the number of types of graphs that can be added to this GraphBoard. Customized graphs (expression graphs) can now be added!

  • Overlay server load statuses for the current week and the previous week and display as one graph
  • Display the calculated maximum value/average value/minimum value of the server load belonging to a particular role

Customized graphs are extremely useful and allow you to implement the above procedures by describing the function expression. We believe that putting this to use will be even easier than before.

Displaying customized graphs is an experimental feature. For more details, check out the following help pages.

mackerel.io

mackerel.io

You can now configure the max number of attempts for host/service metric monitors

As you can see in the image above, the "Max number of attempts before an alert is triggered" can be configured with host metric and service metric monitors.

Up until now, only the monitoring of “Duration” (average of the past n minutes) was supported. For example, if you wanted to have an alert occur when a threshold of 5 is consecutively exceeded, you would configure the following.

  • Threshold of 5
  • Duration of 3 minutes

However, with this configuration, not only would a progression of 756 (a 3min average of 6) lead to an alert occurrence, but also for a progression such as 1161 (in this case, also a 3min average of 6). For this reason, sometimes unintentional alerts would occur.

With this update, you can eliminate cases such as the one above by utilizing the newly added "max_check_attempts". We hope that you can use this feature together with "Duration” monitoring to eliminate unintentional alerts.

Our support window will be closed for the holidays

Both methods of contacting our support team, either through support@mackerel.io or the “contact our support team” option displayed in the upper right header when logged into Mackerel, will be closed during the following dates for the New Year’s holidays.

New Year’s holiday period :Thursday, December 28th, 2017 - Wednesday, January 3rd, 2018 (JST)

All inquiries received during this period will be handled sequentially starting from January 4th, 2018 (Thursday).

Please review your server monitoring settings and contact us if you have any concerns before the end of the year holidays begin!

Target URLs can now be specified directly with the mkr plugin installer etc.

Hello! Mackerel team CRE Inoue (id:a-know) here.

It’s December! And around this time every year, what the IT/Web industry refers to as "Advent Calendars" bustles with various themed blog posts, and Mackerel is no exception. Here is Mackerel’s Advent Calendar!

qiita.com (Japanese only)

But wait, here’s another calendar!

qiita.com (Japanese only)

Be sure to check out this calendar as well. Fellow team CRE Sone (id:Soudai) is going all out!

Now for this week’s update information.

Target URLs can now be specified directly with the mkr plugin installer

We recently made an announcement regarding the recently released command line tool mkr plugin installer feature, including usage details/application methods.

mackerel.io

With this week’s feature update, you can now directly specify a URL as shown below.

mkr plugin install https://www.example.com/mackerel-plugin-foobar.zip

Be sure to try out this plugin installer feature which allows you to easily install useful third-party plugins.

You can now select the API V2 in the PagerDuty notification channel settings

In the integration settings for PagerDuty, one of the notification channels available with Mackerel, you can now select the V2 API.

Although there is no functional difference at this time, it is possible that options only available with V2 will be added in future feature expansion. If this applies to you, please consider making the transition.

Updates for Mackerel related OSS

Updates for various OSS have been released and are listed below. To all of you out there who contributed or reported a problem, thank you very much!

mackerel-agent v0.47.3

  • The output message that occurs when there is a defect in the mackerel-agent.conf plugin command configuration has been improved.
  • The issue of the interface value not being obtained when the network counter is 8 digits or more has been fixed.

mackerel-agent-plugins v0.39.0

  • [docker] The valued obtained by Docker Blkio IOPS has been normalized from per Minute to per Second
  • [jvm] The issue of the option to monitor the remote jvm not functioning properly was fixed.
  • [kinesis-streams] Regarding metrics for the number of records and processing bytes, outputting the "statistics of one record" was changed to "total amount per minute".

Mackerel × Twilio Hands-On Seminar #2

Earlier this year in April, the first Mackerel × Twilio Hands-On Seminar was held and went down as a huge success. Now, the second edition of the seminar is in motion and scheduled to be held on Wednesday, December 13th.

mackerelio.connpass.com (Japanese only)

For the venue of the event, we’ll be renting out the space of RecoChoku, a company which extensively uses the Mackerel - Twilio Integration. This will be a valuable opportunity to learn more about a variety of things ranging from the basics of why server monitoring is important to installing the agent and receiving phone notifications through Twilio.

The event is almost at capacity, but because applications as chosen through a lottery system, there’s still a good chance that you’ll be accepted even if you apply now. You definitely don’t want to miss out on this opportunity!

Azure Integration now supports Virtual Machines

I can’t believe that it’ll already be December next week. It feels like we just celebrated Halloween...

But with December comes the end of year Advent Calendar! Thanks to all of you, Mackerel’s Advent Calendar is filling up little by little.

qiita.com (Japanese only)

There are still spots available, so please join us!

Now onto this weeks update information.

Azure Integration now supports Virtual Machines

Mackerel's integration feature lets you monitor cloud services without having to introduce an agent, and Azure integration now supports Virtual Machines!

For information on available metrics, check out the the following help page.

mackerel.io

This feature is also available if currently using virtual machines with the agent installed.

Even if you use both, they can be managed as one host (the metrics collected with the integration feature will be integrated as "custom metrics" of that host), so it’s possible to aggregate metrics in Mackerel that can’t be collected by the agent alone. And of course, a usage fee for only one host will be charged. Be sure to give it a try!

mkr third party plugin installer feature

In last week's release, we announced the addition of our plugin installer feature to the mkr command. This feature makes it easier for users to install their own third party plugins in the system.

Up until now, in order to install your own plugin on the system, you either had to arrange a method of installation by yourself or submit source code to the following central repository.

Using this new feature, it is now possible to install plugins created in your own GitHub repository using the mkr command. This allows the creator of the plugin the ability to continually perform maintenance.

Additionally, when installing a plugin by submitting code to the central repository, developing with Golang was required. However, with this feature, it’s now possible to develop using the language of your choice.

How to use

Using the mkr plugin install command, you can install the contents of the specified Mackerel plugin package on your system.

% sudo mkr plugin install <owner>/<repo>[@<release_tag>]

By specifying the Github owner name, repository name, as well as the tag name for Github Releases, the Mackerel plugin package is obtained from GitHub Releases and installed.

Mackerel plugin package format

The Mackerel plugin package is a simple zip file. There should be an executable file starting with mackerel-plugin- or check- stored in the archive. As long as it’s an executable file, any language can be used.

The mkr plugin install command regards these executable files as Mackerel plugins and places them in the specific directory. By default, they will be located under /opt/mackerel-agent/plugins/bin.

Uploading the Mackerel plugin package in GitHub Releases

% sudo mkr plugin install <owner>/<repo>[@<release_tag>]

When the above command is executed, mkr attempts to acquire the Mackerel plugin package of the following URL.

https://github.com/<owner>/<repo>/releases/download/<release_tag>/<repo>_<GOOS>_<GOARCH>.zip

If the Mackerel plugin package is placed in GitHub Releases’ <release_tag> according to the naming convention <repo>_<GOOS>_<GOARCH>.zip, you’ll be able to download the Mackerel plugin package from the above URL. Take a look at the following with mackerel-plugin-sample used as an example.

https://github.com/mackerelio/mackerel-plugin-sample/releases

The official plugin registry

https://github.com/mackerelio/plugin-registry

Registering useful plugins in the official plugin registry makes it easier for other users to find plugins. Plugins registered in the plugin registry can be installed using the format mkr plugin install <plugin_name>.

Registration is done by sending a Pull Request to the plugin registry’s Github repository. For the contents of the Pull Request, simply create a file called <plugin_name>.json under the plugins/ directory. The JSON file format is as follows.

You can see a list of the plugins that are currently registered at https://github.com/mackerelio/plugin-registry/tree/master/plugins.

Understand that contribution authority and licensing for the plugins registered in the official plugin registry correspond with the referent repository.

Please do not hesitate to send us a Pull Request.

On a separate note, since package management registries like this often have cool names like CPAN, RubyGems, homebrew, we considered doing the same for our official plugin registry. Hatena CTO id:motemen even came up with the catchy name tsukiji, but it was ultimately dismissed for various reasons, and we settled with the safe name of plugin-registry.

Pull Requests to the existing central repository

Up until now, it was acceptable to register a new plugin with the above-mentioned central repository, but as a general rule from now on, adding new plugins this way will no longer be accepted. Adding a feature to an existing plugin or fixing a bug is of course welcome.

If you create a new plugin, you should create your own plugin repository on GitHub and register it in the official plugin registry.

If you would like to incorporate a registered plugin into Mackerel’s official plugin package distributed by rpm or deb, please contact us through a separate issue etc. Conversely, the Mackerel development team might reach out to plugin creators to ask for permission to incorporate registered packages into the official plugin package. In such a case, we would appreciate your cooperation.

More detailed documentation

For more details on how to install the third party plugin installer described in this entry or how to create a Mackerel plugin package, please refer to the following help pages.

mackerel.io mackerel.io

Future expansion

We are considering the following expansions in the future.

  • A flexible template-like naming configuration, since naming for the Mackerel plugin package is currently defined strictly as <repo>_<GOOS>_<GOARCH>.zip
  • An update feature for installed plugins
  • Metadata plugin support

Of course, we welcome additional proposals and Pull Requests for this feature.

First, create your own plugin, then send a Pull Request to the official plugin registry.

You can now have notifications sent to notification channels for events such as host status changes etc. 

Hello! Mackerel team CRE Inoue (id:a-know) here.

It certainly seems as though winter has arrived as temperatures drop all around the country today. This morning I pulled out my winter gloves without evening thinking about it. The air seems really dry as well. I hope everyone out there is staying healthy.

Anyways, now for this week’s updates.

You can now have notifications sent to notification channels for events such as host status changes and more

A new field labeled "Notification events" is now provided as a configuration item for certain notification channels. With this, in addition to the alert notifications available up to now, it is now possible to have notifications sent the channel for the following events.

  • Host status changes
  • Hosts registered to the organization
  • Hosts retired

Not only is this feature useful for letting you know when events like those mentioned above occur, but combined with Webhook notifications and Amazon API Gateway + Lambda etc., it also gives the user the ability to build a mechanism that will execute arbitrary actions in accordance with the occurrence of these events.

We plan to expand the types of events that can be selected for notification in the future. We are also open to requests / opinions!

OpsGenie V2 REST API now supported

We previously made an announcement regarding the OpsGenie V2 REST API in the entry below, but with this week’s release, support has been completed.

mackerel.io

We apologize for the inconvenience.

Plugins are being added one after another to Mackerel’s official plugin registry

As mentioned in last week’s release and yesterday’s explanation article, a third party plugin installer feature has been added to the CLI tool · mkr.

In order to make it easier for users to find third party plugins, we’ve made an official plugin registry.

github.com

Plugins are being registered one after another. You can register by sending a Pull Request to the plugin registry’s Github repository.

Even plugins such as mackerel-plugin-aws-batch are being added, so be sure to check it out!

【Only a few spots left】Monitoring Seminar in mercari

On Wednesday, November 29th, we are renting out the venue of Mercari (a heavy user of Mackerel) to hold a seminar titled "Monitoring Seminar in mercari"!

mackerelio.connpass.com (Japanese only)

This will be a great chance to exchange knowledge on not only Mackerel but also server monitoring in general. This will also be an opportunity to think about and discuss "server monitoring" in a more familiar way. By all means, join us (Only a few remaining seats)!

Plugin installer feature added to the mkr・CLI tool etc.

Yesterday, we issued a press release as a company announcing the release of a new feature for cloud providers, "Cloud Provider Integration".

hatenacorp.jp (Japanese only)

As the first step in providing this "Cloud Provider integration" feature, we also announced our collaboration with “BIGLOBE Cloud Hosting”, the public cloud of BIGLOBE Co., Ltd.

By developing such features, we hope to be a valuable resource to even more server administrators.

Now onto this week’s update content.

Plugin installer feature added to the mkr・CLI tool

A plugin installer feature has been added to Mackerel’s mkr CLI tool.

Simply put, this "feature makes it easier to install plugins other than official plugin repositories".

All official plugins are developed with Golang, however plugins supported in the plugin installer can use any development language. Mackerel has an official plugin registry, so be sure register your created plugin here!

github.com

Sometime next week, we plan on releasing a separate entry about this plugin installer and how to use, so stay tuned!

Releases for mackerel-agent and more

Releases for various OSS including mackerel-agent have been made. The details follow below. Thank you to all of our contributors!

mackerel-agent v0.47.2

  • Can now be built using Go 1.9.2
  • ERROR (open /sys/block: no such file or directory) level logs that appear in Heroku and other environments have been changed to DEBUG level messages.

mackerel.io

mackerel-agent-plugins v0.38.0

  • [postgres] Processing improved
  • [docker] The graph that displays the CPU usage rate was changed
    • This is an incompatible change
    • Up until now it was a raw counter value, but with this version upgrade it will be displayed in CPU percentage
    • By specifying the -cpu-format option, it is also possible to return to the previous behavior
  • [cloudfront] -metric-key-prefix option now available

The Mackerel advent calendar!

It’s already November and about that time of a year for Mackerel’s technical advent calendar. We made one this year as well.

qiita.com (Japanese only)

As of right now, there are still lots of open spots! We hope that you can join us!

Introducing the Keyboard shortcut feature etc.

On Tuesday, October 31st, Mackerel ran a booth at the AWS CloudRoadShow in Fukuoka.

We received so many visitors! Thank you to everyone who stopped by!

Now for this week’s update information.

Keyboard shortcuts have been implemented

Much like the idea behind the mkr CLI tool, we believe that improving work efficiency for engineers who primarily use Mackerel is a valuable component of our service.

And we believe that most users would prefer to be able to use the keyboard to carry out as many operations as possible when working from the Mackerel web terminal. For this reason, a keyboard shortcut feature was implemented with this week’s release!

The shortcut cheat sheet can be displayed with ?. If you have a chance, give it a try. And if you are one of the many who requested this feature, please give us feedback!

A limit on the number of check monitor posts per request has been set

With this week’s Mackerel API update, a limit has been set for the number of check monitor results that can be posted per request.

mackerel.io

As a result, if you are using a version of mackerel-agent before v0.46.0 and you have more than 101 items configured for check monitoring, you will no longer be able to post check results. This will not affect versions v.0.47.0 and later. Please update your agent if this applies to you.