Mackerel blog #mackerelio

The Official Blog of Mackerel

Regarding the incident that occurred on September 26, 2018 (Wed.)

Thank you for choosing Mackerel.

This is an announcement to report on the incident that occurred today (9/26).

Today at 10:51 am (JST), the API server error rate increased and became unstable.

In terms of this phenomenon, access to the API server failed, and a 5xx status code was most likely returned resulting in an error.

As the API server error rate increased, connectivity monitoring was suspended in order to prevent false reports.

After that, the unstable conditions continued for an extended period of time. At 4:20 pm (JST), recovery measures were taken by adjusting application parameters and reinforcing the server.

We were not able to identify the direct cause and will continue to further investigate this issue. Additionally, starting tomorrow, operations will be implemented to prevent secondary issues from occurring. Please note that depending on the operation, we may temporarily switch to maintenance mode (restricted access to the server).

We apologize for any inconvenience this issue may have caused.

Thank you for your cooperation.