check-log is a plug-in that monitors log files on the server, including applications and middleware.
Pattern matching using regular expressions is performed on the output differences of the monitored log files, and alerts are notified.
Configurable options
Option | Short | Description | Default |
---|---|---|---|
--file | -f | Specify the path of the file to be monitored (can be specified in glob format) | |
--pattern | -p | Specify the pattern of error wording to be detected with a regular expression *1 | |
--exclude | -E | Specify patterns to exclude from detection using regular expressions *1 | |
--warning-over | -w | Raise a Warning alert if the number of rows matching the detection pattern exceeds the specified value | |
--critical-over | -c | Raise a Critical alert if the number of rows matching the detection pattern exceeds the specified value | |
--warning-level | Raise a Warning alert if the value extracted by the detection pattern exceeds the specified value | ||
--critical-level | Raise a Critical alert if the value extracted by the detection pattern exceeds the specified value | ||
--return | -r | Notify of log lines that match the pattern with an alert. *1 | |
--search-in-directory | Specify the directory path where the monitored files are located (in Windows environment, use with --file-pattern ) |
||
--file-pattern | -F | Specify files to be monitored by regular expression | |
--icase | -i | Case-insensitive matching | |
--state-dir | -s | Specify the directory path where the State file is to be saved | |
--no-state | All logs are checked without State file | ||
--encoding | Specify character encoding of monitored files | ||
--missing | Specify alert level if monitored files are missing | UNKNOWN | |
--check-first | Check all contents at the time of the first check of the file | ||
--suppress-pattern | Suppress the display of detection patterns on the host detail screen |
- *1 If multiple conditions are specified, they become AND conditions. (
"--pattern", "A", "--pattern", "B"
) - *2 Content exceeding 1024 characters will be truncated by mackerel-agent.
About state file
The check-log plugin monitors for differences in log file output, so the number of bytes read for each file and the inode number (Linux only) are recorded in the State file.
If the --state-dir
option is not specified, a file in the format of {monitored file}-<hash string>.json
is created in the same directory hierarchy as the monitored log file under the prescribed destination for each OS.
- Linux
/var/tmp/mackerel-agent/check-log
or/tmp/check-log
- Windows
C:\Windows\Temp\check-log
Example configurations
for Linux
The configuration for detecting ERROR
output to /var/log/access.log is as follows
[plugin.checks.access_log] command = ["check-log", "--file", "/var/log/access.log", "--pattern", "ERROR"]
Specify the --return
option to be notified with an alert of detected log lines.
[plugin.checks.access_log] command = ["check-log", "--file", "/var/log/access.log", "--pattern", "ERROR", "--return"]
When a monitored file is checked for the first time, no content check (matching detection patterns) is performed. If you want to check the contents of the file for the first time, use the --check-first
option.
[plugin.checks.access_log] command = ["check-log", "--file", "/var/log/access.log", "--pattern", "ERROR", "--return", "--check-first"]
To check the above settings manually from a terminal or other device, execute as follows
check-log --file /var/log/access.log --pattern "ERROR" --return --check-first
for Windows
The configuration for detecting ERROR
output to C:\log\access.log is as follows
[plugin.checks.access_log] command = ["check-log", "--file", "C:\\log\\access.log", "--pattern", "ERROR"]
If you use regular expressions to specify files with --file-pattern
on Windows, please specify the directory path in the --search-in-directory
option to avoid conflicts between the directory delimiter \
and the regular expression escaping process.
If you want to specify a file of the form C:\log\access.log.{yyyyy}-{mm}-{dd}
with a regular expression, it will look like this.
[plugin.checks.access_log] command = ["check-log", "--search-in-directory", "C:\\log\\", "--file-pattern", "access.log.\\d{4}-\\d{2}-\\d{2}", "--pattern", "ERROR"]
Directory paths cannot be specified using regular expressions. Please configure monitoring rules for each directory.
Tips
Alerts when multiple patterns are matched by AND conditions
Multiple --pattern
options will trigger an alert when log lines containing all keywords are detected.
The following is a condition that must contain PRODUCTION
and ERROR
.
[plugin.checks.access_log] command = ["check-log", "--file", "/var/log/access.log", "--pattern", "PRODUCTION", "--pattern", "ERROR"]
Alerts when any of the OR conditions are matched
The condition specified in the --pattern
option is evaluated as a regular expression.
For example, if you want to detect log lines containing FATAL
or ERROR
, specify them separated by pipe |
as follows
[plugin.checks.access_log] command = ["check-log", "--file", "/var/log/access.log", "--pattern", "FATAL|ERROR"]
Troubleshooting
Log lines matching the condition are output but not detected
Please review your settings, etc., as the following cases are often seen.
- Make sure the
--check-first
option is specified.- If not specified, the contents of the monitored file will not be checked the first time it is checked.
- The same applies to the initial check when a file is switched due to log rotation, etc.
- Make sure that the detection pattern specified in the
--pattern
option is correct.- Since the detection pattern is evaluated as a regular expression, escape it if it contains characters that are interpreted as regular expression meta characters, such as half-width parentheses.
"LOG UNKNOWN: unexpected end of JSON input" alert is issued
The State file in which the check-log plugin records the monitoring status may be corrupted.
Deleting or renaming the State file may solve the problem. Please refer to About the State file for details. Also, since the monitoring state is reset after this action, the behavior will be the same as when it is executed for the first time.
Repository
https://github.com/mackerelio/go-check-plugins/tree/master/check-log