| POSTFIX-LOGWATCH(1) | General Commands Manual | POSTFIX-LOGWATCH(1) |
postfix-logwatch - A Postfix log parser and analysis utility
postfix-logwatch [options] [logfile ...]
The postfix-logwatch(1) utility is a Postfix MTA log parser that produces summaries, details, and statistics regarding the operation of Postfix.
This utility can be used as a standalone program, or as a Logwatch filter module to produce Postfix summary and detailed reports from within Logwatch.
Postfix-logwatch is able to produce a wide range of reports with data grouped and sorted as much as possible to reduce noise and highlight patterns. Brief summary reports provide a quick overview of general Postfix operations and message delivery, calling out warnings that may require attention. Detailed reports provide easy to scan, hierarchically-arranged and organized information, with as much or little detail as desired.
Postfix-logwatch outputs two principal sections: a Summary section and a Detailed section. For readability and quick scanning, all event or hit counts appear in the left column, followed by brief description of the event type, and finally additional statistics or count representations may appear in the rightmost column.
The following segment from a sample Summary report illustrates:
****** Summary ********************************************
81 *Warning: Connection rate limit reached (anvil)
146 Warned
68.310M Bytes accepted 71,628,177
97.645M Bytes delivered 102,388,245
======== ================================================
3464 Accepted 41.44%
4895 Rejected 58.56%
-------- ------------------------------------------------
8359 Total 100.00%
======== ================================================
There are dozens of sub-sections available in the Detailed report, each of whose output can be controlled in various ways. Each sub-section attempts to group and present the most meaningful data at superior levels, while pushing less useful or noisy data towards inferior levels. The goal is to provide as much benefit as possible from smart grouping of data, to allow faster report scanning, pattern identification, and problem solving. Data is always sorted in descending order by count, and then numerically by IP address or alphabetically as appropriate.
The following MX errors segment from a sample Detailed report illustrates the basic hierarchical level structure of postfix-logwatch:
****** Detailed *******************************************
261 MX errors --------------------------------------
261 Unable to look up MX host
222 Host not found
73 foolishspammer.local
60 completely.bogus.domain.example
11 friend.example.com
39 No address associated with hostname
23 dummymx.sample.net
16 pushn.spam.sample.com
The postfix-logwatch utility reads from STDIN or from the named Postfix logfile. Multiple logfile arguments may be specified, each processed in order. The user running postfix-logwatch must have read permission on each named log file.
The options listed below affect the operation of postfix-logwatch. Options specified later on the command line override earlier ones. Any option may be abbreviated to an unambiguous length.
Note: This report may consume a large amount of memory; if you have no use for it, disable the delays report.
The postfix-logwatch utility produces a Summary section, a Detailed section, and additional report sections. With level less than 5, postfix-logwatch will produce only the Summary section. At level 5 and above, the Detailed section, and any additional report sections are candidates for output. Each incremental increase in level generates one additional hierarchical sub-level of output in the Detailed section of the report. At level 10, all levels are output. Lines that exceed the maximum report width (specified with max_report_width) will be cut. Setting level to 11 will prevent lines in the report from being cut (see also --line_style).
Note: if you use parenthesis in your regular expression, be sure they are cloistering and not capturing: use (?:pattern) instead of (pattern).
The default list is "5.. 4.. Warn", which creates three groups of rejects: permanent rejects, temporary reject failures, and reject warnings (as in warn_if_reject).
This feature allows, for example, distinguishing 421 transmission channel closures from 45x errors (eg. 450 mailbox unavailable, 451 local processing errors, 452 insufficient storage). Such a grouping would be configured with the list: "421 4.. 5.. Warn". See RFC 2821 for more information about reply codes.
See also CONFIGURATION FILE regarding using reject_reply_patterns within a configuration file.
The pattern namepat should match the syslog_name configuration parameter specified in the Postfix parameter file main.cf, the master control file master.cf, or the default value as indicated by the output of postconf -d syslog_name.
Note: if you use parenthesis in your regular expression, be sure they are cloistering and not capturing: use (?:pattern) instead of (pattern).
The output of every section in the Detailed report is controlled by a level limiter. The name of the level limiter variable will be output when the sect_vars option is set. Level limiters are set either via command line in standalone mode with --limit limiter=levelspec option, or via configuration file variable $postfix_limiter=levelspec. Each limiter requires a levelspec argument, which is described below in LEVEL CONTROL.
The list of level limiters is shown below.
There are several level limiters that control reject sub-sections (eg. rejectbody, rejectsender, etc.). Because the list of reject variants is not known until runtime after reject_reply_patterns is seen, these reject limiters are shown below generically, with the prefix ###. To use one of these reject limiters, substitute ### with one of the reject reply codes in effect, replacing each dot with an x character. For example, using the default reject_reply_patterns list of "5.. 4.. Warn", three rejectbody variants are valid: --limit 5xxrejectbody, --limit 4xxrejectbody and --limit warnrejectbody. As a convenience, you may entirely eliminate the ### prefix, and instead use the bare rejectXXX option, and all reject level limiter variations will be auto-generated based on the reject_reply_patterns list. For example, the command line segment:
... --reject_reply_patterns "421 5.." \
--limit rejectrbl="1:10:"
would automatically become:
... --reject_reply_patterns "421 5.." \
--limit 421rejectrbl="1:10:" --limit 5xxrejectrbl="1:10:"
See reject_reply_patterns above, and comments in the configuration file
postfix-logwatch.conf.
[ THIS SECTION IS NOT YET COMPLETE ]
The Detailed section of the report consists of a number of sub-sections, each of which is controlled both globally and independently. Two settings influence the output provided in the Detailed report: a global detail level (specified with --detail) which has final (big hammer) output-limiting control over the Detailed section, and sub-section specific detail settings (small hammer), which allow further limiting of the output for a sub-section. Each sub-section may be limited to a specific depth level, and each sub-level may be limited with top N or threshold limits. The levelspec argument to each of the level limiters listed above is used to accomplish this.
It is probably best to continue explanation of sub-level limiting with the following well-known outline-style hierarchy, and some basic examples:
level 0
level 1
level 2
level 3
level 4
level 4
level 2
level 3
level 4
level 4
level 4
level 3
level 4
level 3
level 1
level 2
level 3
level 4
The simplest form of output limiting suppresses all output below a specified level. For example, a levelspec set to "2" shows only data in levels 0 through 2. Think of this as collapsing each sub-level 2 item, thus hiding all inferior levels (3, 4, ...), to yield:
level 0
level 1
level 2
level 2
level 1
level 2
Sometimes the volume of output in a section is too great, and it is useful to suppress any data that does not exceed a certain threshold value. Consider a dictionary spam attack, which produces very lengthy lists of hit-once recipient email or IP addresses. Each sub-level in the hierarchy can be threshold-limited by setting the levelspec appropriately. Setting levelspec to the value "2::5" will suppress any data at level 2 that does not exceed a hit count of 5.
Perhaps producing a top N list, such as top 10 senders, is desired. A levelspec of "3:10:" limits level 3 data to only the top 10 hits.
With those simple examples out of the way, a levelspec is defined as a whitespace- or comma-separated list of one or more of the following:
All three forms of limiters are effective only when postfix-logwatch's detail level is 5 or greater (the Detailed section is not activated until detail is at least 5).
See the EXAMPLES section for usage scenarios.
Postfix-logwatch can read configuration settings from a configuration file. Essentially, any command line option can be placed into a configuration file, and these settings are read upon startup.
Because postfix-logwatch can run either standalone or within Logwatch, to minimize confusion, postfix-logwatch inherits Logwatch's configuration file syntax requirements and conventions. These are:
option = value
To include a command line option in a configuration file, prefix the command line option name with the word "$postfix_". The following configuration file setting and command line option are equivalent:
$postfix_Line_Style = Truncate
--line_style Truncate
Level limiters are also prefixed with $postfix_, but on the command line
are specified with the --limit option:
$postfix_Sent = 2
--limit Sent=2
The order of command line options and configuration file processing occurs as follows: 1) The default configuration file is read if it exists and no --config_file was specified on a command line. 2) Configuration files are read and processed in the order found on the command line. 3) Command line options override any options already set either via command line or from any configuration file.
Command line options are interpreted when they are seen on the command line, and later options will override previously set options. The notable exception is with limiter variables, which are interpreted in the order found, but only after all other options have been processed. This allows --reject_reply_patterns to determine the dynamic list of the various reject limiters.
See also --reject_reply_patterns.
The postfix-logwatch utility exits with a status code of 0, unless an error occurred, in which case a non-zero exit status is returned.
Note: postfix-logwatch reads its log data from one or more named Postfix log files, or from STDIN. For brevity, where required, the examples below use the word file as the command line argument meaning /path/to/postfix.log. Obviously you will need to substitute file with the appropriate path.
To run postfix-logwatch in standalone mode, simply run:
postfix-logwatch file
A complete list of options and basic usage is available via:
postfix-logwatch --help
To print a summary only report of Postfix log data:
postfix-logwatch --detail 1 file
To produce a summary report and a one-level detail report for May 25th:
grep 'May 25' file | postfix-logwatch --detail 5
To produce only a top 10 list of Sent email domains, the summary report and detailed reports are first disabled. Since commands line options are read and enabled left-to-right, the Sent section is re-enabled to level 1 with a level 1 top 10 limiter:
postfix-logwatch --nosummary --nodetail --limit sent='1 1:10:' file
The following command and its sample output shows a more complex level limiter example. The command gives the top 3 Sent email addresses from the top 5 domains, in addition, all level 3 items with a hit count of 2 or less are suppressed (in the Sent sub-section, this happens to be email's Original To address). Ellipses indicate top N or threshold-limited data:
postfix-logwatch --nosummary --nodetail \
--limit sent '1:5: 2:3: 3::2' file
1762 Sent via SMTP -----------------------------------
352 example.com
310 joe
255 joe.bob@virtdomain.example.com
7 info@virtdomain.example.com
21 pooryoda3
11 hot93uh
...
244 sample.net
97 buzz
26 leroyjones
14 sally
...
152 example.net
40 jim_jameson
23 sam_sampson
19 paul_paulson
...
83 sample.us
44 root
39 jenny1
69 dom3.example.us
10 kay
7 ron
6 mrsmith
...
...
The next command uses both reject_reply_patterns and level limiters to see 421 RBL rejects, threshold-limiting level 2 output to hits greater than 5 (level 2 in the Reject RBL sub-section is the client's IP address / hostname pair). This makes for a very nice RBL offenders list, shown in the sample output (note the use of the unambiguous, abbreviated command line option reject_reply_pat):
postfix-logwatch --reject_reply_pat '421 4.. 5.. Warn' \
--nosummary --nodetail --limit 421rejectrbl='2 2::5' file
300 421 Reject RBL ---------------------------------------
243 zen.spamhaus.org=127.0.0.2
106 10.0.0.129 129.0.0.example.com
41 192.168.10.70 hostx10.sample.net
40 192.168.42.39 hostz42.sample.net
15 10.1.1.152 dsl-10-1-1-152.example.us
14 10.10.10.122 mail122.sample.com
7 192.168.3.44 smalltime-spammer.example.com
...
48 zen.spamhaus.org=127.0.0.4
17 10.29.124.92 10-29-124-92.adsl-static.sample.us
...
8 zen.spamhaus.org=127.0.0.11
...
1 zen.spamhaus.org=127.0.0.10
...
Note: Logwatch versions prior to 7.3.6, unless configured otherwise, required the --print option to print to STDOUT instead of sending reports via email. Since version 7.3.6, STDOUT is the default output destination, and the --print option has been replaced by --output stdout. Check your configuration to determine where report output will be directed, and add the appropriate option to the commands below.
To print a summary report for today's Postfix log data:
logwatch --service postfix --range today --detail 1
To print a report for today's Postfix log data, with one level of detail in the Detailed section:
logwatch --service postfix --range today --detail 5
To print a report for yesterday, with two levels of detail in the Detailed section:
logwatch --service postfix --range yesterday --detail 6
To print a report from Dec 12th through Dec 14th, with four levels of detail in the Detailed section:
logwatch --service postfix --range \
'between 12/12 and 12/14' --detail 8
To print a report for today, with all levels of detail:
logwatch --service postfix --range today --detail 10
Same as above, but leaves long lines uncut:
logwatch --service postfix --range today --detail 11
The postfix-logwatch program uses the following (automatically set) environment variables when running under Logwatch:
logwatch(8), system log analyzer and reporter
README, an overview of postfix-logwatch Changes, the version change list history Bugs, a list of the current bugs or other inadequacies Makefile, the rudimentary installer LICENSE, the usage and redistribution licensing terms
Covered under the included MIT/X-Consortium License: http://www.opensource.org/licenses/mit-license.php
Mike Cappella
The original postfix Logwatch filter was written by Kenneth Porter, and has had many contributors over the years. They are entirely not responsible for any errors, problems or failures since the current author's hands have touched the source code.