Update changelog and POD file.

mpaperno · mpaperno · commit 24bd3235d5fe · 2021-08-03T13:39:28.000-04:00
diff --git a/changelog.txt b/changelog.txt
@@ -8,12 +8,20 @@ SpamPD Change Log
     ! : important change, change of default behavior, etc.
 -----------------------------------------------------------
 
-2.61 (30-Jul-21)
+2.61 (3-Aug-21)
 
-Minor bug fixes. Thanks to Simon Matter for reporting and testing.
+Bug fixes, new features, and some optimization. Thanks to Simon Matter for reporting, suggestions, and testing!
 
 * Restore syslog as default logging destination (https://github.com/mpaperno/spampd/issues/31)
 * Fix issues with older Perl versions (https://github.com/mpaperno/spampd/issues/30)
+~ Optimize initial header processing when building message line array in process_message().
+~ Slight optimization to LMTP multi-recipient handling in process_request().
+~ Optimize how rewritten (tagged) message is saved back to temp file.
++ Add detection and logging of "RULESVERSION" tag with SA >= v3.4.0.
++ Add tracking of some per-child runtime statistics which by default are now shown in the child process names.
++ Add ability to provide a custom child process name template string (or not modify the child name at all).
+    Template format documented in POD. (https://github.com/mpaperno/spampd/issues/32)
++ Add _SPAMPDVERSION_ as a "template tag" (macro), eg. for use in SA add_header directives.
 
 
 2.60 (26-Jul-21)
@@ -30,7 +38,7 @@ This version brings quite a few changes, though the base functionality and compa
 + Add --show <thing> argument for printing default option values and other debug.
 * Fix SpamAssassin debug logging with versions 3.1+ (output was going to stderr/wrong syslog/null).
 * Fix for IPv6 addresses being used on --host and --relayhost options (was not possible due to ":<port>" check).
-! SIGHUP will now reload SpamAssassin and SpamPD configuration files, still with graceful child process shutdown.
+! SIGHUP will now reload SpamAssassin and SpamPD configuration files (and all module code), still with graceful child process shutdown.
 ! Use SpamAssassin::Logger module (with SA 3.1+) for all logging. This now inits logging much earlier.
 ! Log to stderr by default if running non-daemonized (with --nodetach).
 ! Child processes are now renamed to "spampd child" to distinguish them from the parent in task lists.
diff --git a/spampd.pod b/spampd.pod
@@ -55,6 +55,7 @@ Options:
   --maxrequests or -r <n>    Maximum requests that each child can process.
   --childtimeout <n>         Time out children after this many seconds.
   --satimeout <n>            Time out SpamAssassin after this many seconds.
+  --child-name-template [s]  Template for formatting child process name.
 
   --pid   or -p <filename>   Store the daemon's process ID in this file.
   --user  or -u <user>       Specifies the user that the daemon runs as.
@@ -70,7 +71,7 @@ Options:
   --set-envelope-from        Set X-Envelope-From header only.
 
   --local-only or -L         Turn off all SA network-based tests.
-  --homedir path             Use the specified directory as SA home.
+  --homedir <path>           Use the specified directory as SA home.
   --saconfig <filename>      Use the file for SA "user_prefs" configuration.
 
   --logfile or -o <dest>     Destination for logs (syslog|stderr|<filename>).
@@ -215,11 +216,19 @@ to peruse the MIAB L<setup|https://github.com/mail-in-a-box/mailinabox/tree/mast
 L<configuration|https://github.com/mail-in-a-box/mailinabox/tree/master/conf> files for reference.
 
 All I<spampd> options have reasonable defaults, especially for a Postfix-centric
-installation.  You may want to specify the --children option if you have an
+installation.  You may want to specify the C<--max-servers> option if you have an
 especially beefy or weak server box because I<spampd> is a memory-hungry
 program.  Check the L<"Options"> for details on this and all other parameters.
 
-Note that B<I<spampd> replaces I<spamd>> from the I<SpamAssassin> distribution
+To show default values for all options, run C<spampd --show defaults>.
+
+B<Since v2.61> I<spampd> injects a C<_SPAMPDVERSION_>
+L<"template tag"|https://spamassassin.apache.org/doc/Mail_SpamAssassin_Conf.html#TEMPLATE-TAGS>
+macro at message processing time. This can be used in an C<add_header> SA config file directive, for example.
+
+  add_header all Filter-Version SpamAssassin _VERSION_ (_SUBVERSION_, Rules: _RULESVERSION_) / SpamPD _SPAMPDVERSION_
+
+Note that B< I<spampd> replaces I<spamd> > from the I<SpamAssassin> distribution
 in function. You do not need to run I<spamd> in order for I<spampd> to work.
 This has apparently been the source of some confusion, so now you know.
 
@@ -303,6 +312,7 @@ Also note that v2.60 added the ability to use a L</"CONFIGURATION FILE"> for spe
     [--max-servers | -mxs  <n>] [--maxsize   <n>   ] [--[no]detach         ]
     [--maxrequests | -r    <n>] [--local-only | -L ] [--[no]setsid         ]
     [--childtimeout        <n>] [--tagall     | -a ] [--log-rules-hit | -rh]
+    [ --child-name-template | -cnt [<template>] ]    [--homedir <path>     ]
     [ [--set-envelope-headers | -seh] | [--set-envelope-from | -sef] ]
 
     [ --logfile | -o (syslog|stderr|<filename>) ][...]
@@ -326,7 +336,7 @@ same option loaded from config file(s).
 Please be sure to also read the general information about specifying option
 arguments in the above L</"USAGE"> section.
 
-To show default values for all options, run C<spampd --show defaults>.
+To view B<default values> for all options, run C<spampd --show defaults>.
 
 =over 5
 
@@ -360,9 +370,9 @@ public interface (IP address) unless you know exactly what you're doing!
 
 =item B<--port> I<<n>>
 
-Specifies what port I<spampd> listens on. By default, it listens on
-port 10025. This is an alternate to using the above --host=ip:port notation.
-Note that a I<port> specified in the C<--host> option will override this one.
+Specifies what port I<spampd> listens on. This is an alternate to using the above
+C<--host=ip:port> notation. Note that a I<port> specified in the C<--host> option
+will override this one.
 
 
 =item B<--socket> I<<socketpath>>
@@ -380,8 +390,7 @@ format, e.g. 700 to specify acces only for the user I<spampd> is run as.
 =item B<--relayhost> I<< (<ip>|<hostname>)[:<port>] >>
 
 Specifies the hostname/IP to which I<spampd> will relay all
-messages. Defaults to 127.0.0.1 (localhost). If the port is not provided, that
-defaults to 25.
+messages. Defaults to 127.0.0.1 (localhost) on port 25.
 
 As of v2.60 this option can also handle IPv6 addresses in the form of
 C<--relayhost n:n:n> or, with port, C<--relayhost [n:n:n]:port> (the square brackets
@@ -392,7 +401,7 @@ Note that the I<port> specified this way implicitly overrides the C<--relayport>
 
 =item B<--relayport> I<<n>>
 
-Specifies what port I<spampd> will relay to. Default is 25. This is an
+Specifies what port I<spampd> will relay to. This is an
 alternate to using the above --relayhost=ip:port notation. Note that a I<port>
 specified in the C<--relayhost> option will override this one.
 
@@ -429,7 +438,7 @@ and C<--max-servers>, based on demand.
 You may want to set your origination mail server to limit the
 number of concurrent connections to I<spampd> to match this setting (for
 Postfix this is the C<xxxx_destination_concurrency_limit> setting where
-'xxxx' is the transport being used, usually 'smtp', and the default is 100).
+'xxxx' is the transport being used, usually 'smtp' or 'lmtp').
 
 See also C<--min-servers>, C<--min-spare>, and C<--max-spare> options.
 
@@ -471,7 +480,7 @@ I<spampd> works by forking child servers to handle each message. The
 B<maxrequests> parameter specifies how many requests will be handled
 before the child exits. Since a child never gives back memory, a large
 message can cause it to become quite bloated; the only way to reclaim
-the memory is for the child to exit. The default is 20.
+the memory is for the child to exit.
 
 
 =item B<--childtimeout> I<<n>>
@@ -481,29 +490,68 @@ a transaction. In an S/LMTP transaction the timer is reset for every command.
 This timeout includes time it would take to send the message data, so it should
 not be too short.  Note that it's more likely the origination or destination
 mail servers will timeout first, which is fine.  This is just a "sane" failsafe.
-Default is 360 seconds (6 minutes).
 
 
 =item B<--satimeout> I<<n>>
 
 This is the number of seconds to allow for processing a message with
 SpamAssassin (including feeding it the message, analyzing it, and adding
 the headers/report if necessary).
+
 This should be less than your origination and destination servers' timeout
-settings for the DATA command. For Postfix the default is 300 seconds in both
-cases (smtp_data_done_timeout and smtpd_timeout). In the event of timeout
-while processing the message, the problem is logged and the message is passed
-on anyway (w/out spam tagging, obviously).  To fail the message with a temp
-450 error, see the --dose (die-on-sa-errors) option, below.
-Default is 285 seconds.
+settings for the DATA command. (For Postfix this is set in C<(smtp|lmtp)_data_done_timeout>
+and C<smtpd_timeout>). In the event of timeout while processing the message, the problem is
+logged and the message is passed on anyway (w/out spam tagging, obviously).  To fail the
+message with a temp 450 error, see the C<--dose> (die-on-sa-errors) option, below.
+
+
+=item B<--child-name-template> or B<-cnt> I<[<template>]> C<new in v2.61>
+
+Template for formatting child process name. Use a blank string (just the argument name
+without a value) to leave the child process name unchanged (will be same as parent command line).
+
+The template uses C<printf()> style formatting, but with named parameter placeholders.
+For example (wrapped for clarity):
+
+  %base_name: child #%child_count(%child_status)
+  [req %req_count/%req_max, time lst/avg/ttl %(req_time_last).4f/%(req_time_avg).4f/%(req_time_ttl).4f,
+  ham/spm %req_ham/%req_spam, rules v%sa_rls_ver)]'
+
+Would produce something like:
+
+  spampd: child #4(D) [req 8/30, time lst/avg/ttl 0.0222/0.0256/0.2045, ham/spm 3/5, rules v1891891]
+
+Parameters are specified like: "Value of %(my_name)s is %(my_float_value).4f", with names
+in parenthesis followed by standard a C<printf()> style formatting specifier (C<s> is default),
+or simply as "Value of %my_name is %my_value" with the default format being a string
+(works for numerics also).
+
+The following variables are available:
+
+    base_name     # Base script name, eg. "spampd"
+    spampd_ver    # SpamPD version, eg. "2.61"
+    perl_ver      # Perl version, eg. "5.28.1"
+    ns_ver        # Net::Server version, eg. "2.009"
+    ns_typ        # Net::Server type, "PreFork" or "PreForkSimple"
+    ns_typ_acr    # Net::Server type acronym, "PF" or "PFS"
+    sa_ver        # SpamAassassin version, eg. "3.4.2"
+    sa_rls_ver    # SpamAassassin rules update version, eg. "1891891" or "(unknown)"
+    child_count   # total number of children launched so far (current child number)
+    child_status  # child status, "C" for connected, or "D" for disconnected
+    req_count     # number of requests child has processed so far
+    req_max       # maximum child requests before exit
+    req_time_last # [s] time to process the last message
+    req_time_ttl  # [s] total processing time for this child
+    req_time_avg  # [s] average processing time for this child (req_time_ttl / req_count)
+    req_ham       # count of ham messages scored by child
+    req_spam      # count of spam messages scored by child
 
 
 =item B<--pid> or B<-p> I<<filename>>
 
 Specifies a filename where I<spampd> will write its process ID so
 that it is easy to kill it later. The directory that will contain this
-file must be writable by the I<spampd> user. The default is
-F</var/run/spampd.pid>.
+file must be writable by the I<spampd> user.
 
 
 =item B<--logfile> or B<-o> I<< (syslog|stderr|<filename>) >> C<new in v2.60>
@@ -567,20 +615,18 @@ The default was C<unix> except on HP-UX and SunOS (Solaris) systems it is C<inet
 
 =item B<--logident> or B<-li> I<<name>> C<new in v2.60>
 
-Syslog identity name to use. This may also be used in log files written directly (w/out syslog). Default is C<spampd>.
+Syslog identity name to use. This may also be used in log files written directly (w/out syslog).
 
 
 =item B<--logfacility> or B<-lf> I<<name>> C<new in v2.60>
 
-Syslog facility name to use. This is typically the name of the system-wide log file to be written to. Default is C<mail>.
+Syslog facility name to use. This is typically the name of the system-wide log file to be written to.
 
 
 =item B<--[no]detach> I<[0|1]> C<new in v2.20>
 
-By default I<spampd> will detach from the console and fork into the
-background ("daemonize"). Use C<--nodetach> to override this.
-This can be useful for running under control of some daemon
-management tools or testing from a command line.
+Tells I<spampd> to detach from the console and fork into the background ("daemonize").
+Using C<--nodetach> can be useful for running under control of some daemon management tools or testing from a command line.
 
 
 =item B<--[no]setsid> I<[0|1]> C<new in v2.51>
@@ -592,25 +638,25 @@ daemonize. Only used if C<--nodetach> isn't specified.
 
 =item B<--maxsize> I<<n>>
 
-The maximum message size to send to SpamAssassin, in KBytes. By default messages
-over 64KB are not scanned at all, and an appropriate message is logged
+The maximum message size to send to SpamAssassin, in KBytes. Messages
+over this size are not scanned at all, and an appropriate message is logged
 indicating this.  The size includes headers and attachments (if any).
 
 
 =item B<--dose> I<[0|1]>
 
-Acronym for (d)ie (o)n (s)pamAssassin (e)rrors.  By default if I<spampd>
-encounters a problem with processing the message through Spam Assassin (timeout
-or other error), it will still pass the mail on to the destination server.  If
-you specify this option however, the mail is instead rejected with a temporary
-error (code 450, which means the origination server should keep retrying to send
-it).  See the related --satimeout option, above.
+Acronym for (d)ie (o)n (s)pamAssassin (e)rrors. When disabled and I<spampd>
+encounters a problem with processing the message through SpamAssassin (timeout
+or other error), it will still pass the mail on to the destination server.
+When enabled, the mail is instead rejected with a temporary error (code 450,
+which means the origination server should keep retrying to send it). See the
+related C<--satimeout> option, above.
 
 
 =item B<--tagall> or B<-a> I<[0|1]>
 
 Tells I<spampd> to have SpamAssassin add headers to all scanned mail,
-not just spam.  By default I<spampd> will only rewrite messages which
+not just spam.  Otherwise I<spampd> will only rewrite messages which
 exceed the spam threshold score (as defined in the SA settings).  Note that
 for this option to work as of SA-2.50, the I<always_add_report> and/or
 I<always_add_headers> settings in your SpamAssassin F<local.cf> need to be
@@ -653,18 +699,18 @@ Turn off all SA network-based tests (DNS, Razor, etc).
 
 Use the specified directory as home directory for the spamassassin process.
 Things like the auto-whitelist and other plugin (razor/pyzor) files get
-written to here.
-Default is /var/spool/spamassassin/spampd.  A good place for this is in the same
-place your bayes_path SA config setting points to (if any).  Make sure this
-directory is accessible to the user that spampd is running as (default: mail).
+written to here. A good place for this is in the same
+place your C<bayes_path> SA config setting points to (if any).  Make sure this
+directory is accessible to the user that spampd is running as.
+
 Thanks to Alexander Wirt for this fix.
 
 
 =item B<--saconfig> I<<filename>>
 
 Use the specified file for SpamAssassin configuration options in addition to the
 default local.cf file.  Any options specified here will override the same
-option from local.cf.  Default is to not use any additional configuration file.
+option from local.cf.
 
 
 =item B<--debug> or B<-d> I<< [<area,...>|1|0] >> C<(updated in v2.60)>
@@ -677,7 +723,7 @@ to 4 (debug), adding yet more info (but not too much) (new in v2.2).
 C<New in v2.60:>
 
 Setting the value to 1 (one) is the same as using no parameter (eg. simply I<-d>).
-The value of 0 (zero) disables debug logging (this is the default).
+The value of 0 (zero) disables debug logging.
 
 The I<area> list is passed on directly to SpamAssassin and controls logging
 facilities. If no I<area>s are listed (and debug logging is enabled), all
@@ -986,15 +1032,18 @@ permissions on the relaysocket!
 =head1 CREDITS
 
 I<spampd> is written and maintained by Maxim Paperno <MPaperno@WorldDesign.com>.
-See L<http://www.WorldDesign.com/index.cfm/rd/mta/spampd.htm> for latest info.
+The open source code repository is located at L<https://github.com/mpaperno/spampd/>.
+See L<http://www.WorldDesign.com/index.cfm/rd/mta/spampd.htm> for historical info.
 
-I<spampd> v2 uses two Perl modules by Bennett Todd and Copyright (C) 2001 Morgan
-Stanley Dean Witter. These are distributed under the GNU GPL (see
-module code for more details). Both modules have been slightly modified
-from the originals and are included in this file under new names.
+I<spampd> v2 uses two Perl modules (I<MSDW::SMTP::Client> and I<MSDW::SMTP::Server>)
+by Bennett Todd and Copyright (C) 2001 Morgan Stanley Dean Witter.
+These are distributed under the GNU GPL (see module code for more details).
+Both modules have been slightly modified from the originals and are included in
+this file under new names (I<SpamPD::Client> and I<SpamPD::Server>, respectively).
 
-Also thanks to Bennett Todd for the example smtpproxy script which helped create
-this version of I<spampd>.  See http://bent.latency.net/smtpprox/ .
+Also thanks to Bennett Todd for the example I<smtpproxy> script which helped create
+this version of I<spampd>.  See L<http://bent.latency.net/smtpprox/> (seems to be down)
+or L<https://github.com/jnorell/smtpprox>.
 
 I<spampd> v1 was based on code by Dave Carrigan named I<assassind>. Trace
 amounts of his code or documentation may still remain. Thanks to him for the
@@ -1019,8 +1068,7 @@ See also: L<https://github.com/mpaperno/spampd/graphs/contributors/>
 
 =head1 COPYRIGHT, LICENSE, AND DISCLAIMER
 
-I<spampd> is Copyright (c) 2002-2006, 2009-2010, 2013, 2018-2019 Maxim Paperno;
-All Rights Reserved.
+I<spampd> is Copyright (c) Maxim Paperno;  All Rights Reserved.
 
 Portions are Copyright (c) 2001 Morgan Stanley Dean Witter as mentioned above
 in the Credits section.
@@ -1109,7 +1157,7 @@ L<Integrating SpamAssassin into Postfix using spampd|https://wiki.apache.org/spa
         return word.replace(word[0], word[0].toUpperCase());
       }).join(' ');
     };
-    var list = document.querySelectorAll("a[href*=podtop] h1, ul#index > li > a, h1 a.u, li.indexItem1 > a");
+    var list = document.querySelectorAll("a[href*=podtop] h1, ul#index > li > a, h1 a.u, body > h1[id], li.indexItem1 > a");
     for (let item of list)
       item.innerText = titleCase(item.innerText);
   }
