MANUAL

fdm ============================================================================

*** Introduction

fdm is a program to fetch mail and deliver it in various ways depending on a
user-supplied ruleset. Mail may be fetched from stdin, IMAP or POP3 servers, or
from local maildirs, and filtered based on whether it matches a regexp, its
size or age, or the output of a shell command. It can be rewritten by an
external process, dropped, left on the server or delivered into maildirs,
mboxes, to a file or pipe, or any combination.

fdm is designed to be lightweight but powerful, with a compact but clear
configuration syntax. It is primarily designed for single-user uses but may
also be configured to deliver mail in a multi-user setup. In this case, it uses
privilege separation to minimise the amount of code running as the root user.

*** Table of contents

##	Installation
##	Quick start
##	The configuration file
%%	Including other files
%%	Macros
%%	Testing macros
%%	Shell commands
##	Invoking fdm
%%	Temporary files
%%	Command line arguments
%%	Running from cron
%%	The lock file
%%	Testing and debugging
##	Fetching mail
%%	Mail tags
%%	POP3 and POP3S
%%	SSL certificate verification
%%	The .netrc file
%%	IMAP and IMAPS
%%	IMAP or POP3 over a pipe or ssh
%%	stdin and local mail
%% 	From maildirs and mboxes
%%	Using NNTP and NNTPS
%%	New or old mail only
##	Defining actions
%%	Drop and keep
%%	Maildirs
%%	Mboxes
%%	IMAP and IMAPS
%%	SMTP
%%	LMTP
%%	Write, pipe, exec and append
%%	stdout
%% 	Rewriting mail
%%	Adding or removing headers
%%	Tagging
%%	Compound actions
%%	Chained actions
##	Filtering mail
%%	Nesting rules
%%	Lambda actions
%%	The all condition
%%	Matched and unmatched
%%	Matching by account
%%	Matching a regexp
%%	Matching bracket expressions
%%	Matching by age or size
%%	Using a shell command
%%	Attachments
%%	Matching tags
%%	Using caches
%%	Cache commands
##	Setting options
##	Archiving and searching mail
##	Using fdm behind a proxy
##	Bug reports and queries
##	Frequently asked questions

### Installation

fdm depends on the Trivial Database library (TDB), available at:

	https://tdb.samba.org

Ensure it is installed, then download the source tarball and build fdm with:

	$ tar -zxvf fdm-?.?.tar.gz
	$ cd fdm-?.?
	$ ./configure && make

Then run 'make install' to install fdm to the default location under
/usr/local. The --prefix argument may be set to specify an alternative
installation location:

	$ ./configure --prefix=/opt/fdm && make
	$ sudo make install

If being run as root, fdm requires a user named "_fdm" to exist. It will drop
privileges to this user and its primary group. The user may be added on
OpenBSD with, for example:

	# useradd -u 999 -s /bin/nologin -d /var/empty -g=uid _fdm

It is not necessary to add a user if fdm is always started by a non-root user.

fdm can be built to use PCRE rather than standard regexps. To do so, add -DPCRE
to the make command:

	$ make -DPCRE

Or PCRE=1 if using GNU make:

	$ make PCRE=1

### Quick start

A simple ~/.fdm.conf file for a single user fetching from POP3, POP3S and IMAP
accounts and delivering to one maildir may look similar to:

 	# Set the maximum size of mail.
	set maximum-size 128M

	# An action to save to the maildir ~/mail/inbox.
	action "inbox" maildir "%h/mail/inbox"

	# Accounts: POP3, POP3S and IMAP. Note the double escaping of the '\'
	# character in the password. If the port is omitted, the default
	# ("pop3", "pop3s", "imap" or "imaps" in the services(5) db) is used.
	account "pop3" pop3 server "my.pop3.server"
		user "my-username" pass "my-password-with-a-\\-in-it"
	account "pop3s" pop3s server "pop.googlemail.com" port 995
		user "my-account@gmail.com" pass "my-password"
	# If the 'folder "my-folder"' argument is omitted, fdm will fetch mail
	# from the inbox.
	account "imap" imap server "my.imap.server"
		user "my-username" pass "my-password" folder "my-folder"

	# Discard mail from Bob Idiot. Note that the regexp is an extended
	# regexp, and case-insensitive by default. This action is a "lambda" or
	# unnamed action, it is defined inline as part of the match rule.
	match "^From:.*bob@idiot\\.net" in headers action drop

	# Match all other mail and deliver using the 'inbox' action.
	match all action "inbox"

A simple initial configuration file without filtering, perhaps to replace
fetchmail or getmail delivering to maildrop, may look similar to:

 	# Set the maximum size of mail.
	set maximum-size 128M

	# Action to pipe directly to maildrop.
	action "maildrop" pipe "/usr/local/bin/maildrop"

	# Account definitions.
	account ....

	# Send all mail to maildrop.
	match all action "maildrop"

To run fdm every half hour from cron, add something like this:

*/30    *       *       *       *       /usr/local/bin/fdm -l fetch

See the fdm.conf(5) man page or the rest of this manual for more detail of the
configuration file format.

### The configuration file

fdm is controlled by its configuration file. It first searches for a .fdm.conf
file in the invoking user's home directory. If that fails, fdm attempts to use
/etc/fdm.conf. The configuration file may also be specified using the '-f'
command line option, see the section on that subject below.

This section gives an overview of the configuration file syntax. Further
details of syntax, and specific keywords, are covered in later sections.

The configuration file has the following general rules:

- Keywords are specified as unadorned lowercase words: match, action, all.
- Strings are enclosed in double quotes (") or single quotes ('). In double
  quoted strings, double quotes may be included by escaping them using the
  backslash character (\). Backslashes must also be escaped ("\\") - this
  applies to all such strings, including regexps and passwords. The special
  sequence '\t' is replaced by a tab character. In single quoted strings no
  escaping is necessary, but it is not possible to include a literal ' or a
  tab character.
- Comments are prefixed by the hash character (#) and continue to the end of
  the line.
- Whitespace is largely ignored. Lines may generally be split, concatenated
  or indented as preferred.
- Lists are usually specified as 'singular item' or 'plural { item item }', for
  example: 'user "nicholas"', 'users { "nicholas" "bob" }'. The singular/plural
  distinction is not required, it is recommended only to aid readability:
  'user { "nicholas "bob" }' is also accepted.
- Regexps are specified as normal strings without additional adornment other
  than the "s (not wrapped in /s). All regexps are extended regexps. They are
  case insensitive by default but may be prefixed with the 'case' keyword to
  indicate case sensitivity is required.
- Strings may be concatenated using plus: "a" + "b" is the same as "ab". This
  is most useful to wrap strings across multiple lines.

Definition/option lines generally follow the following basic form:

	<keyword> <name or command> <parameters>

Example lines that may appear in a configuration file are:

	# This is a comment.

	set lock-types flock

	account "stdin" disabled stdin

	action "strip-full-disclosure"
		rewrite "sed 's/^\\(Subject:.*\\)\\[Full-disclosure\\] /\\1/'"

	match "^X-Mailing-List:.*linux-kernel@vger.kernel.org" in headers
		or "^(To:|Cc:):.*@vger.kernel.org" in headers
		action "linux-kernel"

%%% Including other files

The fdm configuration may be split into several files. Additional files may
be referenced using the 'include' keyword:

	include "my-include-file.conf"

	include	"/etc/fdm.d/shared-conf-1.conf"

%%% Macros

Macros may be defined and used in the configuration file. fdm makes a
distinction between macros which may hold a number (numeric macros) and
those that hold a string (string macros). Numeric macros are prefixed with
the percentage sign (%) and string by the dollar sign ($). Macros are
defined using the equals operator (=):

	%nummacro = 123

	$strmacro = "a string"

Macros may then be referenced in either a standalone fashion anywhere a string
or number is expected, depending on the type of macro:

	$myfile = "a-file"
	include $myfile

	%theage = 12
	match age < %theage action "old-mail"

Or embedded in a string by enclosing the macro name in {}s:

	$myfile2 = "a-file2"
	include "/etc/${myfile2}"

	%anum = 57
	include "/etc/file-number-%{anum}"

Macros are not substituted in strings specified using single-quotes.

%%% Testing macros

The 'ifdef', 'ifndef' and 'endif' keywords may be used to include or omit
sections of the configuration file depending on whether a macro is defined. An
'ifdef' is followed by a macro name (including $ or % type specifier) and if
that macro exists, all following statements up until the next endif are
evaluated (accounts created, rules added, and so on), otherwise they are
skipped. 'ifndef' is the inverse: if the macro exists, the statements are
skipped, otherwise they are included. An example is:

	ifdef $dropeverything
	match all action drop
	endif

These keywords are particularly useful in conjunction with the '-D' command line
option. Any statements between 'ifdef'/'ifndef' and 'endif' must still be valid
syntax.

%%% Shell commands

The value of a shell command may be used at any point in the configuration file
where fdm expects a string or number. Shell commands are invoked by enclosing
them in $() or %(). They are executed when the configuration file is parsed
and if $() is used, any output to stdout is treated as a literal string (as
if the output was inserted directly in the file enclosed in double quotes); %()
attempts to convert the output to a number. For example:

	$mytz = $(date +%Z)

	%two = %(expr 1 + 1)

	$astring = "abc" + $(echo def)

Parts of the command within double quotes (") are subject to tag and macro
replacement as normal (so it is necessary to use %% if a literal % is required,
see the section on tags below); parts outside double quotes or inside single
quotes are not.

### Invoking fdm

fdm accepts a number of command line arguments and may be invoked as needed
from the command line or by a mail transfer agent, such as sendmail, or at
regular times using a program such as cron(8).

%%% Temporary files

As each mail is being processed, it is stored in a temporary file in /tmp, or
if the TMPDIR environment variable exists in the directory it points to.

fdm tries to queue a number of mails simultaneously, so that older can be
delivered while waiting for the server to provide the next. The maximum length
of the queue for each account is set by the 'queue-high' option (the default is
two) and the maximum mail size accepted by the 'maximum-size' option (the
default is 32 MB). In addition, the 'rewrite' action requires an additional
temporary mail. Although fdm will fail rather than dropping mail if the disk
becomes full, users should bear in mind the possibility and set the size of the
temporary directory and the fdm options according to their needs.

%%% Command line arguments

The fdm command has the following synopsis:

	fdm [-klmnqv] [-f conffile] [-u user] [-a account] [-x account]
		[-D name=value] [fetch | poll | cache ...]

The meaning of the flags are covered in the fdm(1) man page, but a brief
description is given below. The flags are also mentioned at relevant points
in the rest of this document.

Flag	Meaning
-k 	Keep all mail (do not delete it from the server). This is useful for
	testing delivery rules without risking mail ending up permanently
	in the wrong place.
-l	Log to syslog(3) using the 'mail' facility rather than outputting to
	stderr.
-m	Ignore the lock file.
-n	Run a syntax check on the configuration file and exit without fetching
	any mail.
-q	Quiet mode. Don't print anything except errors.
-v	Print verbose debugging output. This option may be specified multiple
	times for increasing levels of verbosity. Useful levels are -vv to
	display the result of parsing the configuration file, and -vvvv to copy
	all traffic to and from POP3 or IMAP servers to stdout (note that -l
	disables this behaviour).
-f conffile
	Specify the path of the configuration file.
-u user
	Use 'user' as the default user for delivering mail when started as
	root.
-a account
	Process only accounts with a name matching the given pattern. Note that
	fnmatch(3) wildcards may be used to match multiple accounts with one
	option, and that the option may be specified multiple times.
-x account
	Process all accounts except those that match the given pattern. Again,
	fnmatch(3) wildcards may be used, and the -x option may be specified
	multiple times.
-D name=value
	Define a macro. The macro name must be prefixed with '$' or '%' to
	indicate if it is a string or numeric macro. Macros defined on the
	command line override any macros with the same name defined in the
	configuration file.

If -n is not specified, the flags must be followed by one of the keywords
'fetch' or 'poll' or 'cache'. The 'fetch' keyword will fetch and deliver mail,
the 'poll' keyword print an indication of how many mails are present in each
account, and the 'cache' keyword is followed by one of a set of cache commands
used to manipulate caches from the command-line (see the sections on caches
below). 'fetch' or 'poll' or 'cache' may be abbreviated.

Examples:

	$ fdm -v poll

	$ fdm -vvnf /etc/my-fdm.conf

	$ fdm -lm -a pop3\* fetch

	$ fdm -x stdinacct fetch

	# fdm -u nicholas -vv f

%%% Running from cron

To fetch mail regularly, fdm must be run from cron. This line in a crontab(5)
will run fdm every 30 minutes:

*/30   *      *      *      *      /usr/local/bin/fdm -l fetch

The '-l' option sends fdm's output to syslog(3) rather than having cron mail
it. To keep a closer eye, adding '-v' options and removing '-l' will have
debugging output mailed by cron, or, using a line such as:

*/30   *      *      *      *      fdm -vvvv fetch >>/home/user/.fdm.log 2>&1

Will append extremely verbose fdm output to the ~/.fdm.log file. Note that this
log file can become pretty large, so another cronjob may be required to remove
it occasionally!

%%% The lock file

fdm makes use of a lock file to prevent two instances running simultaneously.
By default, this lock file is .fdm.lock in the home directory of the user who
runs fdm, or /var/db/fdm.lock for root. This default may be overridden in
the configuration file with the 'set lock-file' command:

	set lock-file "/path/to/my/lock-file"

Or disabled altogether by being set to the empty string:

	set lock-file ""

The '-m' command line option may be used to force fdm to ignore the lock file
and run regardless of its existence and without attempting to create it.

%%% Testing and debugging

fdm has some features to assist with testing and debugging a ruleset:

The '-n' command line option. This is particularly useful in conjunction with
'-vv', for example:

	$ cat test.conf
	account "pop3" pop3 server "s" user "u" pass "p"
	action "rw" rewrite "sed 's/\\(Subject:.*\\)\\[XYZ\\]/\1/'"
	action "mbox" mbox "%h/INBOX"
	match all actions { "rw" "mbox" }
	$ fdm -vvnf test.conf
	version is: fdm 0.6 (20061204-1433)
	starting at: Tue Dec  5 15:45:41 2006
	user is: nicholas, home is: /home2/nicholas
	loading configuration from test.conf
	added account: name=pop3 fetch=pop3 server "s" port pop3 user "u"
	added action: name=rw deliver=rewrite "sed 's/\(Subject:.*\)\[XYZ\]/1/'"
	added action: name=mbox deliver=mbox "%h/INBOX"
	finished file test.conf
	added rule: actions="rw" "mbox" matches=all
	configuration loaded
	locking using: flock
	headers are: "to" "cc"
	domains are: "yelena"
	using tmp directory: /tmp

Looking at the output, the parsed strings used by fdm can be seen, and it is
possible to spot that an escape character has been missed in the command.

If '-vvvv' is used, fdm will print all data sent to and received from remote
servers to stdout. Note that this is disabled if the '-l' option is given, and
includes passwords, usernames and hostnames unmodified. The 'fdm-sanitize'
script provided with fdm may be used to remove passwords and usernames from
this output, either while it is being collected:

	fdm -vvvv -a testacct f 2>&1|./fdm-sanitize|tee my-output

Or afterwards:

	./fdm-sanitize <vvvv-output >my-output

Since fdm fetches multiple accounts simultaneously, which may intersperse
debugging output, it is recommended to fetch each account seperately if running
the output through fdm-sanitize. If this is not done, it may not be able to
detect all usernames or passwords.

The '-k' command line option (and the 'keep' keywords on actions and accounts,
covered later) prevent fdm from deleting any mail after delivery. This may be
used to perform any number of test deliveries without risk of losing mail.

### Fetching mail

fdm fetches mail from a set of 'accounts', defined using the 'account'
keyword. Each account has a name, a type, a number of account specific
parameters and a couple of optional flags. The general form is:

	account <name> [<users>] [disabled] <type> [<parameters>] [keep]

The <name> item is a string by which the account is referred in filtering
rules, log output and for the '-a' and '-x' command line options.

The <users> portion specifies the default users to use when delivering mail
fetched from this account as root. It has the same syntax as discussed in
detail in the section below on defining actions.

If the optional 'disabled' keyword is present, fdm ignores the account unless
it is specified on the command line using the '-a' flag.

The optional 'keep' keyword instructs fdm to keep all mail from this account
(not delete it from the server) regardless of the result of the filtering
rules.

The <type> item may be one of: 'pop3', 'pop3s', 'imap', 'imaps', 'stdin',
'maildir' or 'maildirs'.

%%% Mail tags

As mail is processed by fdm, it is tagged with a number of name/value pairs.
Some tags are added automatically, and mail may also be tagged explicitly by
the user (see the later tagging section). Tags may be inserted in strings in a
similar manner to macros, except tags are processed when the string is used
rather than always as the configuration file is parsed. A tag's value is
inserted by wrapping its name in %[], for example:

	match string "%[account]" to "myacct" action "myacctact"

Most of the default tags have a single-letter shorthand which removes the needs
for the []s:

	match string "%a" to "myacct" action "myacctact"

Including a nonexistent tag in a string is equivalent to including a tag with
an empty value, so "abc%[nonexistent]def" will be translated to "abcdef".

The automatically added tags are:

Name		Shorthand	Replaced with
account		%a		The name of the account from which the mail was
				fetched.
home		%h		The delivery user's home directory.
uid		%n		The delivery user's uid.
action		%t		The name of the action the mail has matched.
user		%u		The delivery user's username.
hour		%H		The current hour (00-23).
minute		%M		The current minute (00-59).
second		%S		The current second (00-59).
day		%d		The current day of the month (00-31).
month		%m		The current month (01-12).
year		%y		The current year as four digits.
year2				The current year as two digits.
dayofweek	%W		The current day of the week (0-6, Sunday is 0).
dayofyear	%Y		The current day of the year (000-365).
quarter		%Q		The current quarter (1-4).
rfc822date			The current time in RFC822 date format.
mail_hour			The hour from the mail's date header, converted
				to local time, if it exists and is valid,
				otherwise the current time.
mail_minute			The minute from the mail's date header.
mail_second			The second from the mail's date header.
mail_day			The day from the mail's date header.
mail_month			The month from the mail's date header.
mail_year			The year from the mail's date header as four
				digits.
mail_year2			The same as two digits.
mail_rfc822date			The mail date in RFC822 format.
hostname			The local hostname.
folder				The name of the IMAP folder that the mail
				originated from.

In addition, the shorthand %% is replaced with a literal %, and %1 to %9 are
replaced with the result of any bracket expressions in the last regexp (see
later section on regexps). A leading ~ or ~user is expanded in strings where a
path or command is expected.

Some accounts add additional tags, discussed below.

Tags are replaced in almost all strings (including those in single-quotes!),
some when the configuration file is parsed and some when the string is used.

%%% POP3 and POP3S

Mail may be fetched from a POP3 account. A POP3 account is defined by
specifying the following parameters: the server host and optionally port, and
optionally the user name and password. If the port is not specified, the
default port ('pop3' in the services(5) database) is used. If the user name,
password, or both is omitted, fdm attempts to look it up the .netrc file, see
the next section for details. Optionally fdm can read the password from a
command line program, see below for details.

Examples of a POP3 account definition are:

	account "pop3acct" pop3 server "pop.isp.com" user "bob" pass "pass"

	account "gmx" pop3 server "pop.gmx.net" port 110 user "jim" pass "pass"

	account "acct" pop3 server "10.0.0.1" port "pop3"
		user "nicholas" keep

	account "lycos" disabled pop3 server $localserver port 10110
		pass "password"

Note that the server string is enclosed in double quotes even if it is an IP,
and don't forget to escape any " and \ characters in passwords!

fdm will attempt to use APOP to obscure the password, if the server offers it.
If the server advertises itself as supporting APOP but subsequently refuses
to accept it, fdm will not retry with a cleartext password. Use of APOP can be
disabled for an account using the 'no-apop' flag, for example:

	account "acct" pop3 server "server" user "bob" pass "pass" no-apop

The 'starttls' keyword may be added to a POP3 account to attemp STARTTLS after
connection.

POP3S is specified in exactly the same way, except using the 'pop3s' keyword
for the type, and the default port is 'pop3s' rather than 'pop3':

	account "pop3sacct" pop3s server "pop.isp.com" user "bob" pass "pass"

POP3 accounts automatically tag mail with 'server' and 'port' tags, with the
value of the server and port attributes exactly as specified in the account
definition. A 'server_uid' tag is also added with the server unique id (UIDL).

POP3 adds 'lines', 'body_lines' and 'header_lines' tags with the number of
lines in the complete mail and its body and header. These tags are not updated
to reflect any changes made to the mail by fdm rules.

%%% SSL certificate verification

fdm can verify SSL certificates before collecting mail from an SSL server. This
is enabled globally with the 'verify-certificates' option:

	set verify-certificates

And may be disabled per-account using the 'no-verify' keyword (this applies to
both POP3S and IMAPS accounts):

	account "pop3sacct" pop3s server "pop.isp.com" no-verify

For an introduction to SSL, see:

	http://httpd.apache.org/docs/2.0/ssl/ssl_intro.html

A cert bundle is required to verify SSL certificate chains. For more information
see:

	http://lynx.isc.org/current/README.sslcerts

A pregenerated bundle is available courtesy of the MirOS project:

	http://cvs.mirbsd.de/src/etc/ssl.certs.shar

%%% The .netrc file

If the user name or password is omitted in POP3 or IMAP account definitions,
fdm will attempt to look it up in the .netrc file in the invoking user's home
directory.

The .netrc file format is shared with ftp(1) and some other programs. It
consists of a number of 'machine' sections and optionally one 'default' section
containing a username ('login') and password for that host. fdm accepts entries
only if the machine name matches the POP3 or IMAP server string exactly. If no
matches are found and a 'default' section exists, it is used.

An example .netrc file is:

	machine "my.mail-server.com"
	login "nicholas"
	password "abcdef"

	machine "pop.googlemail.com"
	password "pass1"

	default
	login "bob"
	password "moo"

fdm will abort if the .netrc file is world-writable or world-readable.

%%% Passwords from a command

fdm can read the password from a command by using command substitution
with $(). For example:

	user "fdm@example.com" pass $(gpg --quiet --decrypt ~/.password.gpg)

%%% IMAP and IMAPS

IMAP and IMAPS accounts are defined using exactly the same syntax as for POP3
and POP3S, aside from using the 'imap' or 'imaps' keywords and that the default
port is 'imap' or 'imaps'. There is also an additional, optional 'folders'
option to specify the folders from which mail should be fetched. If omitted,
fdm defaults to the inbox. When the 'folders' tag is used the recieved mail
will have a 'folder' tag with the origin folder name.

Note that with IMAP and IMAPS, mail is still removed from the server unless the
'keep' option is given, or the '-k' command line option used.

Examples of IMAP and IMAPS accounts include:

	account "imapacct" imap server "imap.server.ca" user "bob" pass "pass"

	account "oldimap" disabled imaps server "192.168.0.1" port 10993
		user "nicholas" pass "pass" folders { "Saved" "MyStuff" }

	account "workspam" disabled imap server "my-work.ath.cx"
		user "Nicholas" folder "Junk"

By default, fdm prefers the CRAM-MD5 authentication method, since no passwords
are sent in the clear. If the server does not advertise CRAM-MD5 capability,
and 'oauthbearer' option is not passed the older LOGIN method is used. For
IMAPS connections (which use SSL), the LOGIN method is just as secure. Either
of these methods may be disabled with the 'no-cram-md5' and 'no-login' options.
If the server advertises OAUTHBEARER capability, the 'oauthbearer' option will
use OAuth 2.0 bearer tokens - passed via the 'pass' keyword - as authentication
method.

The 'starttls' keyword may be added to an IMAP account to attemp STARTTLS after
connection.

As with POP3, IMAP adds the 'server', 'port', 'server_uid' and the three line
count tags to mail.

%%% IMAP or POP3 over a pipe or ssh

Mail may be fetched using IMAP or POP3 via a pipe. This is particularly useful
for fetching mail over ssh using public keys.

For IMAP, a user and password may be supplied, but fdm will only use them if
the server asks. If the connection is preauthenticated, the user and password
are unnecessary. For POP3, a user and password must be supplied as usual: due
to the lack of server name, it cannot be read from the .netrc file.

Communication takes place via the pipe program's stdin and stdout. If any
output is found on stderr, fdm will print it (or log it with '-l').

Examples are:

	account "imapssh" imap pipe "ssh jim@myhost /usr/local/libexec/imapd"

	account "imapssh2" imap pipe "/usr/bin/whatever" user "bob" pass "bah"

	account "pop3local" pop3
		pipe "/usr/local/bin/ipop3d" user "me" pass "foo"

%%% stdin and local mail

fdm may be configured to fetch mail from stdin, by specifying an account of
type 'stdin', for example:

	account "stdin" disabled stdin

This is most useful to have fdm behave as a mail delivery agent. To configure
it for single-user use with sendmail, the simplest method it to add:

	"|/usr/local/bin/fdm -m -a stdin fetch"

To the user's ~/.forward file (including the double quotes). Note the use of
'-m' to prevent stdin delivery from interfering with any normal cronjob, and
'-a' to specify that only the disabled "stdin" account should be fetched.

stdin accounts add the three line count tags described in the POP3 section.

%%% From maildirs and mboxes

Fetching from maildirs allows fdm to be used to filter mail on the local
machine. This is covered more detail in the later section on archiving and
searching.

Maildir accounts are specified as follows:

	account "mymaildir" maildir "/path/to/dir"

	account "mymaildirs" maildirs { "/path/to/dir1" "/path/to/dir2" }

Shell glob wildcards may be included in the path names to match multiple
maildirs, but every directory found must be a valid maildir.

Maildir accounts tag mail with a 'maildir' tag which is the basename of the
maildir.

Fetching from mboxes is similar:

	account "mybox" mbox "/path/to/mbox"

	account "mymboxes" mboxes { "/path/to/mbox1" "/path/to/mbox2" }

Note that if an mbox is modified (mail is dropped from it), sufficient disk
space is required to create a temporary copy of the entire mbox.

%%% Using NNTP and NNTPS

fdm can fetch news messages from a news server using NNTP or NNTPS. News
accounts are specified like so:

	account "news1" nntp server "news.server.sk" port 119
		group "comp.unix.bsd.openbsd.misc"
		cache "%h/.fdm.cache/%[group]"

	account "mynews" nntps server "ssl.news.server" port "nntps"
		user "myuser" pass "mypass"
		groups { "alt.test" "alt.humor.best-of-usenet" }
		cache "%h/.fdm.cache"

The cache is a file used to store details of the last article fetched. If only
one group is supplied in the account definition, %[group] tags are replaced by
the name of the group in the cache path. If multiple groups are provided,
%[group] is removed.

Note that whether a message is kept or deleted is irrelevent to NNTP, articles
are always left on the server. The index and message-id of the last article
is recorded in the cache file so that older articles are skipped when the a
newsgroup is again fetched. This happens regardless of any 'keep' keywords or
the '-k' command line option.

As with POP3 and IMAP, NNTP accounts add the 'server' and 'port' tags to mail.
In addition, a 'group' tag is added with the group name. This can ensure
articles are matched purely on the group they are fetched from (trying to do
this using headers is unreliable with cross-posted articles). For example:

match account "news" {
	match string "%[group]" to "comp.lang.c" action "news-%[group]"
	match string "%[group]" to "comp.std.c" action "news-%[group]"
	match all action drop
}

%%% New or old mail only

With POP3 and IMAP, fdm can be set up to fetch only new or old mail. For POP3
this is achieved by recording the current state of the server in a cache file,
which is updated as each mail is fetched. For IMAP it makes use of the 'seen'
server flag which is updated by the server after each mail is fetched.

These options are specified as in the following examples. For POP3:

	account "name" pop3 server "blah" new-only cache "~/.fdm-pop3-cache"

	account "acct" pop3s server "my-server" user "bob"
		new-only cache "my-server-pop3-cache" no-apop

And for IMAP:

	account "imap" imap server "blah" new-only

	account "sslimap" imaps server "imaps.somewhere"
		user "user" pass "pass" old-only no-verify

Note that currently, when using this with IMAP, the server is permitted to flag
the mail as 'seen' before fdm has successfully delivered it, so there is no
guarantee that mail so marked has been delivered, only that it has been
fetched.

### Defining actions

An action is a particular command to execute on a mail when it matches a
filtering rule (see the next section on filtering mail). Actions are named,
similar to accounts, and have a similar form:

	action <name> [<users>] <type> <parameters>

The <users> item may be either:

- the keyword 'user' followed by a single username string or uid, such as:

	user "nicholas"

	user "1000"

- the keyword 'users' followed by a list of users in {}s, for example:

	users { "1001" "nicholas" }

If users are specified, the action will be run once for each user, with fdm
changing to that user before executing the action. Note that fdm will execute
the action once for each user even when not started as root, but will not be
able to change to the user. The user keyword is primarily of use in multiuser
configurations. If users are present on an action, they override any specified
by the account definition.

If running as root and no user is specified on either the action or on the
filtering rule (see the section on filtering below), the default user is
used, see the '-u' command line option and the 'default-user' option in the
setting options section

%%% Drop and keep

The simplest actions are the 'drop' and 'keep' actions. They have no parameters
and are specified like this:

	action "mydropaction" drop

	action "mykeepaction" keep

The 'drop' action arranges for mail to be dropped when rule evaluation is
complete. Note that using 'drop' does not stop further evaluation if the
filtering rule contains a 'continue' keyword, and it may be overridden by a
'keep' option on the account or by the '-k' flag on the command line.

The 'keep' action is similar to 'drop', but it arranges for the mail to be
kept once rule evaluation is complete, rather than dropped.

%%% Maildirs

Mails may be saved to a maildir through a 'maildir' action, defined like so:

	action "mymaildiraction" maildir "/path/to/maildir"

If any component of the maildir path does not exist, it is created, unless the
no-create option is specified. Mails saved to a maildir are tagged with a
'mail_file' tag containing the full path to the file in which they were saved.

%%% Mboxes

An action to deliver to an mbox is defined in the same way as for a maildir:

	action "mymboxaction" mbox "/path/to/mbox"

The same % tokens are replaced in the path. If the mbox does not exist, it
is created. Mboxes may optionally be gzip compressed by adding the 'compress'
keyword:

	action "mymboxaction" mbox "/path/to/mbox" compress

fdm will append .gz to the mbox path (if it is not already present) and append
compressed data. If the mbox exists but is not already compressed, uncompressed
data will be appended.

As with maildirs, if any component of the mbox path does not exist, it is
created, unless the no-create option is set. Mails saved to an mbox are tagged
with an 'mbox_file' tag with the path of the mbox.

%%% IMAP and IMAPS

An action may be defined to store mail in an IMAP folder. The specification is
similar to the IMAP account definition. A server host and optionally port
(default 'imap' or 'imaps') must be specified. A username and password may be
supplied; if they are omitted, fdm will attempt to find a .netrc
entry. Examples include:

	action "myimapaction" imap server "imap.server"

	action "myimapaction" imaps server "imap.server"
		port "8993" user "user" pass "pass" folder "folder"

	action "myimapaction" imaps server "imap.server"
		user "user" pass "pass" no-verify no-login

%%% SMTP

An action may be defined to pass mail on over SMTP. The server host must be
specified and optionally the port and string to pass to the server with the
RCPT TO and MAIL FROM commands. If the port is not specified it defaults to
"smtp". Examples include:

	action "mysmtpaction" smtp server "smtp.server"

	action "mysmtpaction" smtp server "smtp.server" port 587

	action "mysmtpaction" smtp
		server "smtp.server" port "submission" from "bob@server.com"

	action "mysmtpaction" smtp server "smtp.server" to "me@somewhere"

%%% LMTP

An action may be defined to store mail in a LMTP server. UNIX and TCP sockets
are supported. Examples include:

	action "mylmtpaction" lmtp server "/path/to/socket"  to "user@domain"

	action "mylmtpaction" lmtp server "hostname" port 24 to "user@domain"

	action "mylmtpaction" lmtp server "10.1.1.7" port 24 to "user@domain"

%%% Write, pipe, exec and append

Actions may be defined to write or append a mail to a file, to pipe it to a
shell command, or merely to execute a shell command. The append action appends
to and write overwrites the file. % tokens are replaced in the file or command
as for maildir and mbox actions.

Examples are:

	action "mywriteaction" write "/tmp/file"

	action "myappendaction" append "/tmp/file"

	action "mypipeaction" pipe "cat > /dev/null"

	action "domaildirexec" exec "~/.fdm.d/my-special-script %[mail_file]"

Pipe and exec commands are run as the command user (by default the user who
invoked fdm).

%%% stdout

fdm can write mails directly to stdout, using the 'stdout' action:

	action "so" stdout

%%% Rewriting mail

Mail may be altered by passing it to a rewrite action. This is similar to
the pipe action, but the output of the shell command to stdout is reread by fdm
and saved as a new mail. This is useful for such things as passing mail
through a spam filter or removing or altering headers with sed. Note that
rewrite only makes sense on filtering rules where the continue keyword is
specified, or where multiple actions are used (see the next section for details
of this). Possible rewrite action definitions are:

	action "myspamaction" rewrite "bmf -p"

	action "mysedaction" rewrite "sed 's/x/y/'"

%%% Adding or removing headers

Simple actions are provided to add a header to a mail:

	action "lines" add-header "Lines" value "%[lines]"

Or to remove all instances of a header from mail:

	action "del-ua" remove-header "user-agent"

	action "rmhdr" remove-header "x-stupid-header"

	action "remove-headers" remove-headers { "X-*" "Another-Header" }

%%% Tagging

Mails may be assigned one of more tags manually using the tag action type. For
example,

	match account "my*" action tag "myaccts"

	match "^User-Agent:[ \t]*(.*)" action tag "user-agent" value "%1"

The tag is attached to the mail with the specified value, or no value if none
is provided.

%%% Compound actions

Compound actions may be defined which perform multiple single actions.  They
are similar to standard single actions but multiple actions are provided using
{}. For example,

	action "multiple" {
		add-header "X-My-Header" value "Yay!"
		mbox "mbox2"
	}

	action "myaction" users { "bob" "jim" } {
		rewrite "rev"
		maildir "%h/%u's maildir"
	}

Compound action are executed from top-to-bottom, once for each user. Note that
the effects are cumulative: the second example above would deliver a mail
rewritten once to 'bob' and rewritten again (ie, twice) to 'jim'. If this is
not desired, seperate actions must be used.

%%% Chained actions

An action may call other named actions by reusing the 'action' keyword:

	action "abc" action "def"

	action "an_action" {
		rewrite "rev"
		action "another_action"
		action "yet_more_actions"
	}

There is a hard limit of five chained actions in a sequence to prevent infinite
loops.

### Filtering mail

Mail is filtered by defining a set of filtering rules. These rules tie together
mail fetched from an account and passed to one or more actions. Rules are
evaluated from top-to-bottom of the file, and evaluation stops at the first
matching rule (unless the continue keyword is specified).

The general form of a filtering rule is:

	match <conditions> [<users>] <actions> [continue]

The optional <users> item is specified as for an action definition. If users
are specified on a filtering (match) rule, they override any specified on the
action or account.

The <conditions> item is set of conditions against which the match may be
specified, each condition returns true or false. Conditions are described in
the next few sections. Aside from the 'all' condition, which is a special case,
conditions may be chained as an expression using 'and' and 'or', in which case
they are evaluated from left to right at the same precedence, or prepended with
'not' to invert their outcome.

The <actions> item is a list of actions to execute when this rule matches.
It is in the same list format: 'action "name"' or 'actions { "name1" "name2" }'.
It may also be a lambda (inline) action, see the section below.

If a rule with the 'continue' keyword matches, evaluation does not stop after
the actions are executed, instead subsequent rules are matched.

%%% Nesting rules

Filtering rules may be nested by using the special form:

	match <conditions> [<accounts>] {
		match ...
	}

If the conditions on the outer rule match, the inner rules are evaluated. If
none of the inner rules match (or they all specify the 'continue' keyword)
evaluation continues outside to rules following the nested rule, otherwise it
stops.

%%% Lambda actions

Lambda actions are unnamed actions included inline as part of the filtering
rule. This can be convenient for actions which do not need to be used multiple
times. Lambda actions are specified as a combination of the rule and an action
definition. For example:

	match all action maildir "mymaildir"

	match all actions {
		rewrite "rev"
		tag "reversed"
	} continue

%%% The all condition

The all condition matches all mail.

Examples include:

	match all action "default"

	match all rewrite "rewaction" continue

%%% Matched and unmatched

The matched and unmatched conditions are used to match mail that has matched
or has not matched previous rules and been passed on with the 'continue'
keyword. For example,

	match "myregexp" action "act1" continue
	# This rule will match only mails that also matched the first.
	match matched action "act2"
	# This rule will match only mails that matched neither of the first two.
	match unmatched action "act3"

%%% Matching by account

The account condition matches a list of accounts from which the mail was
fetched. It is specified as either a single account ('account "name"') or a list
of accounts ('accounts { "name1" "name2" }'). fnmatch(3) wildcards may also be
used. Examples include:

	match "blah" accounts { "pop3" "imap" } action "go!"

	match matched and account "myacc" action drop

%%% Matching a regexp

Matching against a regexp is the most common form of condition. It takes the
following syntax:

	[case] <regexp> [in headers|in body]

The 'case' keyword instructs fdm to match the regexp case sensitively rather
than the default of case insensitivity. The 'in headers' or 'in body' keywords
make fdm search only the headers or body of each mail, the default is to match
the regexp against the entire mail. Any multiline headers are unwrapped onto
a single line before matching takes place and the process reversed afterwards.
The regexp itself is an extended regexp specified as a simple string, but care
must be taken to escape \s and "s properly.

Examples include:

	match "^From:.*bob@bobland\\.bob" in headers and
	      account "pop3" action "act"

	match ".*YAHOO.*BOOTER.*" in body action "junk"

%%% Matching bracket expressions

The results of any bracket expressions within the last regexp match are
remembered, and may be made use of using the 'string' condition, or used to
construct an action name, maildir or mbox path, etc. The bracket expressions
may be substituted using the %0 to %9 tokens. For example,

	match "^From:.*[ \t]([a-z]*)@domain" in headers action "all" continue
	match string "%1" to "bob.*" action "bobmail"

	match "^From:.*[ \t]([a-z]*)@domain" in headers action "all" continue
	match all action "%1mail"

This is particularly useful in combination with nested rules (see later):
bracket expressions in a regexp on the outer rule may be compared on inner
rules.

Note that %0 to %9 are used only for 'regexp' rules. Regexps that are part of
'command' rules use the 'command0' to 'command9' tags.

%%% Matching by age or size

Mail may be matched based on its age or size. An age condition is specified as
follows:

	age [<|>] <age> [hours|minutes|seconds|days|months|years]

If '<' is used, mail is matched if it is younger than the specified age. If '>',
if it is older. The <age> item may be a simple number of seconds, or suffixed
with a unit. Examples are:

	match age < 3 months actions { "act1" "act2" }

	match age > 100 hours action "tooold"

The age is extracted from the 'Date' header, if possible. To match mails for
which the header was invalid, the following form may be used:

	match age invalid action "baddate"

The size condition is similar:

	size [<|>] <size> [K|KB|kilobytes...]

Where <size> is a simple number in bytes, or suffixed with 'K', 'M' or 'G' to
specify a size in kilobytes, megabytes or gigabytes, such as:

	match size < 1K action "small"

	match size > 2G action "whoa"

%%% Using a shell command

Mail may be matched using the result of a shell command. This condition follows
the form:

	[exec|pipe] <command> returns (<return code>, [case] <stdout regexp>)

If 'exec' is used, the command is executed. If 'pipe', the mail is piped to the
command's stdin. The <command> is a simple string. % tokens are replaced as
normal.

Any of the <return code> or <stdout regexp> or both may be specified. The
<return code> is a simple number which is compared against the return code from
the command, the <stdout regexp> is a regexp that is matched case insensitively
against each line output by the command on stdout. The result of any bracket
expressions in the stdout regexp are saved as 'command0' to 'command9' tags on
the mail.

Any output on stderr is logged by fdm, so 2>&1 must be included in the command
in order to apply the regexp to it.

Examples:

	match exec "true" (0, ) action "act"

	match not pipe "grep Bob" (1, ) action "act"

	match pipe "myprogram" (, "failed") actions { "act1" "act2" }

	match exec "blah" (12, "^Out") action "meep"

%%% Attachments

There are five conditions available to filter based on the size, quantity, type
and name of attachments. They are all prefixed with the 'attachment'
keyword. Two compare the overall number of attachments:

The 'attachment count' conditions matches if the number of attachments is
equal to, not equal to, less than or greater than the specified number:

	match attachment count == 0 action "action"

	match attachment count != 10 action "action"

	match attachment count < 2 action "action"

	match attachment count > 7 action "action"

The 'attachment total-size' condition is similar, but compares the total
size of all the attachments in a mail:

	match attachment total-size < 4096 kilobytes action "action"

	match attachment total-size > 1M action "action"

There are also three conditions which matches if any individual attachment
fulfils the condition: 'any-size' to match if any attachment is less than or
greater than the given size, and 'any-type' and 'any-name' which compare the
attachment MIME type and name attribute (if any) using fnmatch(3):

	match attachment any-size < 2K action "action"

	match attachment any-type "*/pdf" action "action"

	match attachment any-name "*.doc" action "action"

%%% Matching tags

The existence of a tag may be tested for using the 'tagged' condition:

	match tagged "mytag" action "a"

	match tagged "ohno" and size >1K action drop

Or the tags value matched using the 'string' match type (in a similar way to
matching bracket expressions):

	match string "%[group]" to "comp.lang.c" action "clc"

	match string "%u" to "bob" action "bob"

%%% Using caches

fdm has builtin support for maintaining a cache of string keys, including
appending to a cache, checking if a key is present in a cache, and expiring
keys from a cache once they reach a certain age.

These caches should not be confused with the NNTP cache file. Key caches are
referenced by filename and must be declared before use:

	cache "%h/path/to/cache"

	cache "~/.fdm.db" expire 1 month

If the expiry time is not specified, items are never expired from the cache.

Once declared, keys may be added to the cache with the 'add-to-cache' action:

	match all action add-to-cache "~/my-cache" key "%[message_id]"

Or removed with the 'remove-from-cache' action:

	match all action remove-from-cache "~/my-cache" key "%[message_id]"

And the existence of a key in the cache may be tested for using the 'in-cache'
condition:

	match in-cache "~/my-cache" key "%[message_id]" action "foundincache"

Any string may be used as key, but the message-id is most often useful. Note
that the key may not be empty, so care must be taken with messages without
message-id (such as news posts fetched with NNTP).

Caches may be used to elimate duplicate messages using rules similar to those
above:

	$db = "~/.fdm-duplicates.db"
	$key = "%[message_id]"
	cache $db expire 2 weeks

	match not string $key to "" {
		match in-cache $db key $key action maildir "%h/mail/duplicates"
		match all action add-to-cache $db key $key continue
	}

%%% Cache commands

fdm includes a number of commands to manipulate caches from the command-line.
These are invoked with the 'cache' keyword followed by a command. The
following commands are supported:

cache add <path> <string>
cache remove <path> <string>
	These add or remove <string> as a key in the cache <path>.

cache list [<path>]
	This lists the number of keys in a cache, or in all caches declared
	in the configuration file if <path> is omitted.

cache dump <path>
	This dumps the contents of the cache <path> to stdout. Each key is
	printed followed by a space and the timestamp as Unix time.

cache clear <path>
	Delete all keys from a cache.

Examples:

	$ fdm cache list
	/export/home/nicholas/.fdm.d/duplicates: 4206 keys

	$ touch my-cache
	$ fdm cache dump my-cache
	$ fdm cache add my-cache test
	$ fdm cache dump my-cache
	test 1195072403

### Setting options

fdm has a number of options that control its behaviour. These are defined
using the set command:

	set <option> [<value>]

In addition to the options below, some environment variables may be used to
control fdm. If TMPDIR is present, its value will be used instead of /tmp for
saving temporary files.

The possible options are:

- maximum-size <size>

This specifies the maximum size of mail that fdm will accept. The default is 32
MB. Note that fdm may be storing a number of mails simultaneously, up to the
'queue-high' setting (doubled if rewrite is used) for each account, so care
should be taken when increasing this option.

If mail over the maximum size is encountered, fdm will abort with an error,
without deleting the mail from the server (unless 'delete-oversized' is set).

- delete-oversized

If this option is set, fdm will delete oversized mail from the server rather
than leaving it for the user to sort out.

- queue-high <number>

This sets the number of mails fdm will queue simultaneously. The default is
two. Once this limit is reached, fdm will cease fetching mail until the number
queued drops to the value of the 'queue-low' setting.

fdm queues mails so that they may be processed while waiting for data from the
server. Changing this option can increase (or decrease) performance, it's
usefulness varies wildly with the ruleset, speed of connection and remote host,
and the local hardware.

The maximum possible value is currently 50. To ensure fdm fetches and delivers
mail sequentially, set this option to one.

- queue-low <number>

This sets the number of mails fdm will wait for the queue to drop to before
restarting fetching after the 'queue-high' limit has been reached. The default
is three-quarters of the 'queue-high' setting.

- allow-multiple

This option makes fdm ignore the lock file, similar to the '-m' command line
option.

- lock-file <path>

This option specifies a string to use as the path of the lock file. For example:

	set lock-file "/tmp/my-lock-file"

- command-user <user>

This specifies the user used to run exec and pipe actions. By default it is
the user who invoked fdm.

- default-user <user>

This specifies the default user to use if run as root and no users are specified
on the action or filtering rule. This option may be overriden with the '-u'
flag on the command line.

- lock-types [fcntl] [flock] [dotlock]

These specify the type of locking to use when writing to mboxes. fcntl and
flock locking are mutually exclusive, but dotlock may be used with either.
Some NFS servers do not support fcntl. The default is flock only. Example:

	set lock-types fcntl dotlock

- proxy <url>

This specifies a URL to proxy outgoing connections through. See the section
on proxying below.

- unmatched-mail [drop|keep]

This option controls how fdm should deal with mail that reaches the end of
the ruleset (doesn't match any rules, or only rules with the 'continue'
keyword). If 'drop' is specified, such mail is dropped. If 'keep', it is
kept. The default is to keep the mail, and issue a warning.

- purge-after <message count>

The 'purge-after' option instructs fdm to attempt to purge deleted mail after
the specified number of mails has been fetched. This is useful to limit the
number of mails refetched on the next run if the connection fails. It can have
a large effect on performance, particularly if the message count is set to a
low number.

Note that for POP3, purging deleted mail involves disconnecting and
reconnecting from the server; some POP3 servers refuse reconnections if too
many are made too quickly, so this option should be used with care.

- no-received

If this option is present, fdm will not insert a 'Received' header into fetched
mail.

- no-create

When this option is specified, fdm will not attempt to create maildir and
mboxes or directories above them.

- file-umask [user|<umask>]

This specifies the umask to use when creating files. 'user' means to use the
umask set when fdm is started, or it may be specified as a three-digit octal
number. The default is 077.

- file-group [user|<group>]

This option allows the default group ownership of files and directories created
by fdm to be specified. 'group' may be a group name string or a numeric gid. If
'user' is used, or if this option is not set in the configuration file, fdm
does not attempt to set the group of new files and directories.

- timeout <time>

This controls the maximum time to wait for a server to send data before closing
a connection. The default is 900 seconds.

- verify-certificates

This instructs fdm to verify SSL certificates for all SSL connections, see the
previous section on SSL certificate verification.

- ignore-errors

By default, fdm will stop processing mail and exit if any delivery fails; if
this option is set, it will instead continue and try the next mail when
delivery of one fails.

### Archiving and searching mail

As fdm can fetch from maildirs, it can be used to filter mail from one maildir
into another, for example to archive older mail (with drop) or to search for
mail matching a particular pattern (with keep). The 'age' condition, and the
%[maildir] and time/date tags are particularly useful for archiving. For
example, to archive all mail older than 30 days by quarter, something like this
may suffice (the account restriction can be dropped if being used in a
single-purpose configuration file):

	account "archive" disabled maildir "source-maildir"
	match account "archive" and age > 30 days action mbox "%[maildir]q%Q"
	match account "archive" action keep

Then, fdm may be run to move the mail:

	fdm -vaarchive fetch

To search mail, similar rules may be used, but all mail should be kept, in this
example by marking the account with 'keep' so that mail is kept no matter what
rules it matches:

	account "search" disabled maildir "source-maildir" keep
	action "found" mbox "search-results"
	match "^From.*Bob" in headers account "search" action "found"
	match account "search" action keep

All mail matching the regexp will be copied to the target mbox. There are
several other ways to write this ruleset.

### Using fdm behind a proxy

fdm may be used behind a proxy by specifying the proxy URL using the 'proxy'
option. HTTP and SOCKS5 proxies are supported:

	set proxy "http://proxy.server/"

	set proxy "http://proxy.server:port/"

	set proxy "socks5://proxy.server/"

	set proxy "socks5://proxy.server:port/"

	set proxy "socks5://user@proxy.server/"

	set proxy "socks5://user:pass@proxy.server/"

Authentication is not supported for HTTP proxies.

### Bug reports and queries

Bug reports, queries, suggestions and code are best sent by email to:

	nicholas.marriott@gmail.com

Bug reports may also be registered on GitHub.

### Frequently asked questions

%%% Why?

Two main reasons:

1. I didn't like the existing tools. That is not to say that other tools are
   bad or aren't useful, merely that fdm meets my needs better.

And more importantly:

2. I disliked the fact that as a home user with a relatively simple setup I
   had to have five programs to deal with mail: sendmail, fetchmail, procmail,
   archivemail, mutt; all with different, variously broken configuration file
   syntaxes, and weird quirks. I now have three programs: sendmail, fdm and
   mutt, and fewer weird configurations to learn and potential problems. I can
   also do some quite complex things without reaching for additional tools like
   formail and reformail.

%%% How do I write regexps?

See the re_format(7) man page. It is online here (for OpenBSD):

	http://www.openbsd.org/cgi-bin/man.cgi?re_format

There are also a number of books and probably websites on the subject.

%%% Keep doesn't work with GMail!

This is because GMail is broken, see:

	http://pyropus.ca/software/getmail/faq.html#faq-notabug-gmail-bug

In addition, GMail IMAP doesn't set the \Seen flag when mail is fetched with
the UID FETCH BODY[] command (RFC3501 says that the "\Seen flag is implicitly
set"). fdm works around this bug by setting the flag explicitly when mail is
kept.

%%% Why doesn't fdm run as a daemon?

Because that is what cron is for: it comes as standard with all sensible
operating systems, and by using it I avoid a lot of code to deal with signals,
configuration reload, extra configuration options and general other crap
related to running all the time. Daemonising is for servers, for stuff that
needs to run periodically, use cron.

%%% Why does fdm fork child processes when not running as root?

Because one design is much cleaner and easier to work with than two. And it
has a negligible practical effect on performance in any case.

%%% Can fdm get rid of that crap in []s some mailing lists add to subjects?

Yes! Rewrite the mail using sed to remove the crap, for example:

action "full-disclosure" {
	rewrite "sed 's/^\\(Subject:.*\\)\\[Full-disclosure\\] /\\1/'"
	maildir "%h/mail/full-disclosure"
}
match "^List-Id:.*<full-disclosure\\.lists\\.grok\\.org\\.uk>" in headers
	action "full-disclosure"

%%% I'm building on Linux and it complains about REG_STARTEND.

Older versions (pre-2005) of glibc don't support the REG_STARTEND flag to
regexec(3). Either upgrade to a later version of glibc, or use PCRE.

%%% Should I use PCRE or standard POSIX regexps? What's the difference?

PCRE is a library providing "perl-compatible" regexps. These are broadly
compatible with POSIX extended regexps, but with a number of extensions based
on perl regexps.

Unless you want or need some of the extensions, there is generally no
compelling reason to choose one over the other. PCRE is faster than some
POSIX regexp implementations, but few rulesets will include sufficient
regexps for this to make any difference. PCRE has had some security
problems, but most regexp implementations pose a similar risk for their
complexity if nothing else; fdm performs regexp matching as non-root in
any case.

The Debian package and FreeBSD port both use PCRE by default.

Because I hate maintaining and testing two sets of code, there is a strong
possibility that PCRE may become the fdm default at some point.

================================================================================
$Id$