Grepular

Each email you send has a globally unique identifier recorded in the headers of the email under Message-Id. Eg, “Message-ID: <some.code@hostname>”

When replying to a message, most MUA’s look for the Message-Id header in the original email, and then quote it in a new header named In-Reply-To.

So, it occured to me that when sending an email, my MSA (Exim) could record the Message-Id in a database. Then my MX could do lookups against this database using the In-Reply-To headers of incoming email, giving me another whitelist metric. I could then add the Message-Id of the incoming email with the trusted In-Reply-To header to the database, meaning entire threads of conversation become trusted.

This is most useful when your users use mailing lists. For instance, if I send an email to a mailing list, and then somebody replies to it off-list, even though I might never have sent or received email from them before, I know I can trust the message without doing any spam filtering because I know it’s a reply to a trusted message.

Currently, any message ids I add to my database expire after 30 days.

I’m using MySQL as a backend database. Here is a description of the table:

mysql> DESC trusted_message;
+--------------------+------------------+------+-----+---------+----------------+
| Field              | Type             | Null | Key | Default | Extra          |
+--------------------+------------------+------+-----+---------+----------------+
| trusted_message_id | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| message_id_md5     | varchar(32)      | NO   |     | NULL    |                |
| ctime              | datetime         | NO   |     | NULL    |                |
+--------------------+------------------+------+-----+---------+----------------+
3 rows in set (0.02 sec)

I store an MD5 hash of the Message-Id so I know I am dealing with fixed length data. The ctime field contains when the Message-Id was stored, so I can handle expiration.

I created four Macros in Exim (be careful of line wrapping):

SQL_RECORD_TRUSTED_MESSAGE = INSERT INTO trusted_message SET ctime=NOW(), message_id_md5=MD5("${quote_mysql:${sg{$h_Message-Id:}{\N^\s*(\S+).*?$\N}{\$1}}}")

SQL_CHECK_TRUSTED_MESSAGE = SELECT COUNT(*) FROM trusted_message WHERE message_id_md5=MD5("${quote_mysql:${sg{$h_In-Reply-To:}{\N^\s*(\S+).*?$\N}{\$1}}}") AND ctime > DATE_SUB(NOW(),INTERVAL 30 DAY)

RECORD_TRUSTED_MESSAGE = ${lookup mysql{SQL_RECORD_TRUSTED_MESSAGE}{true}{true}}

CHECK_TRUSTED_MESSAGE = ${if and{{!eq{$h_In-Reply-To:}{}}{>{${lookup mysql{SQL_CHECK_TRUSTED_MESSAGE}{$value}{0}}}{0}}}{true}{false}}

The usage of the Exim expansion ${sg} is merely to remove any extraneous whitespace Then in my DATA ACL I do the following before any spam filtering:

accept condition     = CHECK_TRUSTED_MESSAGE
       condition     = RECORD_TRUSTED_MESSAGE
       logwrite      = Response to a trusted message
accept authenticated = *
       condition     = RECORD_TRUSTED_MESSAGE
accept hosts         = +trusted_hosts
       condition     = RECORD_TRUSTED_MESSAGE

I also put this in my NOTSMTP acl to catch email submitted from the command line rather than via SMTP:

accept condition = RECORD_TRUSTED_MESSAGE

Job done.