Tuesday, October 21, 2014

Anti spam plugin

When hosting e-mail, you'll have to deal with spam unfortunately. We use Amavis with spamassassin and ClamAV antivirus. This setup works great fo 95% of the time but there are always some spam messages that pass the spamfilter. For this you can train spamassassin to recognize the spam better.
I came across a nice dovecot plugin that is very simple to implement. Users can classify messages as spam simply by moving the messages into the spam folder. Taking messages out of the spam folder tells spamassassin that this message was in fact not spam.

Here's a short how to on the installation. I assume you have dovecot+spamasssin running already:

$ apt-get install dovecot-antispam

Add the "antispam" plugin to /etc/dovecot/conf.d/20-imap.conf (your plugins list may be different):

mail_plugins = $mail_plugins quota imap_quota antispam

Add the following to /etc/dovecot/conf.d/90-plugin.conf:

plugin {
  antispam_backend = pipe
  antispam_debug_target = syslog
  antispam_verbose_debug = 1
  antispam_signature= X-Spam-Status
  antispam_signature_missing= move
  antispam_mail_sendmail_args= --username=%u
  antispam_mail_spam  = --spam
  antispam_mail_notspam  = --ham
  antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh
  antispam_spam = SPAM;Spam
  antispam_unsure = Virus
  antispam_trash = Trash;trash
}

Create /usr/bin/sa-learn-pipe.sh:

#!/bin/bash
echo /usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt
echo "$$-start ($*)" >> /tmp/sa-learn-pipe.log

#echo $* > /tmp/sendmail-parms.txt
cat<&0 >> /tmp/sendmail-msg-$$.txt

/usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt

rm -f /tmp/sendmail-msg-$$.txt

#echo "$$-end" >> /tmp/sa-learn-pipe.log

exit 0

Make this script executable
$ chmod +x /usr/bin/sa-learn-pipe.sh

Restart dovecot
$ service dovecot restart

Now move a message into the spam folder and watch /var/log/syslog. It should have something like this:
Oct 21 15:16:25 debian imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0
Oct 21 15:16:25 debian imap: antispam: running mailtrain backend program /usr/bin/sa-learn-pipe.sh
Oct 21 15:16:25 debian imap: antispam: running mailtrain backend program /usr/bin/sa-learn-pipe.sh
Oct 21 15:16:25 debian imap: antispam: running mailtrain backend program parameter 1 --username=admin@intermesh.dev
Oct 21 15:16:25 debian imap: antispam: running mailtrain backend program parameter 2 --spam


After this test you should disable debugging in /etc/dovecot/conf.d/90-plugin.conf

17 comments:

  1. spamassassin does not learn and slows down whole imap.

    ReplyDelete
  2. Do you get any errors in the log?
    The only slowdown I notice is when you move mail into the spam or out of it. But that's obvious because it has to copy the mail to a tmp file and feed it to sa-learn.

    ReplyDelete
  3. When I empty a folder it lasts between 10 seconds (less then 10 mails) and about 1 minute (more then 50 mails) before I can access the system again. It depends on how many mails are in the folder.
    And it doesn't learn.

    tail -f /tmp/sa-learn-pipe.log:

    9858-start (--username=xxxxx --spam)
    10002-start (--username=xxxxx --spam)
    10237-start (--username=xxxxx --spam)


    sa-learn -u xxxxx -D --backup:

    Oct 23 11:18:21.308 [9935] dbg: bayes: found bayes db version 3
    v 3 db_version # this must be the first line!!!
    v 0 num_spam
    v 0 num_nonspam

    ReplyDelete
  4. Sorry, my last comment was not specific at all: I meant, when emptying a spam folder it lasts so long and sometimes I have to restart apache because GO hangs and won't come back.

    ReplyDelete
  5. The antispam plugin should not activate when emptying trash or spam. Make sure your trash folder is listed in: antispam_trash = Trash;trash.

    ReplyDelete
  6. Well, I've dnd your howto above and there is the "antispam_trash = Trash;trash" line correctly. The antispam plugin seems to activate when amptying the spam folder. And the plugin does not leran as you can see above. Any idea?

    ReplyDelete
  7. You could enable the debuggin and watch the log files in /tmp. Comment out the echo lines in the pipe script as well.

    ReplyDelete
  8. I purged and deleted everything and reinstalled it. It seems now, that emptying the spam folder works as it should (I have to wait until there are a reasonable amount of mails in there).
    Whether the plugin now learns or not I have to wait a couple of hours to see. I'll be back then.

    does not result in a logfile sendmail-msg-... in /tmp

    ReplyDelete
  9. This comment has been removed by a blog administrator.

    ReplyDelete
  10. Sorry, but sa-learn does not work ->

    Oct 28 15:14:32.715 [12266] dbg: bayes: found bayes db version 3
    v 3 db_version # this must be the first line!!!
    v 0 num_spam
    v 0 num_nonspam

    ReplyDelete
  11. This comment has been removed by a blog administrator.

    ReplyDelete
  12. 1) commented out "rm -f /tmp/sendmail-msg-$$.txt" - line to see what happens
    2) moved a mail from inbox to spam
    3) message is written in /tmp correctly
    4) but sa-learn does not work - spamassasin does not learn ->
    sa-learn -u xxxxx -D --backup ->
    v 3 db_version # this must be the first line!!!
    v 0 num_spam
    v 0 num_nonspam

    5) did this from cli ->
    /usr/bin/sa-learn --username=xxxxx --spam /tmp/sendmail-msg-27545.txt
    6) looked if it learnd ->
    sa-learn -u xxxxx -D --backup ->
    7) Yes, it works ->
    v 3 db_version # this must be the first line!!!
    v 1 num_spam
    v 0 num_nonspam

    It seems, that your script doesn't pass parameters correctly.
    HTH


    ReplyDelete
  13. I'm sory but I can't see what you've done wrong. If I run that command I get a whole bunch of learned messages in the output!

    ReplyDelete
  14. I did nothing wrong with this plugin. I copied and pasted your code and did everything exactly as described.
    Maybe there is something wrong with my dovecot / postfix / spamassassin installation. I will discover and keep you informed.
    Thanks so far.

    ReplyDelete
  15. Finally I think I sorted this out:

    I changed bayes_path in /etc/spamassassin/local.cf:

    from -> bayes_path /home/vmail/
    to -> bayes_path /home/vmail/.spamassassin/bayes

    Now it seems to work:

    sa-learn -u p.stirnberg@stirnberg.net -D --backup ->
    v 3 db_version # this must be the first line!!!
    v 79 num_spam
    v 4087 num_nonspam

    I suppose, this is then a global bayes file. Is that what you intended or was there a MySQL based personal bayes file solution you had in mind?

    ReplyDelete
  16. cartoon HD For iPad Download is a movies app, so you can download and watch HD movies for free. Not only movies, and also watch latest TV shows,

    ReplyDelete
  17. We provide the best service for your Kaspersky Anti-virus Software. For any issue with your Kaspersky Anti Virus, contact our Kaspersky Antivirus Support to get a quick fix for your issue. Our Kaspersky Antivirus Customer Service is open 24x7.

    ReplyDelete