[Avcheck] RFC: Avcheck future

[Michael Tokarev]

Piotr Klaban wrote:
> On Mon, May 24, 2004 at 10:09:31PM +0400, Michael Tokarev wrote:
> 
>> a socket).  The filter may reply with "Ok", in which case the
>> while message is passed to backend smtpd, with "Reject", in
>> which case the content will be rejected, obviously, or with
>> "stolen", in which case the daemon filter will took the
>> responsibility of the delivery.
> 
> I have thought of the similar proxy filter, but made in C.
> Running a proxy filter in Perl would make my server
> unresposible, because of memory overload. With Perl6+parrot
> there would be much less problems with CPU and RAM, but Perl6
> would not be available for a long time yet.

Well... it really depends ;)

Current model I'm using is very light.  Resources needed for
a single proxy process are small enouth, having in mind the
fact that my proxy *forks* from a single process, not being
executed (and read all the perl modules stuff etc) each time.
Sure, avcheck is lighter even.

What avcheck can NOT do, not without a proxy anyway, is to
perform realtime virus checking.  Having in mind amount of
viruses with fake sender information nowadays, I think it
is essential for a mail server to be able to reject malware
right at the smtp level (or sometimes discard it), to stop
worrying about bounces.  In order to be able to perform
"realtime" virusscanning, I may think of - again - a proxy
that writes temp file and when spawns the same avcheck to
do the work... but in this scenario, there's no need in
avcheck at all, because the "missing" functionality in
proxy in this case - consulting the virusscanning daemon -
is very small and easy doable in perl (or whatever lang.
the proxy is written in).

Also, perl code is much easier to extend/modify than C.

> That is why I like avcheck:
> - it is secure
> - it is written in C -> has small CPU and memory footprint
> - it is fast
> 
> That is why I do not like spamassassin (although I run it ...):
> - it is written in Perl: module dependecies (if I reinstall Perl,
>   it is hard to maintain), RAM wasting

Well... You're trying to compare apples and oranges, sort of.
Avcheck does very few things (save input in temp place, consult
av daemon and resubmit input via smtp - this is very simple
stuff).  Spamassassin does quite some processing of input,
decoding, matching, doing it descision - this is complex stuff,
and it should be extendable.  It is very difficult to write it
in C and keep it as useful.  It's interdependancies are here
too just because it's damn large ;) - again, because it does
alot of stuff (and my proxy does not depend on alot of modules,
in fact it does not depend on non-core functionality, and has
no C language components which should be rebuilt on perl update).

> I think avproxy should:
> - be independent from Perl
> - have many AV engines support
> - maybe support spam filtering (there is something called spamd, right?)
> - be written :)

The independance of perl - it seems - is something very difficult
to achieve these days - because of the powers this language gives
us.  The proxy may be written in C just easy, to collect a message
into temp file speaking smtp and execute some perl hooks built into
the proxy itself ("Calling Perl from C" scenario in perlxs) - so that
both startup costs and memory usage will be minimal.  This way, it
should meet all the requiriments.  In my original post, I mentioned
some concurrency controls - the "light" part of a proxy (the one
written in C) may control how many processes are doing the "hard"
work (virus and spam controls) in parallel, and queue the message
without any checking in case of too high load, to be checked at a
later time (using content_filter mechanism as avcheck does).

> Please correct me if I am wrong, but postfix checks RBL/header_checks
> after SMTP proxy filtering, right? I refuse hundrets mails per couple of hours
> with postfix filtering ...

No you're wrong.  Well.. it may be both ways around, you may make
either frontend or backend smtpd to be transparent.  For stuff like
this, usually it is frontend smtpd who does the spam filtering
(smtpd_mumble_restrictions), and it only connects to the proxy
after first *successeful* RCPT TO (i.e. it may reject many RCPTs
before, or not connect to a proxy at all).  Indeed, header checks
and the like are done by cleanup which is behind backend smtpd:

  frontend smtpd => proxy => backend smtpd => cleanup => queue

But with smtp proxy which have ability to perform header_checks
by it's own...  Much more advanced header checks...  I think you
got the idea ;)


/mjt