Scheduled antivirus scans to prevent viral injections on user generated content

When dealing with high traffic sites, especially media based or community based sites, there is always the risk of javascript, virus, XSS or other malicious injection of badness when giving a community of users the ability to upload files to your site.

There are several things to consider when evaluating all “points of entry” that are available to the public, into your systems.

Most content management and community based systems use libraries such as Imagemagick to process images (such as profile pictures) into their proper format and size.

Believe it or not, it is hard to actually inject code or other malicious data into the actual image to survive this sanitizing process. There is still risks , however. The library version you are running may be vulnerable to exploits itself.

As always, a good rule of thumb is to ensure all possible aspects of your systems are up to date and that you are aware of any security vulnerabilities as they come out so they can either be patched or addressed in some other way.

One thing to consider, especially when dealing with thousands of users and even more uploads is a scheduled scan of your user uploads using free virus scanning tools such as clamscan. This is an endpoint reporting strategy that can at least cover your ass in the event that something else was missed or a 0day vulnerability exploited.

It should be noted that the virus scans themselves aren’t intended to protect the linux systems themselves, but rather the opportunistic ‘spreading’ of compromised images and code that having an infected file on a public community based system can provide.

Its very simple to implement clamav (daemonization is not necessary), clamscan is all we need to execute regular scans at 10, 15, 30 or 60 minute intervals.

Once clamscan is implemented, definitions updated (and regular update cronjobs in place) you can roll out a script similar to the one we have here to implement the scheduled scans :

#!/bin/bash
# Scheduled Scan of user uploaded files
# Usage : ./virusscan.sh /folder


SUBJECT="[VIRUS DETECTED] ON `hostname` !"
EMAIL="you@yourdomain.com"
LOG=/var/log/clamav/scan.log

# Clear out old logs -- the email alerts should be archived if we need to go back to old alerts
echo "" > $LOG

# Check if the folder is empty -- only scan if this is an active node in a clustered system
# look for empty dir
if [ "$(ls -A $1)" ]
then
        # Scan files 
        clamscan $1 -r --infected --scan-pdf --scan-elf --log=$LOG

        # Check the last set of results. If there are any "Infected" counts that aren't zero, we have a problem.
        cat $LOG | grep Infected | grep -v 0
        if [ $? = 0 ]
        then
                cat $LOG | mail -s "$SUBJECT" $EMAIL -- -F Antivirus -f antivirus@yourdomain.com
        fi

else
        echo "directory empty -- doing nothing"
        exit 0;
fi

The actual cronjob entry can look something like this :

0 */1 * * * /bin/bash /usr/local/bin/virusscan.sh "/your/path/to/user/uploaded/files/" > /dev/null 2>&1

It seems like a rather simple solution — but it does provide a venue for additional sanitizing of user input. In our experience , it is best to only report on anything that clamscan might find. You can, however tell clamscan to simply delete any suspected infections it finds.

Setup Up Exim with ClamAV and Spamassassin

I decided to post this article on implementing a simple single mail server with anti-spam and anti-virus capabilities.

This guide hopefully will help you on your way to configuring a basic mail system on Linux (specifically Debian).

Installing and configuring Exim 4 on Debian

1. First, install all the necessary Debian packages are on the system as the root user. (The exim4 package will REPLACE the exim package.)

NOTE: If you are using the stable branch, it is suggested to use the debian volatile packages (along with the security packages) so that your system is using the most up-to-date critical packages (like ClamAV) for security purposes. For production servers, you may not want to run a mixed stable/testing/unstable system (though I know some of you do!). To use these packages, see http://volatile.debian.net/ for more information. For those of you who are impatient and don’t want to find the correct mirror, here’s is what I added to my /etc/apt/sources.list file:

deb http://volatile.debian.net/debian-volatile sarge/volatile main contrib

I used aptitude to install these packages, but you could also use the old apt-get method:

apt-get install clamav-daemon 
clamav-freshclam exim4-daemon-heavy exim4 
courier-base courier-authdaemon courier-imap 
courier-pop spamassassin wget spamc sa-exim

When going through the exim4 config, be sure to select the multiple file configuration layout. If you didn’t (or weren’t prompted for it), simply set dc_use_split_config to true in the /etc/exim4/update-exim.conf.conf file. (Thanks Mike!)

2. Create your Maildir directory

maildirmake ~/Maildir/

3. Now we want to make exim4 use Maildir format mailboxes. Modify the file /etc/exim4/update-exim4.conf.conf so that it contains:

dc_localdelivery='maildir_home'

4. We need to Edit /etc/default/spamassassin to enable spamd.

5. Each user can set up their own filters by creating a .forward file in their home directory. If the first line of this file reads

# Exim filter then Exim4 will treat it as a filter.

Here is an example of an Exim filter that checks the headers that SpamAssassin adds and puts the mail in the appropriate Maildir folder:

      # Exim filter
      if $h_X-Spam-Status: CONTAINS "Yes"
           or
        $h_X-Spam-Flag: CONTAINS "Yes"
      then
        save $home/Maildir/.Spam/
        finish
      endif

Exim’s Interface To Mail Filtering (PDF format) – Local copy

6. Many system administrators like to set up the Maildir directories and .forward filter file in the /etc/skel directory so that when they make a new user on the system, everything is automatically copied over. I suggest that you do this as well as it makes things easier.

7. Before going live with the mail server, we will want to test it!

Testing the implementation

1. Generate the new configuration:

update-exim4.conf

If you made it through this, then your config files don’t have any syntax errors.

exim4 -bV

If that works, then there are no config issues

2. Next, start exim by issuing:

/etc/init.d/exim4 start

Above assumes that you are running exim4 as a daemon, and not through inetd

3. Now, check a local address:

            exim4 -bt local_user@example.com

4. Check sending an email:

            exim4 -v mailbox_you_can_check@dom.ain
               From: user@your.domain
               To: mailbox_you_can_check@dom.ain
               Subject: Testing exim
                     
               Testing exim
               .

You should now see some messages to let you know that the email was sent or information about what went wrong.

5. To test with full debug output using a specific config file, use something like:

            exim4 -C /etc/exim/exim_example.conf -d -bt user@example.com

6. To test the config coming from a specified ip address, use:

            exim4 -bh 192.168.1.10

            HELO example.com
               MAIL FROM: 
               RCPT TO: 
               DATA
               Subject: something
               your message here
               .
               QUIT

8. Add the following to your /etc/exim4/conf.d/main/01_exim4-config_listmacrosdefs file:

      # This tells what virus scanner to use
      av_scanner = clamd:/var/run/clamav/clamd.ctl

9. Edit /etc/exim4/conf.d/acl/40_exim4-config_check_data to inlude the following before the “# accept otherwise” line:

      # Reject messages that have serious MIME errors.
         # This calls the demime condition again, but it
         # will return cached results.
         deny message = Serious MIME defect detected ($demime_reason)
         demime = *
         condition = ${if >{$demime_errorlevel}{2}{1}{0}}
         
         # Reject file extensions used by worms.
         # Note that the extension list may be incomplete.
         deny message = This domain has a policy of not accepting certain types of attachments 
                        in mail as they may contain a virus.  This mail has a file with a .$found_extension 
                        attachment and is not accepted.  If you have a legitimate need to send 
                        this particular attachment, send it in a compressed archive, and it will 
                        then be forwarded to the recipient.
         demime = exe:com:vbs:bat:pif:scr
         
         # Reject messages containing malware.
         deny message = This message contains a virus ($malware_name) and has been rejected
         malware = *

10. Then, you need to enable ClamAV.

a) Firstly, you will want to be sure that it is running against messages. In /etc/exim4/sa-exim.conf, search for SAEximRunCond:

SAEximRunCond: ${if and {{def:sender_host_address} {!eq {$sender_host_address}{127.0.0.1}} {!eq {$h_X-SA-Do-Not-Run:}{Yes}} } {1}{0}}

That is simply skipping the scan on anything from the local machine or if the X-SA-Do-Not-Run header in the message is set to Yes. If you just want exim to run ClamAV on all messages, use this:

SAEximRunCond: 1

b) Before restarting ClamAV, we need to be sure that all of the access rights are in place so that the scans actually happen. The best way to handle this is to add the clamav user to the Debian-exim group. Either manually edit /etc/group, or simple run:

adduser clamav Debian-exim

c) Be sure that /etc/clamav/clamd.conf contains a line that reads:

AllowSupplementaryGroups

d) Set the file permissions for the /var/run/clamav directory to allow for the correct user to use it:

            chown Debian-exim.Debian-exim /var/run/clamav
            chmod g+w /var/run/clamav

e) A restart of ClamAV is necessary for the changes to take effect:

/etc/init.d/clamav-daemon restart

11. You should now be able to get your mail via IMAP with a mail client like Mozilla.

Check your headers (View Source) and see that SpamAssassin has added its headers. SMTP-end virus scanning should also be taking place. Check your /var/log/clamav/clamav.log to monitor this.

Multiple Domain Alias Files

The steps below are used to enable support for having multiple virtual domains each with its own alias file.

1. Exim will need to have the alias files for each domain.

a) Create the /etc/exim4/virtual directory.
b) For each virtual domain, create a file that contains the aliases to be used named as the domain.

For example, if I example.com was one of my domains, I’d do the following:

a) Create the /etc/exim4/virtual/example.com file.
b) If my system users were sys1, sys2, and sys3, and their email addresses were to be joe, john, jason, I’d put the following into the domain alias file:

                  joe:    sys1@localhost
                  john:   sys2@localhost
                  jason:  sys3@localhost

If john was also to get all mail addressed to info@example.com, you would add this entry:

info:   sys2@localhost

If you wanted all mail to user1@example.com to go to another email account outside of this domain, you would enter:

user1:  a.user@some.domain

If you wanted all mail directed at any address other than what is defined in the alias file to go to joe, you’d enter:

*:      sys1@localhost

In the above examples, the “@localhost” suffix to the user names forces the delivery to a system user. I found that if you do not include this in the alias files and your machine’s host name is within one of the domains handled by exim, every system user would need an entry in the machine’s domain in order to be delivered corectly.

For instance, if your host name was mail.example1.com and example1.com was handled by this server this would be needed. This would allow delivery to all the system user names at example1.com.

The reason is simple, and I will try to illustrate it for you here:

a) exim receives a message delivered to joe.blow@example3.com
b) The alias file for this domain has joe.blow: jblow in it.
c) This would translate to jblow@domain-of-the-system
d) The process would be repeated using jblow@domain-of-the-system
e) If there was no entry in the domain-of-the-system alias file for jblow, the message would be undeliverable (or non-routable)

You could even have special redirects like the following:

script: "| /path/to/some/script"
prev:   :fail: $local_part left!
kill:   :blackhole:

2. Edit /etc/exim4/conf.d/main/01_exim4-config_listmacrosdefs by replacing the current local_domains line with:

domainlist local_domains = @:localhost:dsearch;/etc/exim4/virtual

3. Create /etc/exim4/conf.d/router/350_exim4-config_vdom_aliases with the following content:

            vdom_aliases:
            driver = redirect
            allow_defer
            allow_fail
            domains = dsearch;/etc/exim4/virtual
            data = ${expand:${lookup{$local_part}lsearch*@{/etc/exim4/virtual/$domain}}}
            retry_use_local_part
            pipe_transport   = address_pipe
            file_transport   = address_file

4. Now, regenerate your exim4 config:

update-exim4.conf

5. If there were no errors, restart exim4:

/etc/init.d/exim4 restart

Domain Dependent Maximum Message Size

The next step for my server is to give each domain a configurable message size limit. Then, when the server get’s a message that is larger than the target domain’s size limit, I want to send a message back to the original sender telling them why the message was not delivered. However, I also want to have that message customized for each domain. That way, the domain owners can provide detailed instructions on how to send large messages to their domain if it is necessary. Of course, there will also need to be some kind of default size limit and message for domains that do not need the customization.

1. Create /etc/exim4/domain-size-limits to contain the list of domains and their maximum message size limits. You can also add a wildcard at the end entry if you want to set a default limit. The file may look something like the following:

      example.com: 20M
      example1.com: 5M
      *: 15M

This provides you a quick way to edit the values. The values will also take effect as soon as the file is saved – no need to restart exim!

2. OK, now we know what domains we want to customize the size for. Now it’s time to create a message to send for those domains. Create /etc/exim4/domain-size-limit-messages with content similar to:

      exmaple.com: The largest acceptable message size for Example.com is
                   ${expand:${lookup{$domain}lsearch*@{/etc/exim4/domain-size-limits}}}.
                   Your message was $message_size. If you feel that $local_part@$domain
                   should really get your message, then visit http://www.example.com/files/
                   where you can upload any large files. If you select $local_part@$domain
                   from the "notify" list, they will receive a message with a link directly
                   to your file.
      *:           The largest acceptable message size for $domain is
                   ${expand:${lookup{$domain}lsearch*@{/etc/exim4/domain-size-limits}}}.
                   Your message size was $message_size. Please revise your message so it
                   does not exceed this maximum file size and resend. If this is not
                   possible, contact the recipient in another way.

As you see, we have one domain that has a custom message sent out, and have defined a default message for all other domains. These messages can be edited at any time and do not need an exim restart to take effect.

3. Now for the fun part! We need a way to catch the messages that are too large for the domain! First, create /etc/exim4/conf.d/router/325_exim4-config_large_messages with the following:

      large_messages:
          driver = accept
          domains = dsearch;/etc/exim4/virtual
          condition = ${if >{$message_size}{${expand:${lookup{$domain}lsearch*@{/etc/exim4/domain-size-limits}}}} {yes}{no}}
          transport = bounce_large_messages
          no_verify

This router dynamically checks which domains are available and what their limits are set to.

4. Now create /etc/exim4/conf.d/transport/40_exim4-config_bounce_large_messages with the following content:

      # This bounces a message to people who send files too large for that domain
      bounce_large_messages:
        driver = autoreply
        from = $local_part@$domain
        to = $sender_address
        subject = Re: ${escape:$h_subject:}
        text = ${expand:${lookup{$domain}lsearch*@{/etc/exim4/domain-size-limit-messages}}}

This transport then sends the original sender a message using the text looked up from the domain-size-limit-messages file for that domain. The From: field is filled in with the intended recipient of the message – appearing to be a reply.

This was actually very simple to put together once I realized what I needed to do. The above is based on what I found in the Exim FAQ

Configuration Tips

Maybe this is something I should have said in the beginning, but at the time or writing this document, I had never set up an exim4 server, and the only exim3 server I had was used with the default debconf install. Therefore, if you see something on this page that could be done in a more elegant, more efficient or just plain better way, please send me a note.

Menu