Clamav space and temporary files about mailu

[kaiyou]

Thanks for reporting, these files should not be stored in the main path, maybe quarantined files could be exported. We should have a closer look and adjust most clamav paths.

Other long-term issue: if the container is not recreated, these files will take space, maybe we should setup a cleanup task or have Clamav clean them.

[kaiyou]

I am just having a close look at this and it seems I cannot reproduce. There is no temporary file in my $PATH/filter directory currently. Do you have error messages in clamav logs, maybe the check fails and temporary files are dumped there.

I will however see that those are not stored in a volume.

[d-fens]

The issue was (I think) on low memory sysstems, freshclam on update would consume a lot of memory. This would crash or be killed:

antivirus_1 | 2016-09-11T17:47:11.375668598Z SelfCheck: Database status OK.
antivirus_1 | 2016-09-11T17:49:36.605641132Z Killed

The result is a .tmp file every time. The solution would be to use a machine with more memory but that isn't really a great option in all cases.

[kaiyou]

How much memory did you use for your tests so that I can reproduce the issue?

[d-fens]

Approximately 450MB in total. However during run time the VPS will only have 30MB free with other containers etc.

The following are the stats for docker for the freeposte containers.

312a3a80efa8        0.00%               848 KiB / 488.5 MiB     0.17%               8.427 MB / 7.779 MB   54.25 MB / 0 B        2
628aa2f5cb7c        0.00%               67.86 MiB / 488.5 MiB   13.89%              2.362 GB / 56.73 MB   3.352 GB / 18.13 MB   4
3cc5be80f4ca        0.00%               1.875 MiB / 488.5 MiB   0.38%               533 kB / 648 B        5.96 GB / 53.49 MB    1
a72d2b5b6e39        0.00%               2.52 MiB / 488.5 MiB    0.52%               10.18 MB / 11.4 MB    8.7 GB / 126.9 MB     5
9c3f947fee48        0.00%               2.691 MiB / 488.5 MiB   0.55%               20.77 MB / 29.77 MB   17.95 GB / 197.5 MB   9
f65bebdeabe8        0.19%               708 KiB / 488.5 MiB     0.14%               4.01 MB / 2.117 MB    51.88 GB / 605.6 MB   3
533ac243ef5f        0.02%               69.92 MiB / 488.5 MiB   14.31%              10.43 MB / 6.548 MB   9.995 GB / 181.9 MB   9

[kaiyou]

Okay, then I believe that the process crashing because it cannot allocate enough memory is quite normal. What one would expect however is the failure to be logged properly and temporary files taking space is an issue as well. I will try and reproduc on a virtual machine with 512MB memory and find a proper solution.

By then, I will add a suggestion to the documentation for users to make sure they have at least 512MB free memory when running Freeposte (and at least 1GB total).

[kaiyou]

There does not seem to be a simple solution to reduce ClamAV memory footprint. The only approache I found was forcing it to swap its unused pages to disk, but unfortunately all pages are equally used everytime a check is performed, thus causing terrible (really terrible) performances.

I did update the documentation to make sure that people run Mailu on a system with at least 1GB of free memory as ClamAV will likely use just under 500M for itself.