Coder Social home page Coder Social logo

the-mcgrail-foundation / mimedefang Goto Github PK

View Code? Open in Web Editor NEW
27.0 11.0 9.0 11.24 MB

MIMEDefang is an e-mail filtering tool that works with the Sendmail “Milter” library. MIMEDefang lets you express your filtering policies in Perl rather than C, making it quick and easy to filter or manipulate your mail.

License: GNU General Public License v2.0

Makefile 1.49% Perl 41.58% PHP 0.22% Shell 2.26% C 31.32% Roff 13.53% Tcl 6.32% M4 3.28%
filter email antispam perl security smtp

mimedefang's Introduction

MIMEDefang

CI for MIMEDefang GitHub license GitHub release

  1. INTRODUCTION

MIMEDefang is an e-mail filter program which works with Sendmail 8.12 and later or Postfix. MIMEDefang filters all e-mail messages sent via SMTP. MIMEDefang splits multi-part MIME messages into their components and potentially deletes or modifies the various parts. It then reassembles the parts back into an e-mail message and sends it on its way.

MIMEDefang is written (mostly) in Perl, and the filter actions are expressed in Perl. This makes MIMEDefang highly flexible and configurable. As a simple example, you can delete all *.exe and *.com files, convert all Word documents to HTML, and allow other attachments through.

MIMEDefang uses the "milter" feature of Sendmail to "listen in" to SMTP connections. It runs a scan once for each message, not once for each recipient (as simpler procmail-based systems do.) Therefore, it is more CPU-friendly than procmail-based systems. In addition, because MIMEDefang can participate in the SMTP connection, you can bounce messages (something impossible to do with procmail-based systems.)

  1. WARNINGS

There are some caveats you should be aware of before using MIMEDefang. MIMEDefang potentially alters e-mail messages. This breaks a "gentleman's agreement" that mail transfer agents do not modify message bodies. This could cause problems, for example, with encrypted or signed messages.

Deleting attachments could cause a loss of information. Recipients must be aware of this possibility, and must be willing to explain to senders exactly why they cannot mail certain types of files. You must have the willingness of your e-mail users to commit to security, or they will complain loudly about MIMEDefang.

  1. PREREQUISITES

MIMEDefang has the following software requirements:

  1. A UNIX-like operating system (MIMEDefang is developed and tested on Linux)

  2. Perl 5.8.0 or higher

  3. Required Perl modules:

    Digest::SHA FindBin MailTools 1.1401 or higher MIME::tools 5.413 or higher MIME::Base64 3.03 or higher MIME::WordDecoder

    These modules are available from http://www.cpan.org

  4. Optional Perl modules:

    Crypt::OpenSSL::Random - Needed to generate a truly random ipheader file HTML::Parser (CPAN) - Needed for append_html_boilerplate function IO::Socket::SSL - Needed for md_check_against_smtp_server SSL checks JSON and LWP::UserAgent - Needed for rspamd support Mail::SpamAssassin (https://www.spamassassin.org/) - spam detector Test::Class, Test::Most and tzdata files - Needed to run regression tests

  5. Sendmail 8.12.x, 8.13.x or Postfix. Get the latest version.

  1. INSTALLATION

There's an excellent MIMEDefang-HOWTO contributed by Mickey Hill at http://www.mickeyhill.com/mimedefang-howto/. It explains everything in this README in much greater detail. Anyway, on with it:

  1. Sendmail

You must be using Sendmail 8.12.x or higher

Obtain the latest Sendmail 8.12.x or higher source release from http://www.sendmail.org. Unpack it. If you are building 8.12.x, add the following lines to devtools/Site/site.config.m4:

    dnl Milter
APPENDDEF(`conf_sendmail_ENVDEF', `-DMILTER')

This enables the mail filter feature. (For 8.13.x and higher versions, Milter is enabled by default.)

Go ahead and build Sendmail following the instructions in the Sendmail documentation. Install and configure Sendmail.

You MUST run a client-queue runner, because MIMEDefang now uses deferred mode to deliver internally-generated messages. We recommend running this command as part of the Sendmail startup:

sendmail -Ac -q5m

Compile and Install Sendmail:

Next, you need to make the Sendmail headers and libraries visible for compiling and linking MIMEDefang. The most reliable way to do this is to run these commands from the main Sendmail directory:

mkdir -p /usr/local/include/sendmail
cp -R include/* /usr/local/include/sendmail
cp -R sendmail/*.h /usr/local/include/sendmail
mkdir -p /usr/local/lib
cp obj.Linux.2.2.14-5.0.i686/*/*.a /usr/local/lib

NOTE: On the last "cp" command, replace "obj.Linux.2.2.14-5.0.i686" with the appropriate "obj.*" directory created by the Sendmail build script.

  1. Obtain and install the necessary Perl modules. These generally build and install as follows:

    perl Makefile.PL make install

If you are using any of the optional Perl modules, install them before starting to build MIMEDefang.

  1. Optionally, obtain and install the "wv" library. Install the wvHtml program in your favourite bin directory (/usr/bin or /usr/local/bin).

  2. Configure, build and install the MIMEDefang software:

    ./configure make make install

NOTE: Unlike most autoconf scripts, the default --sysconfdir for this version of ./configure is "/etc". You can change it to /usr/local/etc as follows:

./configure --sysconfdir=/usr/local/etc

Also, the actual configuration files go in the subdirectory "mail" under --sysconfdir. You can put them elsewhere (eg, /usr/local/etc/mimedefang) like this:

./configure --sysconfdir=/usr/local/etc --with-confsubdir=mimedefang

If you want them right in /usr/local/etc, you'd say:

./configure --sysconfdir=/usr/local/etc --with-confsubdir=

By default, MIMEDefang processes incoming messages in the directory /var/spool/MIMEDefang. You can change this by typing:

./configure --with-spooldir=DIRNAME

By default, MIMEDefang quarantines mail in the directory /var/spool/MD-Quarantine. You can change this by typing:

./configure --with-quarantinedir=DIR2

You should create the spool and quarantine directories with mode 700, owned by the user you run MIMEDefang as.

Summary of useful ./configure options:

--with-sendmail=PATH specify location of Sendmail binary --with-user=LOGIN use LOGIN as the MIMEDefang user --with-milterinc=PATH specify alternative location of milter includes --with-milterlib=PATH specify alternative location of milter libraries

--with-ipheader install /etc/mail/mimedefang-ip-key --with-confsubdir=DIR specify configuration subdirectory (mail) --with-spooldir=DIR specify location of spool directory (/var/spool/MIMEDefang) --with-quarantinedir=DIR specify location of quarantine directory (/var/spool/MD-Quarantine)

--enable-poll Use poll(2) instead of select(2) in multiplexor

--disable-check-perl-modules Disable compile-time checks for Perl modules --disable-embedded-perl Disable embedded Perl interpreter

--enable-debugging Add debugging messages to syslog

--disable-anti-virus Do not search for ANY anti-virus programs

--disable-antivir Do not include support for H+BEDV antivir --disable-vexira Do not include support for Central Command Vexira --disable-uvscan Do not include support for NAI uvscan --disable-sweep Do not include support for Sophos sweep --disable-trend Do not include support for Trend Filescanner/Interscan --disable-AvpLinux Do not include support for AVP AvpLinux --disable-clamav Do not include support for clamav --disable-csav Do not include support for Command Anti-Virus --disable-fsav Do not include support for F-Secure Anti-Virus --disable-fprot Do not include support for F-prot Anti-Virus --disable-fpscan Do not include support for F-prot Anti-Virus v6 --disable-sophie Do not include support for Sophie --disable-nvcc Do not include support for Nvcc

  1. Add the following line to your Sendmail "m4" configuration file. (You DO use the m4 configuration method, right?)

INPUT_MAIL_FILTER(mimedefang', S=unix:/var/spool/MIMEDefang/mimedefang.sock, F=T, T=S:360s;R:360s;E:15m')

(If you keep your spool directory elsewhere, use its location instead of /var/spool/MIMEDefang/mimedefang.sock)

The "T=..." equate increases the default timeouts for milter, which are way too small.

  1. Ensure that mimedefang starts when Sendmail does. In whatever shell script starts sendmail at boot time, add the lines:

    rm -f /var/spool/MIMEDefang/mimedefang.sock /usr/local/bin/mimedefang -p /var/spool/MIMEDefang/mimedefang.sock &

before the line which actually starts Sendmail. When you shut down Sendmail, remember to kill the mimedefang processes. A sample /etc/rc.d/init.d script for Red Hat Linux is in the redhat directory. A sample generic init script which should work on most UNIXes is in the examples directory.

DOCKER

There are docker images available on Docker Hub at https://hub.docker.com/u/mimedefang with MIMEDefang configured with postfix(8) or sendmail(8) MTA.

CONFIGURATION

To configure your filter, you have to edit the file `/etc/mail/mimedefang-filter'. This is a Perl source file, so you have to know Perl. Go ahead and read the man pages mimedefang(8), mimedefang.pl(8) and mimedefang-filter(5). There are some sample filters in the examples directory.

THE MULTIPLEXOR

On a busy mail server, it is too expensive to start a new Perl process for each incoming e-mail. MIMEDefang includes a multiplexor which manages a pool of long-lived Perl processes and reuses them for successive e-mails. Read the mimedefang-multiplexor(8) man page for details. A sample start/stop script is shown in examples/init-script; this script is generic and should work on most flavours of UNIX.

mimedefang's People

Contributors

bigio avatar cstrotm avatar dfskoll avatar grumpybozo avatar pprindeville avatar robert-scheck avatar zbourdeau avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mimedefang's Issues

Problem md_check_against_smtp_server() after update to version 2.85 (CentOS 7)

Hi,

I use MIMEDefang on two server and the function md_check_against_smtp_server() (version mimedefang-2.84-1.el7, perl-IO-Socket-SSL-1.94-7.el7)

If I update one server to version mimedefang-2.85-1.el7 the function return always tempfailed result and the email stay in SMTP queue

mimedefang.pl[1877]: 1B3LSO9x010710: filter_recipient tempfailed recipient [email protected]
Dec 3 21:55:57 mx sendmail[5114]: 1B3KtvRh005114: Milter: to=[email protected], reject=451 4.3.0 Could not connect to other SMTP server smtpserver.xxxx.xxxx

The SMTP destination server has Sendmail 8.14.7 and I see this errors in sendmail with debug level 15

Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: --- 220 smtpserver.xxxx.xxxx ESMTP Fri, 3 Dec 2021 21:55:57 +0100
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: <-- \026\003\001\002
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: Sending "\026\003\001\002" to Milter
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: --- 500 5.5.1 Command unrecognized: "\026\003\001\002"
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: <--
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: Sending "" to Milter
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: --- 500 5.5.1 Command unrecognized: ""
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: <--
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: Sending "" to Milter
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: --- 500 5.5.1 Command unrecognized: ""
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: <--
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: Sending "" to Milter
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: --- 500 5.5.1 Command unrecognized: ""
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: <--
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: Sending "" to Milter
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: --- 500 5.5.1 Command unrecognized: ""
Dec 3 21:55:57 mail sendmail[27552]: 1B3Ktvk8027552: --- 421 4.4.1 smtpserver.xxxx.xxxx Lost input channel from [xxx.xxx.xxx.xxx]

If I copy the file /usr/bin/mimedefang.pl from 2.84 version to the server with version 2.85 the problem disappears and the email is directly send.

It's a TLS problem/incompatibility with the new version of mimedefang.pl file ?

Thanks

Codespell report for "MIMEDefang" (on fossies.org)

A little bit nitpicking:

The FOSS server fossies.org - supporting also the MIMEDefang project - offers among others a feature named "Source code misspelling reports". Such reports are normally only generated on request, but as Fossies administrator I have just created for testing purposes such a codespell based analysis for the new and long-awaited "MIMEDefang" release 2.8.5:

 https://fossies.org/linux/misc/mimedefang/codespell.html

That version-independent (not linked) URL should redirect always to the last report (if available), so currently to

 https://fossies.org/linux/misc/mimedefang-2.85.tar.gz/codespell.html

By the way, the context type of the spelling errors and typos are marked by a single character (within the sortable column "T"): The less interesting, probably not user visible ones (in "comments") are greyed, while the probably more interesting ones are black-colored. If JavaScript is enabled all table columns are sortable.

Although after a first review some obviously wrong matches ("False Positives" = FPs) are already filtered out (ignored) please inform me if you find more of them so that I can force a new improved check if applicable.

Just for information there are also three supplemental pages showing some used "codespell" configuration details, all obvious false positives and an misspelling history (log).

Perhaps more meaningful may be a similar and continuously updated report for the "MIMEDefang GitHub "master" version which could be generated on request within a special restricted "test" folder that isn't really integrated into the standard Fossies services and should also not be accessible to search engines:

 https://fossies.org/linux/test/mimedefang-master.tar.gz/codespell.html

That version independent URL hopefully always redirects to the report for the latest "master" version identified by the short GitHub commit ID and a year-month-day string (YYMMDD) representing the according git pull date (mostly = commit date). The refresh rate is currently half-hourly.

Although the correction of misspellings and typos has probably not a top priority, I hope that the report can nevertheless be a little bit useful.

Off-topic small remarks: In the new release the README file was replaced by a README.md file. Since the textual README file was already a little bit formatted like a Markdown file the according HTML rendering works reasonably but not very well (especially since lines with "--" are handled as an alternative syntax for heading level 2; see e.g. "You must be using Sendmail 8.12.x or 8.13.x"). Additionally, since the item "4. MIME-BASE64 NOTE" was removed the line "5. INSTALLATION" must be now probably numbered "4. INSTALLATION". And in the list before the item "4) Sendmail 8.12.x, 8.13.x or Postfix. Get the latest version." has the same number as the item before.

Test header_timezone_works fails

I am trying to compile and run tests on alpine Linux edge branch with 2.85 release candidate, and get the following error when make test is run. Is it expecting system timezone to be America/Regina?

prove -It/lib --recurse t/unit
t/unit/dates.t .... 1/5
#   Failed test 'Got header timezone for TZ=America/Regina'
#   at t/unit/dates.t line 18.
#   (in MIMEDefang::Unit::dates->header_timezone_works)
#          got: '+0000'
#     expected: '-0600'
# Looks like you failed 1 test of 5.
t/unit/dates.t .... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/5 subtests
t/unit/headers.t .. ok
t/unit/relay.t .... ok
t/unit/sender.t ... ok

Test Summary Report
-------------------
t/unit/dates.t  (Wstat: 256 Tests: 5 Failed: 1)
  Failed test:  3
  Non-zero exit status: 1
Files=4, Tests=51,  4 wallclock secs ( 0.13 usr  0.03 sys +  2.88 cusr  0.69 csys =  3.73 CPU)
Result: FAIL

mimedefang dies on non-ascii filenames (#54 is back)

Hello,
I just found mimedefang (3.3) dying trying to deal with some attachement:
For privacy I won't put the complete attachement filename here but it seems that the problematic part showed up in log as:
re%301sultats

Feb 28 20:39:47 mailhost mimedefang-multiplexor[29992]: 31SJdfJu008556: Worker 0 stderr: open body: Invalid argument at /usr/pkg/lib/perl5/vendor_perl/5
Feb 28 20:39:47 mailhost mimedefang-multiplexor[29992]: 31SJdfJu008556: Worker 0 stderr: .36.0/MIME/Entity.pm line 1897.
Feb 28 20:39:47 mailhost mimedefang-multiplexor[29992]: Worker 0 died prematurely -- check your filter rules
Feb 28 20:39:47 mailhost mimedefang[1954]: 31SJdfJu008556: Error from multiplexor: ERR No response from worker
Feb 28 20:39:47 mailhost sendmail[8556]: 31SJdfJu008556: Milter: data, reject=451 4.3.2 Please try again later

Using the same trick (in attached patch file) as commit 456aeb1 fixed it for me. But is seems that a similar code was present and removed commit 30ff081 ...

patch-modules_lib_Mail_MIMEDefang_MIME.pm.txt

More functions missing in Exports in Action.pm

Hello everyone,

short time ago there has been an error reported on src file MIMEDefang/modules/lib/Mail/MIMEDefang/Actions.pm: #66

This has beem marked as resolved by the commit: e513e46

However the commit only fixes one missing function, but here are more missing.
I've compared the Exports array with functions actually defined in the file and there are 4 functions missing in the Exports array:

  • action_accept_with_warning
  • action_notify_administrator
  • action_replace_with_warning
  • action_sm_quarantine

Commit mentioned earlier only fixes one function: action_sm_quarantine, but there are 3 other still missing.
Filter cannot use those actions as they are not exported.

Many thanks for help

Tomáš Novosad

Built-in tests don't support splitted Perl modules

As of MIMEDefang 3.0, the built-in tests look like this:

$ make test
prove -It/lib --recurse t
t/actions.t .. ok
t/arc.t ...... ok
#   Failed test 't md authres'
#   at t/authres.t line 18.
#   (in Mail::MIMEDefang::Unit::Authres->t_md_authres)
#                   'sa-test.spamassassin.org (MIMEDefang);'
#     doesn't match '(?^:sa\-test\.spamassassin\.org \(MIMEDefang\);(?:.*)\s+dkim=pass \(768\-bit key\) header\.d=sa-test\.spamassassin\.org header\.b="oRxHoP0Y";(?:.*)\s+spf=none \(domain of test\@sa\-test\.spamassassin\.org doesn't specify if 1\.2\.3\.4 is a permitted sender\) smtp\.mailfrom=test\@sa\-test\.spamassassin\.org;)'
# Looks like you failed 1 test of 1.
t/authres.t .. 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests 
#   Failed test 't init globals2'
#   at t/core.t line 20.
#   (in Mail::MIMEDefang::Unit::core->t_init_globals2)
#          got: '1'
#     expected: '0'
# Looks like you failed 1 test of 5.
t/core.t ..... 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/5 subtests 
	(less 1 skipped subtest: 3 okay)
#   Failed test 'gen_msgid_header_works died (Undefined subroutine &main::gen_msgid_header called at t/dates.t line 40.)'
#   at t/dates.t line 55.
#   (in Mail::MIMEDefang::Unit::dates->gen_msgid_header_works)
#   Failed test 'header_timezone_works died (Undefined subroutine &main::header_timezone called at t/dates.t line 12.)'
#   at t/dates.t line 55.
#   (in Mail::MIMEDefang::Unit::dates->header_timezone_works)
#   Failed test 'rfc2822_date_works died (Undefined subroutine &main::rfc2822_date called at t/dates.t line 33.)'
#   at t/dates.t line 55.
#   (in Mail::MIMEDefang::Unit::dates->rfc2822_date_works)
#   Failed test 't_hour_str died (Undefined subroutine &Mail::MIMEDefang::Utils::hour_str called at t/dates.t line 52.)'
#   at t/dates.t line 55.
#   (in Mail::MIMEDefang::Unit::dates->t_hour_str)
#   Failed test 't_time_str died (Undefined subroutine &Mail::MIMEDefang::Utils::time_str called at t/dates.t line 46.)'
#   at t/dates.t line 55.
#   (in Mail::MIMEDefang::Unit::dates->t_time_str)
# Looks like you failed 5 tests of 7.
t/dates.t .... 
Dubious, test returned 5 (wstat 1280, 0x500)
Failed 5/7 subtests 
	(less 2 skipped subtests: 0 okay)
#   Failed test 'dkim verify'
#   at t/dkim.t line 47.
#   (in Mail::MIMEDefang::Unit::DKIM->dkim_verify)
#          got: 'invalid'
#     expected: 'pass'
#   Failed test 'dkim verify'
#   at t/dkim.t line 48.
#   (in Mail::MIMEDefang::Unit::DKIM->dkim_verify)
#          got: '0'
#     expected: '768'
#   Failed test 'dkim verify'
#   at t/dkim.t line 53.
#   (in Mail::MIMEDefang::Unit::DKIM->dkim_verify)
#          got: 'invalid'
#     expected: 'fail'
# Looks like you failed 3 tests of 5.
t/dkim.t ..... 
Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/5 subtests 
#   Failed test 'add_header_ok died (Unable to detect either Unix::Syslog or Sys::Syslog at modules/lib/Mail/MIMEDefang.pm line 218.)'
#   at t/headers.t line 70.
#   (in Mail::MIMEDefang::Unit::headers->add_header_ok)
#   Failed test 'change_header_ok died (Unable to detect either Unix::Syslog or Sys::Syslog at modules/lib/Mail/MIMEDefang.pm line 218.)'
#   at t/headers.t line 70.
#   (in Mail::MIMEDefang::Unit::headers->change_header_ok)
#   Failed test 'delete_header_ok died (Unable to detect either Unix::Syslog or Sys::Syslog at modules/lib/Mail/MIMEDefang.pm line 218.)'
#   at t/headers.t line 70.
#   (in Mail::MIMEDefang::Unit::headers->delete_header_ok)
# Looks like you failed 3 tests of 6.
t/headers.t .. 
Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/6 subtests 
	(less 3 skipped subtests: 0 okay)
t/mime.t ..... ok
#   Failed test 't_get_host_name died (Undefined subroutine &main::get_host_name called at t/net.t line 21.)'
#   at t/net.t line 51.
#   (in Mail::MIMEDefang::Unit::Net->t_get_host_name)
# Looks like you failed 1 test of 9.
t/net.t ...... 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/9 subtests 
	(less 1 skipped subtest: 7 okay)
#   Failed test 'handle_relayok lives'
#   at t/relay.t line 155.
#   (in Mail::MIMEDefang::Unit::filter_relay->accept_and_no_more_filtering)
# died: Undefined subroutine &main::handle_relayok called at t/relay.t line 155.
#   Failed test 'handle_relayok called send_filter_answer with expected arguments'
#   at t/relay.t line 157.
#   (in Mail::MIMEDefang::Unit::filter_relay->accept_and_no_more_filtering)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_relayok lives'
#   at t/relay.t line 155.
#   (in Mail::MIMEDefang::Unit::filter_relay->accept_and_no_more_filtering)
# died: Undefined subroutine &main::handle_relayok called at t/relay.t line 155.
#   Failed test 'handle_relayok called send_filter_answer with expected arguments'
#   at t/relay.t line 157.
#   (in Mail::MIMEDefang::Unit::filter_relay->accept_and_no_more_filtering)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_relayok lives'
#   at t/relay.t line 155.
#   (in Mail::MIMEDefang::Unit::filter_relay->continue)
# died: Undefined subroutine &main::handle_relayok called at t/relay.t line 155.
#   Failed test 'handle_relayok called send_filter_answer with expected arguments'
#   at t/relay.t line 157.
#   (in Mail::MIMEDefang::Unit::filter_relay->continue)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_relayok lives'
#   at t/relay.t line 155.
#   (in Mail::MIMEDefang::Unit::filter_relay->continue)
# died: Undefined subroutine &main::handle_relayok called at t/relay.t line 155.
#   Failed test 'handle_relayok called send_filter_answer with expected arguments'
#   at t/relay.t line 157.
#   (in Mail::MIMEDefang::Unit::filter_relay->continue)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_relayok lives'
#   at t/relay.t line 155.
#   (in Mail::MIMEDefang::Unit::filter_relay->discard)
# died: Undefined subroutine &main::handle_relayok called at t/relay.t line 155.
#   Failed test 'handle_relayok called send_filter_answer with expected arguments'
#   at t/relay.t line 157.
#   (in Mail::MIMEDefang::Unit::filter_relay->discard)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_relayok lives'
#   at t/relay.t line 155.
#   (in Mail::MIMEDefang::Unit::filter_relay->discard)
# died: Undefined subroutine &main::handle_relayok called at t/relay.t line 155.
#   Failed test 'handle_relayok called send_filter_answer with expected arguments'
#   at t/relay.t line 157.
#   (in Mail::MIMEDefang::Unit::filter_relay->discard)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_relayok lives'
#   at t/relay.t line 155.
#   (in Mail::MIMEDefang::Unit::filter_relay->reject)
# died: Undefined subroutine &main::handle_relayok called at t/relay.t line 155.
#   Failed test 'handle_relayok called send_filter_answer with expected arguments'
#   at t/relay.t line 157.
#   (in Mail::MIMEDefang::Unit::filter_relay->reject)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_relayok lives'
#   at t/relay.t line 155.
#   (in Mail::MIMEDefang::Unit::filter_relay->reject)
# died: Undefined subroutine &main::handle_relayok called at t/relay.t line 155.
#   Failed test 'handle_relayok called send_filter_answer with expected arguments'
#   at t/relay.t line 157.
#   (in Mail::MIMEDefang::Unit::filter_relay->reject)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_relayok lives'
#   at t/relay.t line 155.
#   (in Mail::MIMEDefang::Unit::filter_relay->tempfail)
# died: Undefined subroutine &main::handle_relayok called at t/relay.t line 155.
#   Failed test 'handle_relayok called send_filter_answer with expected arguments'
#   at t/relay.t line 157.
#   (in Mail::MIMEDefang::Unit::filter_relay->tempfail)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_relayok lives'
#   at t/relay.t line 155.
#   (in Mail::MIMEDefang::Unit::filter_relay->tempfail)
# died: Undefined subroutine &main::handle_relayok called at t/relay.t line 155.
#   Failed test 'handle_relayok called send_filter_answer with expected arguments'
#   at t/relay.t line 157.
#   (in Mail::MIMEDefang::Unit::filter_relay->tempfail)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
# Looks like you failed 20 tests of 20.
t/relay.t .... 
Dubious, test returned 20 (wstat 5120, 0x1400)
Failed 20/20 subtests 
#   Failed test 'handle_senderok lives'
#   at t/sender.t line 170.
#   (in Mail::MIMEDefang::Unit::filter_sender->accept_and_no_more_filtering)
# died: Undefined subroutine &main::handle_senderok called at t/sender.t line 170.
#   Failed test 'handle_senderok called send_filter_answer with expected arguments'
#   at t/sender.t line 172.
#   (in Mail::MIMEDefang::Unit::filter_sender->accept_and_no_more_filtering)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_senderok lives'
#   at t/sender.t line 170.
#   (in Mail::MIMEDefang::Unit::filter_sender->accept_and_no_more_filtering)
# died: Undefined subroutine &main::handle_senderok called at t/sender.t line 170.
#   Failed test 'handle_senderok called send_filter_answer with expected arguments'
#   at t/sender.t line 172.
#   (in Mail::MIMEDefang::Unit::filter_sender->accept_and_no_more_filtering)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_senderok lives'
#   at t/sender.t line 170.
#   (in Mail::MIMEDefang::Unit::filter_sender->continue)
# died: Undefined subroutine &main::handle_senderok called at t/sender.t line 170.
#   Failed test 'handle_senderok called send_filter_answer with expected arguments'
#   at t/sender.t line 172.
#   (in Mail::MIMEDefang::Unit::filter_sender->continue)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_senderok lives'
#   at t/sender.t line 170.
#   (in Mail::MIMEDefang::Unit::filter_sender->continue)
# died: Undefined subroutine &main::handle_senderok called at t/sender.t line 170.
#   Failed test 'handle_senderok called send_filter_answer with expected arguments'
#   at t/sender.t line 172.
#   (in Mail::MIMEDefang::Unit::filter_sender->continue)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_senderok lives'
#   at t/sender.t line 170.
#   (in Mail::MIMEDefang::Unit::filter_sender->discard)
# died: Undefined subroutine &main::handle_senderok called at t/sender.t line 170.
#   Failed test 'handle_senderok called send_filter_answer with expected arguments'
#   at t/sender.t line 172.
#   (in Mail::MIMEDefang::Unit::filter_sender->discard)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_senderok lives'
#   at t/sender.t line 170.
#   (in Mail::MIMEDefang::Unit::filter_sender->discard)
# died: Undefined subroutine &main::handle_senderok called at t/sender.t line 170.
#   Failed test 'handle_senderok called send_filter_answer with expected arguments'
#   at t/sender.t line 172.
#   (in Mail::MIMEDefang::Unit::filter_sender->discard)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_senderok lives'
#   at t/sender.t line 170.
#   (in Mail::MIMEDefang::Unit::filter_sender->reject)
# died: Undefined subroutine &main::handle_senderok called at t/sender.t line 170.
#   Failed test 'handle_senderok called send_filter_answer with expected arguments'
#   at t/sender.t line 172.
#   (in Mail::MIMEDefang::Unit::filter_sender->reject)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_senderok lives'
#   at t/sender.t line 170.
#   (in Mail::MIMEDefang::Unit::filter_sender->reject)
# died: Undefined subroutine &main::handle_senderok called at t/sender.t line 170.
#   Failed test 'handle_senderok called send_filter_answer with expected arguments'
#   at t/sender.t line 172.
#   (in Mail::MIMEDefang::Unit::filter_sender->reject)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_senderok lives'
#   at t/sender.t line 170.
#   (in Mail::MIMEDefang::Unit::filter_sender->tempfail)
# died: Undefined subroutine &main::handle_senderok called at t/sender.t line 170.
#   Failed test 'handle_senderok called send_filter_answer with expected arguments'
#   at t/sender.t line 172.
#   (in Mail::MIMEDefang::Unit::filter_sender->tempfail)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
#   Failed test 'handle_senderok lives'
#   at t/sender.t line 170.
#   (in Mail::MIMEDefang::Unit::filter_sender->tempfail)
# died: Undefined subroutine &main::handle_senderok called at t/sender.t line 170.
#   Failed test 'handle_senderok called send_filter_answer with expected arguments'
#   at t/sender.t line 172.
#   (in Mail::MIMEDefang::Unit::filter_sender->tempfail)
# Compared array length of $data
#    got : array with 0 element(s)
# expect : array with 6 element(s)
# 0
# Looks like you failed 20 tests of 20.
t/sender.t ... 
Dubious, test returned 20 (wstat 5120, 0x1400)
Failed 20/20 subtests 
t/smtp.t ..... ok
#   Failed test 't_re_match_zip died (Unable to detect either Unix::Syslog or Sys::Syslog at modules/lib/Mail/MIMEDefang.pm line 218.)'
#   at t/utils.t line 113.
#   (in Mail::MIMEDefang::Unit::Utils->t_re_match_zip)
# Looks like you failed 1 test of 9.
t/utils.t .... 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/9 subtests 
	(less 1 skipped subtest: 7 okay)
Test Summary Report
-------------------
t/authres.t (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
t/core.t   (Wstat: 256 (exited 1) Tests: 5 Failed: 1)
  Failed test:  3
  Non-zero exit status: 1
t/dates.t  (Wstat: 1280 (exited 5) Tests: 7 Failed: 5)
  Failed tests:  1-2, 5-7
  Non-zero exit status: 5
t/dkim.t   (Wstat: 768 (exited 3) Tests: 5 Failed: 3)
  Failed tests:  3-5
  Non-zero exit status: 3
t/headers.t (Wstat: 768 (exited 3) Tests: 6 Failed: 3)
  Failed tests:  1, 3, 5
  Non-zero exit status: 3
t/net.t    (Wstat: 256 (exited 1) Tests: 9 Failed: 1)
  Failed test:  2
  Non-zero exit status: 1
t/relay.t  (Wstat: 5120 (exited 20) Tests: 20 Failed: 20)
  Failed tests:  1-20
  Non-zero exit status: 20
t/sender.t (Wstat: 5120 (exited 20) Tests: 20 Failed: 20)
  Failed tests:  1-20
  Non-zero exit status: 20
t/utils.t  (Wstat: 256 (exited 1) Tests: 9 Failed: 1)
  Failed test:  7
  Non-zero exit status: 1
Files=13, Tests=93, 108 wallclock secs ( 0.07 usr  0.02 sys +  2.51 cusr  0.35 csys =  2.95 CPU)
Result: FAIL
make: *** [Makefile:353: test] Error 1
$ 

And yes, there might be one or another missing test dependency, but e.g. &main::gen_msgid_header still doesn't look right.

spamassassin config file ignored in 3.0

My Spamassassin config has been stored in /etc/mail/sa-mimedefang.cf since forever. In the 3.0 upgrade, this file now seems to be ignored. Documentation and man pages still seem to indicate this is an appropriate location.
Kenneth Porter writes "It looks like that logic was lost in spam_assassin_init in modules/lib/Mail/MIMEDefang/Antispam.pm. (That procedure used to be in mimedefang.pl but the code got rearranged into multiple files. It had a fairly elaborate search for config files. Now the optional $config value passed in isn't used and no search is made for one.)"

Use Digest::SHA instead of Digest::SHA1?

As of writing, gen-ip-validator.pl, mimedefang.pl/mimedefang.pl.in are using Digest::SHA1, but I would like suggest a switch to Digest::SHA, because: Digest::SHA is a bit faster than Digest::SHA1, Digest::SHA1 has been removed from some Linux distributions, Digest::SHA is a core library (as of Perl >= 5.10.0) and Digest::SHA1 is not (and never will be). See also: https://stackoverflow.com/questions/3420720/what-are-the-advantages-of-digestsha-over-digestsha1

Unable to convert text in character set `GB18030' to UTF-8

Running git fore some days i found this:

Apr 11 12:17:10 svrh1smtpmx01p mimedefang-multiplexor[556979]: B903C221C11: Worker 1 stderr: Unable to convert text in character set `GB18030' to UTF-8... i
Apr 11 12:17:10 svrh1smtpmx01p mimedefang-multiplexor[556979]: B903C221C11: Worker 1 stderr: gnoring
Apr 11 12:17:10 svrh1smtpmx01p mimedefang-multiplexor[556979]: B903C221C11: Worker 1 stderr:  at /usr/share/perl5/MIME/Head.pm line 799.
Apr 11 12:17:10 svrh1smtpmx01p mimedefang-multiplexor[556979]: B903C221C11: Worker 1 stderr: Unable to convert text in character set `GB18030' to UTF-8... i
Apr 11 12:17:10 svrh1smtpmx01p mimedefang-multiplexor[556979]: B903C221C11: Worker 1 stderr: gnoring
Apr 11 12:17:10 svrh1smtpmx01p mimedefang-multiplexor[556979]: B903C221C11: Worker 1 stderr:  at /usr/share/perl5/MIME/Head.pm line 799.
Apr 11 12:17:10 svrh1smtpmx01p mimedefang-multiplexor[556979]: B903C221C11: Worker 1 stderr: Unable to convert text in character set `GB18030' to UTF-8... i
Apr 11 12:17:10 svrh1smtpmx01p mimedefang-multiplexor[556979]: B903C221C11: Worker 1 stderr: gnoring
Apr 11 12:17:10 svrh1smtpmx01p mimedefang-multiplexor[556979]: B903C221C11: Worker 1 stderr:  at /usr/share/perl5/MIME/Head.pm line 799.

Wishlist: Prepend all md_syslog lines with queueid

Hi,
it would be nice to automaticalkly prepend all md_syslog lines with the queueid of the processing mail. That makes it much easier to find all lines for a specific mail, e.g. MTA + mimedefang and others.

Flo

gen_date_msgid_headers not a public function, but called from MIMEDefang.pm without namespace

Since upgrading to 3.0 or maybe 3.1, I've not been receiving administrator notifications. I traced this via error messages in my mail log which showed that the filter was dying

Sep 21 17:43:16 localhost mimedefang-multiplexor[19273]: 28LGhA2g002233: Worker 4 stderr: Undefined subroutine &Mail::MIMEDefang::gen_date_msgid_headers
Sep 21 17:43:16 localhost mimedefang-multiplexor[19273]: 28LGhA2g002233: Worker 4 stderr: called at /usr/share/perl5/vendor_perl/Mail/MIMEDefang.pm line
Sep 21 17:43:16 localhost mimedefang-multiplexor[19273]: 28LGhA2g002233: Worker 4 stderr: 1106.

Normal messages seem to be OK, but anything which generates a notification is failing, presumably because the notification lacks a msgID...

I added the namespace and all seems OK, notifications are now arriving as expected.

# diff MIMEDefang.pm MIMEDefang.pm.old
855c855
<         $body .= Mail::MIMEDefang::RFC2822::gen_date_msgid_headers();
---
>         $body .= gen_date_msgid_headers();
945c945
<         $body .= Mail::MIMEDefang::RFC2822::gen_date_msgid_headers();
---
>         $body .= gen_date_msgid_headers();
1106c1106
<   $mail .= Mail::MIMEDefang::RFC2822::gen_date_msgid_headers();
---
>   $mail .= gen_date_msgid_headers();

Paul.

action replace_with_url doesn't work correctly with non-ASCII attachment names

When use action replace_with_url attachment names with non-ASCII characters modified like this:

  • all non-ASCII characters in attachment name replace with "#" simbol
  • all ASCII characters in attachment name are intact
    So, if you have attachment name, for example, köül.7z then the name will be transformed to k##l.7z

It seems this behavior occured in versions 2.86 and above when one line was added to sub TakeStabAtFilename (in bold)

sub takeStabAtFilename
{
my ($entity) = @_;
 my $guess = $entity->head->recommended_filename();
 if( defined $guess ) {
   $guess =~ tr/\x00-\x7F/#/c;
return scalar( decode_mimewords( $guess ) );
}  
return '';
}

2.86 - suggested-minimum-filter-for-windows-clients example crash

There appears to be a code defect in the suggested-minimum-filter-for-windows-clients / mimedefang-filter example of 2.86. with the current patches applied.

I have taken the following actions to try to remediate this issue:

  1. I applied the define removal patch to correctly determine RSPAMD but that creates other process crashes.

Mar 11 02:05:59 host mimedefang-multiplexor[4341]: Worker 0 died prematurely -- check your filter rules
Mar 11 02:05:59 host mimedefang-multiplexor[4341]: Reap: worker 0 (pid 8478) exited normally with status 2 (WORKER DIED UNEXPECTEDLY)
Mar 11 02:05:59 host mimedefang-multiplexor[4341]: Worker 0 resource usage: req=1, scans=1, user=3.326, sys=0.181, nswap=0, majflt=0, minflt=26333, maxrss=106732, bi=0, bo=40

  1. I reverted the change to ensure that mailflow continues. In this case I notice that X-Spam header doesn't get applied for any messages.

I haven't been able to investigate the root cause to determine the source and patch the issue so I am creating this issue to flag that this is an open issue that needs work in the event that someone else has the time to investigate this issue.

test failure with mimedefang 3.4

when running the tests on alpine linux edge for mimedefang version 3.4 I get this test failure:

t/arc.t ...... 1/7                                                                                                                 
#   Failed test 'arc sign'                                                                                                         
#   at t/arc.t line 36.                                                                                                            
#   (in Mail::MIMEDefang::Unit::DKIM::ARC->arc_sign)                                                                               
#          got: 'i=1; a=rsa-sha256; cv=none; d=sa-test.spamassassin.org; s=                                                        
#       t0768; t=1682485935; b=Rd3vS+0Y3rvUcZQw7PAfuEELJCZevXuZa7vxQG8x9                                                           
#       phhIzQ55VCJoh5vcv4bxsWUw0ZFBvjHJlkkf0QbiKxelhJk1UnDpSFiaJR1CsAmo                                                           
#       RdGxeg6z38HR3pVgo9NYzMX35w/jJeqJ1FuQrgrPfwBJdNiYmVHZpHgtU/Y8a8Xa                                                           
#       wYSNZp9F+8oN0KxjQUooDTt2rS5dXZ6/D54TfUqOEOe+uZG0wRmBjSKsVM3p6491                                                           
#       P+nb9uNnpe+Ri3q5TguxsmHs2wUbxlBoMcstKJRfWdCmikCsaeW3rKfwlMz6IxLX                                                           
#       E4BtZz4kIyqSnWqtuRwXC+pO3pp/GzWMi+AiFGVTiqYlw=='                                                                           
#     expected: undef                                                                                                              
# Looks like you failed 1 test of 7.

alpine linux edge had Mail::DKIM 1.20230212

Error in MIMEDefang/modules/lib/Mail/MIMEDefang/Actions.pm

in @export = qw{
action_rebuild action_notify_sender action_insert_header action_drop
action_bounce action_accept action_defang action_discard action_add_part
action_tempfail action_add_header action_add_entity action_quarantine
action_quarantine_entire_message action_change_header action_delete_header
action_external_filter get_quarantine_dir
action_replace_with_url action_drop_with_warning action_delete_all_headers
message_rejected process_added_parts add_recipient delete_recipient change_sender
};
this input is missing > "action_sm_quarantine"

this work ok

in @export = qw{
action_rebuild action_notify_sender action_insert_header action_drop
action_bounce action_accept action_defang action_discard action_add_part
action_tempfail action_add_header action_add_entity action_quarantine
action_quarantine_entire_message action_change_header action_delete_header
action_external_filter get_quarantine_dir action_sm_quarantine
action_replace_with_url action_drop_with_warning action_delete_all_headers
message_rejected process_added_parts add_recipient delete_recipient change_sender
};

configure does not handle alternative prefix well

With the command line:

./configure --prefix=/opt/local/ --with-user=defang --with-milterinc=/opt/local/include --with-milterlib=/opt/local/lib/

The configure script works out its libmilter issues like this:

checking for libmilter/mfapi.h... /opt/local/include/libmilter/mfapi.h
checking for libmilter.a... no
checking for libsm.a... no
checking for libmilter.so... /opt/local/lib/libmilter.so
checking whether libmilter requires -lsm... yes
checking whether libsm requires -lldap... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: no: No such file or directory.
no
configure: creating ./config.status

And this in config.log:

configure:6621: checking whether libmilter requires -lsm
configure:6635: gcc -o conftest -g -O2 -fPIC -L/opt/local/lib conftest.c -lpthread -lresolv -lmilter >&5
conftest.c:40:10: fatal error: 'libmilter/mfapi.h' file not found
#include "libmilter/mfapi.h"
^
1 error generated.
configure:6635: $? = 1

The 'make' then fails at link time because there is no libsm. This can be hacked around by copying libsm.a from a sendmail build, but that's BAD.

Adding "CFLAGS=-I/opt/local/include" to the command line fixes BOTH the incorrect detection of the need for '-lsm' and the error from 'nm':

CFLAGS=-I/opt/local/include ./configure --prefix=/opt/local/ --with-user=defang --with-milterinc=/opt/local/include --with-milterlib=/opt/local/lib
[...]
checking for libmilter/mfapi.h... /opt/local/include/libmilter/mfapi.h
checking for libmilter.a... no
checking for libsm.a... no
checking for libmilter.so... /opt/local/lib/libmilter.so
checking whether libmilter requires -lsm... no
checking whether libmilter requires -lldap... no
configure: creating ./config.status

I am sure there must be something trivially easy to do involving autoconf to make $PREFIX propagate down into CFLAGS as it does to LDFLAGS, but I have yet to figure that out. Help appreciated.

systemd units on RH based distro

RedHat based distro uses /etc/sysconfig/mimedefang to store startup preferences instead of /etc/default/mimedefang.
Provided systemd units ignore files under /etc/sysconfig directory and searches only for values in /etc/default/mimedefang.
Provided systemd units should be compatible with RedHat systems or specify on which OS they have been tested.

Increase SpamAssassin size threshold

For the last while, spam has been regularly getting through SpamAssassin because the message contains an attachment larger than 100kB and the message not scanned.

        if (-s "./INPUTMSG" < 100*1024) {
            # Only scan messages smaller than 100kB.    Larger messages
            # are extremely unlikely to be spam, and SpamAssassin is
            # dreadfully slow on very large messages.

Please consider raising the default size. Spammy attachments have been up to 3MB on my server.

I understand the nature of the mimedefang-filter allows the user to modify this themselves. But as a default it would be nice to have it a little higher. It seems spammers are now sending messages with somewhat larger attachments to get around SpamAssassin; perhaps not a new technique, but something I've started seeing more frequently lately.

some tests fail with MIMEDefang 3.1 on alpine linux

Tests pass on x86_x86 ppc64le and s390x, but fail on armv7, armhf, aarch64 with:

t/mime.t ..... ok
[435](https://gitlab.alpinelinux.org/a16bitsysop/aports/-/jobs/814457#L435)#   Failed test 't_relay_is_blacklisted_multi died (Can't use string ("SERVFAIL") as an ARRAY ref while "strict refs" in use at t/net.t line 59.)'
[436](https://gitlab.alpinelinux.org/a16bitsysop/aports/-/jobs/814457#L436)#   at t/net.t line 62.
[437](https://gitlab.alpinelinux.org/a16bitsysop/aports/-/jobs/814457#L437)#   (in Mail::MIMEDefang::Unit::Net->t_relay_is_blacklisted_multi)
[438](https://gitlab.alpinelinux.org/a16bitsysop/aports/-/jobs/814457#L438)# Looks like you failed 1 test of 10.
[439](https://gitlab.alpinelinux.org/a16bitsysop/aports/-/jobs/814457#L439)t/net.t ...... 
[440](https://gitlab.alpinelinux.org/a16bitsysop/aports/-/jobs/814457#L440)Dubious, test returned 1 (wstat 256, 0x100)
[441](https://gitlab.alpinelinux.org/a16bitsysop/aports/-/jobs/814457#L441)Failed 1/10 subtests

full build log:
https://gitlab.alpinelinux.org/a16bitsysop/aports/-/jobs/814457/raw

Mailing list down

I sent a question to the mailing list on Feb. 26 and didn't see it appear on the list. Is that list dead? Is there a new place for operational questions?

MIMEdefang filter process crashes when name of attachment being dropped is in UTF-8

If someone emails you a ZIP file whose name as specified in the MIME headers in UTF-8, then MIMEdefang crashes when trying to drop the ZIP file and add a warning about it to the message. Example logs:

Feb 10 02:05:27 jik4 mimedefang.pl[4134278]: 21A75PbG055031: MDLOG,21A75PbG055031,bad_filename,nova narud%17Eba pdf.zip,application/zip,<[email protected]>,<elided>,=?UTF-8?Q?Re=3A_nova_narud=C5=BEba?=
Feb 10 02:05:27 jik4 mimedefang.pl[4134278]: 21A75PbG055031: MDLOG,21A75PbG055031,mail_in,,,<[email protected]>,<elided>,=?UTF-8?Q?Re=3A_nova_narud=C5=BEba?=
Feb 10 02:05:27 jik4 mimedefang-multiplexor[3749123]: 21A75PbG055031: Worker 0 stderr: open body: Invalid argument at /usr/share/perl5/vendor_perl/MIM         
Feb 10 02:05:27 jik4 mimedefang-multiplexor[3749123]: 21A75PbG055031: Worker 0 stderr: E/Entity.pm line 1892.                                                  
Feb 10 02:05:27 jik4 mimedefang-multiplexor[3749123]: Worker 0 died prematurely -- check your filter rules                                                     
Feb 10 02:05:27 jik4 mimedefang-multiplexor[3749123]: Reap: worker 0 (pid 4134278) exited normally with status 22 (WORKER DIED UNEXPECTEDLY)                   
Feb 10 02:05:27 jik4 mimedefang-multiplexor[3749123]: Worker 0 resource usage: req=304, scans=295, user=3.179, sys=0.240, nswap=0, majflt=600, minflt=13458, maxrss=48816, bi=6040, bo=46080                                                  
Feb 10 02:05:27 jik4 mimedefang[3749140]: 21A75PbG055031: Error from multiplexor: ERR No response from worker                                                  

The relevant MIME headers from the attachment in the spam message logged above:

Content-Transfer-Encoding: base64                                               
Content-Type: application/zip;                                                  
 name="=?UTF-8?Q?nova_narud=C5=BEba_pdf=2Ezip?="                                
Content-Disposition: attachment;                                                
 filename*=UTF-8''nova%20narud%C5%BEba%20pdf.zip;                               
 size=705397                                                                    

I don't know if this is the right fix—in fact, I'm pretty sure that it's not—but it fixed the issue for me, so perhaps it will give someone else who knows more about all this stuff than I do an idea what the right fix should be:

--- mimedefang.pl	2022/02/10 16:46:42	1.1
+++ mimedefang.pl	2022/02/10 16:53:29
@@ -75,6 +75,7 @@
 use Sys::Hostname;
 use File::Spec qw ();
 use Errno qw(ENOENT EACCES);
+use Encode;
 
 # Detect these Perl modules at run-time.  Can explicitly prevent
 # loading of these modules by setting $Features{"xxx"} = 0;
@@ -989,10 +990,17 @@
 
     my ($part);
 
+    my $charset = undef;
+    if (Encode::is_utf8($data)) {
+        $data = encode 'UTF-8', $data;
+        $charset = 'utf-8';
+    }
+
     $part = MIME::Entity->build(Type => $type,
 				Top => 0,
 				'X-Mailer' => undef,
 				Encoding => $encoding,
+                                Charset => $charset,
 				Data => ["$data"]);
     defined ($fname) && $part->head->mime_attr("Content-Type.name" => $fname);
     defined ($disposition) && $part->head->mime_attr("Content-Disposition" => $disposition);

Need backscatter detector/handler for receive-only addresses

Feature request: I'm getting a lot of backscatter to receive-only "role" addresses like webmaster, hostmaster, and noc, addresses I use to register at ARIN or my domain registrar. I get maybe 20-50 of these a day. I'd like to have some way of detecting and rejecting them at the MTA. For example, I could provide a list of such addresses and the filter would check for common NDR patterns and bounce those for these addresses. Or perhaps a fancier filter could spot that the NDR is for an obvious forgery.

[Originally sent to the mailing list, which seems to be bouncing right now.]

Requirement for modern Perl Mail::DKIM > 1.2020

Since MIMEDefang 3.0, the modern Perl Mail::DKIM > 1.2020 is required, which impacts especially long-term Linux distributions:

  • CentOS/RHEL 7: Mail::DKIM 0.39 (distribution reaches end of life at June 30, 2024)
  • CentOS/RHEL/Rocky 8: Mail::DKIM 0.54 (distribution reaches end of life at May 31, 2029)
  • CentOS/RHEL/Rocky 9: Mail::DKIM 1.20200907
  • Fedora 35: Mail::DKIM 1.20200907
  • Fedora 36: Mail::DKIM 1.20200907
  • Fedora Rawhide: Mail::DKIM 1.20220520

From what I can see, the Mail::DKIM requirement is not really conditional/optional right now, which would leave CentOS/RHEL/Rocky before version 9 on MIMEDefang 2.86 until the distribution goes end of life. If this is intended by upstream, it's fine for me, too. I just wanted to raise this.

One might argue that Mail::DKIM is not really required, however without Mail::DKIM, make test fails and the unconditional installation of Mail/MIMEDefang/DKIM.pm and Mail/MIMEDefang/DKIM/ARC.pm still leads to a run-time dependency generated by RPM. I did not yet test, if I just could drop these two files myself during installation (and/or if just dropping one file might be still good enough to provide the maximum functionality with the old Mail::DKIM versions).

P.S.: Mail/MIMEDefang/Authres.pm depends on Mail::MIMEDefang::DKIM as I noticed now, so ripping just out the Mail::DKIM part doesn't seem to work as mentioned above.

In the end I would like to hear your upstream opinion and/or suggestions here.

Mimedefang 3.3 WORKER DIED UNEXPECTEDLY / line 1892

After i found an older issue for 2.84 i upgraded to current mimedefang 3.3 (Recompiled bookworm package for bullseye)

I still see issues like this:

Mar 31 16:55:41 svrh1smtpmx01p mimedefang-multiplexor[96084]: 6C716220ECB: Worker 0 stderr: open body: Invalid argument at /usr/share/perl5/MIME/Entity.pm
Mar 31 16:55:41 svrh1smtpmx01p mimedefang-multiplexor[96084]: 6C716220ECB: Worker 0 stderr: line 1892.
Mar 31 16:55:41 svrh1smtpmx01p mimedefang-multiplexor[96084]: Worker 0 died prematurely -- check your filter rules
Mar 31 16:55:41 svrh1smtpmx01p mimedefang-multiplexor[96084]: stats 1680274541.278 EndFilter worker=0 nworkers=2 nbusy=0 numRequests=153
Mar 31 16:55:41 svrh1smtpmx01p mimedefang-multiplexor[96084]: Reap: worker 0 (pid 96087) exited normally with status 22 (WORKER DIED UNEXPECTEDLY)
Mar 31 16:55:41 svrh1smtpmx01p mimedefang-multiplexor[96084]: Worker 0 resource usage: req=153, scans=153, user=13.225, sys=0.658, nswap=0, majflt=0, minflt=20610, maxrss=106016, bi=0, bo=843712
Mar 31 16:55:41 svrh1smtpmx01p mimedefang[96106]: 6C716220ECB: Error from multiplexor: ERR No response from worker
root@svrh1smtpmx01p:/var/log# dpkg -l mimedefang libmime-tools-perl
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name               Version      Architecture Description
+++-==================-============-============-=========================================
ii  libmime-tools-perl 5.509-1      all          Perl5 modules for MIME-compliant messages
ii  mimedefang         3.3-1        amd64        e-mail filter program for sendmail

Flo

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.