Coder Social home page Coder Social logo

snawoot / postfix-mta-sts-resolver Goto Github PK

View Code? Open in Web Editor NEW
117.0 15.0 24.0 419 KB

Daemon which provides TLS client policy for Postfix via socketmap, according to domain MTA-STS policy

License: MIT License

Makefile 1.14% Python 94.63% Shell 3.58% Dockerfile 0.65%
postfix mta-sts sts smtp-sts postfix-helper postfix-policy-server postfix-smtp

postfix-mta-sts-resolver's Introduction

postfix-mta-sts-resolver

Build Status Coverage PyPI - Downloads PyPI PyPI - Status PyPI - License postfix-mta-sts-resolver CodeQL

Daemon which provides TLS client policy for Postfix via socketmap, according to domain MTA-STS policy.

Current support of RFC 8461 is limited:

Server has configurable cache backend which allows to store cached STS policies in memory (internal), file (sqlite) or in Redis database (redis).

Requirements

  • Postfix 3.4+ (or Postfix 2.10+ if missing Postfix SNI feature is tolerable. In that case you have to set zone option require_sni to false in MTA-STS daemon config)
  • Python 3.5.3+ (see "Systems without Python 3.5+" below if you haven't one, or use Docker installation method)
  • aiodns
  • aiohttp
  • aiosqlite
  • redis-py
  • PyYAML
  • (optional) uvloop

All dependency packages installed automatically if this package is installed via pip.

Installation

Method 1. System-wide install from PyPI (recommended for humans)

Run:

sudo python3 -m pip install postfix-mta-sts-resolver[redis,sqlite,postgres]

If you don't need redis or sqlite support, you may omit one of them in square brackets. If you don't need any of them and you plan to use internal cache without persistence, you should also omit square brackets.

Package scripts shall be available in standard executable locations upon completion.

pip user install

All pip invocations can be run with --user option of pip installer. In this case superuser privileges are not required and package(s) are getting installed into user home directory. Usually, script executables will appear in ~/.local/bin.

Method 2. System-wide install from project source

Run in project directory:

sudo python3 -m pip install .[redis,sqlite,postgres]

If you don't need redis, sqlite or postgres support, you may omit one of them in square brackets. If you don't need any of them and you plan to use internal cache without persistence, you should also omit square brackets.

Package scripts shall be available in standard executable locations upon completion.

Method 3. Install into virtualenv

See "Building virtualenv"

Method 4. Docker

Run

docker volume create mta-sts-cache
docker run -d \
    --security-opt no-new-privileges \
    -v mta-sts-cache:/var/lib/mta-sts \
    -p 127.0.0.1:8461:8461 \
    --restart unless-stopped \
    --name postfix-mta-sts-resolver \
    yarmak/postfix-mta-sts-resolver

Daemon will be up and running, listening on local interface on port 8461. Default configuration baked into docker image uses SQLite for cache stored in persistent docker volume. You may override this configuration with your own config file by mapping it into container with option -v my_config.yml:/etc/mta-sta-daemon.yml.

Method 5. Snap Store

Get it from the Snap Store

sudo snap install postfix-mta-sts-resolver

NOTE: in snap layout mta-sta-daemon program is named postfix-mta-sts-resolver.daemon and mta-sts-query is named postfix-mta-sts-resolver.query.

Common installation notes

See also contrib/README.md for RHEL/OEL/Centos and FreeBSD notes.

See contrib/ for example of systemd unit file suitable to run daemon under systemd control.

Running

This package provides two executables available after installation in respective locations.

mta-sts-query

mta-sts-query is a command line tool which fetches and outputs domain MTA-STS policies. Intended to be used for debug purposes.

Synopsis:

$ mta-sts-query --help
usage: mta-sts-query [-h] [-v {debug,info,warn,error,fatal}]
                     domain [known_version]

positional arguments:
  domain                domain to fetch MTA-STS policy from
  known_version         latest known version (default: None)

optional arguments:
  -h, --help            show this help message and exit
  -v {debug,info,warn,error,fatal}, --verbosity {debug,info,warn,error,fatal}
                        logging verbosity (default: warn)

mta-sts-daemon

mta-sts-daemon is a daemon which provides external TLS policy for Postfix SMTP client via socketmap interface.

You may find useful systemd unit file to run daemon in contrib/.

Synopsis:

$ mta-sts-daemon --help
usage: mta-sts-daemon [-h] [-v {debug,info,warn,error,fatal}] [-c FILE]
                      [-l FILE] [--disable-uvloop]

optional arguments:
  -h, --help            show this help message and exit
  -v {debug,info,warn,error,fatal}, --verbosity {debug,info,warn,error,fatal}
                        logging verbosity (default: info)
  -c FILE, --config FILE
                        config file location (default: /etc/mta-sts-
                        daemon.yml)
  -l FILE, --logfile FILE
                        log file location (default: None)
  --disable-uvloop      do not use uvloop even if it is available (default:
                        False)

Seamless restart/upgrade/reload and load balancing

By default mta-sts-daemon allows its multiple instances to share same port (on Linux/FreeBSD/Windows). Therefore, restart or upgrade of daemon can be performed seamlessly. Set of unit files for systemd in contrib/ directory implements "reload" by mean of running backup instance when main instance is getting restarted.

Also on Linux and FreeBSD, load distributed across all processes (with SO_REUSEPORT and SO_REUSEPORT_LB respectively).

MTA-STS Daemon configuration

See configuration man page and config_examples/ directory. Default config location is: /etc/mta-sts-daemon.yml, but it can be overridden with command line option -c FILE.

All options is self-explanatory, only exception is strict_testing option. If set to true, STS policy will be enforced even if domain announces testing MTA-STS mode. Useful for premature incorporation of MTA-STS against domains hesitating to go enforce. Please use with caution.

Postfix configuration

SMTP client of your Postfix instance must be able to validate peer certificates. In order to achieve that, you have to ensure smtp_tls_CAfile or smtp_tls_CApath points to system CA bundle. Otherwise you'll get Unverified TLS connection even for peers with valid certificate, and delivery failures for MTA-STS-enabled destinations. Also note: even enabled tls_append_default_CA will not work alone if both smtp_tls_CAfile and smtp_tls_CApath are empty.

Once certificate validation is enabled and your Postfix log shows "Trusted TLS connection ... " for destinations with valid certificates signed by public CA, you may enable MTA-STS by adding following line to main.cf:

smtp_tls_policy_maps = socketmap:inet:127.0.0.1:8461:postfix

If your configuration already has some TLS policy maps, just add MTA-STS socketmap to list of configured maps accordingly to smtp_tls_policy_maps syntax. TLS policy tables are searched in the specified order until a match is found, so you may have table with local overrides of TLS policy prior to MTA-STS socketmap. This may be useful for skipping network lookup for well-known destinations or relaxing security for broken destinations, announcing MTA-STS support.

Reload Postfix after reconfiguration.

Warning: MTA-STS policy overrides DANE TLS authentication

Due to Postfix's limitations, a resolved MTA-STS policy overrides DANE TLS authentication (RFC 6698), because DANE is an internal feature of Postfix, and the postfix-mta-sts-resolver always responds with a (smtp_tls_policy_maps) lookup result secure for Secure server certificate verification.

  • The resulting behaviour is against RFC 8461, 2:

    However, MTA-STS is designed not to interfere with DANE deployments when the two overlap; in particular, senders who implement MTA-STS validation MUST NOT allow MTA-STS Policy validation to override a failing DANE validation.

    Domains implementing both MTA-STS and DANE probably want DANE to be preferred:

    • DANE allows strict binding of certificates; the policy can authorize only a certain certificate or certificates from a certain CA. With MTA-STS, a certificate from any trusted CA is automatically trusted; RFC 8461, 10.1:

      SMTP MTA-STS relies on certificate validation via PKIX-based TLS identity checking [RFC6125]. Attackers who are able to obtain a valid certificate for the targeted recipient mail service (e.g., by compromising a CA) are thus able to circumvent STS authentication.

    • Based on DNSSEC, DANE not vulnerable to downgrade attack that could prevent policy discovery. MTA-STS security considerations acknowledges this weakness in RFC 8461, 10.2:

      Since MTA-STS uses DNS TXT records for policy discovery, an attacker who is able to block DNS responses can suppress the discovery of an MTA-STS Policy, making the Policy Domain appear not to have an MTA-STS Policy.

      Resistance to downgrade attacks of this nature -- due to the ability to authoritatively determine "lack of a record" even for non-participating recipients -- is a feature of DANE, due to its use of DNSSEC for policy discovery.

  • The postfix-mta-sts-resolver does not intent to implement policy lookups for DANE, and responses other than secure with match= would not verify the TLS certificate as required by RFC 8461, 4,2.

If you wish to meet this requirement:

  • List a DANE policy resolver responding with dane-only (for Mandatory DANE) before postfix-mta-sts-resolver in smtp_tls_policy_maps lookup table list.

  • Alternatively, you could use a static lookup table for domains known to implement both MTA-STS & DANE, e.g.,

    smtp_tls_policy_maps = hash:/etc/postfix/tls_policy,socketmap:inet:127.0.0.1:8461:postfix
    

Operability check

Assuming default MTA-STA daemon configuration. Following command:

/usr/sbin/postmap -q dismail.de socketmap:inet:127.0.0.1:8461:postfix

should return something like:

secure match=mx1.dismail.de

Postfix log should show Verified TLS connection established to ... instead of Untrusted ... or Trusted TLS connection established to ... when mail is getting sent to MTA-STS-enabled domain.

Special cases of deployment

Systems without Python 3.5+

Some people may find convenient to install latest python from source into /opt directory. This way you can have separate python installation not interfering with system packages by any means. Download latest python source from python.org, unpack and run in unpacked source directory:

./configure --prefix=/opt --enable-optimizations && make -j $[ $(nproc) + 1 ] && make test && sudo make install

Python binaries will be available in /opt/bin, including pip3. You may install postfix-mta-sts-resolver using /opt/bin/pip3 without interference with any system packages:

sudo /opt/bin/pip3 install postfix-mta-sts-resolver[sqlite,redis]

Executable files of postfix-mta-sts-resolver will be available in /opt/bin/mta-sts-query and /opt/bin/mta-sts-daemon

Building virtualenv

Run make in project directory in order to build virtualenv. As result of it, new directory venv shall appear. venv contains interpreter and all required dependencies, i.e. encloses package with dependencies in separate environment. It is possible to specify alternative path where virtualenv directory shall be placed. Specify VENV variable for make command. Example:

make VENV=~/postfix-mta-sts-resolver

Such virtual environment can be moved to another machine of similar type (as far python interpreter is compatible with new environment). If virtualenv is placed into same location on new machine, application can be run this way:

venv/bin/mta-sts-daemon

Otherwise, some hacks required. First option - explicitly call virtualenv interpreter:

venv/bin/python venv/bin/mta-sts-daemon

Second option - specify new path in shebang of scripts installed in virtualenv. It is recommended to build virtualenv at same location which app shall occupy on target system.

Credits

Inspired by this forum thread.

postfix-mta-sts-resolver's People

Contributors

acteru avatar bjoe2k4 avatar frederick888 avatar galme avatar jschauma avatar ldelouw avatar lxp avatar martb avatar mzch avatar oh2fih avatar rafork avatar sisheogorath avatar snawoot avatar timgates42 avatar tohojo avatar toxicvengeance avatar vexelnet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

postfix-mta-sts-resolver's Issues

MTA-STS fails with gmail.com

Sending mail to Google
It is not possible with mta-sts to send mail to Google, it is remaining in the mail queue.

To Reproduce

  1. Redis enabled MTA-STS installed using method 2 including Redis: postfix-mta-sts-resolver (1.0.1)
  2. Postfix mail_version = 3.3.0
  3. /usr/sbin/postmap -q gmail.com socketmap:inet:127.0.0.1:8461:postfix

secure match=.gmail-smtp-in.l.google.com:gmail-smtp-in.l.google.com servername=hostname

Expected behavior
It works with MS/Outlook but not with Google (see Output)

Output listings

  1. maillog
Steps to reproduce the behavior:
Oct 17 20:55:42 smtp.<myserver>.com postfix/smtp[22718]: warning: smtp_tls_policy_maps, next-hop destination "gmail.com": invalid attribute name: "servername"
Oct 17 20:55:42 smtp.<myserver>.com postfix/smtp[22718]: warning: TLS policy lookup for gmail.com/gmail-smtp-in.l.google.com: client TLS configuration problem

  1. mta-sts.log
2021-10-17 20:55:42 DEBUG    STS: Lookup skipped: domain = gmail.com
2021-10-17 20:55:42 DEBUG    STS: Future await complete: data=b'90:OK secure match=gmail-smtp-in.l.google.com:.gmail-smtp-in.l.google.com servername=hostname,'
2021-10-17 20:55:42 DEBUG    STS: Wrote: b'90:OK secure match=gmail-smtp-in.l.google.com:.gmail-smtp-in.l.google.com servername=hostname,'

Environment (please complete the following information):

  • OS: [GNU/Linux]
  • Distro: [Ubuntu]
  • Distro version: [18.04.6]
  • Python version: [3.6.9]

Support for redis sentinel

Is your feature request related to a problem? Please describe.
We would like to be able to connect to Redis sentinel cluster that we are already have in place for rspamd. But it looks like this is currently not implemented/supported. (Please correct me if I am wrong).

Describe the solution you'd like
The option to discover the current (r/w) master instance of Redis. It looks like this is possible using the library already in use. https://aioredis.readthedocs.io/en/latest/api/high-level/#aioredis.client maybe implemented via an alternative connection string.

Describe alternatives you've considered
At the moment, we can absolutely live with a single redis instance or sqlite for the cache, but still would be nice to have this.

Additional context
thanks for this project and your work!

Allow configuring the unix socket permissions

Is your feature request related to a problem? Please describe.

By default the socket is created with the permissions 0o755 on my system and postfix won't be able to connect since it runs as a different user. Putting the postfix user in the mta-sts group won't help either.

Describe the solution you'd like

I'd be nice to be able to configure the permissions via the config file.

Describe alternatives you've considered

For now I've added ExecStartPost=/usr/bin/chmod 666 socketfd to the systemd unit file. This works, but you have to configure the socket twice, once in the config, once in the systemd file.

Allow configuring the unix socket owner and group

Is your feature request related to a problem? Please describe.
Similar to #45 I'd like to implement finer access to the mta-sts-daemon's unix socket. Right now I must add Postfix to the primary group of the user running mta-sts-daemon to use more restricted permissions like 660 for the socket. However, this still gives Postfix more access than it actually needs, because Postfix can then not just access the socket, but any other group-accessible file - i.e. usually all files of the user running mta-sts-daemon.

Following the principle of least privilege (POLP), I'd like to change the unix socket's owner and group, not just its mode.

Describe the solution you'd like
Since chowning files isn't possible for unprivileged users, it must be possible to start mta-sts-daemon with root privileges and let the daemon drop its permissions to a configurable unprivileged user after creating the socket. As a side effect, this also allows users to bind to ports < 1024 without permanently running the daemon as root.

Accordingly there should be three more config options: Two options to set the unix socket's owner and group respectively, and another option to set the unprivileged user to run the daemon with. Since the naming of the config options will likely cause confusion (owner of the unix socket vs. unprivileged user running the daemon), I'd recommend to change the config structure as follows (also move host, port, and reuse_port to the listen_on section, but still support the old configs to preserve BC):

listen_on:
    path: /path/to/unix.sock
    user: mta-sts-sock
    group: mta-sts-sock
    mode: 0660
user: mta-sts

When running mta-sts-daemon as root, it will first create the socket /path/to/unix.sock (owned by mta-sts-sock:mta-sts-sock, mode 0600) and then drop its privileges to user mta-sts. When trying to start mta-sts-daemon as an unprivileged user, the daemon should bail, unless the executing user matches both user and listen_on.user.

Describe alternatives you've considered
In theory, one could add the mta-sts user to the mta-sts-sock group and then add ExecStartPost=/usr/bin/chown mta-sts-sock:mta-sts-sock /path/to/unix.sock to the Systemd unit file, but I didn't test it.

On a side note, when using the fork syscall for a native implementation as suggested before, the mta-sts user isn't required to be a member of the mta-sts-sock group, because the unprivileged child process would inherit the open socket from the privileged root process.

Additional context
A lot of ❤️ for your awesome project, keep up the great work, thank you! 👍

Notice for Postfix 3.4+ users

There is a bug in Postfix 3.4+ which makes Postfix unable to load the key+cert+chain.

Make sure you have smtpd_tls_eecdh_grade in it's default value auto or you have Postfix version which contains fix for this bug.

Client tls issue when using unix socket instead of inet tcp

Hi,

When i use the standard inet configuration everything works ok, the daemon runs ok and i can send/receive emails fine. /usr/sbin/postmap -q dismail.de socketmap:inet:127.0.0.1:8461:postfix returns the correct response.

When i configure the daemon to use a unix socket instead of inet tcp i get a client tls configuration error and i cannot send email.

-Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient-------
865A235FB 2902 Mon Jun 12 10:40:38 [email protected]
(client TLS configuration problem)
[email protected]

However when i run /usr/sbin/postmap -q dismail.de socketmap:unix:/var/spool/postfix/run/mta-sts/mta-sts-resolver.sock:postfix it returns the same response as the above inet command, indicating it's working ok.

My mta-sts daemon file has the below config parameters for the socket:
path: /var/spool/postfix/run/mta-sts/mta-sts-resolver.sock
mode: 0666

And my postfix main.cf has the line (when trying to use the unix socket instead of inet):
smtp_tls_policy_maps = socketmap:unix:/var/spool/postfix/run/mta-sts/mta-sts-resolver.sock:postfix
I've also tried just using run/mta-sts-mta-sts-resolver.sock as the path but got the same result.

I'm assuming it's a configuration problem but i'm not sure what i could have done wrong.

I'm using the inet socket for now as that is working ok.

Thank you for your time!

MTA-STS Overrides DANE

I don't know of a good domain to generate a test-case for, sadly, so apologies if this isn't true, but from my understanding of the integration flow, it seems like using this in smtp_tls_policy_maps would result in MTA-STS overriding DANE, which seems like a pretty severe security downgrade for domains which use both (eg protonmail). Doing a DANE lookup in postfix-mta-sts-resolver before returning MTA-STS results seems somewhat overcomplicated, so it would certainly be more ideal if postfix had a way to capture this policy in its config (ie "DANE-but-fall-back-to-verify"), but presuming both use the same (caching) DNS resolver and the TLSA record has a reasonable TTL, doing a DANE check first shouldn't have much of a performance hit nor be too brittle.

KeyError: 'url' with Debian Buster and python 3.7

Describe the bug
I installed mta-sts-daemon with sudo python3 -m pip install postfix-mta-sts-resolver[redis] and installed the systemd files.

If i run mta-sts-daemon on command line i get:

2022-05-30 20:28:25 INFO MAIN: MTA-STS daemon starting...
2022-05-30 20:28:25 INFO MAIN: Starting eventloop...
2022-05-30 20:28:25 INFO MAIN: uvloop enabled.
2022-05-30 20:28:25 INFO MAIN: Eventloop started.
Traceback (most recent call last):
File "/usr/local/bin/mta-sts-daemon", line 10, in
sys.exit(main())
File "/usr/local/lib/python3.7/dist-packages/postfix_mta_sts_resolver/daemon.py", line 123, in main
evloop.run_until_complete(amain(cfg, evloop))
File "uvloop/loop.pyx", line 1448, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.7/dist-packages/postfix_mta_sts_resolver/daemon.py", line 65, in amain
await cache.setup()
File "/usr/local/lib/python3.7/dist-packages/postfix_mta_sts_resolver/redis_cache.py", line 38, in setup
url = self._opts['url']
KeyError: 'url'

Expected behavior
I expected no crash ;)

Environment (please complete the following information):

  • OS: Linux
  • Distro: Debian
  • Distro version: Buster (10)
  • Python version: 3.7

dependencies unclear

setup.py and requirements.txt have diverging dependency information. It's not a problem for me, but perhaps it's an oversight and these should be kept in sync?

Email deliverability fails to protonmail.com

Describe the bug
I've installed postfix-mta-sts-resolver on my iRedMail email server but email deliverability fails when I send an email to ProtonMail, however I can send emails with a verified TLS connection to Gmail and Outlook.
I tried to install via pip and snap but both versions don't work, /etc/mta-sts-daemon.yml is missing, no systemd service is created and nothing is listening on port 8461 in both cases so I'm not sure if this is an issue with my installation method or an actual bug.

To Reproduce
Steps to reproduce the behavior:

  1. Install via apt sudo apt install postfix-mta-sts-resolver on Ubuntu 22.04 with iRedMail
  2. Add smtp_tls_policy_maps = socketmap:inet:127.0.0.1:8461:postfix into postfix main.cf
  3. Use postfix-mta-sts-resolver config provided below
  4. Send an email to protonmail

postfix-mta-sts-resolver config

host: 127.0.0.1
port: 8461
reuse_port: true
shutdown_timeout: 20
cache:
  type: internal
  options:
    cache_size: 10000
default_zone:
  strict_testing: false
  timeout: 20
proactive_policy_fetching:
  enabled: true

Expected behavior
I should be able to send emails to protonmail no problem, according to hardenize mta-sts is setup correctly for protonmail.com.

Output listings

Logs

May 26 22:06:22 mail postfix/smtp[80090]: CA certificate verification failed for mail.protonmail.ch[176.119.200.128]:25: num=2:unable to get issuer certificate
May 26 22:06:22 mail postfix/smtp[80090]: Untrusted TLS connection established to mail.protonmail.ch[176.119.200.128]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
May 26 22:06:22 mail postfix/smtp[80090]: 4QSNgT0FWpz9wJP: Server certificate not verified

mta-sts-query domain protonmail.com

(<STSFetchResult.NONE: 0>, None)

Environment (please complete the following information):

  • OS: [Ubuntu 22.04]
  • Python version: [Unknown]

TypeError

Describe the bug
In 1.1.0 TypeError occurred.

To Reproduce
Steps to reproduce the behavior:

  1. pip install postfix-mta-sts-resolver[redis,sqlite]
  2. mta-sts-daemon --config $HOME/daemon.yml --logfile $HOME/logs/daemon.log

Expected behavior
Run successfully.

Output listings

Traceback (most recent call last):
  File "/srv/mta-sts/.pyenv/versions/3.9.9/bin/mta-sts-daemon", line 8, in <module>
    sys.exit(main())
  File "/srv/mta-sts/.pyenv/versions/3.9.9/lib/python3.9/site-packages/postfix_mta_sts_resolver/daemon.py", line 123, in main
    evloop.run_until_complete(amain(cfg, evloop))
  File "uvloop/loop.pyx", line 1501, in uvloop.loop.Loop.run_until_complete
  File "/srv/mta-sts/.pyenv/versions/3.9.9/lib/python3.9/site-packages/postfix_mta_sts_resolver/daemon.py", line 65, in amain
    await cache.setup()
import json
import uuid
  File "/srv/mta-sts/.pyenv/versions/3.9.9/lib/python3.9/site-packages/postfix_mta_sts_resolver/redis_cache.py", line 39, in setup
    self._pool = await aioredis.from_url(**self._opts)
TypeError: from_url() missing 1 required positional argument: 'url'

Environment (please complete the following information):

  • OS: GNU/Linux
  • Distro: Debian
  • Distro version: 11.1
  • Python version: 3.9.9

Additional context

  • daemon.yml
host: 127.0.0.1
port: 8461
reuse_port: true
shutdown_timeout: 20
proactive_policy_fetching:
  enabled: true
cache:
  type: redis
  options:
    address: "redis://127.0.0.1:6379/5?timeout=5"
    minsize: 5
    maxsize: 25
default_zone:
  strict_testing: false
  timeout: 4
zones:
  mail-services.net:
    strict_testing: false
    timeout: 4
  • Fixed?
    I add a line to add url argument, it seems good...?
--- redis_cache.py.orig	2021-12-18 14:22:14.674578264 +0900
+++ redis_cache.py	2021-12-18 14:21:50.126583375 +0900
@@ -35,7 +35,8 @@
         self._pool = None

     async def setup(self):
-        self._pool = await aioredis.from_url(**self._opts)
+        url = self._opts['address']
+        self._pool = await aioredis.from_url(url, **self._opts)

     async def get(self, key):
         assert self._pool is not None

Is this correct?

No "Trusted TLS connection established to" message by default

As far as I know, most people run at the smtp_tls_security_level = may, and some at dane. As far as I know, that will never result in a "Trusted TLS connection established to" message. I get Untrusted and Anonymous connections, and Verified for the DANE enabled domains. Could you please update README.md to not suggest that people should already get Trusted connections?

mta-sts-query returns NONE instead of FETCH_ERROR when DNS error

Describe the bug
When invalid DNS server is set or if DNS server is not responding, error mta-sts-query returns STSFetchResult.NONE instead of STSFetchResult.FETCH_ERROR

To Reproduce
Steps to reproduce the behavior:

  1. Set invalid DNS server in /etc/resolv.conf
  2. Run for example mta-sts-query dismail.de
  3. Check command output

Expected behavior
(<STSFetchResult.FETCH_ERROR: 2>, None)

Output listings
(<STSFetchResult.NONE: 0>, None)

Environment (please complete the following information):

  • OS: GNU/Linux
  • Distro: Ubuntu
  • Distro version: 20.04
  • Python version: 3.8

Additional context
Error returned for DNS request:
(11, 'Could not contact DNS servers')

Version installed from Ubuntu repo.

# apt info postfix-mta-sts-resolver
Package: postfix-mta-sts-resolver
Version: 0.7.5-1
Priority: optional
Section: universe/mail
Origin: Ubuntu
Maintainer: Ubuntu Developers <[email protected]>
Original-Maintainer: Benjamin Hof <[email protected]>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 124 kB
Pre-Depends: init-system-helpers (>= 1.54~)
Depends: python3-aiodns, python3-aiohttp, python3-yaml, python3:any, adduser, ca-certificates, python3-aiosqlite, python3-aioredis
Recommends: python3-uvloop
Enhances: postfix
Homepage: https://github.com/Snawoot/postfix-mta-sts-resolver
Download-Size: 30.0 kB
APT-Manual-Installed: yes
APT-Sources: http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages
Description: Daemon providing MTA-STS map to Postfix

How to test successful postfix-mta-sts-resolver setup?

I just installed Sendmail 8.17.2 and postfix-mta-sts-resolver . I want to verity my setup, by sending to a site, which announces MTA-STS support but does not offer STARTTLS. The only site I found was https://mtasts.xyz/ , however its policy cannot be fetched, as the certificate for the web and smtp servers are expired. As such the policy is ignored. The site writes “Please send more suggestions so we can list them here!” and “If you know of anything else similar, please let us know!”, without saying how to contact the site owners.

Please extend the setup instructions for postfix-mta-sts-resolver, clarifying how the setup can be validated. E.g. by mentioning a misconfigured mail domain, which announces MTA-STS, but not not offer MTA-STS.

Support unix sockets for daemon

Right now, the config supports host and port options for the responder. It makes sense to also offer a path option for unix sockets. This wouldn't be too hard, as the only thing that really needs to be changed is the create_custom_socket function in utils.py and its call in responder.py`.

mta-sts-query documentation mentions verbosity, but it's not implemented

Describe the bug
mta-sts-query documentation mentions verbosity, but it's not implemented

client's ArgParse setup lacks -v, but daemon's clearly has it.

and using docker to run the client, -h output shows that -v is not implemented.

# docker exec postfix-mta-sts-resolver mta-sts-query -h
usage: mta-sts-query [-h] domain [known_version]

positional arguments:
  domain         domain to fetch MTA-STS policy from
  known_version  latest known version (default: None)

optional arguments:
  -h, --help     show this help message and exit

Probably the docs should be fixed first, and later the client could be enhanced to support debug logging.

Move to aioredis v2

Is your feature request related to a problem? Please describe.
From the docs of v2: As of December, 2019, this library had been abandoned

v2 started hitting repos of rolling release distros (e.g. Archlinux) a while ago. I'd prefer to install that instead of an outdated and unmaintained version from two years ago which conflicts with the current version unless docker or pyenv is used.

Describe the solution you'd like
make a clear brake and adapt to new aioredis API

Describe alternatives you've considered
support both aioredis versions

Additional context
I've managed to do the necessary changes in redis_cache.py. Please let me what solution is preferred so that I may prepare a pull request.

Postfix client TLS configuration problem when sending emails to google.com

Describe the bug
Postfix shows client TLS configuration problem when sending emails to google.com and postfix-mta-sts-resolver is enabled.

To Reproduce
Steps to reproduce the behavior:

  1. Enable postfix-mta-sts-resolver
  2. Send an email to google.com
  3. Postfix shows client TLS configuration problem and emails are deferred

Expected behavior
No errors.

Output listings
Postfix configuration:

smtp_tls_CAfile =
smtp_tls_CApath = /etc/ssl/certs
smtpd_tls_CAfile =
smtpd_tls_CApath =
smtp_tls_policy_maps = socketmap:inet:127.0.0.1:8461:postfix

postfix-mta-sts-resolver (Docker) configuration:

host: 0.0.0.0
port: 8461
reuse_port: true
shutdown_timeout: 20
cache:
  type: redis
  options:
    address: "redis://172.17.0.1/1?timeout=5"
    minsize: 2
    maxsize: 25
default_zone:
  strict_testing: false
  timeout: 4

Postfix logs:

Jun 22 10:27:40 Fred-Linode postfix/smtp[28944]: warning: smtp_tls_policy_maps, next-hop destination "gmail.com": invalid attribute name: "servername"
Jun 22 10:27:40 Fred-Linode postfix/smtp[28944]: warning: TLS policy lookup for gmail.com/gmail-smtp-in.l.google.com: client TLS configuration problem
Jun 22 10:27:40 Fred-Linode postfix/smtp[28944]: warning: TLS policy lookup for gmail.com/gmail-smtp-in.l.google.com: client TLS configuration problem
Jun 22 10:27:40 Fred-Linode postfix/smtp[28944]: warning: smtp_tls_policy_maps, next-hop destination "gmail.com": invalid attribute name: "servername"
Jun 22 10:27:40 Fred-Linode postfix/smtp[28944]: warning: TLS policy lookup for gmail.com/alt1.gmail-smtp-in.l.google.com: client TLS configuration problem
Jun 22 10:27:40 Fred-Linode postfix/smtp[28944]: warning: TLS policy lookup for gmail.com/alt1.gmail-smtp-in.l.google.com: client TLS configuration problem
Jun 22 10:27:40 Fred-Linode postfix/smtp[28944]: warning: smtp_tls_policy_maps, next-hop destination "gmail.com": invalid attribute name: "servername"
Jun 22 10:27:40 Fred-Linode postfix/smtp[28944]: warning: TLS policy lookup for gmail.com/alt2.gmail-smtp-in.l.google.com: client TLS configuration problem
Jun 22 10:27:40 Fred-Linode postfix/smtp[28944]: 5DA2A181C5: to=<[email protected]>, relay=none, delay=3.9, delays=0.76/0.01/3.2/0, dsn=4.7.5, status=deferred (client TLS configuration problem)

postfix-mta-sts-resolver logs:

2020-06-22 10:27:36 DEBUG    STS: len(self._children) = 1
2020-06-22 10:27:36 DEBUG    STS: Read: b'17:postfix gmail.com,'
2020-06-22 10:27:36 DEBUG    STS: Enq request: b'postfix gmail.com'
2020-06-22 10:27:36 DEBUG    STS: Got new future from queue
2020-06-22 10:27:36 DEBUG    STS: Lookup PERFORMED: domain = gmail.com
2020-06-22 10:27:36 DEBUG    RES: Got STS resolve request: sts_txt_domain=_mta-sts.gmail.com, known_id=20190429T010101
2020-06-22 10:27:40 DEBUG    RES: Parsed STS record for domain 'gmail.com': {'v': 'STSv1', 'id': '20190429T010101'}
2020-06-22 10:27:40 DEBUG    STS: Future await complete: data=b'90:OK secure match=.gmail-smtp-in.l.google.com:gmail-smtp-in.l.google.com servername=hostname,'
2020-06-22 10:27:40 DEBUG    STS: Wrote: b'90:OK secure match=.gmail-smtp-in.l.google.com:gmail-smtp-in.l.google.com servername=hostname,'
2020-06-22 10:27:40 DEBUG    STS: Read: b'17:postfix gmail.com,'
2020-06-22 10:27:40 DEBUG    STS: Enq request: b'postfix gmail.com'
2020-06-22 10:27:40 DEBUG    STS: Got new future from queue
2020-06-22 10:27:40 DEBUG    STS: Lookup skipped: domain = gmail.com
2020-06-22 10:27:40 DEBUG    STS: Future await complete: data=b'90:OK secure match=.gmail-smtp-in.l.google.com:gmail-smtp-in.l.google.com servername=hostname,'
2020-06-22 10:27:40 DEBUG    STS: Wrote: b'90:OK secure match=.gmail-smtp-in.l.google.com:gmail-smtp-in.l.google.com servername=hostname,'
2020-06-22 10:27:40 DEBUG    STS: Read: b'17:postfix gmail.com,'
2020-06-22 10:27:40 DEBUG    STS: Enq request: b'postfix gmail.com'
2020-06-22 10:27:40 DEBUG    STS: Got new future from queue
2020-06-22 10:27:40 DEBUG    STS: Lookup skipped: domain = gmail.com
2020-06-22 10:27:40 DEBUG    STS: Future await complete: data=b'90:OK secure match=.gmail-smtp-in.l.google.com:gmail-smtp-in.l.google.com servername=hostname,'
2020-06-22 10:27:40 DEBUG    STS: Wrote: b'90:OK secure match=.gmail-smtp-in.l.google.com:gmail-smtp-in.l.google.com servername=hostname,'
2020-06-22 10:27:50 DEBUG    STS: Client disconnected

Redis:

172.17.0.1:6379[1]> ZRANGE "google.com" 0 -1
1) "R\xc7\x11;T\xfcJ\x8c\xaa\xd5\xaa\x91\x0fx\x83\xc4[\"20190429T010101\", {\"mx\": [\"aspmx.l.google.com\", \"*.aspmx.l.google.com\"], \"version\": \"STSv1\", \"mode\": \"enforce\", \"max_age\": 86400}]"

Tests:

$ postmap -q dismail.de socketmap:inet:127.0.0.1:8461:postfix
secure match=mx1.dismail.de:mx2.dismail.de servername=hostname
$ postmap -q google.com socketmap:inet:127.0.0.1:8461:postfix
secure match=aspmx.l.google.com:.aspmx.l.google.com servername=hostname

Environment (please complete the following information):

  • OS: Linux
  • Distro: Ubuntu
  • Distro version: 18.04
  • Python version: 3.8 (I'm using yarmak/postfix-mta-sts-resolver ba7ca013cc35)
  • Postfix 3.3.0-1ubuntu0.2

Querying IDN domains causes an error

Describe the bug
When querying an IDN domain with utf-8 characters in them, the connection gets closed prematurely and an error gets logged.

To Reproduce
Steps to reproduce the behavior:

  1. Run a lookup with a domain that has an utf-8 character in it: /usr/sbin/postmap -q ätsch.de socketmap:inet:127.0.0.1:8461:postfix

Expected behavior
The domain should be converted to it's ascii counterpart (in this example xn--tsch-koa.de) and resolved correctly.

Output listings

Aug 20 18:52:28 proxy mta-sts-daemon[283]: 2022-08-20 18:52:28 ERROR    STS: Exception in sender coro: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128)
Aug 20 18:52:28 proxy mta-sts-daemon[283]: Traceback (most recent call last):
Aug 20 18:52:28 proxy mta-sts-daemon[283]:   File "/usr/lib/python3/dist-packages/postfix_mta_sts_resolver/responder.py", line 149, in sender
Aug 20 18:52:28 proxy mta-sts-daemon[283]:     data = await fut
Aug 20 18:52:28 proxy mta-sts-daemon[283]:   File "/usr/lib/python3/dist-packages/postfix_mta_sts_resolver/responder.py", line 167, in process_request
Aug 20 18:52:28 proxy mta-sts-daemon[283]:     req_zone, _, req_domain = raw_req.decode('ascii').partition(' ')
Aug 20 18:52:28 proxy mta-sts-daemon[283]: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128)
Aug 20 18:52:28 proxy mta-sts-daemon[283]: 2022-08-20 18:52:28 ERROR    STS: Exception in sender coro: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128)
Aug 20 18:52:28 proxy mta-sts-daemon[283]: Traceback (most recent call last):
Aug 20 18:52:28 proxy mta-sts-daemon[283]:   File "/usr/lib/python3/dist-packages/postfix_mta_sts_resolver/responder.py", line 149, in sender
Aug 20 18:52:28 proxy mta-sts-daemon[283]:     data = await fut
Aug 20 18:52:28 proxy mta-sts-daemon[283]:   File "/usr/lib/python3/dist-packages/postfix_mta_sts_resolver/responder.py", line 167, in process_request
Aug 20 18:52:28 proxy mta-sts-daemon[283]:     req_zone, _, req_domain = raw_req.decode('ascii').partition(' ')
Aug 20 18:52:28 proxy mta-sts-daemon[283]: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128)
Aug 20 18:52:28 proxy postfix/postmap[11247]: warning: table socketmap:inet:127.0.0.1:8461:postfix lookup error: unexpected disconnect
Aug 20 18:52:28 proxy postfix/postmap[11247]: fatal: table socketmap:inet:127.0.0.1:8461:postfix: query error: Operation now in progress

Environment (please complete the following information):

  • OS: GNU/Linux
  • Distro: Debian
  • Distro version: sid
  • Python version: 3.10

Policies for Smarthosts

Describe the bug
When a client is configured to use a smarthost/relay/relayhost for its mail delivery, the policy lookup (and validation) is not initiated.

To Reproduce
Steps to reproduce the behavior:

  1. configure postfix to use a relay (example config line in main.cf: relayhost = [myrelayhost.example.com]:123)
  2. attempt to send a valid email to someone/somewhere
  3. mail.log says Trusted TLS connection established (but not Verified as it should be) and there are no logs in mta-sts.log to signify a lookup on a domain for policies.

Expected behavior
The proposed MTA-STS standard (RFC 8461) section 3.4. Policy Selection for Smart Hosts and Subdomainsin section (link) states:

When sending mail via a "smart host" -- an administratively configured intermediate SMTP relay, which is different from the message recipient's server as determined from DNS -- compliant senders MUST treat the smart host domain as the Policy Domain for the purposes of policy discovery and application.

My understanding is that there should be a policy lookup and validation for the example.com domain in this case.

Note: had the configuration been without the brackets (like relayhost = myrelayhost.example.com:123), I would expect for myrelayhost.example.com to be looked up and validated (since myrelayhost isn't really a host then but rather a subdomain due to the allowed MX lookups from the absence of []).

Output listings
mta-sts.log:
STS: Enq request: b'postfix [myrelayhost.example.com]:123'

Environment (please complete the following information):

  • OS: Linux
  • Distro: Ubuntu
  • Distro version: 18.04.3 LTS
  • Python version: 3.6.9

Additional context
I found the short-circuit code responsible for this behaviour here: link. If I hack around this (extract the host's domain), the TLS connection shows as verified and the lookups & validations are done as expected.

P.S. Thank you for the work put into this project!

[CERT-Bund#2023092728001552] Vulnerability report regarding postfix and postfix-mta-sts-resolver

I came across some traces of a CERT-Bund vulnerability report:

From postfix-devel  Tue Oct 24 11:58:20 2023
From: Wietse Venema via Postfix-devel <postfix-devel () postfix ! org>
Date: Tue, 24 Oct 2023 11:58:20 +0000
To: postfix-devel
Subject: [pfx-dev] Re: [CERT-Bund#2023092728001552] Vulnerability report regarding postfix and postfix-mta-sts-resolver
Message-Id: <4SF9cx74KKzJrP1 () spike ! porcupine ! org>
X-MARC-Message: https://marc.info/?l=postfix-devel&m=169814870008296

see also:

Since I found nothing in the issues/pull-requests tracker/readme I'm wondering whether they managed to contact you and whether current postfix-meta-sts-resolver is vulnerable (if it was a valid report, in the first place).

shutdown_timeout undocumented

shuttdown_timeout is currently not documented.

Adding adding a description of the shutdown_timeout parameter would be helpful complete the documentation.

Make systemd dependency optional

Is your feature request related to a problem? Please describe.
systemd is only available on GNU/Linux systems, but users of other kernels / operating systems (e.g. FreeBSD) might still get benefits from this software.

Describe the solution you'd like
The sdnotify dependency added in #16 is unconditional: if sdnotify cannot be used, the daemon will not run. Making this an optional dependency would allow to run the software on FreeBSD unpatched.

Thanks !

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.