The kumomta from kumocorp

Tasks

Beta Give feedback

DKIM validation #82

receiving
SPF validation #83

enhancement receiving
DMARC support #84

Needs Funding enhancement receiving
Options

TLSRPT Report Generation

TLSRPT: https://datatracker.ietf.org/doc/html/rfc8460

Add logging for Outbound TLS

We should add these logging items

The TLS version that was negotiated with the remote end during STARTTLS
The TLS cipher that was negotiated with the remote end during STARTTLS

Prebuilt Ability to re-route messages to a different KMTA instance.

As a MailOps admin, I need to be able to move a queued message between nodes in my cluster to ensure any given message can reach the node that can handle its delivery.

Smarthost routing should resolve to the the quoted domain without a training "dot"

When using a "smarthost" route, it seems required to add a trailing dot to prevent resolving to the local cloud host.

In my test, i needed to resolve this issue by adding a training dot to the domain. The domain has an A record but not an MX record, but I thought that would be fine since A would be a fallback from a missing MX anyway.

This is in the config:

kumo.on('smtp_server_message_received', function(msg)
     msg:set_meta('queue', 'az.ksink.aasland.com')
end)

and I can resolve the address with a ping:

PING az.ksink.aasland.com (20.9.144.182) 56(84) bytes of data.

But...

DEBUG     localset-4 insert{name="az.ksink.aasland.com"}:insert: kumod::queue: insert_ready: failed to resolve az.ksink.aasland.com: no record found for Query { name: Name("az.ksink.aasland.com.qjcierrxmllupc5mn3fhg4m1uf.xx.internal.cloudapp.net."), query_type: A, query_class: IN }

In order to fix this I needed to do this: (notice the training dot)

     msg:set_meta('queue', 'az.ksink.aasland.com.')

Users should not need to add the dot.

RabbitMQ Event Publishing

Incoming TLSRPT Support

Kafka Integration

Lone LF in Body Handling

Depends On:

#17

Lone LF in Headers Handling

Depends On:

#17

Outbound HTTP delivery support

CLI

Automated adjustment of messages per hour.

As a Deliverability Engineer I have to handle responses from remote MTAs that indicate our server is sending too quickly.

For example:

451 4.7.650 The mail server [x.xx.xx.xx] has been temporarily rate limited due to IP reputation. For e-mail delivery information, see https://postmaster.live.com (S843)[#.prod.protection.outlook.com]

I need to throttle traffic down, wait an appropriate period of time to see if the response stops being returned, then try and throttle down again.

While there are ISP responses to say I need to throttle down, there are no anticipated responses to tell me to throttle up, so I either need to make the throttle down a temporary measure, or be able to define a cadence at which the server should attempt to throttle back up, likely based on age of the IP or time since the last throttle down response.

Postgres Support

Add Granular Queue Depth to Metrics API

Webhook Support

As a DevOps engineer, I need to be able to integrate my MTA's event data against internal systems without having to poll and read the logs. Our platform currently has webhook ingestion and I want to use it with my KumoMTA servers.

As a cloud relay provider, I need to provide my tenants with webhook event data, so that they don't have to constantly visit my API. I need to dynamically configure a large quantity of webhooks on a per-tenant basis, with each tenant able to select which events they receive in their webhook.

HAProxy Support

Automated Suspension of a pathway.

As a Deliverability Engineer, there are numerous responses I can encounter that are best addressed by suspending my traffic on a given pathway for a period of time.

One example of this is the common Yahoo TS04 error:

421 4.7.0 [TSS04] Messages from 0.0.0.0 temporarily deferred due to user complaints - 4.16.55.1; see https://postmaster.yahooinc.com/error-codes

When such an error is encountered, the MTA needs to stop sending over that IP to the Yahoo MX group for two hours, as per Yahoo's recommendation.

Add option to force fsync per-message.

As an admin using deferred spooling, I need a way to flag a message for immediate fsync, so that I don't lose critical messages in the event of a server failure.

While RocksDB gives the safety of a disk spool with the performance of memory spooling, many will opt for disk spooling with deferred spooling, but will have high-priority messages that must be spooled immediately. We need a way to flag those messages on injection to ensure an immediate spool to disk.

MailWizz Integration

Max Line Length Violation Handling

Depends On:

#17

Add logging for inject and delivery types

We should add these logging items

source type from which the message was received, either api or smtp
delivery method, one of smtp, HTTP, etc (Or whatever protocol we use to deliver the message)

MySQL Support

SMTP Trace Logging

Mautic Integration

Clustered Traffic Shaping

DANE Support

DANE for SMTP: https://datatracker.ietf.org/doc/html/rfc7672
DANE in General: https://datatracker.ietf.org/doc/html/rfc6698

TLSA resolution psuedocode: https://datatracker.ietf.org/doc/html/rfc6698#appendix-B.2

Implementing this requires solid DNSSEC support, and I'm not sure whether it currently exists.
Relevant issues in the upstream resolver library:

In addition, rustls doesn't have first-class support for DANE in general:

rustls/rustls#816

and it isn't clear how/if we can adapt the data and feed it into it verifier implementation

KumoProxy - SOCKS5 Proxy

While HAProxy support is a good step toward our users being able to leverage more IPs than can typically be allocated on a single instance, it remains limited as a solution due to how it handles connection issues.

We will improve on the current functionality available via HAProxy by implementing a more tightly integrated proxy using SOCKS5.

Consider building off:

https://github.com/dizda/fast-socks5
https://github.com/ajmwagar/merino
https://github.com/saiko-tech/mmproxy-rs/ (this has some more sophisticated piping/splicing that should have better throughput)

MTA-STS Support

MTA-STS: https://datatracker.ietf.org/doc/html/rfc8461

The gist for us is:

We need to manage a persistent cache of policies; this is not really optional like most caches. The purpose is to resist various attacks as described in rfc8461. In particular, we need to refresh the cached policy at an unpredictable time before the policy expires, in order to resist timing based attacks.
Fetching the policy itself appears to be straightforward; resolve a TXT record, and if it looks valid, perform an HTTPS get from a well-known endpoint, parse and cache.
Introduce an enable_mta_sts option in kumo.make_egress_path. This should default to true. Setting it to false will skip resolving the MTA STS policy, and is present as a kill-switch in case something goes wrong.
In the smtp dispatcher, we will use the content of the policy to effectively override the enable_tls setting, but additionally applying MX host validation to the MX names as described in https://datatracker.ietf.org/doc/html/rfc8461#section-4.1. Failed hosts are treated as transient connection failures and allow the dispatcher to continue with the next candidate MX.

#15

OnGage Integration

Automated Handling of ISP Responses

As a Deliverability Engineer, I need to comply with the feedback being provided by the remote MTAs I connect to in order to maintain deliverability and sender reputation.

For example, if the server received the following response:

451 4.7.650 The mail server [x.xx.xx.xx] has been temporarily rate limited due to IP reputation.

The server would want to lower the rate of sending on the IP address for a window of time.

This feature is composed of a number of elements as serves as the master ticket.

Ensure logs are flushed before shutdown

When shutting down the service, logs should write out any buffered entries before allowing the service to actually shut down.

Add event for message loading from spool

Dynamic source configuration

RFC 2549 Support

KumoMTA is for all use cases, which means having wide support for all RFCs pertaining to data transmission.

Lack of RFC2549 support is a glaring omission in all other MTAs, whether commercial or Open Source.

Non-Compliant Recipient Address Handling

Depends On:

#17

Configuration of traffic shaping automation rules.

As a Deliverability Engineer, I need to be able to record the rules I want used for my traffic shaping automation.

I would like to know that I can start from a default set of rules that are updated by the community, but can then augment/replace existing rules with logic I have created from my own research and experience.

While I would like to be able to manage those rules through a UI that is either provided or developed myself, I typically expect that the rules do not change on a very regular basis and therefore am fine with manually editing a file with my rules contained in it.

I want to be able to define rules based on a site name that represents a rollup of all domains that deliver to the same MX servers, but also may need to explicitly define rules for a domain in spite of it being a member of a given site name.

For each rule, I want to be able to specify the response in question and one or more actions to take as a result of the response.

I should be able to leave comments in the rules file so that I can reference those comments in the future to determine the logic used by myself, my peers, or my predecessors.

Add logging for Source IP

We should report on the IP address of the egress source. Currently can report on the name only.

Logging currently includes egress_source. We should be able to report egress_source.name and egress_source.ip

Add DKIM explainer diagram

We should add a DKIM explainer diagram. Like this:

Pattern Matching of Remote Responses

As a Deliverability Engineer, I need to be able to define the events I'm interested in automating, based on the response returned by the remote MTA.

For this to work, I need to be able to define the response string of interest using expressions, because remote servers can return customized responses that are personalized to the individual interaction.

For example, I may receive a response such as:

451 4.2.0 68.69.14.27 Throttled - try again later. SyEWdGKFD38hOSyEbdIFU1 Please see [http://postmaster.comcast.net/smtp-error-codes.php#RL000002](http://postmaster.comcast.net/smtp-error-codes.php#RL000002%C2%A0)

In this case, the random string is not of interest, and the IP address is specific to the individual connection. My real interest is in the initial IANA response code and the RL000002 category code.

Missing Header Handling

Depends On:

#17

kumocorp / kumomta Goto Github PK

kumomta's Introduction

KumoMTA

Documentation

Community

Developers

Reporting Bugs

Getting Help

Talk to Us

kumomta's People

Contributors

Stargazers

Watchers

Forkers

kumomta's Issues

Tasks

Recommend Projects

Recommend Topics

Recommend Org