efforg / starttls-everywhere

A system for ensuring & authenticating STARTTLS encryption between mail servers

STARTTLS Everywhere

STARTTLS Everywhere is an initiative for upgrading the security of the email ecosystem. Several sub-projects fit underneath the general umbrella of "STARTTLS Everywhere". The name itself is a bit of a misnomer, since the original idea for the project came about in 2014, when STARTTLS support hovered around 20% across the internet. Since then we've come a long way, with Gmail's transparency report citing ~90% of inbound and outbound mail are encrypted with TLS, as of 2018.

We still have a long way to go. STARTTLS (opportunistic TLS) is vulnerable to trivial downgrade attacks that continue to be observed across the Internet. As of 2018, a quick Zmap query for a common STARTTLS-stripping fingerprint (a banner that reads "250 XXXXXXXX" rather than "250 STARTTLS") reveals around 8 thousand hosts. This is likely an under-estimate, since active attackers can perform the stripping in a less detectable way (simply by omitting the banner, for instance, rather than replacing its body with X's).

In addition, STARTTLS is also vulnerable to TLS man-in-the-middle attacks. Mailservers currently don't validate TLS certificates, since there has only recently been an attempt to standardize the certificate validation rules across the email ecosystem.

Absent DNSSEC/DANE, STARTTLS by itself thwarts purely passive eavesdroppers. However, as currently deployed, it allows either bulk or semi-targeted attacks that are very unlikely to be detected. We would like to deploy both detection and prevention for such semi-targeted attacks.

Goals

• Prevent TLS stripping from revealing email contents to the network, when in transit between major MTAs that support STARTTLS.
• Prevent active MITM attacks at the DNS, SMTP, TLS, or other layers from revealing contents to the attacker.
• Zero or minimal decrease to deliverability rates unless network attacks are actually occurring.
• Create feedback-loops on targeted attacks and bulk surveilance in an opt-in, anonymized way.

Non-goals

• Prevent fully-targeted exploits of vulnerabilities on endpoints or on mail hosts.
• Refuse delivery on the recipient side if sender does not negotiate TLS (this may be a future project).
• Develop a fully-decentralized solution.
• Initially we are not engineering to scale to all mail domains on the Internet, though we believe this design can be scaled as required if large numbers of domains publish policies to it.

Solution stacks

A solution needs the following things:

• Server can advertise TLS support and MX data
• In a non-downgrade-able way
• Minimize deliverability impact
• Widely deployed

DNSSEC, DANE, and TLSRPT

With DNSSEC and DANE for email, mailservers can essentially publish or pin their public keys via authenticated DNS records. If a domain has DNSSEC-signed their records, the absence/presence of a DANE TLSA record indicates non-support/support for STARTTLS, respectively.

Our goals can also be accomplished through use of DNSSEC and DANE, which is a very scalable solution. DANE adoption has been slow primarily since it is dependent on upstream support for DNSSEC; operators have been very slow to roll out DNSSEC. Making DNSSEC easier to deploy and improving its deployment is, for now, outside the scope of this project, though making DANE easier to deploy may be in-scope.

The mention of TLSRPT is due to the fact that several operators consistently deploy DNSSEC or DANE incorrectly. We want to close the misconfiguration reporting feedback loop. TLSRPT is an RFC for publishing a "reporting mechanism" to DNS. This endpoint can be an email address or a web endpoint; it is expected that senders will publish to these when failures occur, and that receivers will aggregate these reports and fix their configurations if problems arise.

• Server can advertise TLS support and MX data (DANE TLSA records)
• In a non-downgrade-able way (NSEC3 for DNSSEC)
• Minimize deliverability impact (TLSRPT, ideally)

MTA-STS, Preloading, and TLSRPT

MTA-STS is a specification for mailservers to publish their TLS policy and ask senders to cache that policy, absent DNSSEC. The policy can be discovered at a .well-known address served over HTTPS at the email domain (for instance, Gmail's record). MTA-STS support is discovered through an initial DNS lookup.

There is value in deploying an intermediate solution, perhaps through MTA-STS, that does not rely on DNSSEC. This will improve the email security situation more quickly. It will also provide operational experience with authenticated SMTP over TLS that will make eventual rollout of DANE-based solutions easier.

However, MTA-STS, unlike DNSSEC + DANE, is trust-on-first-use. Since MTA-STS assumes no DNSSEC, the initial DNS query to discover MTA-STS support is downgradable. A full solution would include distributing an MTA-STS preload list via our email security database.

• Server can advertise TLS support and MX data (MTA-STS)
• In a non-downgrade-able way (Preloading)
• Minimize deliverability impact (TLSRPT, ideally)

Project scope

The project scope is very large, though our development team is extremely small. The following is a list of things that we care about doing, and afterwards is a short-term timeline of the currently prioritized tasks.

If you are working next to or directly on one or more of these things, feel free to shoot us an email at [email protected].

Email security database (STARTTLS policy list)

• Usage guidelines (for configuration of sending mailservers)
• Validation guidelines (for configuration of receiving mailservers)
• Detailed spec of list format.
• Submission site for adding mail domains
• Maintaining and encouraging use of starttls-policy-cli.

Tracking and encouraging deployment of existing standards.

• DANE
  • Several other folks do a great job of monitoring DANE deployment and misconfigurations on the internet.
• MTA-STS
  • Encouraging MTA-STS validation support in popular MTA software.
  • Encouraging mailservers to publish their policies.
• TLSRPT
  • Encouraging reporting support in popular MTA software.
  • Encouraging mailservers to host reporting servers/endpoints.

Currently actively maintaining/building

• Maintaining the email security database and submission site
• Building out integrations for starttls-policy-cli
• Encouraging MTA-STS validation support in other MTA software.

Contributing

• Announcements mailing list: [email protected], https://lists.eff.org/mailman/listinfo/starttls-everywhere
• Developer mailing list: [email protected], https://lists.eff.org/mailman/listinfo/starttls-everywhere-devs
• IRC/dev channel: TBD
• We host a weekly development call every Thursday at 11AM Pacific Time. Shoot the mailing list an email if you're interested in joining or just sitting in.

starttls-everywhere's Issues

Re-enable merging policies from local configs

Mailserver operators should be able to provide their own local policy configurations that merge nicely with the global policy file.

This functionality was lost in the recent policy format refactors & changes-- let's bring it back!

SPF record for starttls-everywhere.org doesn't match probe IP address

starttls-everywhere.org publishes a record restricting it to a single IP address

$ dig +short starttls-everywhere.org txt
"v=spf1 ip4:67.212.170.242 -all"

This is not the address that the probes come from, but this server identifies in HELO as starttls-everywhere.org thus fails the SPF HELO check (RFC 7208).

Systems that normally require valid reverse DNS (which the checking IP address does not have) will sometimes fall back to an SPF HELO validation. But in the case of the probe, both fail.

Jun 25 20:53:22 util01 exim[12330]: 2018-06-25 20:53:21 no host name found for IP address 178.128.188.40
Jun 25 20:53:31 util01 exim[12330]: 2018-06-25 20:53:31 acl_local_auth_spf_helo: fail: Please see http://www.openspf.org/Why?id=postmaster%40starttls-everywhere.org&ip=178.128.188.40 : Reason: mechanism

Canonical URL in config.json returns 404

The comment in config.js with a canonical URL (https://eff.org/starttls-everywhere/config) for the latest config version returns a 404.

Is this intentional? Is there a better URL to include in the comment?

split data & code into separate repositories

This would be easier to package in the long run and has all sorts of other nice properties. We can also separate list linting from the CLI tool's CI.

letsencrypt-postfix/PostfixConfigGenerator.py Not Found

Certbot is installed, and is working fine on Ubuntu 16.04 x64.
Following the Directions:

git clone https://github.com/EFForg/starttls-everywhere
downloads it ok.

cd starttls-everywhere

letsencrypt-postfix/PostfixConfigGenerator.py examples/starttls-everywhere.json /etc/postfix /etc/letsencrypt/live/YOUR.DOMAIN.EXAMPLE.COM

Error:
-bash: letsencrypt-postfix/PostfixConfigGenerator.py: No such file or directory

Searched everywhere on the Server, Github, and Google, cannot find:
PostfixConfigGenerator.py

Update README

The README still strongly discurages to use this plugin on production systems.
As the last activity was over a year ago, I'm just wondering if the plugin maybe is now stable?
If yes, please update the README accordingly, so that newcomers are well informed.
Thanks a lot!

Wrong ruleset for bitmessage.org

There is a ruleset for bitmessage.com which has .org links in it.
That's kinda broken.. The right one is .org

Needs a source license

I reccomend GPL3

Vagrantfile broken `synched_folder`.

Running a vagrant up with the provided Vagrantfile barfs an error:

Bringing machine 'sender' up with 'virtualbox' provider...
Bringing machine 'valid' up with 'virtualbox' provider...
There are errors in the configuration of this machine. Please fix
the following errors and try again:

vm:
* The host path of the shared folder is missing: vagrant-shared/starttls-everywhere

This appears to be the result of a config.vm.synched_folder directive in the Vagrantfile without a corresponding folder on disk.

Based on the name I'm guessing that it's intended to be the starttls-everywhere src? It's not clear what should be placed in ./vagrant-shared/starttls-everywhere. It doesn't look like vagrant-bootstrap.sh touches anything in /vagrant/starttls-everywhere so maybe the offending Vagrantfile line needs to be deleted?

Sign list + specify verification process

Generate signature for the policy file using offline signing key.

This issue also includes prominently documenting how exactly consumers of STARTTLS Everywhere's policy list should verify the signature on the file, and against what public key. The update function in our Python tooling must also verify this signature.

investigate overlap with AME workshop outcome

There's been a workshop on automatic mail encryption (AME2016) in berlin last week. Unfortunately none of 'our' contributors attended. Hence we should investigate if there's possible overlap and interest in taking up some of their ideas and implementing them in certbot/an autoconfiguration tool in the future as AME projects evolve.

• workshop: https://github.com/mailencrypt/ame2016
• resulting projects:
  • INBOME (in-band opportunistic mail encryption):
    1. code: https://github.com/mailencrypt/inbome
    2. docs: https://inbome.readthedocs.io/en/latest/

Generate a transport policy only

Hi,

I have been interested in improving my work's outgoing mail setup to get above opportunistic encryption. I have been loosely following starttls-everywhere and I would love to be able to use your current lists for this purpose.

What makes me hesitant is that the current tooling seems to take over the entire postfix configuration, which I doubt we would very happy with. Notably, the particular SMTP server is not receiving mails from the internet.

• Do you have an example of how to only generate/maintain the transport map with the TLS rules?

object has no attribute 'Config'

It sounds like Python is mis-finding a Config.py module running Debian Sid with python2

# code_legacy/PostfixConfigGenerator.py examples/starttls-everywhere.json /etc/postfix /etc/letsencrypt/live/YOUR.DOMAIN.EXAMPLE.COM
Traceback (most recent call last):
  File "code_legacy/PostfixConfigGenerator.py", line 440, in <module>
    c = config.Config()
AttributeError: 'module' object has no attribute 'Config'
# find . -name Config.py
# 

Allow people to test outbound STARTTLS support

I've seen a few instances where admins have configured STARTTLS support for inbound connections but not enabled it for outbound connections. Outbound STARTTLS is not visible from a user perspective, and even if they have access to accounts on different servers, verification usually relies on being able to read an interpret trace headers.

It would be great if there were a tool to check outbound support, or at least some reference to this configuration issue in the faq, with a link to a tool such as checktls.com/TestSender.

Add TLS protocols to the policy list

It would be nice if the policy list included the versions of TLS supported. This would allow requiring TLS 1.2 (or TLS 1.3) for domains that support it, without needing to require TLS 1.2 for all domains in the policy list.

An implementation might be for the policy list to include the highest version of TLS supported. Then the postfix integration should invert that. For example, if a domain supports TLS 1.2 in the policy list, the Postfix policy map should have "protocols=!SSLv2:!SSLv3:!TLSv1:!TLSv1.1". In this way, downgrades are prevented, but the policy map does not preclude a newer version from being used. If the site starts supporting TLS 1.3, it can be instantly used without needing a policy update.

Standardize a dedicated SMTP-over-TLS port

STARTTLS is subject to attacks during opportunistic negotiation in-protocol, and MTA-STS just pushes the same concerns into the DNS and HTTPS stages. .well-known addresses only work because we standardized on port 443 for HTTPS. We should standardize a port with the numbering authorities for TLS-only SMTP connections so that e-mail can benefit from the same advantages that HTTPS currently enjoys.

golden-domains vs policy.json

We use the golden-domains to enforce secure connections.
The project starts to push rules to policy.json. I guess policy.json is the new source to be trusted. Any plans to migrate golden-domains to the policy.json?
I guess golden-domains will be abondend and not updated in the furure.

Add a "DANE" mode to the policy list

Some people want to tie their certificates to DANE TLSA DNS RRs. We should support this, by adding an optional "DANE" mode, which would indicate that the given recipient server's cert chains to DANE, not a CA root in some root program.

Clarify min-tls-version property

If the min-tls-version property is present, sending mail to domains under this policy should fail if the sending MTA cannot negotiate a TLS version equal to or greater than the listed version. Valid values are TLSv1, TLSv1.1, and TLSv1.2.

Assume, I pin my domain with TLSv1.2. Does that mean that old MTAs which only support TLS1.0, but use starttls-everywhere cannot contact me anymore?

Would it be a good idea then to define a local property max-tls-version which overrides min-tls-version globally? Otherwise it feels like I'm always going to use TLS1.0 to ensure maximal connectivity.

Add the front-end to sentry

Add Piwik support

Some README.md comments

1. In "File Format", the example JSON no longer matches the description. For instance, there is no explanation of the tls-policies field.
2. There is no explanation of what *.example.com refers to - presumably arbitrary-deep subdomains of example.com and example.com itself?
3. How does conflict resolution work if there exist conflicting policies (either tls-policies or acceptable-mxs) for *.eff.org and lists.eff.org?
4. "A user of this file format may choose to accept multiple files." - how does conflict resolution work in this case?
5. "enforce" and "log-only" appear to be mutually-exclusive values, but can domains that specify "enforce" also opt-in to logging? Or is this the default behavior? What if a domain doesn't specify the error-notification URL?
6. Can a domain specify multiple error-notification URLs? I think this would be useful.
7. Are there reasonable restrictions on the error-notification URL? Ex: should be HTTPS, same base domain as the URL that the policy applies to.
8. "The accept-pinset field references an entry in the pinsets list, which has the same format and semantics as Chrome's pinning list." - Chrome's pinning list at https://src.chromium.org/chrome/trunk/src/net/http/transport_security_state_static.json maps "static_spki_hashes" to a list of names for the hashes, not the hashes themselves (which is what "accept-spki-hashes" is). It is probably worth discussing explicitly how chain validation works - a cert is accepted if at least one of the "accept-spki-hashes" is found in the chain.
9. Chrome's pinning list now has a "bad_static_spki_hashes" value such that a cert only validates if none of these are in the chain. Is it worth adding something analagous here?

Validate reporting endpoints and MXs

Issue to track more validation for the JSON file: reporting endpoints should be mailto: or https: and mxs should be valid FQDNs.

Installation fails

I followed the steps in your README.md, but the installation fails on the following step:

$ pip install starttls-policy
Collecting starttls-policy
  Could not find a version that satisfies the requirement starttls-policy (from versions: )
No matching distribution found for starttls-policy

Reporting website issues

Please advise how I should report website issues?

The following https://starttls-everywhere.org/faq/#downgrades features "hthem":

"Mailserver operators would sign their MX records with DNSSEC, and publish a record alongside hthem containing the public key that the mailserver is expected to use in TLS."

Suggestion: Use NameCoin

I love this idea of utilizing Dane & DNSSEC but as you noted these issues have to be done out of band. Please consider using or adding the ability for namecoin to do the lookups. Namecoin allows anyone to cryptographic create TLD but additionally they can pin certificates under identities that is available to anyone who downloades the client and blockchain which is exactly what you want here.

https://wiki.namecoin.info/?title=Identity

Here is an implementation doing exactly what you want:

https://github.com/okTurtles/dnschain

Crontab for fetching updates

Should provide a sample crontab for fetching updates to the policy once a week.

Currently, the expiration date is set to one month from the update time, but we will eventually shorten this expiration to about 2x the expected time between client updates.

"update-only" option does not exist (anymore)

https://github.com/EFForg/starttls-everywhere/tree/master/starttls-policy#manually-updating-the-policy-list

Unclear policy specification

Current specification is too vague and contradictory. For example it has different key names for actual set of policies. Most likely it's traces of recent changes.

Outdated links

Specification refers to Chrome HSTS preload list and MTA-STS policy as starting point for understanding policy format, but links are outdated: first one is not accessible anymore, second is outdated because points to draft of recently accepted standard.

Key pinning

Most important: it doesn't clearly defines how implementer should conduct verifications and how to threat results. For example, pin defined as follows:

Should match a key of pinsets in the higher-level config. For a given pinset, a certificate for this mail domain should be accepted if at least one of the static_spki_hashes SPKIs is found in the trust chain.

Site sheds light:

We also accept requests to pin intermediate certificate public keys. Although this option gives operators flexibility in trust, key pinning carries higher risks of breakage and is more difficult to do correctly. As such, these requests will be judged on a case-by-case basis.

It is not clear: can I pin end entity cert? If so, is it OK to skip PKI certification path which precedes matching pinned key?

MTA-STS

mta-sts

Default: false

If set, then senders should expect this recipient domain to support MTA-STS.

What sender should do with his expectations? Especially, what sender should do with domains which included into STARTTLS-E list, but having mode: none MTA-STS policy? Why STARTTLS-E list has to interfere with MTA-STS or DANE at all?

I think it can use some polish.

add dev setup instructions

There don't seem to be instructions on how to set up a dev environment for STARTTLS Everywhere.

add support for WKS

If we opt to implement MTA-STS as is, we need to serve HTTPS, as a further enhancement we could add support for WKS (OpenPGP Web Key Service) which would allow users to automagically look up OpenPGP keys of a given e-mail address local to the MTA.

Implementation should be rather straight-forward. WKS key discovery roughly works like that:

For example the URI to lookup the key for [email protected] is:

  https://example.org/.well-known/openpgpkey/
  hu/iy9q119eutrkn8s1mk4r39qejnbu3n5q

(line has been wrapped for rendering purposes)

which is served simply via a /.well-known URL, where the account (e-mail address) is encoded as follows: z-base-32(SHA1(${address-local_part})).

---

• z-base-32: "human-oriented base-32 encoding"
• RFC5785: "Defining Well-Known Uniform Resource Identifiers (URIs)"

Please clarify relation of starttls-everywhere and MTA-STS

I hope this is the right place to raise this discussion. I'm confused about what the intention of STARTTLS Everywhere is in relation to MTA-STS.

In a blogpost in June here
https://www.eff.org/deeplinks/2018/06/technical-deep-dive-starttls-everywhere
EFF writes:
"In order to detect downgrade attacks, we’re hosting a policy list of mailservers that we know support STARTTLS. This list acts essentially as a preload list of MTA-STS security policies."

That sounds good. Yet from all I can tell this is not what's happening right now. The current STARTTLS Everywhere web page doesn't even mention MTA-STS.
When I try to submit a domain to the list I am asked to provide my MX hostnames. It provides no option to say "get the MX records from my MTA-STS policy file", which would be what an MTA-STS preload list would do.

I believe this way of operating the list is actively dangerous. Some people may start using the list without regularly updating it. So if you add your MX records you may end up in a situation where you can't change your MX records any more without breaking stuff.

Future transition to IETF MTA-MTA security standards and supporting DEEP

Hi,

There's currently a lot of work being done within UTA ("Utilising TLS in applications" - https://datatracker.ietf.org/wg/uta/documents) regarding mail security.

Strong contenders are:

• SMTP-STS (to be splitted into multiple documents)
• DEEP (MUA-MTA)

These standards are being actively worked on and aren't finished yet. They will be by around end of the year. Once they are RFCs and implemented in software, we should support these. Hence: we need to start thinking now about how we transition users from our current approach to these new security standards once they're deployment-ready.

I'm looking for comments, especially from @pde as he's intimately familiar with Certbot.

Aaron

Current postfix policy is useless

The current policy generates only a may. may means 'use TLS if possible' and is already the recommend default. This does still allow downgrade and MITM attacks.

Here are some more secure options:

example.com verify protocols=TLSv1.2
example.edu encrypt protocols=SSLv3 #at least not unencrypted
example.net verify match=mx #certificate does not match all MX hosts
example.org dane-only protocols=TLSv1.3

Is this usable?

The README doesn't clearly indicate whether this is an experimental idea, or something that can currently be used by domain admins (nor clearly indicate how to do so, if yes).

Could you update to reflect the current state of things?

Figure out what postfix does with simultaneous "encrypt" and a cert Subject name constraint

List signature on website points to an invalid URL

Hey, not sure if this is the right place to raise this issue, please redirect me somewhere if it isn't.

On https://starttls-everywhere.org/policy-list/ the link for the signature in

The list is hosted here, and the corresponding signature is here.

points to https://dl.eff.org/starttls-everywhere/policy.jsona.asac which has two weird as in it and throws a 404 not found. I guess this is expected to point to https://dl.eff.org/starttls-everywhere/policy.json.asc

Capitalization and comparing cert names

I thought domain names were case insensitive [1], so I created some test records:
turtle.chocovax.net. 300 IN MX 5 TURTLE.CHOCOVAX.NET.
turtle.chocovax.net. 300 IN MX 10 aspmx2.googlemail.com.
turtle.chocovax.net. 300 IN MX 1 ASPMX.L.GOOGLE.COM.

Not sure how this website (caches?) results, but the 1 and 10 should present valid certificates, the 10 is presented as 'aspmx2.googlemail.COM.' on https://starttls-everywhere.org/results/?turtle.chocovax.net they both say 'In order for your certificate to be valid for your email domain, it should be unexpired, chain to a valid root, and one of the names on the certificate should … match the … the server’s hostname (the name of the server, as indicated by an MX record).

[1] except for DKIM txt records (and maybe others?)

Disabling of SSLv3 transport support within LE/starttls-everywhere

Hi,

[ SSLv2 will also be disabled - no discussion there, see www.drownattack.com ]

During a call @pde, @dmwilcox and myself decided to move forward with completely removing SSLv3 support in our supported software mail daemons. In the past there used to be a lot of discussion if disabling of older protocol versions causes plaintext fallback with SMTP transfer. We just had a look at these two different sources of information..

1. http://arxiv.org/pdf/1510.08646v2.pdf (Table III, page 6)
2. http://conferences2.sigcomm.org/imc/2015/papers/p27.pdf (Table 2, page 4)

This suggests that SSLv3 is rarely used in the real world with as little as 0.2% of IPv4-connected servers on the Internet offering exclusive support for it and as little as 0.06% establishing connections with this particular protocol version with Google's SMTP servers. The Google dataset further highlights that only a single SSLv3 cipher-suite is established, namely: 0x5 (TLS_RSA_WITH_RC4_128_SHA) which, in addition, should be considered insecure because it uses RSA as key-exchange method and RC4 as stream cipher for encryption. While attacks on RC4 - currently - do not affect typical SMTP mail transfer, we've seen these attacks improving a lot over a very short time span. RFC7525 explicitly forbids the use of RC4 cipher-suites in TLS, we should adopt this recommendation here with disabling SSLv3 in general. Added to that, IETF published RFC7568 that prohibits use of SSLv3 in online communication (it's draft was aptly named draft-ietf-tls-sslv3-diediedie).

Please discuss. Pull requests forthcoming.

Switching File Formats

JSON with comments is nice, but not an approved standard - we should probably switch to something that's universally used and maybe a bit more terse in the future (low priority).

What do you think? @pde @dmwilcox

Suggestions?

Aaron

Honor reporting endpoints

Like we mention in the spec, policies can specify a reporting endpoint during their testing phase. Here's an issue to track progress on how we'll actually aggregate and send failure reports to those endpoints.

Add post-install check for open-relay in running-config

It might be a good idea to write a short script that runs a few local SMTP connect-checks if the configuration permits, in any way, open-relaying. Which will cause mail servers to be blacklisted within hours and for long periods of time in certain cases.

I need to write such a script anyway, will contribute and integrate once I have something ready.

Support for CAA DNS RRs

Hi,

RFC6844 defines a method by which a domain owner can restrict the CA that's allowed to issue certificates for their domain. AFAICT checking for this is not yet implemented by any mail daemons. In combination with MTA-STS this would make a lot of sense. I will bring this up in the relevant IETF WGs to get more input but I appreciate community input in here as well.

/cc @pde @dmwilcox

Remove pinsets (prev: Discuss and clarify key-pinning mechanism)

As @Snawoot mentions in issue #99,

it doesn't clearly defines how implementer should conduct verifications and how to threat results.
It is not clear: can I pin end entity cert? If so, is it OK to skip PKI certification path which precedes matching pinned key?

The key-pinning in STARTTLS Everywhere has been in an odd "limbo" state for quite a while. Removing the capability entirely was discussed at some point in the past-- do we want to repeat the HPKP debacle that web faced? I can see this feature breaking more things than it solves, but it's true that without DANE, there isn't a way to pin trust in email. If we want to keep it, it's time to clarify this behavior.

Move main.cf security settings into the 'enhance' framework, and make them user controllable

At the moment we set global postfix settings in wrangle_existing_config. Probably those should be set and called as enhancements in the iInstaller APIs (enhance and supporeted_enhancements).

Set up CI

to lint the list and run unit tests.

`PostfixConfigGenerator.py` fails on Ubuntu 12.04.5 because of missing `main.cf` file

The Postfix package in Ubuntu 12.04.5 doesn't seem to contain a main.cf file, thus PostfixConfigGenerator.py currently fails:

(precise)azet@localhost ..tarttls-everywhere/letsencrypt-postfix (git)-[master] % ./PostfixConfigGenerator.py ../examples/starttls-everywhere.json /etc/postfix /etc/letsencrypt/live/example.com
<subprocess.Popen object at 0x1fa5ed0>
Traceback (most recent call last):
  File "./PostfixConfigGenerator.py", line 293, in <module>
    pcgen.deploy_cert("example.com", cert, key, chain, fullchain)
  File "./PostfixConfigGenerator.py", line 188, in deploy_cert
    self.wrangle_existing_config()
  File "./PostfixConfigGenerator.py", line 67, in wrangle_existing_config
    self.raw_cf = open(self.fn).readlines()
IOError: [Errno 2] No such file or directory: '/etc/postfix/main.cf'
1 (precise)azet@localhost ..tarttls-everywhere/letsencrypt-postfix (git)-[master] % lsb_release -a                                                                                          :(No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 12.04.5 LTS
Release:        12.04
Codename:       precise
(precise)azet@localhost ..tarttls-everywhere/letsencrypt-postfix (git)-[master] % file /etc/postfix/main.cf/etc/postfix/main.cf: ERROR: cannot open `/etc/postfix/main.cf' (No such file or directory)
1 (precise)azet@localhost ..tarttls-everywhere/letsencrypt-postfix (git)-[master] % ls /etc/postfix                                                                                         :(dynamicmaps.cf  master.cf  post-install  postfix-files  postfix-script  sasl

@pde mentioned a userbase of about 3%, so we might want to support that version.

This is the package in question: http://packages.ubuntu.com/precise/postfix

Postfix chokes on our TLS pins

After trying this out on a first real mailserver (woohoo!) I'm seeing delivery failures to gmail (less awesome). Log entries are:

Feb 17 20:28:01 ip-172-31-50-10 postfix/pickup[24437]: 777D0425EA: uid=1000 from=<ubuntu>
Feb 17 20:28:01 ip-172-31-50-10 postfix/cleanup[24935]: 777D0425EA: message-id=<[email protected]>
Feb 17 20:28:01 ip-172-31-50-10 postfix/qmgr[24436]: 777D0425EA: from=<[email protected]>, size=771, nrcpt=1 (queue active)
Feb 17 20:28:01 ip-172-31-50-10 postfix/smtp[25002]: warning: smtp_tls_policy_maps, next-hop destination "gmail.com": malformed attribute/value pair "!SSLv3": missing '=' after attribute name
Feb 17 20:28:01 ip-172-31-50-10 postfix/smtp[25002]: connect to gmail-smtp-in.l.google.com[2607:f8b0:400d:c07::1b]:25: Network is unreachable
Feb 17 20:28:01 ip-172-31-50-10 postfix/qmgr[24436]: warning: private/smtp socket: malformed response
Feb 17 20:28:01 ip-172-31-50-10 postfix/qmgr[24436]: warning: transport smtp failure -- see a previous warning/fatal/panic logfile record for the problem description
Feb 17 20:28:01 ip-172-31-50-10 postfix/master[7332]: warning: process /usr/lib/postfix/smtp pid 25002 killed by signal 11
Feb 17 20:28:01 ip-172-31-50-10 postfix/master[7332]: warning: /usr/lib/postfix/smtp: bad command startup -- throttling
Feb 17 20:28:01 ip-172-31-50-10 postfix/error[25005]: 777D0425EA: to=<[email protected]>, relay=none, delay=0.26, delays=0.01/0.24/0/0.01, dsn=4.3.0, status=deferred (unknown mail transport error)
Feb 17 20:29:49 ip-172-31-50-10 postfix/anvil[24933]: statistics: max connection rate 1/60s for (smtp:209.85.214.172) at Feb 17 20:26:29
Feb 17 20:29:49 ip-172-31-50-10 postfix/anvil[24933]: statistics: max connection count 1 for (smtp:209.85.214.172) at Feb 17 20:26:29
Feb 17 20:29:49 ip-172-31-50-10 postfix/anvil[24933]: statistics: max cache size 1 at Feb 17 20:26:29

support postscreen

I am not sure if this is the right place to mention this, please close this issue if its not relevant.

I tried the test tool (https://www.starttls-everywhere.org/) with a few domains under my control and the report came back all wrong.

Apparently, the tool doesn't behave very well when postscreen (a feature of postfix) is enabled. Here are the relevant postscreen options that I use:

postscreen_blacklist_action = drop
postscreen_cache_cleanup_interval = 24h
postscreen_cache_retention_time = 30d
postscreen_command_time_limit = 10s
postscreen_greet_action = enforce
postscreen_greet_wait = 15s
postscreen_dnsbl_action = enforce
postscreen_dnsbl_threshold = 3

I am guessing that the long greet delay of 15 seconds, does not allow the tool the properly detect starttls.

Thank you.

MTAConfigGenerator.py fixes TLS version instead of setting a minimum

In Postfix TLS policy tables to enforce at least TLSv1, the syntax is:
protocols=!SSLv2:!SSLv3

Similarly to enforce at least TLSv1.1:
protocols=!SSLv2:!SSLv3:!TLSv1

Avoid "protocols=TLSv1.1" or "protocols=TLSv1" as these rule out "TLSv1.2" and possibly also "TLSv1.1".

See: http://www.postfix.org/postconf.5.html#smtp_tls_protocols