trumail / trumail

:email: :white_check_mark: A Fast and Free Email Verification API written in Go

Trumail

Trumail is a free and open source email validation/verification system. It is available in three forms, the Golang client library verifier for use in your own Go projects, a public API endpoint (more info: https://trumail.io), and a public Docker image on DockerHub (see: https://hub.docker.com/r/sdwolfe32/trumail/).

NOTE: While we do offer a managed, enterprise level service to paying customers, it is highly recommended that you host the service yourself either using a Docker image or by forking and serving this project on your own instance. Please keep in mind, self-hosting Trumail requires bidirectional communication on port 25 which most residential ISPs restrict - AWS and Digitalocean both permit this sort of communication.

Using the API (public or self-hosted)

Using the API is very simple. All that's needed to validate an address is to send a GET request using the below URL with one of our three supported formats (json/jsonp(with "callback" (all lowercase) queryparam)/xml).

https://api.trumail.io/v2/lookups/{format}?email={email}&token={token}

Using the library

package main

import (
	"log"

	trumail "github.com/sdwolfe32/trumail/verifier"
)

func main() {
  v := trumail.NewVerifier("YOUR_HOSTNAME.COM", "[email protected]")
  
  // Validate a single address
  log.Println(v.Verify("[email protected]"))
}

Running with Go

go get -d github.com/sdwolfe32/trumail/...
go install github.com/sdwolfe32/trumail
trumail

Running with Docker

docker run -p 8080:8080 -e [email protected] sdwolfe32/trumail

How it Works

Verifying the deliverability of an email address isn't a very complicated process. In fact, the process Trumail takes to verify an address is really only half that of sending a standard email transmission and is outlined below...

First a TCP connection is formed with the MX server on port 25.

HELO my-domain.com              // We identify ourselves as my-domain.com (set via environment variable)
MAIL FROM: [email protected]     // Set the FROM address being our own
RCPT TO: [email protected] // Set the recipient and receive a (200, 500, etc..) from the server
QUIT                            // Cancel the transaction, we have all the info we need

As you can see we first form a tcp connection with the mail server on port 25. We then identify ourselves as example.com and set a reply-to email of [email protected] (both these are configured via the SOURCE_ADDR environment variable). The last, and obviously most important step in this process is the RCPT command. This is where, based on the response from the mail server, we are able to conclude the deliverability of a given email address. A 200 implies a valid inbox and anything else implies either an error with our connection to the mail server, or a problem with the address requested.

The BSD 3-clause License

Copyright (c) 2018, Steven Wolfe. All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

• Neither the name of Trumail nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

trumail's Issues

Few Questions

Hey @sdwolfe32 😊

Thanks for making such script. I have few questions:

1. If I install on my server am I dependent on your or any other service?
2. Do I have to pay or I can contribute some as per my reach?
3. Does this script can be installed on CentOS 6 server?

Thanks

Number of requests

Stabilize and document API response schemas

It would be great if it'd be possible to stabilize, document and freeze schemas for successful and error API responses at least per version.

There were a few changes lately that broke API consumers. I.e.

• hostExists field was removed and then reinstated a few days later
• gravatar field removed
• the error response changed

Thank you so much for your effort, time and creating a great service.

getJSON Method - Unexpected token :

Using AJAX getJSON method to validate email addresses and I am receiving the following syntax error

jQuery.ajax({url: "https://trumail.io/json/"+addr, dataType: 'jsonp', success: function(result){ console.log("Stuff is here: "+result); }});

Uncaught SyntaxError: Unexpected token :

Result from api and library not sample

I tried some email address on https://trumail.io/ and run example

func main() {
  v := trumail.NewVerifier("YOUR_HOSTNAME.COM", "[email protected]")
  
  // Validate a single address
  log.Println(v.Verify("[email protected]"))
}

Result I got doesn't sample.
Almost gmail it's OK but yahoo mail or hot mail or aol mail log null

Whats about proxy?

Hi

What do you think about proxy support? Maybe TOR?

False Yahoo Results

Hey,

Gmail/Outlook/Hotmail/Aim seem to work fine but for Yahoo it seems to give false reports like the below:

{ "address": "[email protected]", "username": "dsgdfh23.sdfg", "domain": "yahoo.com", "hostExists": true, "deliverable": true, "fullInbox": false, "catchAll": true, "disposable": false, "gravatar": false }

That email address is not valid but your script is showing it as valid?

@ character can be url encoded

This works

curl -s "https://api.trumail.io/v1/json/super%23user@example" | python -m json.tool
{
    "address": "super#user@example",
    "username": "super#user",
    "domain": "example",
    ...
}

but this is broken

curl -s "https://api.trumail.io/v1/json/super%23user%40example" | python -m json.tool
{
    "address": "super%23user%40example",
    "username": "",
    "domain": "",
    ...
}

Since the email is being passed in the url itself (which is convenient in a lot of cases) the API probably shouldn't make assumptions which part of the url can or can't be url encoded.

Connection reset by peer

Hi, I've set up the server and got «connection reset by peer» trying to verify @amazon.com email. Other domains are working. The whole response is «read tcp 160.96.41.64:48768->52.94.124.7:25: read: connection reset by peer». Can you please help me figure this out?

404 page not found on heroku.com

Hi,

Many congratulations for this work.

It can be an opensource alternative to neverbounce.com, datavalidation.com etc.

I am new to GO though I tried to run via heroku seems static site is not loading. it says 404 page not found is it normal?

thanks

You should say how it works in the README

"This marks an email as valid if it has an @ character in it"

"This checks DNS MX records for the domain"

"This sends a test email and if there is no bounce it marks the email as verified"

"This requires the email owner to RSVP"

"This uses a DAT file purloined from the dark net containing all pwned emails"

"This leverages quantum computing resources in a parallel universe and just effing knows 1ms before being asked whether an email address exists or not"

Batch Verification

Eh. You have no database. WTF?!?! Raw heroku

use golang + whatever mysql , nosql

or maybe try if you see a future . Postgres10 + ElasticSearch :)
This way already verified stuff you have stored already.

but what do I know. My github is empty :(

Minimum Length E-Mail

Love what you are doing,
One idea I have is setting the minimum length for an email to be valid.
For example checking [email protected] returns that its valid, even though its too short to be an actual yahoo email address. I can work on figuring out the numbers if needed

Stripe API

IMO a stripe API would be niice + login system for customers.

Different API (public/selfosted) answers

Hi! Thank you for your service. We used it on a paid terms and now want to use selfhosted docker version. Almost all work well but when I try to check email without MX entries or nonexistent host, latest version returns:

{
  "message": "Mail server does not exist",
  "details": "lookup 4356.iy on 8.8.8.8:53: no such host"
}

but test API on https://trumail.io returns:

{
    "address": "[email protected]",
    "username": "andrey",
    "domain": "4356.iy",
    "md5Hash": "c720cf8b88312322fa71c777979cc355",
    "validFormat": true,
    "deliverable": false,
    "fullInbox": false,
    "hostExists": false,
    "catchAll": false,
    "gravatar": false,
    "role": false,
    "disposable": false,
    "free": false
}

is there any way to get same message format from selfhosted version?

Add X-RateLimit-* headers to 429 responses

I'd suggest considering adding X-RateLimit-* headers to 429 responses to help automatically taking corrective measures and stopping hammering the API unnecessarily.

Of course there are different ways to go about it. I.e.

• X-RateLimit-Limit - Request limit per time window
• X-RateLimit-Remaining - The number of requests left for the time window window
• X-RateLimit-Reset - The remaining window before the rate limit resets in UTC epoch seconds

Thanks for consideration and a great service!

Simple run

{"time":"2018-04-10T01:19:50.92199198Z","level":"ERROR","prefix":"echo","file":"echo.go","line":"340","message":"code=404, message=Not Found"}

I have this error when i try to run trumail source code(without any changes except source address in config) on inteligo but i dont have any problems when i use docker, any ideas where is the problem ?

Not able to verify with updated build

Hi

Thanks for such a great product.

Recently I updated my module to latest docker image and it seems its not able to verify the emails anymore.

I am sending the request in this format "http://api.localhost:8080/v1/json/[email protected]". Every time I hit it responds back with deliverable as FALSE.

I also tested the same on my Heroku and still same issue (to make sure if my local IP was black listed).

Old version of same project is working fine in Heroku, could you please help me understand what I am doing wrong?

Thanks

Join TDV

ThunderDataVerify is truly a honest email verification company.

I want to get you paid for work you do. I’m not greedy.

2 man project you and https://github.com/vodolaz095

https://thunderdataverify.com

Got 100k usd in servers I think you can use and I know passion is everything that’s what I have aswell. I don’t do it for $

You need a few dns servers etc.

Result detail

Hi,
Realy nice work from you. Thanks a lot. One more thing while checking the email, output is ok if you also add the result detail it will be to more better. For example, if I verify this email( [email protected]) my deliverable status is false. Its fine but why this email gets failed, for that, if you display a Result Detail message it really helps to understand.

ReturnPath relationship

I can get this if you are willing to work.

People would love this once I contact sparkpost mailgun etc to integrate it

JSONP Support

Is there any chance for implementation of jsonp support?

Error Compling the new update version

command-line-arguments

.\main.go:25:27: not enough arguments in call to verifier.NewVerifier
have (string, string)
want (int, string, string)
.\main.go:26:7: undefined: api.NewService
.\main.go:35:22: cannot convert ":" to type int
.\main.go:35:22: invalid operation: ":" + config.Port (mismatched types string and int)

Support internationalized domain names without punycode

Testing addresses that use internacionalized domain names fail with {..., "error":"Mail server does not exist", "errorDetails":"lookup [domain]: no such host"}.

This issue don't happen if the internacionalized part is in the user part, only if it's in the domain part and if punycode isn't used.

Compile error when using Echo.

After installed all Dependencies when i run trumail I got this error.

github.com\labstack\echo\echo.go:624: e.AutoTLSManager.HTTPHandler undefined (type autocert.Manager has no field or method HTTPHandler).

Port 25

Port 25 is already a suspect. Even commercial accounts block if API's activity is detected. How about using other ports such as 587? How could that be done?

400 Bad request

...

add rethinkDB

Add rethinkDB driver with Golang.

https://github.com/vodolaz095 Is my dev.

Reason rethinkDB and and storing a database is because it makes it easier to verify a email once you know it’s verified.

nolisting / multiple mx

I tested with gmail, works great.

When the target has multiple MX and the 1st timeout (nolisting), there's no fallback and the result is timeout

Steps to build custom docker image

Hello, I'm trying to docker build .
I did make glide
maybe I'm missing something

Sending
build context to Docker daemon 51.79MB
Step 1/6 : FROM alpine:latest
---> 3fd9065eaf02
Step 2/6 : RUN apk add --no-cache ca-certificates
---> Using cache
---> c2853ab3189f
Step 3/6 : ADD trumail /usr/local/bin/trumail
ADD failed: stat /var/lib/docker/tmp/docker-builder162298542/trumail: no such file or directory

Docker version 18.03.0-ce, build 0520e24

False Negatives

Getting some false negatives: emails that are good but getting "deliverable":false.

Self-Hosted in Windows does not pass

Great work. When I run the main.go in Windows 10 environment, I received the following errors:

golang.org/x/net/context

trml\src\golang.org\x\net\context\context.go:45:19: undefined: Context
trml\src\golang.org\x\net\context\context.go:46:9: undefined: background
trml\src\golang.org\x\net\context\context.go:54:13: undefined: Context
trml\src\golang.org\x\net\context\context.go:55:9: undefined: todo

golang.org/x/net/idna

trml\src\golang.org\x\net\idna\punycode.go:31:42: undefined: labelError

EU GDPR

Since email addresses are personally identifiable information, how do you meet the requirements of the GDPR?

Disposable e-mail domains

Hi, I've been using trumail for some time and it's doing a great job, one of the features that I used most is to check if an e-mail is disposable, why did you remove it?

Do you recommend any alternative to check this?

Email provider support

Hey I wanted to know if you are focusing on specific providers because currently a lot of the german email providers aren't working with your API, for ex. (GMX, Web.de, 1&1)

I think they have different responses and behaviours than the standard (Gmail, G-suite, Yahoo, iCloud)

Do you intend to support those as well in the future?

Thanks ;)

Return Path Reputation

IMO Return Path Reputation seems interesting.

to be honest all this is cool and stuff but tbh maybe you should shift direction DNS way.

e.g. golang dev maybe research golang dns server :)

Yahoo

Uses Catch all.

IMO maybe try idk looking at reverse dns and see what you can do with that :)

Bounce server

Ever consider a bounce server to see the email exists?

Unable to send request to local machine

Hello

I am trying to use the docker image, the installation worked perfectly. The system says "http server started on [::]:8080".

But when I make a GET request to the system I get "{"message": "Not Found"}".

My request is in this format:
Type: GET
URL: http://localhost:8080/[email protected]

Is there anything wrong with my request? Any help will be great, thanks.

How to handle webmail emails.?

When i used to check webmails like( Hotmail, outlook) emails my IP address got blocked, Is there any solution for this. Please share some idea.

PROXY request

Is there any way to send trumail local server requests through PROXY?

serverless?

Have you considered using your client lib in an AWS lambda function? Using something like Apex to upload the Go function and your verifier lib? Maybe use SNS to send the deliverability email?

Just wondering if you've thought about it as a possible deployment option.

Thankyou for creating this and sharing it 👍

Receiving false negatives

I tested out the api from trumail.io and received false negatives on "deliverable" for known working email addresses.

What steps should I take to look into what prompted the false negative?