gmr / email-normalize Goto Github PK

Hi there;
When I try to call blocking normalize function. Call fails with

RuntimeError: There is no current event loop in thread 'Thread-7'.

Normalizer.normalize() breaks when it is used in a program that contains multiple event loops, for example because it's multithreaded. A common example where this might be the case is Django + uvicorn.

Example:

import asyncio
import threading
import time

import email_normalize

def fn():
    asyncio.set_event_loop(asyncio.new_event_loop())

    print(email_normalize.normalize("[email protected]"))

t1 = threading.Thread(target=fn)
t2 = threading.Thread(target=fn)

t1.start()

time.sleep(1)

t2.start()

t1.join()
t2.join()

Output:

Result(address='[email protected]', normalized_address='[email protected]', mx_records=[(5, 'gmail-smtp-in.l.google.com'), (10, 'alt1.gmail-smtp-in.l.google.com'), (20, 'alt2.gmail-smtp-in.l.google.com'), (30, 'alt3.gmail-smtp-in.l.google.com'), (40, 'alt4.gmail-smtp-in.l.google.com')], mailbox_provider='Google')
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "reproduce.py", line 10, in fn
    print(email_normalize.normalize("[email protected]"))
  File "<<xxx/site-packages>>/email_normalize/__init__.py", line 259, in normalize
    return loop.run_until_complete(normalizer.normalize(email_address))
  File "/usr/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete
    return future.result()
  File "<xxx/site-packages>/email_normalize/__init__.py", line 192, in normalize
    mx_records = await self.mx_records(domain_part)
  File "<xxx/site-packages>/email_normalize/__init__.py", line 153, in mx_records
    records = await self._resolver.query(domain_part, 'MX')
RuntimeError: Task <Task pending coro=<Normalizer.normalize() running at <xxx/site-packages>/email_normalize/__init__.py:192> cb=[_run_until_complete_cb() at /usr/lib/python3.7/asyncio/base_events.py:153]> got Future <Future pending> attached to a different loop

This is caused by the fact that a single aiodns.DNSResolver is being shared across event loops due to it being a singleton.

For now, I've worked around it in my own project by doing this:

async def normalize_email(email_address: str) -> str:
    normalizer = email_normalize.Normalizer()
    email_normalize.Normalizer._instance = None
    return (await normalizer.normalize(email_address)).normalized_address

Not work

email = '[email protected]'
normalized = normalize(email)
print(email, normalized)

#return '[email protected]'

Firstly, this is a great library. Thank you!

Event loops on Windows don't support all methods, particularly aiodns calls add_reader which is unsupported, and leads to a hang if you're using ProactorEventLoop (the default on Windows):

See more: https://docs.python.org/3.9/library/asyncio-platforms.html#windows

Hi, I love this library and the rules it brings, I faced some issues when running this in an offline setting though, and was wondering if we can add an option to avoid mxrecords lookup (happy to submit a PR if you are open to the idea). This can also work even if online but you don't care about 100% accuracy (i.e. taking the domain input at face value). Do you think we can add that?

When running email_normalize.normalize() in a django project we get the following error:

RuntimeError: There is no current event loop in thread 'ThreadPoolExecutor-0_0'.

The RuntimeError is thrown at loop = asyncio.get_event_loop() (line 257, in normalize)

Creating a 'hello-world' Django project with the following view will reproduce the problem:

from django.shortcuts import render
from django.http import HttpResponse
import email_normalize

# Create your views here.

def index(request):
    email_normalize.normalize('[email protected]')
    return HttpResponse('Hello world!')

using:

• email-normalize == 1.0.4
• Django == 3.0.7