lasselindqvist / robotframework-imaplibrary2

IMAP email testing library for Robot Framework

IMAP email testing library for Robot Framework

Docs

Introduction

Note: This is a fork of https://github.com/rickypc/robotframework-imaplibrary

ImapLibrary2 is a IMAP email testing library for Robot Framework.

More information about this library can be found in the Keyword Documentation.

If you are interested to contribute back to this project, please see Contributing section.

Examples

*** Settings ***
Library    ImapLibrary2

*** Test Cases ***
Email Verification
    Open Mailbox    host=imap.domain.com    [email protected]    password=secret
    ${LATEST} =    Wait For Email    [email protected]    timeout=300
    ${HTML} =    Open Link From Email    ${LATEST}
    Should Contain    ${HTML}    Your email address has been updated
    Close Mailbox

Multipart Email Verification
    Open Mailbox    host=imap.domain.com    [email protected]    password=secret
    ${LATEST} =    Wait For Email    [email protected]    timeout=300
    ${parts} =    Walk Multipart Email    ${LATEST}
    :FOR    ${i}    IN RANGE    ${parts}
    \\    Walk Multipart Email    ${LATEST}
    \\    ${content-type} =    Get Multipart Content Type
    \\    Continue For Loop If    '${content-type}' != 'text/html'
    \\    ${payload} =    Get Multipart Payload    decode=True
    \\    Should Contain    ${payload}    your email
    \\    ${HTML} =    Open Link From Email    ${LATEST}
    \\    Should Contain    ${HTML}    Your email
    Close Mailbox

Installation

Using pip

The recommended installation method is using pip:

pip install robotframework-imaplibrary2

The main benefit of using pip is that it automatically installs all dependencies needed by the library. Other nice features are easy upgrading and support for un-installation:

pip install --upgrade robotframework-imaplibrary2
pip uninstall robotframework-imaplibrary2

Notice that using --upgrade above updates both the library and all its dependencies to the latest version. If you want, you can also install a specific version:

pip install robotframework-imaplibrary2==x.x.x

Proxy configuration

If you are behind a proxy, you can use --proxy command line option or set http_proxy and/or https_proxy environment variables to configure pip to use it. If you are behind an authenticating NTLM proxy, you may want to consider installing CNTML to handle communicating with it.

For more information about --proxy option and using pip with proxies in general see:

• http://pip-installer.org/en/latest/usage.html
• http://stackoverflow.com/questions/9698557/how-to-use-pip-on-windows-behind-an-authenticating-proxy
• http://stackoverflow.com/questions/14149422/using-pip-behind-a-proxy

Manual installation

If you do not have network connection or cannot make proxy to work, you need to resort to manual installation. This requires installing both the library and its dependencies yourself.

• Make sure you have Robot Framework installed.
• Download source distributions (*.tar.gz) for the library:
  • https://pypi.python.org/pypi/robotframework-imaplibrary2
• Download PGP signatures (*.tar.gz.asc) for signed packages.
• Find each public key used to sign the package:

gpg --keyserver pgp.mit.edu --search-keys D1406DE7

• Select the number from the list to import the public key
• Verify the package against its PGP signature:

gpg --verify robotframework-imaplibrary2-x.x.x.tar.gz.asc robotframework-imaplibrary2-x.x.x.tar.gz

• Extract each source distribution to a temporary location.
• Go to each created directory from the command line and install each project using:

python setup.py install

If you are on Windows, and there are Windows installers available for certain projects, you can use them instead of source distributions. Just download 32bit or 64bit installer depending on your system, double-click it, and follow the instructions.

Directory Layout

doc/ : Keyword documentation

src/ : Python source code

test/ : Test files

utest/
:   Python unit test

Usage

To write tests with Robot Framework and ImapLibrary, ImapLibrary must be imported into your Robot test suite.

*** Settings ***
Library    ImapLibrary2

See Robot Framework User Guide for more information.

More information about Robot Framework standard libraries and built-in tools can be found in the Robot Framework Documentation.

Building Keyword Documentation

The Keyword Documentation can be found online, if you need to generate the keyword documentation, run:

make doc

or

python -m robot.libdoc src/ImapLibrary2 doc/ImapLibrary2.html

Run Unit Tests, and Test Coverage Report

Test the testing library, talking about dogfooding, let's run:

make test

Deploying to Pypi

python setup.py sdist bdist_wheel
twine upload dist/*

Contributing

If you would like to contribute code to Imap Library project you can do so through GitHub by forking the repository and sending a pull request.

When submitting code, please make every effort to follow existing conventions and style in order to keep the code as readable as possible. Please also include appropriate test cases.

That's it! Thank you for your contribution!

License

Copyright (c) 2015-2016 Richard Huang.

This library is free software, licensed under: Apache License, Version 2.0.

Documentation and other similar content are provided under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

robotframework-imaplibrary2's Issues

Waiting for emails based on Subject doesn't work - error: UID command error: BAD [b'expected search-key instead of "{5}<search_term>']

Since the UTF8 subject encoding fix was merged all my "Wait for email" based on Subject are failing with following error. Seems as if the subject is not being provided properly to the underlying python lib. My guess would be, that it has something to do with the fact, that for the SUBJECT criterion the string value is not interpolated as for other text based criteria:

Example extract from a robot framework log

KEYWORD ${email} = ImapLibrary2 . Wait For Email subject=${subject}, timeout=${timeout}, status=UNSEEN
Documentation:
Wait for email message to arrived base on any given filter criteria. Returns email index of the latest email message received.

Start / End / Elapsed: 20220712 09:56:03.809 / 20220712 09:56:03.996 / 00:00:00.187
09:56:03.809 TRACE Arguments: [ subject="'Job'" | timeout='600' | status='UNSEEN' ]
09:56:03.996 FAIL error: UID command error: BAD [b'expected search-key instead of "{5}\'JOB\'"']
09:56:03.996 DEBUG Traceback (most recent call last):
File "path\venv\lib\site-packages\ImapLibrary2_init_.py", line 382, in wait_for_email
self._mails = self.check_emails(**kwargs)
File "path\venv\lib\site-packages\ImapLibrary2_init.py", line 433, in _check_emails
typ, msgnums = self._imap.uid('search', None, *criteria)
File "path\Python\Python39\lib\imaplib.py", line 890, in uid
typ, dat = self._simple_command(name, command, *args)
File "path\Python\Python39\lib\imaplib.py", line 1230, in _simple_command
return self._command_complete(name, self._command(name, *args))
File "path\Python\Python39\lib\imaplib.py", line 1055, in _command_complete
raise self.error('%s command error: %s %s' % (name, typ, data))
imaplib.IMAP4.error: UID command error: BAD [b'expected search-key instead of "{5}\'JOB\'"']

make test fails

make test fails to

2019-12-18T19:38:55.3955474Z ERROR: test_should_delete_all_emails (test_imaplibrary.ImapLibraryTests)
2019-12-18T19:38:55.3955875Z Delete all emails.
2019-12-18T19:38:55.3956844Z ----------------------------------------------------------------------
2019-12-18T19:38:55.3957162Z Traceback (most recent call last):
2019-12-18T19:38:55.3959368Z   File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/mock/mock.py", line 1330, in patched
2019-12-18T19:38:55.3959989Z     return func(*args, **keywargs)
2019-12-18T19:38:55.3960863Z   File "/home/runner/work/robotframework-imaplibrary2/robotframework-imaplibrary2/test/utest/test_imaplibrary.py", line 321, in test_should_delete_all_emails
2019-12-18T19:38:55.3961503Z     self.library.delete_all_emails()
2019-12-18T19:38:55.3962860Z   File "/home/runner/work/robotframework-imaplibrary2/robotframework-imaplibrary2/src/ImapLibrary2/__init__.py", line 104, in delete_all_emails
2019-12-18T19:38:55.3963272Z     self._get_all_emails()
2019-12-18T19:38:55.3963940Z   File "/home/runner/work/robotframework-imaplibrary2/robotframework-imaplibrary2/src/ImapLibrary2/__init__.py", line 413, in _get_all_emails
2019-12-18T19:38:55.3964495Z     typ, mails = self._imap.uid('search', None, 'ALL')
2019-12-18T19:38:55.3964880Z ValueError: not enough values to unpack (expected 2, got 0)
2019-12-18T19:38:55.3965062Z 
2019-12-18T19:38:55.3965246Z ======================================================================
2019-12-18T19:38:55.3965465Z ERROR: test_should_mark_all_emails_as_read (test_imaplibrary.ImapLibraryTests)
2019-12-18T19:38:55.3965685Z Mark all emails as read.
2019-12-18T19:38:55.3966172Z ----------------------------------------------------------------------
2019-12-18T19:38:55.3966427Z Traceback (most recent call last):
2019-12-18T19:38:55.3967562Z   File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/mock/mock.py", line 1330, in patched
2019-12-18T19:38:55.3967945Z     return func(*args, **keywargs)
2019-12-18T19:38:55.3968675Z   File "/home/runner/work/robotframework-imaplibrary2/robotframework-imaplibrary2/test/utest/test_imaplibrary.py", line 340, in test_should_mark_all_emails_as_read
2019-12-18T19:38:55.3968986Z     self.library.mark_all_emails_as_read()
2019-12-18T19:38:55.3969616Z   File "/home/runner/work/robotframework-imaplibrary2/robotframework-imaplibrary2/src/ImapLibrary2/__init__.py", line 218, in mark_all_emails_as_read
2019-12-18T19:38:55.3969914Z     self._get_all_emails()
2019-12-18T19:38:55.3970534Z   File "/home/runner/work/robotframework-imaplibrary2/robotframework-imaplibrary2/src/ImapLibrary2/__init__.py", line 413, in _get_all_emails
2019-12-18T19:38:55.3971088Z     typ, mails = self._imap.uid('search', None, 'ALL')
2019-12-18T19:38:55.3971334Z ValueError: not enough values to unpack (expected 2, got 0)
2019-12-18T19:38:55.3971508Z 
2019-12-18T19:38:55.3971717Z ======================================================================
2019-12-18T19:38:55.3971941Z ERROR: test_should_mark_all_emails_as_read_from_deprecated_keyword (test_imaplibrary.ImapLibraryTests)
2019-12-18T19:38:55.3972143Z Mark all emails as read using deprecated keyword.
2019-12-18T19:38:55.3972616Z ----------------------------------------------------------------------
2019-12-18T19:38:55.3972874Z Traceback (most recent call last):
2019-12-18T19:38:55.3973457Z   File "/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/mock/mock.py", line 1330, in patched
2019-12-18T19:38:55.3973742Z     return func(*args, **keywargs)
2019-12-18T19:38:55.3974451Z   File "/home/runner/work/robotframework-imaplibrary2/robotframework-imaplibrary2/test/utest/test_imaplibrary.py", line 350, in test_should_mark_all_emails_as_read_from_deprecated_keyword
2019-12-18T19:38:55.3974936Z     self.library.mark_as_read()
2019-12-18T19:38:55.3975567Z   File "/home/runner/work/robotframework-imaplibrary2/robotframework-imaplibrary2/src/ImapLibrary2/__init__.py", line 226, in mark_as_read
2019-12-18T19:38:55.3975858Z     self.mark_all_emails_as_read()
2019-12-18T19:38:55.3976494Z   File "/home/runner/work/robotframework-imaplibrary2/robotframework-imaplibrary2/src/ImapLibrary2/__init__.py", line 218, in mark_all_emails_as_read
2019-12-18T19:38:55.3976789Z     self._get_all_emails()
2019-12-18T19:38:55.3978111Z   File "/home/runner/work/robotframework-imaplibrary2/robotframework-imaplibrary2/src/ImapLibrary2/__init__.py", line 413, in _get_all_emails
2019-12-18T19:38:55.3979907Z     typ, mails = self._imap.uid('search', None, 'ALL')
2019-12-18T19:38:55.3980201Z ValueError: not enough values to unpack (expected 2, got 0)

as these use mocking, we just need to update the tests.

Wait For Email: UID SEARCH Failure due to backend error

Wait For Email is failing if specify sender argument.

${latest}= Wait For Email subject=Welcome! timeout=30

Works fine but if you specify sender then it starts failing.

${latest}= Wait For Email [email protected] subject=Welcome! timeout=30

Fails with error:

FAIL : imap.search error: NO, [b'[UNAVAILABLE] UID SEARCH Failure due to backend error.

Walk Multipart Email: initial_value must be str or None, not bytes

Walk Multipart Email is failing with error:

TypeError: initial_value must be str or None, not bytes

Wait For Email is returning index as bytes but converting the index into string doesn't help.

    ${latest}=    Wait For Email    subject=Welcome!    timeout=60
    ${type}=    Evaluate    type($latest)    
    ${latest_str}=    Convert To String    ${latest}
    ${type}=    Evaluate    type($latest_str)    
    ${parts}=    Walk Multipart Email    ${latest_str}

20191121 10:17:59.267 : INFO : ${latest} = 12
20191121 10:17:59.284 : INFO : ${type} = <class 'bytes'>
20191121 10:17:59.288 : INFO : ${latest_str} = 12
20191121 10:17:59.292 : INFO : ${type} = <class 'str'>
20191121 10:17:59.533 : FAIL : TypeError: initial_value must be str or None, not bytes

Multipart message body cannot be read with Python 3

Get Multipart Payload keyword seems to fail for email body if the e-mail has an attachment. I get this error:
AttributeError: 'str' object has no attribute 'decode'

I googled for that error, and it seems that the message cannot be decoded in the same way in Python 3.
https://stackoverflow.com/questions/28583565/str-object-has-no-attribute-decode-python-3-error

Add proxy support

In many companies the IMAP ports 143 and 993 are blocked by firewall and to access internet it is needed to use proxy so current version of this library can not be used there. The IMAP4 and IMAP4_SSL classes does not support directly passing the proxy setting however it is possible to inherit from these classes and program proxy support e.g. like shown on example: https://gist.github.com/sstevan/efccf3d5d3e73039c21aa848353ff52f

Keyword Documentation links point to old imap library

The Keyword Documentation links all point to the old imap library. I get that the keywords may be the same, but the version information is different.

Create proper tests

Create proper tests.

Possibly:
Create 2 Gmail accounts. Send email from one to the other. Use the framework code then to act on the emails.

Github actions are free, so start with them.

Fix decoding problems once and for all

If email payload is base64 encoded or if email is encoded using different encoding than UTF-8 or ISO-8859-1 method get_email_body return's not correctly decoded body. Solution is instead use email module, which can parse/decode whole email and gives ability to access properties and methods OOP way.

Fix was inspired by https://www.thepythoncode.com/article/reading-emails-in-python .

Update proxy documentation

Hello @lasselindqvist, thx for merging and release new version containing proxy implementation from #16, but could you pls update as well Documentation to contain new parameters in open_mailbox method:

        - ``proxy_host``: Proxy host to connect via. (Default None)
        - ``proxy_port``: Proxy port to connect via. (Default None)
        - ``proxy_user``: Proxy username to connect via. (Default None)
        - ``proxy_password``: Proxy password to connect via. (Default None)
        - ``proxy_type``: Proxy type to connect via. Available values are: http, socks4, socks5 (Default http)

non-ascii characters in email subject cannot be encoded

When I call the function "Wait For Email" with the parameter "subject" containing special characters (umlaute), I receive the following error: "UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 5: ordinal not in range(128)" This is caused by the following line in _check_emails: typ, msgnums = self._imap.uid('search', None, *criteria), which assumes ascii characters. Is there a way to account for utf-8 characters?
I have tried this solution: https://stackoverflow.com/questions/63685324/searching-for-utf-8-encoded-subjects-with-imaplib
It worked, but then I run into a follow-up issue with the function "Get Links From Email", receiving the error: " get_email_body called on multipart email 'Bestätigen'. Please first use method walk_multipart_email"

Release new version

Release new version.
Currently #7 is unreleased.

Delete All Emails keyword is not deleting all emails

For some reason keyword is deleting only one email but not all emails in inbox folder. No any errors or exceptions.

Test scripts cannot be run successfully from jenkins

Whenever we run the following test case from command console, it always tests pass:
login test
Open Mailbox host=imap.host.cn user=@.cn password=pwd port=993

but always failed whenever we run it from jenkins, and the jenkins can correctly run all other scripts using other RS's libraries including robotframework-seleniumlibrary.
The error message is:
[FAIL] OSError: [WinError 10038] An operation was attempted on a non socket.

I wonder whether robotframework-imaplibrary2 should be adjusted to fix the above problem.

Get Email Body decodes URLs with issues

URLs are decoded with issues, for instance URL

<a href="https://example.com/verify-account?uid=5dd50bc5d3b1f228474551d7&t=98e1ee0c-a159-475e-a0e3-f74a533d48c8">

is decoded as

<a href="https://example.com/verify-account?uid]d50bc5d3b1f228474551d7&t�e1ee0c-a159-475e-a0e3-f74a533d48c8">

Here is Python code of function get_email_body():

        if self._is_walking_multipart(email_index):
            body = self.get_multipart_payload(decode=True)
        else:
            encoded_body = self._imap.uid('fetch', email_index, '(BODY[TEXT])')[1][0][1]
            try:
                body = decode(encoded_body, 'quopri_codec').decode('UTF-8')
            except:
                body = decode(encoded_body, 'quopri_codec').decode('ISO-8859-1')
        return body

Wait for email fails with error: UID command error: BAD [b'The specified message set is invalid.']

While Updating from RF-ImapLibrary2 from Version 0.4.0 to 0.4.4 we are getting this error:
error: UID command error: BAD [b'The specified message set is invalid.']
In Version 0.4.0 all worked fine.

We are using Python 3.8.0 and RF 5.0.1 on Windows 10.

${email_index} = ImapLibrary2 . Wait For Email sender=${email_sender}, subject=${decoded_subject}, timeout=360, recipient=${email_recipient}, status=UNSEEN, folder=${email_folder}

As e-mail prover we are using outlook.com.
Subject is something like this: S1 Dein neues Passwort für domain.de
And the email folder: S1/ChangePassword

The Email Types are:
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit