iamleot / transferwee Goto Github PK

If the file has a fully qualified path name, such as /tmp/hello.txt, it cannot be found. For instance, I created this file:

paulo:~/src/transferwee (master)$ ls -l /tmp/hello.txt
-rw-rw-r-- 1 paulo paulo 12 mai 31 13:11 /tmp/hello.txt

and when I try to upload it, I get this error:

paulo:~/src/transferwee (master)$ python3 transferwee.py upload -f [email protected] -t [email protected] -- /tmp/hello.txt
Traceback (most recent call last):
  File "transferwee.py", line 325, in <module>
    print(upload(args.files, args.m, args.f, args.t))
  File "transferwee.py", line 275, in upload
    _prepare_email_upload(filenames, message, sender, recipients)['id']
  File "transferwee.py", line 149, in _prepare_email_upload
    "files": [_file_name_and_size(f) for f in filenames],
  File "transferwee.py", line 149, in <listcomp>
    "files": [_file_name_and_size(f) for f in filenames],
  File "transferwee.py", line 133, in _file_name_and_size
    filesize = os.path.getsize(file)
  File "/usr/lib/python3.6/genericpath.py", line 50, in getsize
    return os.stat(filename).st_size
FileNotFoundError: [Errno 2] No such file or directory: 'hello.txt'

If I run transferwee from the same directory as the file, it works:

paulo:~/tmp$ python3 ~/src/transferwee/transferwee.py upload -f [email protected] -t [email protected] -- hello.txt
https://we.tl/t-cuohfc9FOP

It seems there is a problem if too many files are sent at the same time. Maybe a well positioned time.sleep could solve this problem.

Checking for no duplicate filenames
Preparing to upload
Could not find any csrf-token
From storm_upload_token WETRANSFER_STORM_PREFLIGHT URL is: https://storm-us-east-1.wetransfer.net/api/v2/batch/preflight
From storm_upload_token WETRANSFER_STORM_BLOCK URL is: https://storm-us-east-1.wetransfer.net/api/v2/blocks
From storm_upload_token WETRANSFER_STORM_BLOCK URL is: https://storm-us-east-1.wetransfer.net/api/v2/batch
Get transfer id eac9fe195aa5***0231128094504
Doing preflight storm
Preparing storm block upload

Traceback (most recent call last):
  File "/upload_wetransfer.py", line 18, in upload_wetransfer
    url = upload(files)
          ^^^^^^^^^^^^^
  File "/transferwee/transferwee.py", line 588, in upload
    for f, b in zip(files, blocks["data"]["blocks"]):
                           ~~~~~~^^^^^^^^
KeyError: 'data'

The request is sending this response :

>>> print(blocks)
>>> {'ok': False, 'error': {'client_action_required': False, 'error_code': 'TOO_MANY_REQUESTS', 'message': 'You are creating too many blocks too quickly, please try again after waiting for a little bit.'}}

I am trying to send 636 files at the same time.

For now, the module download as synchronously (one-by-one).
Can we use as asynchronously(using asyncio or etc) on batch download and download more fastly?

Hi there!

I spent a bit of time figuring out how to best use this tool from a python package I am developing.

in order to use transferwee from a python package, the script needs to be cloned and executed programmatically.

transferwee_path = os.path.join(__working_path__, 'transferwee/transferwee.py')
if not os.path.isfile(transferwee_path):
    Repo.clone_from("[email protected]:iamleot/transferwee.git", os.path.join(__working_path__, 'transferwee/'))
    subprocess.call([sys.executable, transferwee_path, "download", "-o", zip_name, "url", source_url])

If the most recent version of this script is uploaded to PyPi #31, the module could be imported rather than cloned to the file system.

from transferwee import transferwee

transferwee_path = transferwee.__file__

subprocess.call([sys.executable, transferwee_path, "download", "-o", zip_name, "url", source_url])

If this isn't helpful, feel free to close. I just thought this may save someone some time down the road 💖

-- ✨ Noelle ✨

Traceback (most recent call last):
File "transferwee.py", line 393, in
download(u, args.o)
File "transferwee.py", line 136, in download
file = _file_unquote(urllib.parse.urlparse(dl_url).path.split('/')[-1])
TypeError: 'str' does not support the buffer interface

I'm finally plugging transferwee into my bash script that sends albums from quodlibet, but I'm not being able to upload any files. I manage to upload the same files using the Web interface on a browser. Perhaps it's a problem with the WeTransfer API, I don't know.

Here's the test file I have been trying to upload:

paulo:~/tmp$ ls -l foo/bar
-rw-rw-r-- 1 paulo paulo 9 jun  2 18:58 foo/bar

and here's the output:

paulo:~/tmp$ transferwee upload -f [email protected] -t [email protected] -- foo/bar
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/paulo/.local/lib/python3.6/site-packages/requests/adapters.py", line 445, in send
    timeout=timeout
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3/dist-packages/six.py", line 692, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/paulo/.local/bin/transferwee", line 325, in <module>
    print(upload(args.files, args.m, args.f, args.t))
  File "/home/paulo/.local/bin/transferwee", line 282, in upload
    _upload_chunks(transfer_id, file_id, f)
  File "/home/paulo/.local/bin/transferwee", line 217, in _upload_chunks
    'Access-Control-Request-Method': 'PUT',
  File "/home/paulo/.local/lib/python3.6/site-packages/requests/api.py", line 85, in options
    return request('options', url, **kwargs)
  File "/home/paulo/.local/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/paulo/.local/lib/python3.6/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/paulo/.local/lib/python3.6/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "/home/paulo/.local/lib/python3.6/site-packages/requests/adapters.py", line 495, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

I'm running latest master (5d921f6) on Xubuntu 18.04.02

Hi @iamleot,

Again thanks for this tool that allows a great level of automation.
I just forked and added a windows version to produce a single windows exe (so easiest way to deploy on windows), if you want to link to my repo : https://github.com/outilslibre/transferwee-win somewhere for windows users. Many thanks

Hello,

It's apparently not compatible with versions < 3.9 anymore, contrary to what is assumed in 4f7cd22. Here's the error in 3.8:

Traceback (most recent call last):
  File "./transferwee.py", line 175, in <module>
    def _file_name_and_size(file: str) -> dict[str, Union[int, str]]:
TypeError: 'type' object is not subscriptable

I'm not qualified in Python to understand exactly what's going on here, as long as the import seems to be present at the top: from typing import Any, List, Optional, Union.

When trying to download a file I get following error message:
file = urllib.parse.urlparse(dl_url).path.split('/')[-1] TypeError: a bytes-like object is required, not 'str'

I tried to print out "r" from the "download_url" class
(r = requests.post(WETRANSFER_DOWNLOAD_URL.format(transfer_id=transfer_id), json=j),
but only got <Response [403]>.

I believe this is because I am not sending an authorization key with the request, right?
(https://developers.wetransfer.com/documentation#send-request)

I want to upload multiple .zip files that are each 1.5 GB in size.
I expected to get multiple URLs back but instead it just ran and uploaded for a really long time.
Then it only displayed one URL that was expired.
I think it tried to upload all files as one link but since the upload limit is 2 GB it expired.

Can you maybe add a check if the selected files are more than 2 GB in total and if so start multiple upload threads?

Traceback (most recent call last): File "/root/wetransfer/transferwee.py", line 650, in <module> print(upload(args.files, args.n, args.m, args.f, args.t)) File "/root/wetransfer/transferwee.py", line 551, in upload _storm_preflight(transfer["storm_upload_token"], files) File "/root/wetransfer/transferwee.py", line 329, in _storm_preflight return r.json() File "/usr/lib/python3.9/site-packages/requests/models.py", line 891, in json return complexjson.loads( File "/usr/lib64/python3.9/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/usr/lib64/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

As of a couple of days back transferee stopped working and I get 403 errors for all requests. Adding a "User-Agent" header to requests seems to solve the problem.

hi @iamleot

Great work! I've been using it for quite a while and it was working great, but recently I keep receiving below errors:

Checking that all files exists
Checking for no duplicate filenames
Preparing to upload
Get transfer id b8b1cb5fb7a594c69dca110589851adf20230403142526
Uploading test.txt as part of transfer_id b8b1cb5fb7a594c69dca110589851adf20230403142526
{'message': 'Transfer already closed (state: processing)'}
Traceback (most recent call last):
  File "/Volumes/workspace/Robox/ROS/ws/cloud_tools/transferwee.py", line 443, in <module>
    print(upload(args.files, args.n, args.m, args.f, args.t))
  File "/Volumes/workspace/Robox/ROS/ws/cloud_tools/transferwee.py", line 380, in upload
    file_id = _prepare_file_upload(transfer_id, f, s)['id']
KeyError: 'id'

Are you aware of this issue? is it because the wetransfer server changed recently?

Thanks.

the file only 68MB

python transferwee.py upload -v "audio.m4a"

Checking that all files exists
Checking for no duplicate filenames
Preparing to upload
Could not find any csrf-token
From storm_upload_token WETRANSFER_STORM_PREFLIGHT URL is: https://storm-us-east-1.wetransfer.net/api/v2/batch/preflight
From storm_upload_token WETRANSFER_STORM_BLOCK URL is: https://storm-us-east-1.wetransfer.net/api/v2/blocks
From storm_upload_token WETRANSFER_STORM_BLOCK URL is: https://storm-us-east-1.wetransfer.net/api/v2/batch
Get transfer id 84d0a30e9763ab565da61da02ae126c920230819085919
Doing preflight storm
Preparing storm block upload
Traceback (most recent call last):
  File "/content/transferwee.py", line 684, in <module>
    print(upload(args.files, args.n, args.m, args.f, args.t))
  File "/content/transferwee.py", line 588, in upload
    for f, b in zip(files, blocks["data"]["blocks"]):
KeyError: 'data'

i try add this code in 587 logger.debug(blocks)

{'ok': False, 'error': {'message': 'Block at position 0 ({:content_length=>71567915, :content_md5_hex=>"3dbb5db155ffebc8d7ce20c804ada891"}) is too large (max is 62914560)'}}

The required python version is hardcoded at 3.7:

#!/usr/bin/env python3.7

so that when I try to run transferwee, I get:

paulo:~/src/transferwee (master)$ ./transferwee.py --help
/usr/bin/env: ‘python3.7’: No such file or directory

I'm running Xubuntu 18.0.02 LTS and the installed python 3 version is:

paulo:~/src/transferwee (master)$ python3 --version
Python 3.6.7

Shouldn't the env shebang require only python3? Like this:

#!/usr/bin/env python3

I'm not a Python programmer but I had a look around other python 3 programs I have installed, like quodlibet, and they all have a shebang like that.

Hello Leonardo,
thanks for this good project. I'm using it a lot. My native language is Chinese, when using subcommand 'download', filename is not decoded as Chinese when uploaded filename contains Chinese characters. Could you decode filename before saving it? After line #123, it's better to add "file = urllib.parse.unquote(file)".

thanks again for this good project!

It would be useful if transferwee set the exit code to something other than 0 when an error occurs. As of now, it seems that the exit code is always set to 0, even when an error occurs. For instance, trying to upload a non-existent file:

paulo:~/tmp$ ls foo
ls: cannot access 'foo': No such file or directory
paulo:~/tmp$ transferwee upload foo
None
paulo:~/tmp$ echo $?
0

Of course, I could also capture stdout and check if it's a valid URL: if it isn't, it means there was an error. Checking the exit code feels more natural.

Thanks for sharing transferwee! I couldn't find a license, it would be helpful to add one so that potential users know how you intend it to be used.

I'm trying to use this from Terminal on Mac.

I installed with pip install transferwee, which seemed to work and I'm on Python 3.10.

I've tried:

transferwee upload file /Users/noisyneil/Desktop/Test.txt

...but I get: ModuleNotFoundError: No module named 'transferwee.main'

I'm a python beginner so sorry if I'm doing something silly.

It seems like WeTransfer changed something in the past couple of days and download_url now raises ConnectionError('Could not prepare session').

_prepare_session reports "Could not find any csrf-token".

Hey! About two hours ago wetransfer started reporting incorrect X-CSRF Tokens despite them correct. This can be fixed by supplying additional headers.

def get_headers(session):
    return {
        'x-csrf-token': session.headers['x-csrf-token'],
        'x-requested-with': 'XMLHttpRequest',
    }

This can simply be supplied with each post request, here for example in _prepare_link_upload:

r = session.post(WETRANSFER_UPLOAD_LINK_URL, json=j, headers=get_headers(session))

If you'd like me to submit a pull request for this fix just let me know.

The script stopped working today. I've been using every day for a while and today the script decided it would not work anymore. I tried a few thing and found out that the method that generates the downloadable url (download_url(url) at line 64) is probably the culprit. this would be a good time to say I'm using this to download only, not upload. At the end of this method the JSON object (j) on has this: {'message': 'Missing parameter :intent'}. Hope you can fix this soon. If you need more info feel free to contact me

Hi,

It would be great to have an option to show download/upload progress. Currently when using transferwee download <link>, there is no output while the file downloads.

Would be nice to be able to provide credentials for every upload request to be able to store files in your account, rather than just as a guest.

Can anyone advise on how to install this? I'm a python/github noob.

I've run the following:

python -m pip install requests
git clone https://github.com/iamleot/transferwee.git
cd transferwee
transferwee upload file /Users/noisyneil/Desktop/Test.txt

...and got zsh: command not found: transferwee.

Any help appreciated.

Great tool! It would be nice to have an option to specify a name for the output-file. Example:

transferwee -o /tmp/output-file download https://wetransfer.com/downloads/123lkjfd

Cheers

I'm trying to upload a file with a sender address and a recipient address, but the script doesn't work as expected.

gilberto@margot:~/$ ./transferwee/transferwee.py upload -m "New CV created" -f <sender_email> -t <recipient_email> FILE 
usage: transferwee upload [-h] [-m message] [-f from] [-t to [to ...]]
                          file [file ...]
transferwee upload: error: the following arguments are required: file

Am I wrong in calling the script?

Hello,

  This tool was awesome and I would request to add one more to this tool. 

  Add the upload/download progress/status and Speed of the transfer. 

as soon as the file size exceeds 15 MB, I get this error:

$ ./transferwee.py upload /tmp/toto.mp3 -v
Checking that all files exists
Checking for no duplicate filenames
Preparing to upload
Could not find any csrf-token
From storm_upload_token WETRANSFER_STORM_PREFLIGHT URL is: https://storm-eu-west-1.wetransfer.net/api/v2/batch/preflight
From storm_upload_token WETRANSFER_STORM_BLOCK URL is: https://storm-eu-west-1.wetransfer.net/api/v2/blocks
From storm_upload_token WETRANSFER_STORM_BLOCK URL is: https://storm-eu-west-1.wetransfer.net/api/v2/batch
Get transfer id 6f0651d79de2f01c4c14ed0d93eb185120231022124836
Doing preflight storm
Preparing storm block upload
Traceback (most recent call last):
File "/tmp/transferwee/./transferwee.py", line 683, in
print(upload(args.files, args.n, args.m, args.f, args.t))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/transferwee/./transferwee.py", line 587, in upload
for f, b in zip(files, blocks["data"]["blocks"]):
~~~~~~^^^^^^^^
KeyError: 'data'

use master version and python3.11

thanks !

Hello

Thank you for your work.
I encounter this error when I add the path I want the files to download.
For example;

web2@serve: transferwee -o /var/downloadedFiles/transferwee/ -v https://wetransfer.com/abcdef
Traceback (most recent call last):
  File "we.py", line 438, in <module>
    download(u, args.o)
  File "we.py", line 155, in download
    with open(file, 'wb') as f:
IsADirectoryError: [Errno 21] Is a directory: '/var/downloadedFiles/transferwee/'

Why I am getting this error IsADirectoryError?