rndusr / torf-cli Goto Github PK

Hi @rndusr, is it possible include the options:

--nopublisher
--nopublisher-url

Some trackers include these informations, but is not necessary.

It would be useful to have a command-line option to force a formatted output even when piping the output.

Here's why: I'm replacing transmission-show with dorf to show torrent files attached to mail messages and also on mc. In both cases, the output is piped to an internal pager, so the result is, for instance, with mc:

It's perfect for parsing, I can see there's a single TAB character between each header and its value, but not for viewing. mc pipes the output to its interval viewer. With neomutt, of course, the result will be the same, since it also pipes the output to its internal viewer:

An explicit option to force formatted output would be ideal for these cases.

It might be useful to be able to expand environment values in the config file. For instance, I set the comment for sef-created torrents in a bash script:

COMMENT=$USER@$HOSTNAME

It would be nice to be able to set it in torf's config file, like:

comment = $USER@$HOSTNAME

I've just created a biggsh torrent with many files and noticed that the progress/ETA status line keeps moving across the screen and ends up garbled at the far right. For instance, here's what ended up on the screen after creating this big torrent:

When piping the output, the progress/ETA status line also moves across the screen, which results in garbled output when piped into a text file.

Running torf 6537c03 on Xubuntu 18.04.

I know it may be possible to script it in, but if there is a flag that can get it automatically and also use the tracker defined via -t flag.

Like:

torf 'folder name' \
--tracker-url https://ngosang.github.io/trackerslist/trackers_best.txt \
-t udp://tracker.internetwarriors.net:1337/announce \
-t udp://tracker.opentrackr.org:1337/announce \
-t udp://tracker.torrent.eu.org:451/announce \
-o "test.torrent"

I've just installed torf on Xubuntu 18.04 with Python 3.65:

paulo:~/src$ pip3 install --user git+https://github.com/rndusr/torf-cli.git
Collecting git+https://github.com/rndusr/torf-cli.git
  Cloning https://github.com/rndusr/torf-cli.git to /tmp/pip-kssluewc-build
Collecting pyxdg (from torf-cli==1.2)
  Using cached https://files.pythonhosted.org/packages/39/03/12eb9062f43adb94e30f366743cb5c83fd15fef026500cd4de42c7c12280/pyxdg-0.26-py2.py3-none-any.whl
Collecting torf>=1.3 (from torf-cli==1.2)
  Using cached https://files.pythonhosted.org/packages/dc/9b/8b82bb81dc06753921c02205c3eadf600b1ceec002761baab6188f115675/torf-1.3.tar.gz
Collecting bencoder.pyx<2.0.0,>=1.1.1 (from torf>=1.3->torf-cli==1.2)
  Using cached https://files.pythonhosted.org/packages/90/96/aed139c3b893ce09e8d1033303e5451c2799a9741d29028d3c80c5c66853/bencoder.pyx-1.2.1-cp36-cp36m-manylinux1_x86_64.whl
Building wheels for collected packages: torf
  Running setup.py bdist_wheel for torf ... done
  Stored in directory: /home/paulo/.cache/pip/wheels/55/19/37/eeaf4874c7718d0d3b29c85d980b946d5d29706863853e2400
Successfully built torf
Installing collected packages: pyxdg, bencoder.pyx, torf, torf-cli
  Running setup.py install for torf-cli ... done
Successfully installed bencoder.pyx-1.2.1 pyxdg-0.26 torf-1.3 torf-cli-1.2

and if I run torf with any option/arg, I always get the same error, for instance, running torf -h:

paulo:~/src$ torf -h
Traceback (most recent call last):
  File "/home/paulo/.local/bin/torf", line 11, in <module>
    load_entry_point('torf-cli==1.2', 'console_scripts', 'torf')()
  File "/home/paulo/.local/lib/python3.6/site-packages/torfcli/__init__.py", line 13, in run
    from ._cli import run, CLIError
  File "/home/paulo/.local/lib/python3.6/site-packages/torfcli/_cli.py", line 39, in <module>
    _DEFAULT_PROFILE_FILE = os.path.join(BaseDirectory.load_first_config(_vars.__appname__), 'config')
  File "/usr/lib/python3.6/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Thank you! This tool is too powerful, but there is a small regret that it is impossible to view and modify the "source" field of the private torrent file. Can you add this feature? Because the "source" field is important for private trackers.
Ps: You can view and modify the ‘’source’’ field of the torrent file via https://github.com/drizt/torrent-file-editor this, but this tool cannot be operated by commander.

Right now it's only possible to add multiple files to a torrent by specifying a directory, but this includes the top level directory in the torrent as the root.

For example, running torf /my/files creates this:

files/file1.txt
files/directory/file2.txt

when for my use case, I want this instead:

file1.txt
directory/file2.txt

Is it possible to allow for torf-cli to have multiple file arguments specified? Such that this would work:

torf /my/files/* ...

This would solve my issue as above.

Alternatively, a switch for excluding the top level directory and having the contents as the root(s) would be acceptable.

torfcli/_vars_.py was renamed to torfcli/_vars.py but setup.py is still referencing torfcli/_vars_.py:

paulo:~/src/torf-cli$ python3 setup.py install --user
Traceback (most recent call last):
  File "setup.py", line 3, in <module>
    with open('torfcli/__vars__.py') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'torfcli/__vars__.py'

Running 435d328.

I was trying to create a torrent of a directory which contained a soft symlink to another file also in the target directory:

$ ls
OVMF_CODE.fd
OVMF_VARS.fd
snapshot.qcow2
base.qcow2
latest.qcow2 -> snapshot.qcow2

Attempting to create a torrent with torf resulted in the following error:

       Name  stuff
       Size  28.48 GiB
    Created  2022-01-05 14:29:00
 Created By  torf 3.4.0
    Private  yes
 Piece Size  8 MiB
Piece Count  3646
 File Count  5
      Files  stuff
             ├─OVMF_CODE.fd [3.48 MiB]
             ├─OVMF_VARS.fd [528 KiB]
             ├─latest.qcow2 [2.55 GiB]
             ├─base.qcow2 [23.38 GiB]
             └─snapshot.qcow2 [2.55 GiB]
   Progress  90.04 % ▕snapshot.qcow2             ▏ 5.69 GiB/s
torf: Invalid metainfo: ['info']['pieces'] is empty

On a hunch, I removed the symlink latest.qcow2 and was able to successfully create the torrent.

I have no suggestion for how to resolve this, I just wanted to document that a symlink will cause torf to fail with an error message which does not clearly indicate the root cause of the failure.

Using the following command

torf 'sample' \
-t http://nyaa.tracker.wf:7777/announce \
-t http://open.nyaatorrents.info:6544/announce \
-t http://anidex.moe:6969/announce	\
-t http://tracker.anirena.com:80/announce \
-t udp://open.stealth.si:80/announce \
-t udp://tracker.opentrackr.org:1337/announce \
-t udp://tracker.uw0.xyz:6969 \
-t udp://tracker.coppersurfer.tk:6969/announce \
-t udp://tracker.openbittorrent.com:80/announce \
-P -o sample.torrent

shows the following error

Traceback (most recent call last):
  File "/usr/local/bin/torf", line 8, in <module>
    sys.exit(run())
  File "/usr/local/lib/python3.6/dist-packages/torfcli/__init__.py", line 21, in run
    run()
  File "/usr/local/lib/python3.6/dist-packages/torfcli/_main.py", line 43, in run
    _create_mode(cfg)
  File "/usr/local/lib/python3.6/dist-packages/torfcli/_main.py", line 71, in _create_mode
    created_by=None if cfg['nocreator'] else _config.DEFAULT_CREATOR
TypeError: __init__() got an unexpected keyword argument 'exclude'

When creating a magnet link (torf -m), it would be more useful to output only the magnet link if stdout is piped, to avoid unnecessary parsing, so that this (on bash):

magnet=$(torf -m foo)

wouldn't need any extra parsing. Of course, it's trivial to parse the output, but it feels more natural/intuitive if torf -m piped only the magnet link.

I'm getting a test failure when building version 3.2.0:

============================= test session starts ==============================
platform linux -- Python 3.8.3, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
rootdir: /build/torf-cli/src/torf-cli-3.2.0
collected 189 items

tests/test_basics.py ...                                                 [  1%]
tests/test_configfile.py .............                                   [  8%]
tests/test_configformat.py .......                                       [ 12%]
tests/test_create.py .................................................   [ 38%]
tests/test_edit.py .................................                     [ 55%]
tests/test_errors.py .....                                               [ 58%]
tests/test_info.py ....................                                  [ 68%]
tests/test_json.py ........                                              [ 73%]
tests/test_metainfo.py .......F..                                        [ 78%]
tests/test_profiles.py .....                                             [ 80%]
tests/test_progress.py ........                                          [ 85%]
tests/test_stdin.py ....                                                 [ 87%]
tests/test_utils.py ..                                                   [ 88%]
tests/test_verify.py ......................                              [100%]

=================================== FAILURES ===================================
______________________ test_metainfo_when_editing_torrent ______________________

capsys = <_pytest.capture.CaptureFixture object at 0x7fdea15c6fa0>
create_torrent = functools.partial(<function _create_torrent at 0x7fdea1875940>, local('/tmp/pytest-of-builduser/pytest-0/test_metainfo_when_editing_tor0'), local('/tmp/pytest-of-builduser/pytest-0/test_metainfo_when_editing_tor0/My Torrent'))

    def test_metainfo_when_editing_torrent(capsys, create_torrent):
        with create_torrent(trackers=['http://foo', 'http://bar']) as orig_torrent:
            run(['-i', str(orig_torrent),
                 '--comment', 'This comment was not here before.',
                 '--date', '1999-07-23 14:00',
                 '--nowebseed', '--webseed', 'https://new.webseeds',
                 '-o', 'new.torrent', '--metainfo', '-v'])
        cap = capsys.readouterr()
        assert cap.err == ''
        j = json.loads(cap.out)
        assert 'info' in j
        assert 'name' in j['info']
        assert 'pieces' not in j['info']
        assert j['comment'] == 'This comment was not here before.'
>       assert j['creation date'] == 932731200
E       assert 932749200 == 932731200

/build/torf-cli/src/torf-cli-3.2.0/tests/test_metainfo.py:135: AssertionError
=========================== short test summary info ============================
FAILED tests/test_metainfo.py::test_metainfo_when_editing_torrent - assert 93...
======================== 1 failed, 188 passed in 1.80s =========================

System information:

• OS: Arch Linux x86_64
• python: 3.8.3
• python-torf: 3.0.2
• python-pyxdg: 0.26
• python-flatbenconde: 0.2.1

On a tracker one of the requirements is set entropy field to a random number.

Is this possible with torf-cli?

First time i ever heard of entropy field and i cant find anything online about it with any torrent creator tool

It would be useful to generate a magnet link when creating a new torrent, I guess by default, or perhaps actioned by a command line option.

Here's the situation: I want to send someone a file, so I create a torrent, start seeding it, and would like to send the corresponding magnet link to this someone.

When adding trackers to a .torrent via -t t1,t2,t3, it adds them all under one block.
When adding trackers to a .torrent via -t t1 -t t2 -t t3, it does the same.

Adding via separate -t parameters, it should add that -t set + and CSV into one block, and the next -t into a new block.

some torrent clients only use the first tracker in a block.
separating each tracker into separate blocks uses all trackers at once.

Traceback (most recent call last):
  File "/usr/bin/torf", line 33, in <module>
    sys.exit(load_entry_point('torf-cli==3.2.0', 'console_scripts', 'torf')())
  File "/usr/bin/torf", line 22, in importlib_load_entry_point
    for entry_point in distribution(dist_name).entry_points
  File "/usr/lib/python3.9/importlib/metadata.py", line 524, in distribution
    return Distribution.from_name(distribution_name)
  File "/usr/lib/python3.9/importlib/metadata.py", line 187, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: torf-cli

Running Arch and using the torf-cli AUR package.

Wouldn't it make more sense to set the default creation date to now, instead of today ? I know I can set it in the config file, but it has just occurred to me that now seems a more natural, expected default. I only noticed when I was testing stig filtering rules against locally created torrents.

It would be a beneficial feature to be able to remove files from a torrent file as it would mean we do not have to create a new torrent.

Use Case:
You have a torrent that contains a large video file and some small text files.
You want to upload the torrent on another site but do not want to include the small test files only the video file.
Currently we would have to recreate the torrent with just the large video file and this can be time consuming.
As we already have the large file pieces in the exisiting torrent it would be great if we could remove the small files from the exsiting torrent.

This tool appears to have this feature - https://torrent-file-editor.github.io/

Please disregard.

Currently I'm creating very large torrents (in excess of 300GB) and I need a bigger piece size than 16MiB.

Normally this wouldn't be an issue, but due to rtorrent's poor programming it crashes when trying to force check piece hashes for a large amount of piece sizes (almost 20,000 in my case at 16 MiB)

Would it be possible to override this when explicitly giving an argument of a higher piece size than 16MiB? I understand that the piece size logic is done in the base library, but it also has the capability of being overriden by any tool that uses it (in this case torf-cli)

After doing some research, it seems that a lot of recent clients support piece sizes larger than 16MiB, and the 16MiB cap is just a result of a limitation with an old version of uTorrent.

I understand that this may lead to undesirable effects like more wasted data and much less control over selected files to download, but I don't really have a choice here.

i have a torrent that i wanted to reuse but change source and trackers but am getting the following error.

torf: Invalid metainfo: ['info']['piece length'] is invalid: 827392

torrent seems to add fine and download fine in qbittorrent.

question: any way to reuse a torrent that gives this error ['info']['piece length'] is invalid

Could it be possible to add an option like --threads or --workers to control the number of hashing threads?
As part of my own tool I am creating a benchmark of multiple bittorrent metafile utilities.
I want to include this tool but I need to control the number of hashing threads for a fair comparison.

something like --piece-size, -ps 16MiB

Is this possible with CLI? Only seeing example to make torrent/magnet from existing files.

I'm getting the following result when running the tests on torf-cli 3.1.0:

=================================== FAILURES ===================================
______________________________ test_creation_date ______________________________

capsys = <_pytest.capture.CaptureFixture object at 0x7f2545432100>
create_torrent = functools.partial(<function _create_torrent at 0x7f25456668b0>, local('/tmp/pytest-of-builduser/pytest-0/test_creation_date0'), local('/tmp/pytest-of-builduser/pytest-0/test_creation_date0/My Torrent'))
human_readable = functools.partial(<function human_readable.<locals>._human_readable at 0x7f254531d790>, <_pytest.monkeypatch.MonkeyPatch object at 0x7f25453bb250>)
clear_ansi = <function clear_ansi.<locals>._clear_ansi at 0x7f254533aee0>
regex = <class 'conftest.regex.<locals>._regex'>

    def test_creation_date(capsys, create_torrent, human_readable, clear_ansi, regex):
        date = datetime(2000, 5, 10, 0, 30, 45)
        with create_torrent(creation_date=date) as torrent_file:
            with human_readable(True):
                run(['-i', torrent_file])
                cap = capsys.readouterr()
                assert clear_ansi(cap.out) == regex(r'^\s*Created  2000-05-10 00:30:45$', flags=re.MULTILINE)
    
            with human_readable(False):
                run(['-i', torrent_file])
                cap = capsys.readouterr()
>               assert cap.out == regex(r'^Created\t957911445$', flags=re.MULTILINE)
E               AssertionError: assert 'Name\tMy Torrent\nInfo Hash\t5351b9e1a1da3e752a39ff22af929a7c47a85d90\nSize\t27\nComment\tOriginal Comment\nCreated\t...51b9e1a1da3e752a39ff22af929a7c47a85d90&dn=My+Torrent&xl=27&tr=http%3A%2F%2Fsome.tracker&ws=http%3A%2F%2Fsome.webseed\n' == ^Created\t957911445$
E                +  where 'Name\tMy Torrent\nInfo Hash\t5351b9e1a1da3e752a39ff22af929a7c47a85d90\nSize\t27\nComment\tOriginal Comment\nCreated\t...51b9e1a1da3e752a39ff22af929a7c47a85d90&dn=My+Torrent&xl=27&tr=http%3A%2F%2Fsome.tracker&ws=http%3A%2F%2Fsome.webseed\n' = CaptureResult(out='Name\tMy Torrent\nInfo Hash\t5351b9e1a1da3e752a39ff22af929a7c47a85d90\nSize\t27\nComment\tOriginal ...a3e752a39ff22af929a7c47a85d90&dn=My+Torrent&xl=27&tr=http%3A%2F%2Fsome.tracker&ws=http%3A%2F%2Fsome.webseed\n', err='').out
E                +  and   ^Created\t957911445$ = <class 'conftest.regex.<locals>._regex'>('^Created\\t957911445$', flags=re.MULTILINE)
E                +    where re.MULTILINE = re.MULTILINE

/build/torf-cli/src/torf-cli-3.1.0/tests/test_info.py:147: AssertionError
=========================== short test summary info ============================
FAILED tests/test_info.py::test_creation_date - AssertionError: assert 'Name\...
======================== 1 failed, 173 passed in 1.61s =========================

System Information:

• OS: Arch Linux x86_64
• python: 3.8.2
• torf-cli: 3.1.0
• python-torf: 3.0.0
• python-flatbencode: 0.2.1
• python-pyxdg: 0.26
• python-pytest: 5.4.1

Hi, I have a folder with 50 .torrent files. How can I run torf-cli to change the annouce and the source of these 50 .torrent files in batch mode? And, is there any way to run torf-cli in Windows terminal? Thanks in advance.

PS: I installed pipx in my window and then I installed torf. But I got this error below when run torf. Any idea how to solve this problem?

C:\>torf -i Mirc.torrent Traceback (most recent call last): File "c:\python38\lib\runpy.py", line 193, in _run_module_as_main return _run_code(code, main_globals, None, File "c:\python38\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\torf.exe\__main__.py", line 7, in <module> File "C:\Users\xandynho\AppData\Roaming\Python\Python38\site-packages\torfcli\__init__.py", line 16, in run from . import _config, _errors, _main, _ui File "C:\Users\xandynho\AppData\Roaming\Python\Python38\site-packages\torfcli\_ui.py", line 24, in <module> from . import _term, _utils, _vars File "C:\Users\xandynho\AppData\Roaming\Python\Python38\site-packages\torfcli\_term.py", line 14, in <module> import termios ModuleNotFoundError: No module named 'termios'

For example,
If the input is: torf -z public demo-folder
then the resulting .torrent file should be named something like [public] demo-folder.torrent or PUBLIC.demo-folder.torrent
and if the input is: torf -z private demo-folder
then the resulting .torrent file should be named something like [private] demo-folder.torrent or PRIVATE.demo-folder.torrent

Basically appending the profile name at the start of the torrent file to prevent overwriting and help recognizing what .torrent file is made with what profile when making multiple torrents using multiple profiles for the same folder/file.

torf 5.1.0
python 3.12.2
Windows 10 Pro

Doing a simple torf -x -p DIRECTORY will include Thumbs.db in that directory, which is undesirable and unexpected. Many users have hidden files showing in Explorer, but very few have "protected operating system files" set to show. From a quick test, I observed that qBittorrent's torrent creator excludes hidden files, both "protected system files" like Thumbs.db and just plain hidden files.

If you edit the source and a tracker at the same time the resulting torrent will be NEW and not an edited version of the original.

Steps to reproduce
1). Create new torrent if you do not have an existing one to edit.
torf somefile -t http://bar:123/announce -t http://baz:321/announce --private
2). add torrent to your torrent client
3). edit the torrent by changing the source and adding a new tracker
torf -i somefile.torrent -t http://test:123/announce -s "test" -o new-somefile.torrent
4). add torrent to your torrent client

you will see that the edited torrent did not update the original torrent in your client and instead created a new torrent.

Torrent client tested with: transmission 2.94

Can you please ad support for creating hybrid torrents as a default option but, with the option to create a v1 one torrent if required.

hybrid torrent test at the bottom:
https://blog.libtorrent.org/2020/09/bittorrent-v2/