afpy / powrap

Migrated to a free and open-source forge: https://git.afpy.org/AFPy/powrap

powrap

Script to fix indentation of given .po files. If --modified is given, it will only fix modified files according to git (useful if your .po files are versioned).

if --quiet is given, the progress bar will not be shown

Powrap is part of poutils!

Poutils (.po utils) is a metapackage to easily install useful Python tools to use with po files and powrap is a part of it! Go check out Poutils to discover the other tools!

Dependencies

powrap relies on msgcat from gettext so you'll have to install gettext first, for example on Debian run:

apt install gettext

Contributing

Start by creating a venv and pip install -r requirements-dev.txt in it.

To run the tests, use tox -p auto.

To install powrap in the current venv run pip install -e ..

Dependencies

We're using pip-tools to pin our dependencies, but in the setup.cfg our dependencies are not pinned, the goal is to ensure powrap can easily be installed along with other tools.

Dependencies pinning is only done to have a reproducible development environment and corresponding env in the CI:

pip-compile setup.py  # generates requirements.txt
pip-compile requirements-dev.in  # generates requirements-dev.txt

It's possible to upgrade pinned dependencies with the --upgrade flag of pip-compile.

powrap's Issues

powrap doesn't accept files containing undecodable characters

When I try to wrap c-api/stable.po, library/decimal.po, howto/unicode.po, etc. it throws:

usage: powrap [-h] [--modified] [-C GIT_ROOT] [--quiet] [--diff] [--check] [--version] [--no-wrap] [po_files ...]
powrap: error: argument po_files: invalid path value: 'library/decimal.po'

Also, msgcat can't write into the created NamedTemporaryFile, when run with --check.

EgeAk@egelaptop powrap on main 🐍 v3.10.9  in powrap
🕙 15:20:48 ❯ python -m powrap about.po --check
Checking wrapping of po files:   0%|                                                             | 0/1 [00:00<?, ?it/s]msgcat: cannot create output file "C:\Users\EgeAk\AppData\Local\Temp\tmp0megkadg": Permission denied
Error processing 'about.po': Command '['msgcat', '-', '-o', 'C:\\Users\\EgeAk\\AppData\\Local\\Temp\\tmp0megkadg']' returned non-zero exit status 1.
Checking wrapping of po files: 100%|█████████████████████████████████████████████████████| 1/1 [00:02<00:00,  2.52s/it]

It also adds whitespace at the top of the first msgid-msgstr pair on every file. (This turns out to be about MSYS msgcat not powrap itself)

# Python Documentation Turkish Translation
# Copyright (C) 2001-2023, Python Software Foundation
# This file is distributed under the same license as the Python package.
-#
+# 
msgid ""
msgstr ""

I tried this on different installations, different computers, and various shells (Git Bash, PowerShell 5 and 7, cmd). WSL (Ubuntu) seemed to have no problem though.

Any idea why this is happening?

powrap chokes on non-ascii characters on Windows

Calling powrap on a translated file on Windows gives the error below. It chokes on the first accentuated characters before giving up. Calling directly msgcat from the Windows shell works.
It might be something like Python subprocess.Popen function is feeding the msgcat subprocess with native Windows encoding (which something like utf-16) instead of utf-8. Passing encoding='utf-8' to subprocess.run fixes it BUT I can not garante yet it will not break another platform.

Fixing indentation in po files:   0%|          | 0/1 [00:00<?, ?it/s]<stdin>:37:25: invalid multibyte sequence
<stdin>:37:26: invalid multibyte sequence
<stdin>:38:26: invalid multibyte sequence
<stdin>:38:28: invalid multibyte sequence
<stdin>:38:29: invalid multibyte sequence
<stdin>:38:46: invalid multibyte sequence
<stdin>:38:47: invalid multibyte sequence
<stdin>:38:50: invalid multibyte sequence
<stdin>:38:51: invalid multibyte sequence
<stdin>:38:55: invalid multibyte sequence
<stdin>:38:56: invalid multibyte sequence
<stdin>:38:64: invalid multibyte sequence
<stdin>:38:65: invalid multibyte sequence
<stdin>:40:14: invalid multibyte sequence
<stdin>:40:15: invalid multibyte sequence
<stdin>:48:37: invalid multibyte sequence
<stdin>:48:38: invalid multibyte sequence
<stdin>:49:6: invalid multibyte sequence
<stdin>:49:7: invalid multibyte sequence
<stdin>:49:29: invalid multibyte sequence
msgcat: too many errors, aborting

Traceback (most recent call last):
  File "c:\program files\python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\program files\python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\mdupuy\AppData\Roaming\Python\Python36\Scripts\powrap.exe\__main__.py", line 9, in <module>
  File "C:\Users\mdupuy\AppData\Roaming\Python\Python36\site-packages\powrap\powrap.py", line 56, in main
    fix_style(args.po_files, args.modified, args.no_wrap)
  File "C:\Users\mdupuy\AppData\Roaming\Python\Python36\site-packages\powrap\powrap.py", line 33, in fix_style
    run(args, universal_newlines=True, check=True, input=po_content)
  File "c:\program files\python36\lib\subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['msgcat', '-', '-o', 'faq/design.po']' returned non-zero exit status 1.

Bad handling of errors

powrap throws an exception when msgcat fails. IMO, it would be better to print an error message and exit with an appropriate code.
See this python-docs-fr documentation build.

Dependabot couldn't authenticate with https://pypi.python.org/simple/

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Implement an option to wrap only modified paragraphs

As msgcat now have two distinct wrapping depending on the version of gettext, powrap can no longer enforce a "common True wrapping".

Trying to enforce a common True wrapping in this condition looks wrong, but users could still have to use powrap (users of emacs po-mode that does not wrap at all need a tool to wrap, so at least me.).

But if a repo (like python-docs-fr) have no "common True wrapping" and I edit a typo on a file and I powrap the file, I may rewrap the whole file, leading to an unacceptable diff for a typo.

So maybe powrap can have an option to rewrap only paragraphs that were edited AND need rewrapping, just to minimize diffs.

See python/python-docs-fr#1825 for context.

Also while there no longer a "common True wrapping", there no longer a real need to use msgcat, we may fall back to textwrap for portability?

Keep getting this error

Traceback (most recent call last):
  File "/home/jighdan/github/env/bin/powrap", line 8, in <module>
    sys.exit(main())
  File "/home/jighdan/github/env/lib/python3.8/site-packages/powrap/powrap.py", line 125, in main
    fix_style(args.po_files, args.no_wrap, args.quiet)
  File "/home/jighdan/github/env/lib/python3.8/site-packages/powrap/powrap.py", line 45, in fix_style
    run(args, encoding="utf-8", check=True, input=po_content)
  File "/usr/lib/python3.8/subprocess.py", line 489, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'msgcat'

I have used different environment managers (venv and conda), install powrap from source; but still getting the same error.

Missing dependency in requirements.in

Looks like install dependency tqdm defined in setup.cfg is missing from requirements.in file.
It is however included 'by sheer luck' in requirements.dev as it is a dependency of twine.
On a side note, blake is duplicated.

powrap does not work on Windows

Steps to reproduce:

• git clone https://github.com/yeyeto2788/python-docs-es.git
• git checkout traduccion-misc
• pip install powrap
• powrap .\library\misc.po

Error:

(venv) C:\Users\yeyeto2788\workspace\python-docs-es\library>powrap "misc.po"
Fixing wrapping of po files:   0%|                                                                                                                                                            | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "C:\Program Files\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\yeyeto2788\workspace\python-docs-es\venv\Scripts\powrap.exe\__main__.py", line 9, in <module>
  File "c:\users\yeyeto2788\workspace\python-docs-es\venv\lib\site-packages\powrap\powrap.py", line 142, in main
    fix_style(args.po_files, args.no_wrap, args.quiet)
  File "c:\users\yeyeto2788\workspace\python-docs-es\venv\lib\site-packages\powrap\powrap.py", line 55, in fix_style
    run(args, encoding="utf-8", check=True, input=po_content)
  File "C:\Program Files\Python36\lib\subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Program Files\Python36\lib\subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "C:\Program Files\Python36\lib\subprocess.py", line 971, in _execute_child
    args = list2cmdline(args)
  File "C:\Program Files\Python36\lib\subprocess.py", line 461, in list2cmdline
    needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: argument of type 'WindowsPath' is not iterable

OS: Windows 10
Python: Python 3.6.5

No matter what path I pass over the arguments that I would give me the same error.

P.D: Might not be the best OS in the world but that's what I have for now 😢

Progress bar when nothing

powrap -m affiche une barre de progression même quand il ne fait rien (et ça confuse les débutants qui ont déja commit leurs changements)

other directory + `--modified` = "fatal: not a git repository"

When using --modified flag to wrap po files in another directory (e.g. powrap --modified some-dir/*.po), powrap 0.3.0 seems to expect the current directory to be a Git checkout (instead of the specified one) and fails with the following output:

fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
  File "/home/runner/.local/bin/powrap", line 8, in <module>
    sys.exit(main())
  File "/home/runner/.local/lib/python3.8/site-packages/powrap/powrap.py", line 139, in main
    git_status = check_output(
  File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'status', '--porcelain', '--no-renames']' returned non-zero exit status 128.

Note:

• This doesn't happen when running on the same dir (e.g. powrap --modified *.po)
• This doesn't happen without the flag (e.g. powrap some-dir/*.po)

Return 1 if a file has been modified in a CI environement

python/python-docs-fr#602

see issue above and black. So we can add it to python-docs-fr travis

powrap exits status 1 when done what is expected

powrap returns exit status 1 when at least one file was modified. When running in a Makefile this can problematic because it may end with error status and without any clue. It is possible to work around it with powrap *.po **/*.po || true, but then real problems will not be spotted.

NOTE: This is documented in --help for the --check flag, but not documented for normal run (e.g. powrap *.po **/*.po).

Please consider adding a flag that disable this (or disable by default and add flag that enable this).

Remove msgcat dependency

Polib library is clossplatform and has a stable API. Can be used to create the differences between pofiles without the need of install gettext.