mbdevpl / typed-astunparse Goto Github PK

Python 3 AST unparser with type comments support.

License: Apache License 2.0

Python 91.67% Jupyter Notebook 7.34% Shell 0.99%

code-generation code-generator abstact-syntax-tree type-annotations

typed-astunparse's Issues

Neither newlines nor triple quotes preserved… for docstrings

import typed_ast.ast3
import typed_astunparse

code = '''
class Snake(object):
    """
    Om nom nom
    
    and some more nom nom nom
    """
    
    def bite(me: str) -> bool:
        return me == 'cython'
'''
roundtrip = typed_astunparse.unparse(typed_ast.ast3.parse(code))
print(roundtrip)

Outputs:

class Snake(object):
    '\n    Om nom nom\n    \n    and some more nom nom nom\n    '

    def bite(me: str) -> bool:
        return (me == 'cython')

Use-case: I'm writing a docstring transformer

Problem with version-query

On my environment version-query raises ValueError. More details on mbdevpl/version-query#2

I temporally fixed issue on my environment by putting actual version number on '_version.py',

Syntax Error on Python 3.7

I am trying to run this on python 3.7.0b3 and am getting a SyntaxError when I import it. Is async a reserved keyword now?

$ python3.7
Python 3.7.0b3 (v3.7.0b3:4e7efa9c6f, Mar 29 2018, 08:11:44)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import typed_ast.ast3
>>> import typed_astunparse
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/typed_astunparse/__init__.py", line 12, in <module>
    from .unparser import Unparser
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/typed_astunparse/unparser.py", line 248
    def _generic_FunctionDef(self, t, async=False):
                                          ^
SyntaxError: invalid syntax
>>>

It does not preserve per-argument type comments

This strips per-argument type comments (example taken from PEP 484):

code = '''
def send_email(address,     # type: Union[str, List[str]]
               sender,      # type: str
               cc,          # type: Optional[List[str]]
               bcc,         # type: Optional[List[str]]
               subject='',
               body=None    # type: List[str]
               ):
    # type: (...) -> bool
    """Send an email message.  Return True if successful."""
    pass
'''

import typed_ast.ast3
import typed_astunparse

print(typed_astunparse.unparse(typed_ast.ast3.parse(code)))

Output is:

def send_email(address, sender, cc, bcc, subject='', body=None):
    # type: (...) -> bool
    'Send an email message.  Return True if successful.'
    pass

release sdist

Carry over from python/typed_ast#25 (comment)

unparsing of complex and/or nested f-strings fails sometimes

It fails due to the fact that sub-strings and values in the expressions in f-strings are unparsed without knowledge what string delimiters should be used, and therefore running

import typed_ast.ast3
import typed_astunparse
print(typed_astunparse.unparse(typed_ast.ast3.parse(
    """f'length of "hello" is {len("hello")}'""", mode='eval')))

prints

f'length of "hello" is {len(\'hello\')}'

but parsing it again

typed_ast.ast3.parse("""f'length of "hello" is {len(\\'hello\\')}'""", mode='eval')

results in an error.

Traceback (most recent call last):

  File "/home/mateusz/.local/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2910, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

  File "<ipython-input-12-b854d28c44ec>", line 1, in <module>
    typed_ast.ast3.parse("""f'length of "hello" is {len(\\'hello\\')}'""", mode='eval')

  File "/home/mateusz/.local/lib/python3.6/site-packages/typed_ast/ast3.py", line 58, in parse
    return _ast3._parse(source, filename, mode, feature_version)

  File "<unknown>", line 1
SyntaxError: f-string expression part cannot include a backslash

This would be fixed by implementing an f-string unparser that is aware which string delimiters should be used to generate code for each sub-string and expression within an f-string.

per-argument type comments in function definitions are unparsed incorrectly

The way those type comments are unparsed goes against rules of PEP484. This leads to errors when unparsed files are parsed again by typed_ast - no errors with ast.

def function_b8(spam=0,  # type: int
                ham='',  # type: str
                eggs=0.0):  # type: float
    """function with args with type comments"""
    spam, ham, eggs = None, None, None

After parsing and unparsing the above, it looks like:

def function_b8(spam  # type: int
        =0, ham  # type: str
        ='', eggs=0.0):
    # type: float
    'function with args with type comments'
    (spam, ham, eggs) = (None, None, None)

Which is valid code and runs, but it definitely is not a correct unparsing! And moreover, it looks ugly.

mbdevpl / typed-astunparse Goto Github PK

typed-astunparse's Issues

Neither newlines nor triple quotes preserved… for docstrings

Problem with version-query

Syntax Error on Python 3.7

It does not preserve per-argument type comments

release sdist

unparsing of complex and/or nested f-strings fails sometimes

per-argument type comments in function definitions are unparsed incorrectly

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent