purcell / airspeed Goto Github PK

View Code? Open in Web Editor NEW

91.0 91.0 38.0 147 KB

A lightweight Python template engine compatible with Velocity, used in OpenStack

License: Other

Python 99.42% Nix 0.58%

airspeed's People

Stargazers

Watchers

airspeed's Issues

Remove dev suffix from release versions

As can be seen in PyPI release history, all releases look like developmental releases (see https://www.python.org/dev/peps/pep-0440/#developmental-releases) but they are actually standard releases.

So, developmental releases require (pip) --pre to install, but airspeed releases install without it.

It's just a bit confusing :)

(P.S. How mature/stable/reliable do you think the library is? Is it used in production somewhere? The version numbers just give impression that it might not be the case?).

Upgrade the version of the dependent library - "cache-tools"

I have upgrade the version to 2.0.1, everything is ok.

Velocity 1.7

Hi all
Are you planning on making airspeed compliant with Velocity 1.7?
There are some deprecated constructs, such as $velocityCount (now available as $foreach.count or $foreach.index)

The changes are outlined in
http://velocity.apache.org/engine/releases/velocity-1.7/upgrading.html

best wishes,
Boris

Deprecation warning due to invalid escape sequences.

Deprecation warning due to invalid escape sequences. Using raw strings or escaping them again helps in resolving this. Check https://github.com/asottile/pyupgrade/ for automatic fix of this.

find . -iname '*.py' | grep -Ev 'rdf4|doc|benchmark|tool' | xargs -P4 -I{} python3.8 -Wall -m py_compile {}
./airspeed/__init__.py:628: DeprecationWarning: invalid escape sequence \.
  DOT = re.compile('\.(.*)', re.S)
./airspeed/__init__.py:753: DeprecationWarning: invalid escape sequence \*
  '#(?:#.*?(?:\n|$)|\*.*?\*#(?:[ \t]*\n)?)(.*)$',

Support #define directive block

Hi @purcell, I'm having issues handling the #define directive (example below), and wondering if it's in your plans to support it?

import airspeed

def test_templating():
    template = """
  #define( $loop )
  #end
 """
    t = airspeed.Template(template)
    replaced = t.merge(locals())

if __name__ == "__main__":
    test_templating()

Currently it raises this exception airspeed.TemplateSyntaxError: line 5, column 3: expected block element in template body, got: #end and navigating the code I couldn't figure a handler for it

Wrong newline/whitespace behavior

See the section "Formatting Issues" in https://velocity.apache.org/engine/releases/velocity-1.5/user-guide.html, about gobbling up excess whitespace. It's not the behavior of airspeed.

airspeed-0.5.2dev_20150417 seems to not always be able to find variables in namespace.

airspeed-0.5.2dev_20150417 seems to not always be able to find variables in namespace.
airspeed-0.4.2dev_20141218 works fine.

The namespace dictionary contains the 'assembly' key.
In some cases everything worked fine (airspeed-0.5.2dev_20150417) when generating the output
files, in others (namespace dictionary contains some larger objects) it gave the following crash.

template_base_folder = os.path.dirname(template_filename)

with open(template_filename, 'r') as input_fh:
    template = airspeed.Template(input_fh.read())

output_fh.write(str(template.merge(namespace, loader=airspeed.CachingFileLoader(template_base_folder))))
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 70, in merge
    self.merge_to(namespace, output, loader)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 81, in merge_to
    self.root_element.evaluate(fileobj, namespace, loader)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 329, in evaluate
    return self.evaluate_raw(*args)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 1175, in evaluate_raw
    self.block.evaluate(stream, namespace, loader)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 329, in evaluate
    return self.evaluate_raw(*args)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 1203, in evaluate_raw
    child.evaluate(stream, namespace, loader)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 329, in evaluate
    return self.evaluate_raw(*args)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 917, in evaluate_raw
    self.block.evaluate(stream, namespace, loader)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 329, in evaluate
    return self.evaluate_raw(*args)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 1203, in evaluate_raw
    child.evaluate(stream, namespace, loader)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 329, in evaluate
    return self.evaluate_raw(*args)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 1159, in evaluate_raw
    self.block.evaluate(stream, namespace, loader)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 329, in evaluate
    return self.evaluate_raw(*args)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 1203, in evaluate_raw
    child.evaluate(stream, namespace, loader)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 335, in evaluate
    TemplateExecutionError(self, exc_info), exc_info[2])
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 329, in evaluate
    return self.evaluate_raw(*args)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 698, in evaluate_raw
    value = self.expression.calculate(namespace, loader)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 633, in calculate
    value = self.part.calculate(namespace, loader, global_namespace)
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 579, in calculate
    result = current_object[self.name]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]

...

  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 213, in __getitem__
    return self.parent[key]
  File "/virtenv/local/lib/python2.7/site-packages/airspeed-0.5.2dev_20150417-py2.7.egg/airspeed/__init__.py", line 211, in __getitem__
    return dict.__getitem__(self, key)
TemplateExecutionError: Error in template '<string>' at position 1487-1498 in expression: ${assembly}
RuntimeError: maximum recursion depth exceeded

When reverting back to airspeed-0.4.2dev_20141218 (which I used before) everything works fine.

I tried to isolate a test case with the contents of namespace dictionary that causes the crash, but in
the isolated case it worked.

pip install --pre airspeed now fails in virtual environment without version qualifier when installing latest

Pip install is failing from within a new virtualenv, as shown below. Python version 2.7.6 on Ubuntu 14.04.1 LTS was used in this test.

The exact pattern shown below succeeded on airspeed-0.5.2dev-20150216.tar.gz, before today's update.

Using pip install --pre airspeed==0.5.3dev-20150421 seems to resolve the issue for Ubuntu virtual environments.

Strange that pip install --pre picks up macosx version of package on Ubuntu, but only within a virtualenv, not within global Python environment, which seems to pick the appropriate package.

Failing virtualenv test case on Ubuntu for install of latest:

root@saltmaster:# mkdir test_virtualenv
root@saltmaster:# virtualenv test_virtualenv
New python executable in test_virtualenv/bin/python
Installing setuptools, pip...done.
root@saltmaster:# source test_virtualenv/bin/activate
(test_virtualenv)root@saltmaster:# pip install --pre airspeed
Downloading/unpacking airspeed
Downloading airspeed-0.5.3dev-20150421.macosx-10.10-x86_64.tar.gz
Running setup.py (path:/root/test_virtualenv/build/airspeed/setup.py) egg_info for package airspeed
Traceback (most recent call last):
File "", line 17, in
IOError: [Errno 2] No such file or directory: '/root/test_virtualenv/build/airspeed/setup.py'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):

File "", line 17, in

IOError: [Errno 2] No such file or directory: '/root/test_virtualenv/build/airspeed/setup.py'

Cleaning up...
Command python setup.py egg_info failed with error code 1 in /root/test_virtualenv/build/airspeed
Storing debug log for failure in /root/.pip/pip.log

#define - Assigns a block of VTL to a reference

import airspeed
d = """
#define( $hello ) Hello $who #end 
#set( $who = "World!") $hello 
"""
t = airspeed.Template(d)
o = t.merge({   })
print o

http://velocity.apache.org/engine/devel/vtl-reference-guide.html

newlines in macro call

this is working in java but not with airspeed

import airspeed

d = """
#macro( somemacro $a $b $c )
    Funky
#end

#somemacro("a", "b",
         "c") 

"""

t = airspeed.Template(d)
o = t.merge({   })
print o

order of operations non-standard

import airspeed
import sys

testooo_template = """
#set ($should_be_6_05 = 7.8 - (1 * 2 - 0.125 * 2) )
#set ($will_be_6_05 = 7.8 - ((1 * 2) - (0.125 * 2)) )

this should_be_6_05: >>> $should_be_6_05 <<<  \#set (\$should_be_6_05 = 7.8 - (1 * 2 - 0.125 * 2) )

this will_be_6_05: >>> $will_be_6_05 <<<       \#set (\$will_be_6_05 = 7.8 - ((1 * 2) - (0.125 * 2)) )

"""

partfile_template = airspeed.Template(testooo_template)
sys.stdout.write(partfile_template.merge({}))


python3 /home/nathan/ . . . /airspeed_ooo_test.py

this should_be_6_05: >>> 4.05 <<<  #set ($should_be_6_05 = 7.8 - (1 * 2 - 0.125 * 2) )

this will_be_6_05: >>> 6.05 <<<       #set ($will_be_6_05 = 7.8 - ((1 * 2) - (0.125 * 2)) )

Csv Reader not treated as Iterable?

I'm not an experienced Python developer but if I do the following (Python 2.7)

with open(csvfile, 'rb') as inputfile:
        reader = csv.DictReader(inputfile) # or csv.reader(inputfile)
        variables['reader'] = reader

print template.merge(variables, loader = loader)

with this template

#foreach($row in $reader)
## use $row here
#end

I get an error

ValueError: value for $row is not iterable in #foreach: <csv.DictReader instance at 0x02C38030>

but I have to change

variables['reader'] = reader

variables['reader'] = list(reader)

to get it to work. Is that expected behaviour? It was my impression that the csv reader could be treated as iterable, seeing as in standard python this works:

for row in reader
    print row

#set directive: Assignment value lost after foreach loop

Variables that are created via #set prior to a foreach block are readable from within that block, but any assignments to that variable are forgotten after the foreach block. Here's a test case (flofuchs@bc1c280) that illustrates the behaviour:

def test_outer_variable_assignable_from_foreach_block(self):
    template = airspeed.Template(
        "#set($var = 1)"
        "#foreach ($i in $items)$var,#set($var = $i)#end"
        "$var")
    self.assertEquals("1,2,3,4", template.merge({"items": [2, 3, 4]}))

AssertionError: '1,2,3,4' != u'1,2,3,1'

Using the original Java engine $var is rendered as 4 after the loop.

Other than that: Thanks for doing a Python implementation of Velocity. It's been helping our team greatly in a long-term project.

Florian

Prefer dict items over built-in methods

Currently it's not possible to use keys like 'items' in dictionaries (maps), because it has a method items. I'd suggest to prefer dictionary items over methods in case they collide.

Cut 0.6.0 release on github

Hi, on PyPI there is already a 0.6.0 release, but on github there is no corresponding tag. Would it be possible to create this tag here as well, so the distros that use github as reference are notified about the update?

Support for Map.put and Array.add

I'm going to implement Java Velocity Map.put method.

Airspeed treats single number sign (hash) as the beginning of a single line comment

Airspeed treats single number sign (hash) as the beginning of line comment. Velocity documentation says that single line comments should start with two hashes.

I was trying to use airspeed to generate markdown content. I've tried it with similar template:

# $now Some Text

Missing support for trim method.

I do not sure if this is a bug or a feature (according to #47 (comment)). But if I execute:

import airspeed

t = airspeed.Template("""
#set($some="      some  ")
$some.trim()
$some.strip()
""")
people = [{'name': 'Bill', 'age': 100}, {'name': 'Bob', 'age': 90}, {'name': 'Mark', 'age': 25}]
print(t.merge(locals()))

the output is:


$some.trim()
some

so, for some reason the trim method used to remove white spaces in Java need to be changed to its Python equivalent.

Support for identifiers using hyphens

Velocity supports hyphenated identifiers if configured by changing the parser configuration. I made a PR that adds support for that. Still, I'd like to know from you what would be the best approach to make that conditional, similar to what Velocity does at the moment - only support dashed identifiers if the parser is configured to do so.

We have a couple of options:

Create two conditional regex attributes, depending on the configuration setting (defaulting to the existing behaviour).
Do some attribute reassignment using __call__, but probably to much magic for what it's worth.

Nevertheless, we would need some transversal way to pass the parser configuration into the parsing classes. Any ideas?

https://velocity.apache.org/engine/2.3/vtl-reference.html#set-establishes-the-value-of-a-reference - parser.allow_hyphen_in_identifiers

Macro referenced before used issue

Airspeed throws an exception if the macro is referenced in the template before it is defined. This works normally in Apache Velocity.

Code example:

import airspeed

t = airspeed.Template("""
#set($value = 'my test')
${instancesuper}
#if(!$!{instancesuper})
missing end
#end
Value is '$value'
#new_macro('A','B')
#macro(new_macro $name $surname)
Your name is $name $surname
#end
Testing macro
#new_macro('A','B')"""
)


print(t.merge(locals()))

This throws a following exception:

raise Exception('no such macro: ' + self.macro_name)
airspeed.TemplateExecutionError: Error in template '<string>' at position 102-121 in expression: #new_macro('A','B')
Exception: no such macro: new_macro

Add feature to check VTL syntax

I would be handy to have the ability to check the syntax of VTL documents without actually trying to render them. For example, check the syntactical validity of logic in a VTL document without actually executing it. I need to check the syntax of some VTL documents but I do not have data for the variables that the template uses.