marrow / cinje Goto Github PK
View Code? Open in Web Editor NEWA Pythonic and ultra fast template engine DSL.
License: MIT License
A Pythonic and ultra fast template engine DSL.
License: MIT License
Looking for ideas on how to do this one. Stop-gap measure: include a "source line number" comment on every generated line originating in the source file? We are decorating each produced function, so in theory we can actually grab the exception itself and manipulate it. Comments are stripped from bytecode in optimized execution modes, so we'd need to store line number mapping somewhere else…
Can be used by a wide range of programmers' editors.
urwid
or similar curses
interface for exploring the benchmarks:
WebCore tutorial step using Bless #{} will return:
SyntaxError: decoding with 'cinje' codec failed (AssertionError: Unable to identify handler for line; this should be impossible!)
Test case (based on WebCore tutorial)
https://www.irccloud.com/pastebin/raw/fJubH7dP
This allows us to execute the encoding import and registration on Python startup.
An example approach can be seen in the setup.py
for pytest-cov
.
The following example produces strangeness:
: def template
: if False
Hi.
This generates the following code (pre-1.0):
def template(*, _escape=_escape, _bless=_bless, _args=_args):
if False:
_buffer = []
__w, __ws = _buffer.extend, _buffer.append
__ws(' Hi.\n')
yield "".join(_buffer)
The _buffer
allocation being itself conditional results in problems. Lame, naive solution: have all template functions call ensure_buffer
immediately.
They have no effect, an only clutter the generated code.
For examples:
: use myapp.templates.master:master_template
: using foo.bar:baz
if
, elif
, and else
.while
and for
, with an optional else
run-off-the-end chained block.if
, elif
, and else
.while
and for
, with an optional else
run-off-the-end chained block.According to nedbat
on IRC, coverage.py
version 4.0 adds the capability to write plugins. We should totally do this.
Simplest test case:
# encoding: cinje
: def testcase
: for i in ('hello', 'world')
${i}
Ignoring default imports, optimizations, and line mapping, the transformed result is currently:
def testcase():
for i in ('hello', 'world'):
_buffer = []
__w, __ws = _buffer.extend, _buffer.append
__w((
'\t\t',
_escape(i),
'\n'
))
yield "".join(_buffer)
Note the dangerous location of buffer construction. If the loop were empty, it would not be called, but the code transformer considers the buffer prepared after that point regardless. In the empty test case, the subsequent yield
will explode as _buffer
would be undefined. Additionally, the buffer is overwritten on each iteration here.
Temporary workaround: include some emitted content prior to a function's first iterator. An HTML comment works well.
Simple fix: force buffer construction after function declaration.
Some resources to review to facilitate this:
Allow and adapt to async def
usage. This ticket, vs. #9, is more about integration into an asyncio
environment and less about inverting the template rendering call tree.
Simple test case:
# encoding: cinje
: def tmpl
A sample of Unicode™ text.
Without the ™ symbol, all is well. With it, a substantial traceback is produced:
/Users/amcgregor/Projects/cegid/cinje/cinje/encoding.pyc in cinje_decode(input, errors, final)
18 def cinje_decode(input, errors='strict', final=True):
19 if not final: return '', 0
---> 20 output = transform(bytes(input).decode('utf8', errors))
21 return output, len(input)
22
/Users/amcgregor/Projects/cegid/cinje/cinje/encoding.pyc in transform(input)
12 def transform(input):
13 #__import__('pudb').set_trace()
---> 14 translator = Context(input)
15 return '\n'.join(str(i) for i in translator.stream)
16
/Users/amcgregor/Projects/cegid/cinje/cinje/util.pyc in __init__(self, input)
449
450 def __init__(self, input):
--> 451 self.input = Lines(input.decode('utf8') if isinstance(input, bytes) else input)
452 self.scope = 0
453 self.flag = set()
/Users/amcgregor/Projects/cegid/cinje/cinje/util.pyc in __init__(self, input, Line)
397
398 else:
--> 399 self.source = list(self.Line(i + 1, j) for i, j in enumerate(input.split('\n')))
400 self.buffer = deque(self.source)
401
/Users/amcgregor/Projects/cegid/cinje/cinje/util.pyc in <genexpr>((i, j))
397
398 else:
--> 399 self.source = list(self.Line(i + 1, j) for i, j in enumerate(input.split('\n')))
400 self.buffer = deque(self.source)
401
/Users/amcgregor/Projects/cegid/cinje/cinje/util.pyc in __init__(self, number, line, scope, kind)
318 def __init__(self, number, line, scope=None, kind=None):
319 if isinstance(line, str):
--> 320 line = line.decode('utf-8')
321
322 self.number = number
/Users/amcgregor/Projects/cegid/rita/.venv/lib/python2.7/encodings/utf_8.pyc in decode(input, errors)
14
15 def decode(input, errors='strict'):
---> 16 return codecs.utf_8_decode(input, errors, True)
17
18 class IncrementalEncoder(codecs.IncrementalEncoder):
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2122' in position 19: ordinal not in range(128)
Looks like a double decoding: util.py:451, util.py:320 (incorrect isinstance
check if normalized str
has been imported from compat
)
Note: Unlike #2, this ticket is more about the inversion of the template include pattern and less about the pure integration of cinje into an asyncio
environment, though such integration does need to be kept in mind.
Currently the yield
mechanics are used to identify an insertion point for additional content in a template, creating a "wrapping" template function. This gives you an entry point of the most specific template, which then typically dives into its wrapper, then deeper into whatever that is wrapped by internally, etc., etc.
The inverse approach, where you have a "layout" template which you populate with content blocks, is also extremely useful. Because templates are first-class functions, you can easily pass individual, or whole lists of template functions around, using functools.partial
to bind values to them for use at render-time. This is a start, as during template processing the layout template would descend into each content block to render, blocking the overall process. Instead, when a child template is encountered, if it's an async def
it can be sent up to the reactor for processing. In the resulting HTML insert a placeholder or marker (a la the animated bars on Facebook posts as they load), finally streaming the layout template, then individual content blocks, as MIME multipart. Should also optionally be able to execute the template more classically, without async deferral.
Streaming chunks to the browser as completed can be accomplished via MIME multipart AJAX, via something like mpAjax. A rough p-code example using Futures:
def layout(reactor, *blocks):
"""Mock boostrap row/column layout."""
# Prepare some content.
_buffer = []
_tasks = []
for block in blocks:
if not _buffer or not block:
if _buffer:
_buffer.extend(('</div><div class="row">\n', ))
_buffer.extend(('<div class="row">\n', ))
if not block:
continue
_tasks.append(reactor.submit(block))
_buffer.extend(('<div class="placeholder" data-await="', id(_tasks[-1]), '"></div>\n'))
return (''.join(_buffer), _tasks)
def render_page():
identifier = "gc0p4Jq0M2Yt08jU534c0p"
response.content_type = "multipart/mixed; boundary=" + identifier
page, content = layout(executor, [
"some render function, first column",
"some other render function, second column",
None,
"lastly a full width footer",
])
yield page
for chunk in as_completed(content):
yield '--' + identifier + '\nIdentifier: ' + str(id(chunk)) + '\n\n' + chunk.result()
yield '--' + identifier + '--\n'
Currently only one directive attempts to split anything on an expression, the cinje.inline.text
processor when handling %{expr arg}
formatted string replacements. This is broadly useful, notably for the : use
and : using
directives amongst others, and better aligns with the documentation.
cinje.inline.text
"expression split" code into the Line
class. (As a companion for the partitioned
attribute.)cinje.inline.use
directive to utilize the expression split.cinje.block.using
directive to utilize the expression split.Where currently you require multiple lines with explicit flow control statements:
: for item in items
: if item % 2 == 0
: continue
: end
Permit inclusion of filtering conditions (one or more) in the form:
: for item in items if item % 2
There are several avenues for translation to plain Python for these. Unwrap and reconstruct the multiple-line form, or alternatively wrap in a generator comprehension:
for item in (i for i in items if item % 2):
Hey there,
Seems not to run on python 3.9:
$ python benchmark.py
Traceback (most recent call last):
File "/home/ybon/Code/py/cinje/example/benchmark.py", line 445, in <module>
import bigtable
File "/home/ybon/Code/py/cinje/example/bigtable.py", line 106
__gzmapping__ = b"eJxjYGJkAANGMEBmMDAxMTAjCWETAwsRlmBgAAAPKwBL"
^
SyntaxError: unexpected EOF while parsing
or am I missing something ?
Thanks :)
Python 3.7 has removed this as an acceptable behaviour, as such, cinje breaks on Python 3.7 as demonstrated after adding 3.7 to the Travis-CI test rig, and exploring explosions locally (which are either hilariously non-existent or egregiously verbose exception chains).
Some ideas for flags:
__w((
to yield "".join((
.)Thinking something like an extruded _⊓_⊓_
corrugated tin sheet to match the meanings of the name.
Notably pip
installation directly from the Git repo for those not needing to modify or contribute, and a brief outline of how to add Git-based dependencies to the application setup.py
.
Reminder: Update the Marrow project blueprint README template with this information, too.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.