smfoote / tornado

HTML templates with asynchronous rendering, built with JavaScript.

Tornado—Fast, Secure, HTML templates

Tornado is an HTML templating language for the server and the browser. In the browser, Tornado renders DOM immediately and dynamic values asynchronously as the data becomes available.

Trying Tornado

The easiest way to try Tornado is with the sandbox. More complete documentation is coming soon.

Using Tornado

To install the Node.js package:

$ npm install tornado.js

Compiling templates in Node

var tornado = require('tornado.js');
var fs = require('fs');

var templateString = '<h1>Hello, {name}</h1>';
var templateName = 'hello';

var compiledTemplate = tornado.compile(templateString, templateName);
fs.writeFileSync('hello.js', compiledTemplate);

Compiling templates from the command line

Use tornado --help for more information on CLI options

$ tornado hello.td > hello.js

Using your compiled templates

Now your templates are compiled and you are ready to use them on your page. You need both the Tornado runtime and the compiled templates on the page, and the Tornado runtime must be included first. The runtime can be downloaded from the latest release.

var container = document.getElementById('content-container');
var template = td.getTemplate('hello');
container.appendChild(template.render({ name: 'World' }));

Contributing

If you are reading this section, you are awesome! Thanks for your interest in contributing. For more information on contributing, please review the Tornado Constitution. A more comprehensive contributors' guide will be ready soon. In the meantime, following the steps below will get you up and running.

Building

To build the project, run (this needs to be run after each change is made to code in the src/ directory):

$ grunt

Sandbox

You can use the Sandbox to try out your changes and see output, compiled template, and the AST. To prepare the Sandbox, run:

$ grunt sandbox

Then open /test/sandbox/index.html in your browser.

Tests

Acceptance tests are found in test/acceptance/. If you make changes to the tests, run:

$ grunt acceptance

To see the results of the tests, open tests/testRunner.html in your browser.

tornado's Issues

HTML comments are not created/inserted

HTML comments don't (and shouldn't) affect how the page renders. However, they should be inserted on the page. Only Tornado comments can safely be ignored.

Sections don't insert into the correct position.

If a section (array-style) is not the last item in an HTML element, it will overwrite all of the subsequent items in the element.

For example:

<p>My friends ({friends}{name}, {/friends}) are so <span>great</span>!</p>

with context:

{
  friends: [
    {name: 'Jimmy'},
    {name: 'Prash'},
    {name: 'Seth'},
    {name: 'Kate'}
  ]
}

becomes:

<p>My friends (Jimmy, Prash, Seth, Kate, </p>

Move svg attribute adjustment to a pass

Create a {:pending} body to display while waiting for a promise to resolve

For example:

{#apiData}
  {! Render the page !}
{:pending}
  {! Show the loading page !}
{/apiData}

SVG doesn't work at all

Because SVG requires an XML namespace, elements within the SVG element must be created within the appropriate namespace, using document.createElementNS. Hopefully this will only require a change to the compiler.

Remove excess whitespace in a transform pass

Leading whitespace is insignificant in certain HTML elements. Removing this whitespace can make the compiled templates a lot smaller.

Use a proper testing framework.

It would be a shame to have to rewrite all of the tests, so hopefully that's not necessary, but we should use an established test runner (along with the compareNode code) for testing.

Is it possible to not use tdBody ?

experiment with not having to keep track of tdBody index

`@eq`, `@ne`, and their friends fail when `key` or `val` is 0

Dotted paths with no value cause an error

If your template contains {a.b}, and your context is {}, an error is thrown.

This needs:

• A test
• A fix

Create a performance dashboard

The dashboard should have:

• comparisons to other templating languages for compiling and rendering
• comparisons to past versions of Tornado for compiling and rendering
• runtime file size comparisons to other languages and past versions of Tornado
• helpers file size comparisons to other languages and past versions of Tornado
• compiled template size comparisons to other languages and past versions of Tornado

HTML Entities that are hex or decimal are not recognized

For example 𝔟, 𝔟, and 𝔟 should all render the same character (𝔟), but only 𝔟 works.

Compiler does not compile blocks or inline partials

Add looping helpers

• @sep
• @idx
• @fisrt
• @last

Create version of compiled output that requires no runtime

I want to be able to compile a template that I can import without requiring any additional dependencies.

import blogPostTemplate from './templates/blogPost';

...
  render(model) {
    // for demonstration purposes only
    container.innerHTML = '';
    container.appendChild(blogPostTemplate.render(model));
  },

...

Let the AST visitor do the walking

Some node handlers (TORNADO_BODY, HTML_ELEMENT, buildElementAttributes) currently call walk, walkAttr or step. Can this logic be moved into the parser instead?

Add `td.assert` to check param validity (and other stuff like that)

Instead of having a bunch of logic in the function code to check that a param exists and is of the right type, handle that logic in td.assert, that throws errors with useful descriptions when something is not right.

Asynchronous values at the root of a fragment are not rendered

The issue is that the parentNode is cached as the document fragment, and when the document fragment is appended, the document fragment is no longer the parentNode of the placeholder node.

Move HTML entities out of grammar, into a pass

SVG and mathML

Parsing stuff inside of SVG is silly

https://html.spec.whatwg.org/multipage/syntax.html#parsing-main-inforeign

https://html.spec.whatwg.org/multipage/syntax.html#adjust-svg-attributes

Create comparison helpers

• @eq
• @ne
• @gt
• @gte
• @lt
• @lte

Sibling tornado bodies don't render correctly

The tdIndex in the compiler only accounts for nested tornado bodies. If there are siblings, only the last sibling will compile.

Compiler does not compile helpers

Move prod/dev short method name logic into an optimization pass

Allow templates be compiled for dev (verbose) and production (more minifiable)

This will probably require creating aliases for all of the runtime methods and a mapping between aliases and full method names in the compiler.

Compiler does not compile partials

consider using document.implementation.createHTMLDocument

... instead

IE9+
https://developer.mozilla.org/en-US/docs/Web/API/DOMImplementation
we could add a fallback to fragment but IE9+ seems sufficient.

it's what jQuery uses and is more secure since it won't execute scripts/download images immediately.

http://stackoverflow.com/questions/7738046/what-for-to-use-document-implementation-createhtmldocument

Provide an API for compiler pass authors to modify the AST

Make it easy to snip/add a leaf (during a visit ignore this child)
Make it easy to prune/add a branch
Make it easy to modify the identity of a node etc.

Use es6 module instead of module.exports

HTML comments break tornado references

I think it has something to do with index counting in the compiler.

Allow visitor to pass arguments

Server Side Rendering

Tornado templates should be able to render on the server with Node. There are Node implementations of the DOM API (e.g. simple-dom) that should get us most of the way there, but there are still a few things lacking.

• tornado#render may need to return a promise that resolves when all internal promises have resolved, because values that resolve after the server responds would otherwise not be included in the HTML payload.
• simple-dom does not support the innerHTML property, which is used within the Tornado runtime, and seems like the easiest way to turn the DOM returned by Tornado into a string.
• because Tornado creates DOM instead of strings, it seems like streaming (and thus early flush) is not possible. This needs further investigation.

Start writing documentation

• Decide where the documentation should live
• Write template syntax docs
• Write Tornado API docs
• Write command line docs
• Write guides

Create a command line tool for compiling Tornado templates.

template#render should return a promise.

Because there are so many promises being resolved in a Tornado template, the template's render method should return a promise that resolves when the template is ready to be used.

convert the visualizer pass into a debug utility

Model this after gulp-debug

• a compiler pass author can use this utility to see how a branch of the tree looks at a given point in time
• a compiler pass author can use this utility to see how the entire tree looks at a given point in time

Create select helpers

The @select and its children will certainly be requested, but Tornado is not Dust. Should these helpers be included in core?

A rendered template should have an "all done" promise

When a Tornado template renders, it does not wait for promises to resolve before returning. In most cases this is an advantage, in that the developer doesn't have to track each promise individually and update the UI as each promise resolves. However, Tornado doesn't have a way of informing the developer that all promises have resolved and the template is fully rendered in its final state.

Error handling

Build consistent, intuitive error handling

Set up continuous integration

Travis CI (or something like it) and Saucelabs would be ideal. Depends on #34.

Spec for Helper and other language extensions for compiler authors

This can be part of the constitution but a formal spec for compiler authors would also be helpful. IMO sections can also be thought of as extension of the core. I like the idea that everything is a helper or at least everything is an extension. By extension, I mean i can turn on/off a compiler flag and have it included/excluded from the output.

How should compiler authors think about helpers? Are there other extensions beyond helpers? How does one verify that the helper is compiled correctly? How does one verify that a compiler is correct?

setAttribute is still trying to use the cache'd DOM

Due to cloneNode, caching the DOM was no longer a possibility, but setAttribute wasn't updated.

Add JavaScript Linting

Since the code is mostly ES6, we will need to use either JSHint or ESLint plus babel-eslint. This task includes a lint configuration file (e.g. .jshintrc) and a grunt task.

The parser should generically look at Tornado "code blocks"

Today the peg grammar handles distinguishing between a exists section etc.

e.g.

  var tornadoBodyTypes = {
    '?': 'exists',
    '^': 'notExists',
    '#': 'section',
    '@': 'helper',
    '+': 'block',
    '<': 'inlinePartial',
    '%': 'pragma'
  };

Now that we have an AST transform pass, we should be able to push this logic out into a separate pass and keep the grammar generic.

proposal:

• a codeBlock is { followed by a codeBlockIdentifier ...
• a codeBlockIdentifier is one of backtick~!@#%^&*_+-=;,.? -- what's on a US QWERTY keyboard minus dollar, parens, braces, brackets, pipe, colon, double and single quotes, <, > and slash and back slash since these are going to be confusing or are used for other things already

issues:

• we might need to revisit what bodies are

Add debugging helpers

• @contextDump
• @debugger?

Elements with Tornado references inside of textarea are breaking

<textarea>
  <div>{?hello}hello{/hello}{?hello}{.}{/hello}</div>
</textarea>

When an element inside of a textarea contains a Tornado body or reference, the element is stringified before being inserted into the textarea, and the placeholders where the references would be inserted are lost.

escapable raw is not preserved when entering tornado_body state

this currently fails.

<textarea>
  {#foo}
     <span>should be raw text here </span>
  {/foo}
</textarea>

There is a similar issue in attrs but we have codes to fix all this.

Ideally, we should have a mechanism that knows how to connect a tornado body fragment to it's parent element placeholder based on the state it came from. Perhaps we can solve this generically for escapable_raw and attrs and namespaces and etc.

Support an extensible compiler/parser

See also #66. In order to simplify the current codebase + allow for future improvements, please support some kind of extension/plugin for the compiler. For example, pegjs/pegjs supports --plugins(see https://github.com/pegjs/pegjs/blob/master/lib/peg.js).

This simplifies our codebase since it allows us to separate our compilation code into passes. Logic can be pulled out (if it doesn't change the AST or require some ordering) into a separate pass (e.g. check the AST).