Transfering the repository and package to me about twing-loader

[ericmorand]

A few words about what a Webpack loader is and why current implementation of twing-loader arguably isn't a Webpack loader.

First, it is important to understand that a Webpack loader is basically a transform: it takes a resource and transform it into a piece of code that can be interpreted and executed by a Javascript engine - i.e. a valid Javascript code. Browserify uses the name transform to talk about that kind of plugin and their terminology is much more appropriate than loader and in sync with nodejs's naming of streams that transform their input - aka Transform Streams.

With that in mind let's see what twing-loader actually does:

• It receives a source and its belonging path
• It renders that source using Twing
• It returns the result of the rendering

Going back to the definition of a loader, the returned result is expected to be a piece of code that can be interpreted and executed by a Javascript engine - i.e. a valid Javascript code. What we see here is that the piece of code returned by the loader (the result of the rendering) is most probably not a valid Javascript piece of code.

Let's take the simplest example in the form of a Twig template named index.twig:

FOO

This Twig template would be transformed by twing-loader to this piece of Javascript code:

FOO

And except if earlier in the bundle the FOO variable has been declared, this Javascript code is not valid and throws the following error:

ReferenceError: FOO is not defined

This is exactly whats happens when the following module is bundled by Webpack using twing-loader and the bundle is executed:

require('./index.twig')

Now, let's see how another loader, string-loader, transform index.twig:

module.exports = "FOO";

As you can see, the same Twig template has been transformed into a perfectly valid Javascript code: a module that exports a string that contains FOO.

So, the main noticeable and problematic thing with twing-loader is that it most of the time doesn't transform its input into a valid Javascript piece of code.

For fun, let's create a Twig template that would be transformed into a valid piece of code by twing-loader:

let FOO = "FOO";

return FOO;

This template would be transformed to:

let FOO = "FOO";

return FOO;

But that wouldn't hold any kind of interest.

Let's see now what we expect twing-loader to do:

• It receives a source and its belonging path
• It compiles it into a template
• It returns the template that can be executed at runtime

Let's see the kind of code we expect our index.twig template to be transformed into:

let {TwingEnvironment, TwingLoaderArray} = require('twing');

let loader = new TwingLoaderArray({
	'index.twig': 'FOO'
});
let env = new TwingEnvironment(loader);
let template = env.loadTemplate('index.twig');

module.exports = template;

Here, we not only returns a perfectly valid Javascript code but also a Twing template that can be used in the bundle. Since we export the template as a node module, this transform allows us to write the following piece of code and expect it to run in a browser after being bundled by Webpack:

/** @type TwingTemplate **/
let template = require('./index.twig');

let render = template.render({});

console.warn(render);

As you can see, we required a Twig source and were returned a Twing template.

That's what we expect form a Twig loader. And that's what my proposal does. Well, not exactly, because instead of returning a Twing template, it returns a function that, when executed (with optional data as parameter), renders the template. It would allow to write that kind of code:

/** @type Function **/
let template = require('./index.twig');

let render = template();

console.warn(render);

I made that conception choice to have twing-loader homogeneous with the conception choices of most other templating languages around - including twig-loader itself: requiring a Twig source returns a function that can be executed to render the template. And also to be compliant with html-webpack-plugin expectations - see jantimon/html-webpack-plugin#1120.

This could be easily improved in the future if someone needs the Twing template alongside the function, like twig-loader does.

[ericmorand]

By passing them to the function returned by the require:

/** @type Function **/
let template = require('./index.twig');

let render = template({
    foo: "bar"
});

console.warn(render);

I didn't get how the data is passed to the renderer in the current implementation though.

[ericmorand]

And now that the concept has been explained, let's go a bit further: ideally, we want to cache the compiled template instead of having Twing compile it again and again and again everytime we require it. After all, once bundled, a Twig template will never change. So it would be an overkill to compile it everytime we require it: let's get to the next step and have the loader bundle the compiled template.

That's what my proposal does. It transform a Twig source into something like this:

const {cache, loader, getEnvironment} = require('/home/ericmorand/Projects/twing-loader/dist/runtime.js');
const env = getEnvironment(require('/home/ericmorand/Projects/twing-loader/test/integration/include/function/environment.js'));
cache.write('__HASHED__0def5982d22ff8720764e91334e84a6d9403b7ee40d7d4350908632baf647963', (() => {let module = {
    exports: undefined
};

module.exports = (Runtime) => {
    let templates = {};
    

    /* __HASHED__0def5982d22ff8720764e91334e84a6d9403b7ee40d7d4350908632baf647963 */
    templates.__HASHED__0def5982d22ff8720764e91334e84a6d9403b7ee40d7d4350908632baf647963 = class __HASHED__0def5982d22ff8720764e91334e84a6d9403b7ee40d7d4350908632baf647963 extends Runtime.TwingTemplate {
        constructor(env) {
            super(env);

            this.source = this.getSourceContext();

            this.parent = false;

            this.blocks = new Map([
            ]);
        }

        doDisplay(context, blocks = new Map()) {
            // line 1, column 1
            Runtime.echo(`FOO`);
        }

        getTemplateName() {
            return `__HASHED__0def5982d22ff8720764e91334e84a6d9403b7ee40d7d4350908632baf647963`;
        }

        getDebugInfo() {
            return new Map([[20, {"line": 1, "column": 1}]]);
        }

        getSourceContext() {
            return new Runtime.TwingSource(``, `__HASHED__0def5982d22ff8720764e91334e84a6d9403b7ee40d7d4350908632baf647963`, ``);
        }
    };

    return templates;
};

return module.exports;})());

loader.addTemplateKey('__HASHED__0def5982d22ff8720764e91334e84a6d9403b7ee40d7d4350908632baf647963', '__HASHED__0def5982d22ff8720764e91334e84a6d9403b7ee40d7d4350908632baf647963');

let template = env.loadTemplate('__HASHED__0def5982d22ff8720764e91334e84a6d9403b7ee40d7d4350908632baf647963');

module.exports = function(context = {}) {
    return template.render(context);
};

This is a bit more complicated that previously shown but it executes much faster. Because the template has already been compiled by the loader and at runtime the bundle only execute the template.

The price is obviously the size of the bundle, as always. It would be great to add an option to twing-loader to choose if we want a compiled template to be bundled or to have the template being compiled in the bundle. Since Twing works perfectly in the browser, both solutions should work equally - except in term of speed and size of cource.

On thing to note: the pre-compiled solution allows for template anonymization, as you can see with the __HASHED__[...] names. This is to prevent the path to the original source code to show in the bundle.

[nicolasRdr]

Hi @ericmorand,

Nice proposition, I'm going to give you the ownership of this repo.
No doubt the project will evolve in your hand.

As the documentation regarding transferring repository stated, "The target account must not have a repository with the same name, or a fork in the same network.".

Can you please make sure that these conditions are ok ?
Then I will be able to make the transfer.

[ericmorand]

Good point!

I had a fork. I deleted it.

[DerekRoth]

Wow, thanks for the explanation, i kind of get it now. Quick question then: how do you pass variables to the template ?