balena-io-modules / balena-temen Goto Github PK

View Code? Open in Web Editor NEW

2.0 5.0 1.0 490 KB

Balena TEMplating ENgine

License: Apache License 2.0

Rust 91.92% Shell 4.71% JavaScript 3.38%

rust template-engine template-library template-language

balena-temen's Introduction

balena temen

A templating engine for JSON.

Provides facilities to:

evaluate JSON
evaluate JSON with custom evaluation keyword, functions and filters
evaluate single expression (result is of any type)
evaluate single logical expression (result is a boolean)
retrieve AST of any expression / identifier

_{an open source 📡 project by balena.io}

Goal

balena-temen crate is one small piece of the balena.io configuration project. This project has no public / open specification yet, but we're working on it and it will be public once finished.

Supported platforms

This library is written in the Rust language and can be used:

directly, as a Rust crate
as an isomorphic NPM package (NodeJS & browser)

Documentation

Breaking changes

Version 0.3

Python style kwargs were removed in favor of positional arguments. All functions and filters do accept positional arguments only.
NOW() function accepts one boolean argument only

Version 0.2

All function and filter identifiers were uppercased
- An example uuidv4() -> UUIDV4()

Version 0.1

$$eval keyword was renamed to $$formula. You can still use $$eval if you want by instantiating your own Engine with the EngineBuilder and custom eval keyword registration.

Usage

Rust

Add as a dependency to your Cargo.toml:

[dependencies]
balena-temen = "0.1"

Evaluate simple JSON:

use balena_temen::evaluate;
use serde_json::json;

let data = json!({
    "wifi": {
        "ssid": "Balena Ltd",
        "id": {
            "$$formula": "super.ssid | slugify"
        }
    }
});
let evaluated = json!({
    "wifi": {
        "ssid": "Balena Ltd",
        "id": "balena-ltd"
    }
});

assert_eq!(evaluate(data).unwrap(), evaluated);

Javascript

Install via npm

npm install --save balena-temen

Evaluate simple JSON:

const bt = require('balena-temen');

console.log(
    bt.evaluate({
        "ssid": "Some Cool SSID!",
        "id": {
            "$$formula": "super.ssid | slugify"
        }
    })
);

An example of using this module in nodeJS is available in the examples/node folder:

cd examples/node
npm install
npm start

An example of using this module in the browser is available in the examples/browser folder:

cd examples/browser
npm install
npm start

Open localhost:8080 in your browser and you should see evaluated JSON in the browser console.

Support

If you're having any problem, please raise an issue on GitHub or contact us, and the balena.io team will be happy to help.

License

balena-temen is open source software, and may be redistributed under the terms specified in the license.

balena-temen's People

Contributors

Stargazers

Watchers

balena-temen's Issues

uuidv4() generator doesn't work

Reproducible Node case:

const bt = require('balena-temen');

console.log(
    bt.evaluate({
        "uuid": {
            "$$eval": "uuidv4()"
        }
    })
);

The error message:

panicked at 'could not initialize thread_rng: No entropy sources available (permanently unavailable)', /Users/robertvojta/.cargo/registry/src/github.com-1ecc6299db9ec823/rand-0.5.5/src/rngs/thread.rs:84:17
wasm://wasm/004488ea:574

RuntimeError: unreachable
    at __rust_start_panic (wasm-function[573]:1)
    at rust_panic (wasm-function[570]:30)
    at std::panicking::rust_panic_with_hook::h6790e8431e342a60 (wasm-function[565]:444)
    at std::panicking::continue_panic_fmt::h3b66c044a41127bf (wasm-function[564]:120)
    at std::panicking::begin_panic_fmt::h522439ba8a0fbac4 (wasm-function[561]:95)
    at rand::rngs::thread::THREAD_RNG_KEY::__init::_$u7b$$u7b$closure$u7d$$u7d$::hdd28f475617c5274 (wasm-function[357]:91)
    at rand::rngs::thread::thread_rng::hef4638d394e756b1 (wasm-function[356]:1024)
    at uuid::v4::_$LT$impl$u20$uuid..Uuid$GT$::new_v4::h9644a31e3bf57f8d (wasm-function[345]:14)
    at balena_temen::builtin::function::uuidv4::uuidv4::h3ce10819a87f445f (wasm-function[200]:18)
    at balena_temen::engine::Engine::eval_function::he1b50994ba503a6b (wasm-function[233]:943)

Same error in both nodeJS & browser.

Replace error-chain

We should replace error-chain with custom error type allowing us to provide more precise error messages like which part of the expression is wrong, etc.

Reorganise tests & write more of them

Go through the grammar and test all possibilities (= expressions)

Add relative -> absolute identifier conversion

We have to provide a public API for users with the ability to normalize identifier. It means conversion from relative identifier to an absolute one (remove this and super keywords).

Add helper function to evaluate the whole JSON

Imagine you have a JSON like this one ...

{
  "uuid": {
    "$$eval": "uuidv4()"
  },
  "id: {
    "$$eval": "ssid | slugify"
  },
  "ssid": "Zrzka 5G"
}

... and you'd like to evaluate everything at once to get something like ...

{
  "uuid": "2a0b9f5a-04e8-483e-bb3c-c889822afc67",
  "id": "zrzka-5g",
  "ssid": "Zrzka 5G"
}

... we should provide a helper function for this, so, one can do it with one call. It's possible today, but one has to find all $$eval and evaluate them one by one.

@cyplo WDYT?

Add benchmarks

We need to add benchmarks, but we have to check with CI guys how stable in terms of performance is our CI, etc. Or we should run them locally on our machines with predictable performance.

Remove now() fn workaround

chrono doesn't support wasm32 (see chronotope/chrono#243 & chronotope/chrono#287)
we have added a workaround from the jjpe/wasm branch, which works, but ...

src/context.rs:

#[cfg(not(target_arch = "wasm32"))]
fn utc_now() -> DateTime<Utc> {
    Utc::now()
}

// chrono crate doesn't support wasm32 arch yet, workaround
#[cfg(target_arch = "wasm32")]
fn utc_now() -> DateTime<Utc> {
    let now = js_sys::Date::new_0();
    let millisecs_since_unix_epoch: u64 = now.get_time() as u64;
    let secs = millisecs_since_unix_epoch / 1000;
    let nanos = 1_000_000 * (millisecs_since_unix_epoch - 1000 * secs);
    let naive = NaiveDateTime::from_timestamp(secs as i64, nanos as u32);
    DateTime::from_utc(naive, Utc)
}

Stabilise API

Till 0.1, even patch version can break public API. Once all features will be in place, working, API should be stabilised, 0.1 released and then we should follow semantic versioning.

Add fuzzer

~~Current implementation of the fuzzer is kind of useless. There's no way how to get the crashing input, etc.~~ Useless fuzzer was removed.

We have to come up with some fuzzing strategy like:

when to run it,
where to run it,
for how long,
how often,
etc.

contains unnecessary allocations,
does not leverage references, lifetimes, cow,
etc.

Just an update, here is a list of DSL generators (+validators ideally) we use/ll be using in balena configuration:

DH-Param (could be the output of require('dhparam')(bits))
RSA Keypair
x.509 CACert
x.509 Cert
EC PrivateKey
EC keyid
EC Cert

More info about EC *.

slugify
trim
date
time
datetime

See Supported Filters.

Remove uuidv4() fn workaround

Once new uuid crate is released and #352 fixed, we can remove our uuid & rand workaround.

Remove rand dependency
Enable v4 feature (uuid crate)
Enable wasm-bindgen feature (uuid crate, wasm32 arch)
Replace custom uuidv4() fn body with Uuid::new_v4()

base64encode
base64decode