Coder Social home page Coder Social logo

nx-js / compiler-util Goto Github PK

View Code? Open in Web Editor NEW
163.0 8.0 19.0 115 KB

An NX utility, responsible for executing code in the context of an object.

Home Page: http://www.nx-framework.com/

License: MIT License

JavaScript 100.00%
javascript eval scoped es6 compile

compiler-util's Introduction

The compiler util

This library is part of the NX framework.

The main purpose of this library is to allow the execution of strings as code in the context of an object.

Installation

$ npm install @nx-js/compiler-util

Platform support

  • Node: 6 and above
  • Chrome: 49 and above (after browserified)
  • Firefox: 38 and above (after browserified)
  • Safari: 10 and above (after browserified)
  • Edge: 12 and above (after browserified)
  • Opera: 36 and above (after browserified)
  • IE is not supported

Usage

const compiler = require('@nx-js/compiler-util')

Compiling code

compiler.compileCode(string) creates a function from a string. The returned function takes an object as argument and executes the string as code in the context of the passed object. The string can be any valid JavaScript code.

const code = compiler.compileCode('return prop1 + prop2')
const sum = code({prop1: 1, prop2: 2}) // sum is 3

Temporary variables

The returned function also accepts a second object argument, that may contain temporary variables. Temporary variables are added to the context object while the code is executing. They are favored over the permanent context variables.

const code = compiler.compileCode('return prop1 + prop2')
const context = {prop1: 1, prop2: 2}
const temporary = {prop1: 2}
const sum = code(context, temporary) // sum is 4, context is still {prop1: 1, prop2: 2}

Limiters

Limiters are functions, which can defer or block code execution. Some popular limiters are debounce and throttle for example. Limiters can be registered by name with compiler.limiter(name, function) and used at the end of the code with the & symbol.

// next is the code or the next limiter
compiler.limiter('delay', next => setTimeout(next, 1000))

const code = compiler.compileCode('console.log(message) & delay')
const context = {message: 'Hello World'}
code(context) // prints 'Hello World' to the console after a second

Limiters accept a context object, which can be used to share a context between executions of the code. It makes the creation of rate limiters - like throttle and debounce - straightforward.

compiler.limiter('debounce', debounce)

function debounce (next, context) {
  clearTimeout(context.timer)
  context.timer = setTimeout(next, 200)
}

After the context argument limiters accept any number of custom arguments. These can be passed after the limiter name in the code, separated by spaces.

compiler.limiter('delay', (next, context, amount) => setTimeout(next, amount))

const code = compiler.compileCode('console.log(message) & delay 2000')
const code2 = compiler.compileCode('console.log(message) & delay amount')

const context = {message: 'Hello World', amount: 3000}
code(context) // prints 'Hello World' to the console after 2 seconds
code2(context) // prints 'Hello World' to the console after 3 seconds

Multiple limiters can be piped with the & symbol.

const code = compiler.compileCode('console.log(message) & delay 1000 & throttle 100')

// this logs 'Hello World' a second after you click the button
// and it logs a message once per 100 milliseconds at most, excess messages are not logged
button.addEventListener('code', () => code({message: 'Hello World'}))

You can find some commonly used limiters in this repo.

Compiling expressions

compiler.compileExpression(string) creates a function from a string. The returned function takes an object as argument and executes the string as an expression in the context of the passed object. It returns the result of the evaluated expression. The string can be any javascript expression that may come after a return statement.

const expression = compiler.compileExpression('prop1 || prop2')
const result = expression({prop2: 'Hello'}) // result is 'Hello'

Expressions return undefined instead of throwing a TypeError on invalid property access. This allows lazy initialization of your data.

const expression = compiler.compileExpression('item.name')
const context = {}

let result = expression(context) // result is undefined, no error is thrown

context.item = {name: 'item name'}
result = expression(context) // result is 'item name'

Filters

Filters are functions, which can filter and modify expression result. Some popular filters are upperCase and trim for example. Filters can be registered by name with compiler.filter(name, function) and used at the end of the expression with the | symbol.

// txt is the result of the expression
compiler.filter('upperCase', txt => txt.toUpperCase())

const expr = compiler.compileExpression('message | upperCase')
const context = {message: 'Hello World'}
console.log(expr(context)) // prints 'HELLO WORLD' to the console

Filters accept any number of custom arguments. These can be passed after the filter name in the expression, separated by spaces.

compiler.filter('splice', (txt, start, end) => txt.splice(start, end))

const expr = compiler.compileExpression('message | splice 0 6')
const context = {message: 'Hello World'}
console.log(expr(context)) // prints 'Hello' to the console

Multiple filters can be piped with the | symbol.

const expr = compiler.compileExpression('message | splice 0 6 | upperCase')
const context = {message: 'Hello World'}
console.log(expr(context)) // prints 'HELLO' to the console

You can find some commonly used filters in this repo.

Handling globals

compiler.expose('String, String, ...') exposes globals by name for the compiler. Non of the globals are exposed by default.

const code = compiler.compileCode('console.log(Math.round(num))')
compiler.expose('console', 'Math')
code({num: 1.8}) // logs 2 to the console

Context variables are always favored over global ones, when both are present with the same name.

compiler.hide(String, String, ...) hides globals by name, while compiler.hideAll() hides all globals.

const code = compiler.compileCode('console.log(Math.round(num))')
compiler.expose('console', 'Math')
code({num: 1.8}) // logs 2 to the console
compiler.hide('console', 'Math')
code({num: 1.8}) // throws an error, console and Math are undefined

Alternative builds

This library detects if you use ES or commonJS modules and serve the right format to you. The exposed bundles are transpiled to ES5 to support common tools - like UglifyJS. If you would like a finer control over the provided build, you can specify them in your imports.

  • @nx-js/compiler-util/dist/es.es6.js exposes an ES6 build with ES modules.
  • @nx-js/compiler-util/dist/es.es5.js exposes an ES5 build with ES modules.
  • @nx-js/compiler-util/dist/cjs.es6.js exposes an ES6 build with commonJS modules.
  • @nx-js/compiler-util/dist/cjs.es5.js exposes an ES5 build with commonJS modules.

If you use a bundler, set up an alias for @nx-js/compiler-util to point to your desired build. You can learn how to do it with webpack here and with rollup here.

Contributions

This library has the very specific purpose of supporting the NX framework. Features should only be added, if they are used by the framework. Otherwise please fork.

Bug fixes, tests and doc updates are always welcome. Tests and linter (standardJS) must pass.

Authors

License

MIT

compiler-util's People

Contributors

gimenete avatar solkimicreb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

compiler-util's Issues

Roadmap

This is my proposed roadmap for the project:

nx-compile.2.1.0

Replace the current deep freezing of exposed globals with Object.freeze() with a toggleable alternative. Ideally secure() should not modify any behavior for normal code. By recursively wrapping things in Proxies and replacing properties with accessors we can create a `toggleable Object.freeze()' that only freezes the object when a hidden flag is turned on.

Moving the docs

Move the docs to the NX home page. This would have to benefit of editable examples. Finding security leaks would be a lot faster with these examples, than having to clone and run things locally.

Large scale testing

Fork some big projects (frameworks for example), run them through webpack and pass the whole source to compileCode() to see if its works in big. Also a benchmarking suite would be nice for these to test performance.

Others

In the meantime I continues to fix the new security issues if you find any.

Some usability questions regarding the API

Question about non-existent variables on the sandbox object

What should be the behavior when someone tries to use a non existent variable (not present on the sandbox object and not exposed from global scope)? Should it throw and error (as normal strict mode in global scope does) or should it return undefined?

First solution would match with the spec, while the second one is more forgiving and nicer for small expressions in HTML code. For example angular does the second solution in its in-line expressions in the view to not throw when a resource is not fetched yet.

Question about global vs local security

I moved secure() to be a global function that can only be called once. I did it because I think security should be handled globally. If one part of the project is leaking and the others are secured the whole project is leaking. Do you think security settings should be more fine grained (toggleable per code snippet)? This would be more useful for developers who know what they are doing and a bit more risky for the newbier ones.

Failure to build project dependant on compiler-util

When trying to make a production build of a project which depends on @nx-js/compiler-util using create-react-apps build system I get the following error:

Failed to minify the code from this file: 

 	./node_modules/@nx-js/compiler-util/src/modifiers.js:7 

Read more here: http://bit.ly/2tRViJ9

Following that link I'm instructed to ask you to publish a new version of your package to npm, but with ES6 features compiled out of the source code and replaced by ES5 analogues. Can I assist you in any way to reach the point where you can do this?

Security issues

If you find a new security leak with secure() please post it here. Thanks!

Sandbox escape possibility (`this`)

With the default setup it is easily possible to escape the sandbox:

const compiler = require('@nx-js/compiler-util')
compiler.compileCode('return this')({})
// returns Object [global]

There is a way to protect against this vulnerability:

compiler.compileCode('return this').bind({})({})
// returns {}

But this library should be "secure by default".

Occurs in both nodejs and browser.
Version: 2.0.0

this === window ?

Not sure if there is a problem or not, but this seems to resolve to window, so i guess i can access global stuff through that. Not sure if this is something that is ok or not or whether a .call(null or .apply(null would help...

the blog article https://blog.risingstack.com/writing-a-javascript-framework-sandboxed-code-evaluation/ has some typos too, so copy/pasting the code to the devtools doesnt always work.

Another thing i was thinking about is, that maybe someone could "pass in" an object that contains a has or get method which is "destructured" by the with (...) { ... } statement and maybe can cause problems too?

Security issues are ignored

I've posted several comments pointing out security issues with nx-compile, and I've pointed at solutions to those problems, but every time, my comments get deleted.

It's clear that the maintainers behind nx-compile have no concern for achieving actual security. Rather, they are more interested in their pride.

I'll post it again.

The global object can easily be leaked (even in "version 2.0")

compiler.secure();
var code = compiler.compileCode('return 2..constructor.constructor("return this")()', {});
code(); // => the global object is returned

To solve this problem, please take inspiration from my expression-sandbox module.

Bypass nx-compile 2.0 with hoisting

In the proposal at:
#2 (comment) and #3 you talk about "Early closing the with block with a single '}' character is now not possible."

It looks like the idea for the fix was based on the fact that even if you break out of the "with"
statement you are after the return statement, and code after the return is assumed unreachable. However a functions declared after the return statements are hoisted into scope(above the with statement). If we name that function "sandbox" we replace the Proxy with our function bypassing the sandbox.

resulting in the following code in the new Function(

// this gets hoisted 
function sandbox(){}// }) }

with (sandbox) { 
    return (() => { 
        'use strict'; global.isSecure=false
    })
};

I created the following failing test as a POC:

it('should protect against "early mustache closing" string manipulation and hoisting', () => {
    const code = compiler.compileCode('global.isSecure=false})};function sandbox(){}//', {})
    code()
    expect(global.isSecure).to.be.true
})

Tested against #3

Vulnerability

You can escape the sandbox using something like this:

const compiler = require('@nx-js/compiler-util');

const code = compiler.compileCode('} { return global.process.env.LOGNAME');
const user = code({});
console.log(user);

Threading

Thanks for sharing this library!

Reading through the code it seems like that user code is run on the main thread, right?

Any known issues or recommendations about Running it in a worker thread?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.