toshok / echojs Goto Github PK

from _ejs_Array_prototype_join:

    ejsval* strings = (ejsval*)malloc (sizeof (ejsval) * num_strings);
    int i;

    for (i = 0; i < num_strings; i ++) {
        strings[i] = ToString(EJS_DENSE_ARRAY_ELEMENTS(_this)[i]);
        result_len += EJSVAL_TO_STRLEN(strings[i]);
    }

If a collection is run during the course of that loop, the strings allocated prior to the allocation causing the collection will become garbage.

At the time I last checked OSX wasn't supported. Seems it is now?

hi toshok,

I began to write a JavaScript front-end base on LLVM a half and a month ago, and I found this open source project about a month ago, thank you very much since I get a lot of help from this project.

I use code of runtime in my project, but I modify the file names, struct names and function names of them, to match the code style of mine. and reconstruct some structs and functions to match with my code. So I am so intertwined with this when I decide to open source my project.

So, could I open source my project with part-modified code of you ? If I can't do that, I will rewrite this part of code in my project later, thanks.

I can't find your email address so I post a message here, and you can contact me with:
zhangheng607 at 163 dot com

by the way, my site is:
eddid/jslang

This is one form of optimization we'd get with closure analysis, but it's such an idiomatic pattern in JS, and it's easy easy easy to remove.

take the following:

var foo = (function() {
    var y = 5;
    return y;
})();

This is done because the only way to achieve lexical block scoping with var is by using these IIFE's.

but ejs supports let, and we transform vars away during compilation. so this would become:

let foo = (function() {
    let y = 5;
    return y;
})();

at which point we don't need the IIFE at all, and we can convert the entire thing to essentially this:

let $y = 5;
let foo = $y;

this requires a lot of possible code motion. If the IIFE is an argument to a function, for instance:

  let x = 5;
  let foo = callFunction(x, (function () {
                                var arg1 = 5;
                                x = 10;
                                return arg1;
                             })());

we need to preserve order of evaluation, so this would become something like:

let x = 5;
let $arg0 = x;
let $arg1 = 5;
x = 10;
let foo = callFunction($arg0, $arg1)

more here...

echojs at present fails to build even under osx (running Yosemite) have also have very little sucsess on linux.

It would be good if we could strip it back to basics and split out the node-llvm and esj-llvm stuff into seperate modules and also upgrade node-llvm to use nan so it can easily build on 0.10, 0.12 and iojs

The following methods aren't implemented in ejs yet:

Array.prototype.toLocaleString (22.1.3.26)
Array.prototype.sort (22.1.3.24)

Right now we do absolutely nothing with super.

It should be an syntax error outside of functions corresponding to MethodDefinitions that aren't property accessors, and (i think?) aren't static.

e.g.:

let a,b;
[a,b] = [1,2];
({a,b} = {a:"hello", b:"goodbye"});

The version of esprima we're using doesn't parse them yet, and we also don't support them. We'll need to introduce a fresh binding for the rhs to desugar things properly, so we'll end up with something like this for the above:

let a,b;
let destruct_tmp01, destruct_tmp02;
destruct_tmp01 = [1,2],a=destruct_tmp01[0],b=destruct_tmp01[1];
destruct_tmp02 = {a:"hello", b:"goodbye"}, a=destruct_tmp02.a, b=destruct_tmp02.b;

right now all strings constants (including identifiers) are created in the .data segment of the resulting executable. that way we don't pay any heap cost for their usage (yay.)

This mostly works, but there are a couple of problems with the approach:

1. they're not aggregated across different .js files - each .js file gets its own string constants. I haven't measured how much duplication we have, but switching to es6 modules with each file in the bindings (115+ in UIKit) having their own strings.. I expect we could be saving some memory.
2. They have a startup cost associated with them - or rather a toplevel func invocation cost. This is because we need them to be represented as ejsvals. so we generate three globals in llvm IR during compilation - an ejsval for the actual value we'll be using for a given constant, an EJSPrimString, and a ucs2 buffer. The initialization of a given constant string looks something like:

  ejs_prim_string_%1235.length = <compile-time-known-constant>;
  ejs_prim_string_%1235.data.flat = ucs2_buffer_%1235;
  ejsval_%1235 = STRING_TO_EJSVAL_IMPL(&ejs_prim_string_%1235);

it would be lovely if we could use relocs + other magic to have all this happen at link time.

Fixing 1 up there is as easy as just keeping a map during compilation and spitting out the right stuff after we've processed all the .js, similar to what we do for the require map. We could even spit it out as a .c file like the require map.. maybe the C compiler will do a better job of linking everything together without requiring runtime initialization.

similar to #10.

pirouette does the following to map objective-c enums:

var enum = Object.create(null);
Object.defineProperty (enum, "foo", 5);
...
Object.freeze(enum);

esprima does the same with its syntax object, which is a set of constant strings.

If we can statically determine that an object is constructed and populated with constants - and then frozen, we can move the construction/population out of code entirely, and put it in the data segment of the executable instead.

It looks like SpiderMonkey, in the case of sparse arrays, fallbacks to simple property access* (i.e. arr[450] gets the '450' property, if any).

Wouldn't that work here for now, instead of creating a huge array? (for arrays > SOME_HUGE_VALUE, that is).

• It looks like SP uses some checks here and there to detect this case, based on the use-case.

Right now there are a mix of references like // ECMA262: x.y.z.w and // ES6: x.y.z.w in comments around code, and we've directly copied (and converted to comments) the spec algorithms in numerous places.

Some of the ECMA262 references are to the ES5 spec, some to ES6. And ES6 recently changed all its numbering so the ES6 references are often out of date as well.

There are also a number of other more fundamental changes (addition of Symbol in ES6 caused all the ToStrings to become ToPropertyKeys when dealing with properties, and there are a host of other subtle changes.

All these need to be updated (and we need to put a stake in the ground as far as ES6 spec date is concerned and say "we implement ES6 as speced in the <date/version> draft." We can then update when we need to by looking at diffs between different draft versions and making only those changes, but what we have right now is kind of a mess.

IMO ejs shouldn't allow a writable global object, and all normal JS objects should be non-writable as well, at least by default. If we disable all writes (i.e. globals are frozen) then we can inline accesses for a ton of things at compile time.

I.e.

let a = Object.create (null);

This gets transformed to contain %getGlobal(Object) (which is itself a property lookup, just with implicit obj parameter) + a normal property lookup.

If everything is frozen, then we could directly invoke the function _ejs_Object_create (not even using _ejs_invoke_closure, since we can annotate the function such that the compiler knows it doesn't close over anything.)

Consult the list of ES6 features seen here to see what still needs implementing.

http://kangax.github.io/compat-table/es6/

The builtin functions provided in the environment have 'length' properties spelled out in ecma262. we should do a pass over all methods and use new macro EJS_INSTALL_ATOM_FUNCTION_LEN_FLAGS (check in ejs-array.c or ejs-string.c for usage) everywhere. then likely change the macro's name and get rid of the other macros.

Hey Chris,

Trying to install everything from zero (previous machine died), I realized I forgot to mention that pcre and parson are not mentioned in the requirements list (and they are not being fetched by default either).

see: https://mail.mozilla.org/pipermail/es-discuss/2012-November/026126.html

Right now we do nothing with the superClass when a class is declared as extends superClass.

We could c&p the coffeescript (without the static property copying) into the desugar pass.

facepalm

the compiler translates update assignments like "a += b" to "a = a + b"

this works for simple lvals like member expressions and identifiers, but it fails when there is an expression like: "a[i++] += b"

There are no binaries available for this, and no commands how to compile a project using this either. Like, if I guess right, I need to compile this repository, then use this to compile JavaScript into machine code for Windows. So, how do I use this? Can you provide a step for step guide with using commands to use this?

Until the compiler is fully self hosting and we can move to it for more self-hosting work, (or node gets es6 module support) we need to support require().

The original require() support was pretty ugly:

1. save off value of JS global "exports" into prev_exports
2. set JS global "exports" to a new object
3. run toplevel for require()'d module
4. cache exports object so future require() calls for same module just return it
5. reinstance prev_exports.

all this broke with the additional of es6 module support.

ES6 module support changed the toplevel function to return the default export. default exports in ES6 bear a lot of similarity to the thing returned by require(), so I think we should do this to reinstate require behavior:

1. if the compiler sees any import/export/module forms in a given file, set a bool flag for it called "es6" to true. Otherwise set it to false.
2. if "es6" is true for a given file, we add two things:

• a formal parameter to the toplevel, called "exports" - no more saving/restoring a global.
• a "return exports;" statement at the end of the toplevel.

1. in require(), check if "es6" for the required()'d module is true.

• if true, create a new JS object and pass it as the exports parameter.
• cache the return value from the toplevel for future require() calls

This should solve interoperability in both directions:

1. module foo from "foo" will work on foo.js that use exports.bar = baz;
2. let foo = require('foo'); will work on foo.js that uses export default { ... }

According to http://kangax.github.io/compat-table/es6/ this feature is missing.

The parser for integer literals needs updating, not to be confused with parseInt's implementation, which is currently implemented according to ECMA262 15.1.2.2 (note that ES6 draft 18.2.5 (https://people.mozilla.org/~jorendorff/es6-draft.html#sec-parseint-string-radix) doesn't update this to allow 0[bB] or 0[oO] as string prefixes where radix is detected [marked as wontfix in https://bugs.ecmascript.org/show_bug.cgi?id=1585]).

There are a of number places in the spec (e.g. dealing with Proxies and Promises) where spec level functions [[Call]] user-defined functions. These can throw. We need to catch those exceptions and report them back to these spec-level functions. There isn't a way to do this directly in the code of the caller (or in C at all, given that our exceptions have a different personality), so what we need is a wrapper around _ejs_invoke_closure, written in llvm-ir.

It could look something like this llvm IR/JS pseudo-code:

// returns TRUE if invoke succeeded, with return value in result
// returns FALSE if exception thrown, with exception in result
EJSBool _ejs_invoke_closure_exc(ejsval* result, ...args,) {
  *result = _ejs_undefined;
  %1 = invoke (_ejs_invoke_closure, ...args), @success_bb, @thrown_bb
@success_bb:
  *result = %1
  return EJS_TRUE;
@thrown_bb:
  *result = %the-exception
  return EJS_FALSE;
}

using an out param for retval should get us around the platform abi problem with sret returns. actually if we typedef ejsval to int64 here we might be able to type pun our way to freedom and use a single llvm-ir file for everything.

$ make
Making all in external-deps
Makefile:32: target `clean-double-conversion' doesn't match the target pattern
(cd pcre-iossim && \
    PATH=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin:$PATH \
    CC="clang -arch i386 -DTARGET_CPU_X86=1 -DEJS_BITS_PER_WORD=32 -DIS_LITTLE_ENDIAN=1 -miphoneos-version-min=8.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator8.3.sdk" \
    CXX="clang++ -arch i386 -DTARGET_CPU_X86=1 -DEJS_BITS_PER_WORD=32 -DIS_LITTLE_ENDIAN=1 -miphoneos-version-min=8.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator8.3.sdk" \
    LD="clang" \
    AS="/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/as" \
    ../pcre/configure --host=i386-apple-darwin --enable-pcre16 --enable-utf --disable-cpp) && touch .stamp-configure-pcre-iossim
configure: WARNING: if you wanted to set the --build type, don't use --host.
    If a cross compiler is detected then cross compile mode will be used
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for i386-apple-darwin-strip... no
checking for strip... strip
checking for a thread-safe mkdir -p... ../pcre/install-sh -c -d
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for i386-apple-darwin-gcc... clang -arch i386 -DTARGET_CPU_X86=1 -DEJS_BITS_PER_WORD=32 -DIS_LITTLE_ENDIAN=1 -miphoneos-version-min=8.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator8.3.sdk
checking whether the C compiler works... no
configure: error: in `/Users/noah/src/echojs/external-deps/pcre-iossim':
configure: error: C compiler cannot create executables
See `config.log' for more details
make[1]: *** [.stamp-configure-pcre-iossim] Error 77
make: *** [all-recurse] Error 1

It seems to be compiling for the iPhone Simulator. I'm just trying to compile it for my own computer.

For purposes of the test626 integration, I'm considering a 'run' mode in ejs itself:

1. Compile to a temporary output, i.e. '/tmp/xasdadssdasd.out'
2. Simple invocation on that file.
3. Removal of that file.

This would look something like:

./ejs --run mysample.js [mysample args]

Simple, but I'm wondering if it's clean enough, and desirable - we may not want to let our users (for now) to do that directly.

(This is an alternative to creating a bash wrapper to do this for us, and then be passed to the test262 infrastructure).

Right now Map's are implemented word-for-word from the spec drafts, including a linear insertion-order lookup.

Maps should at the very least be a linked hash map for the string case, which would bring the compiler speed back up to what it was before switching from Objects to Maps.

osx-test/hello-osx.js has this block:

            this.xmlhttp = new XMLHttpRequest();
            this.xmlhttp.onreadystatechange = () => {
        console.log ("readyState == " + this.xmlhttp.readyState);
        if (this.xmlhttp.readyState === 4) {
                    console.log ("woohoo!");
            console.log (this.xmlhttp.responseText);
        }
            };
        console.log (this.xmlhttp.onreadystatechange);
            this.xmlhttp.open('GET', 'http://www.google.com/', true);
            this.xmlhttp.send();

'xmlhttp' should be a local variable, but if you make it one we crash, since the .send() doesn't cause it to be rooted by the GC. The GC frees it and we crash when attempting to call the onreadystatechange callback.

the kangax test for Symbol.toPrimitive fails for ejs because a few of the ops don't call ToPrimitive.

Probably most importantly, we're missing an implementation of Abstract Relational Comparison.

these don't currently parse with esprima/harmony, so we can't do much on that front.

After your latest change I cannot build echo-js anymore: it fails when building pcre with the iosdev profile:
...
"configure: error: You need a C++ compiler for C++ support."
...

Sadly it seems this is also seen under Linux for some versions of pcre (even if g++ is around :/) Out of curiosity: what version of Mac OS are you using?

(I'm using Mountain Lion, with the latest command line tools, 6.* simulators installed, etc)

I plan to add patches towards this goal.

http://encoding.spec.whatwg.org/

the following function:

function foo() {
  var a = [1,2,3,4];
  var b = 1;
  var c = 2;

  var new_a = a.map (function (el) { return el * b; }

  return function () {
    return c;
  }
}

will create a single closure environment holding both b (closed over by the function passed to map) and c (closed over by the returned function.)

In this totally disjoint case, ejs could split the closure environments, but I need to investigate if the overhead of splitting the closure makes sense for small environments (I'm guessing no for this particular case, particularly since the closed over variables are never set to anything other than primitives..)

Also, what happens if the sets are not totally disjoint? Take the following:

function foo() {
  var a = [1,2,3,4];
  var b = 1;
  var c = 2;
  var d = 4;

  var new_a = a.map (function (el) { return el * b + d; }

  return function () {
    return c + d;
  }
}

so now both functions close over d as well.

we could make 3 environments in this case;

env_0 = { d: 4 }
env_1 = { env: env_0, b: 1 }
env_2 = { env: env_0, c: 2 }

accesses to d get replaced with something different from within each function:

1. env_1.env.d in the function passed to map
2. env_2.env.d in the returned function.

This also has a cost in the increased indirection to get to the d binding, so need some heuristic based on the number of accesses to d to determine if we should just create 1 environment.

we don't currently autodetect sdk versions and rely on the user to set it.

We also don't update the SDK version very often (since tracking bleeding edge is equally likely to cause problems if users don't also upgrade.)

The README currently doesn't say anything any of this variable. we should fix the README, and add autodetection (the variable would presumably override the autodetected version if there are multiple versions.)

we're using ES6 Map in the compiler now, so we really need a node that is capable of it. does >= 0.10.24 have Map.forEach?

right now the ComputeFree pass uses its own recursive tree walk code, where the return value of its visit (called "free") function is the free vars in that node's, instead of the node itself.

it needs to be rewritten to be a NodeVisitor like all the others. It should pass false to the NodeVisitor ctor so that a copy of the tree isn't made during traversal.r, but changing all the