emscripten-core / emscripten Goto Github PK

Emscripten: An LLVM-to-WebAssembly Compiler

License: Other

CMake 0.19% Python 3.52% Shell 0.08% Makefile 0.05% HTML 0.55% JavaScript 9.90% C 35.29% C++ 49.75% Assembly 0.21% Batchfile 0.28% Pawn 0.05% NASL 0.11% PowerShell 0.01% CSS 0.01% Roff 0.01% TypeScript 0.01%

hacktoberfest emscripten wasm webassembly

emscripten's Introduction

Main project page: https://emscripten.org

GitHub CI status:

Chromium builder status: emscripten-releases

Overview

Emscripten compiles C and C++ to WebAssembly using LLVM and Binaryen. Emscripten output can run on the Web, in Node.js, and in wasm runtimes.

Emscripten provides Web support for popular portable APIs such as OpenGL and SDL2, allowing complex graphical native applications to be ported, such as the Unity game engine and Google Earth. It can probably port your codebase, too!

While Emscripten mostly focuses on compiling C and C++ using Clang, it can be integrated with other LLVM-using compilers (for example, Rust has Emscripten integration, with the wasm32-unknown-emscripten and asmjs-unknown-emscripten targets).

License

Emscripten is available under 2 licenses, the MIT license and the University of Illinois/NCSA Open Source License.

Both are permissive open source licenses, with little if any practical difference between them.

The reason for offering both is that (1) the MIT license is well-known and suitable for a compiler toolchain, while (2) LLVM's original license, the University of Illinois/NCSA Open Source License, was also offered to allow Emscripten's code to be integrated upstream into LLVM. The second reason became less important after Emscripten switched to the LLVM wasm backend, at which point there isn't any code we expect to move back and forth between the projects; also, LLVM relicensed to Apache 2.0 + exceptions meanwhile. In practice you can just consider Emscripten as MIT licensed (which allows you to do pretty much anything you want with a compiler, including commercial and non-commercial use).

See LICENSE for the full content of the licenses.

emscripten's People

Contributors

Stargazers

Watchers

Forkers

albertz robterrell jpraher festercluck rozgo mchorfa jeromeclerico atoun axiomsofchoice charles-cai gijs opichals tzuryby oxromantic crgosselin reobwuein sigmundvik ipv6 kuccello codeinpeace poutyface berkus benoitt ssssssssssss alard tonyg eclecticdave bemasc longjohncoder adel-e zmaril fishingcactus jterrace lzyzsd madbarua sophrinix ehsan sunragav gero3 alberthrocks royk bartuer adetaylor jperla richardassar r2d2v1 renenyffenegger roselone srinivasarao549 jscheid nbp behdad davidben kevnz landakram brson whilet lt1946 linghuye mpjeff hellcoderz diclophis zhuzhuaicoding spacebat juj hsingh23 jingtaow xymeng toshok h4ck3rm1k3 dineshkummarc tazzkiller spencerx caiiiycuk rodneywhite-at-theaccelerationagency comforx imvu hachque-emscripten victusfate secretrobotron vibster crispywalrus zazk nandhp forandom ziedus sreekumar-kr dopefishjustin stechz hubei lgstarn zhuomingliang rharkeadsk andreabedini mrrio mnaamani arunsirrpi xrmx flufy3d vvuk

emscripten's Issues

time test failure

The new 'time' test fails on my machine, in all settings (clang, llvm_gcc, optimize, etc.). The output is

stime: -1
tzname[0] set: 1
tzname[1] set: 1
sec: 43
min: 22
hour: 3
day: 25
mon: 11
year: 102
wday: 3
yday: 358
dst: 0
off: 0
zone: GMT
timegm <-> gmtime: 1
old year: 102
new year: 70
old year again: 102
localtime timezone: 1
localtime daylight: 1
localtime tzname: 1
localtime <-> mktime: 1
old year: 102
new year: 69
old year again: 102
time: 1
difftime+: 268848637.000000
difftime-: -268848637.000000
1854 days: 365
2000 days: 366
2001 days: 365
2004 days: 366
asctime: Tue Dec 24 19:22:43 2002
old asctime: Tue Dec 24 19:22:43 2002
new asctime_r: Sat Jul 2 12:33:20 2011
old asctime again: Tue Dec 24 19:22:43 2002
clock: 0
ctime: 0

Only things I can think of offhand that could cause it to fail for me but succeed elsewhere, are headers? Or possibly different time values in JavaScript due to different local settings?

Pointer casting support

This might be a bit going against the strategy outlined in the README. It is stated that different sized pointer casting code will not work properly in emscripten.

However I wanted to compile a specific file format parsing library (st-guilde .HYP hepertext files http://code.google.com/p/libhyp/) into javascript along with its LH5 decompression algorithm. That required the pointer casting to work properly.

After adding some aliases and methods to src/library.js I succeeded to build a working application available at http://opichals.github.com/hypview/ (.hyp file viewer, one can download some .hyp file from http://libhyp.atari.org for tests).

The pointer casting changes are available at https://github.com/opichals/emscripten/tree/correct-pointers branch (on top of the nodejs branch I posted about before elsewhere). The core of the idea is that the resulting generated code emulates bigendian HEAP access for different size datatypes like e.g. "HEAP[v+1]<<8 | HEAP[v]" for reading 16bit int. No performance or other optimizations were accounted for - just the result correct behaviour.

Just wanted to make this available. I tend to think that this is the closest we can get to real LLVM bytecode memory management. Not sure whether this is something to be merged to master later on (perhaps as an option?).

QUANTUM_SIZE setting is unintuitive

Since QUANTUM_SIZE can only be set to 1 or 4 at this time, it makes sense to have it set from a boolean flag in settings.js instead of the current numerical value, to show that other values (e.g. 2, 8) are not supported. This can be done by either replacing QUANTUM_SIZE completely or creating a new flag and setting QUANTUM_SIZE to the right integer on start up.

Build lua from source in test runner

To improve our Lua build, we should build it from source in the test runner. This will allow building with clang, as well as debugging the existing issues we have with Lua.

See test_the_bullet (the bullet test) in tests/runner.py for how we do this for Bullet. The same approach should be done for Lua.

node.js support and npm repository

It should be simple (and very useful) to have node.js running emscripten, as a npm module

fscanf, reading and writing file streams

Hey!

I've tried emscripten for converting our c++ projects to javascript. These programs read and write files simple text files. I have the content of the input files in javascript, and I want to process them with the converted c++ program, and receive the content of the output files in javascript.

I saw you have a STDIO object for reading and writing file streams, but I found that fscanf doesnt exist at all. There is an scanf and a function called fread.

Could you please implement a simple fscanf function?
How can I set the initial content of the input files? Can it be done with STDIO.prepare(...)?
I saw that there is an scanf implementation which does not supprt %f format yet. Do you think you can make that too?

What would happen if I open an input file for writing, change the content, and open it again for reading in the same running session? Can it store the file changes during the running of the program? I could add persistance on javascript side to save them to the HTML5 local storage, but at the moment I'm not sure if this idea would work.

Thanks,
Henko

Windows: test_conststructs failure

On my Windows 7 system, running:

runner clang_0_0.test_conststructs

gives this output:

======================================================================
ERROR: test_conststructs (__main__.clang_0_0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Y:\emscripten\tests\runner.py", line 1117, in test_conststructs
    self.do_test(src, '*70,97,15,3,3029,90*')
  File "Y:\emscripten\tests\runner.py", line 322, in do_test
    self.assertContained(expected_output, js_output)
  File "Y:\emscripten\tests\runner.py", line 276, in assertContained
    raise Exception("Expected to find '%s' in '%s'" % (limit_size(value), limit_size(string)))
Exception: Expected to find '*70,97,15,3,3029,90*' in '*70,97,9900,0.12,3029,90*
'

----------------------------------------------------------------------
Ran 1 test in 0.391s

FAILED (errors=1)

Here are the generated src.cpp.o.ll and src.cpp.o.js files:
http://pastebin.com/krJWXweS
http://pastebin.com/bkwFWGjy

The problem seems to be that the double member variable in the IUB struct is not correctly aligned/padded. Emscripten seems to think that the size of the IUB struct is 16 bytes, wheras clang will align and pad the double in a way that makes the struct 24 bytes.

The problem can be observed in the generated src.cpp.o.js file where Emscripten allocates 64 "bytes" (16 "bytes" for each entry in the iub array):

var $iub=__stackBase__+16;
var $after=__stackBase__+80;

but then goes ahead and copies 96 "bytes" when initializing the array:

_llvm_memcpy_p0i8_p0i8_i32($4, __ZZ4mainE3iub, 96, 8, 0);

Bindings test failure

In a clean checkout, test_scriptaclass fails on my mahcine due to being unable to find bindingtest.c. Did you forget to check it in?

JS Math.* equivalent cstdlib functions are not in FUNCTION_TABLE

The addresses of the corresponding cstdlib maths functions are not added to FUNCTION_TABLE. This causes any dereferenced function pointer to a cstdlib maths function to die.

$ cat test.c
#include <math.h>
#include <stdio.h>

int
main(void) {
  float (*fn)(float) = &sqrtf;
  printf("fn(10) = %f\n", fn(10));
  return 0;
}
$ ~/downloads/llvm-gcc-4.2-2.9.source/cbuild/install/bin/llvm-gcc -emit-llvm -c -o test.o test.c
$ ~/downloads/llvm-2.9/cbuild/Release/bin/llvm-link -o test test.o 
$ ~/downloads/llvm-2.9/cbuild/Release/bin/llvm-dis -show-annotations test
$ ~/reposs/emscripten/emscripten.py test.ll > test.js
$ ~/downloads/v8/d8 test.js 
test.js:727: TypeError: Property 'function sqrt() { [native code] }' of object 0,0 is not a function
        var $2=FUNCTION_TABLE[$1](10);
                                ^
TypeError: Property 'function sqrt() { [native code] }' of object 0,0 is not a function
    at _main (test.js:727:33)
    at Object.callMain (test.js:762:10)
    at run (test.js:779:18)
    at test.js:790:3

I'm going to guess they're not in FUNCTION_TABLE since they're normally dynamically linked functions. In this case however, since emscripten is providing equivalent functions for these cstdlib function calls, they should be added to FUNCTION_TABLE.

argparse is Python 2.7+, so we fail on 2.6-

emscripten.py fails on Python 2.6 and below due to argparse being 2.7+.

If there is no easy fix, then we should have a clear error message for this at least. Also, the test runner's error is even less clear, but that's a separate issue to fix.

Whether we need to rewrite anything depends on how popular 2.7 is, I guess. I wonder how many people still use 2.6. (I noticed this on a machine running Ubuntu 10.04, which is 2.6. I guess that's pretty old, but it is an LTS release...)

test_zlib failed on python runner.py

emscripten latest git version

/dev/shm/tmp$ uname -a
Linux ubuntu32 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:50 UTC 2011 i686 i686 i386 GNU/Linux
/dev/shm/tmp$ clang -v
clang version 2.9 (tags/RELEASE_29/final)
Target: i386-pc-linux-gnu
Thread model: posix
/dev/shm/tmp$ llvm-gcc -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: /home/duncan/llvm-2.9/32/llvmgcc42-2.9-release.src/configure --prefix=/home/duncan/llvm-2.9/32/Phase2/Release/llvmgcc42-2.9-release.install --program-prefix=llvm- --enable-llvm=/home/duncan/llvm-2.9/32/Phase2/Release/llvmCore-2.9-release.obj --enable-languages=c,c++,fortran
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2.9)
:/opt/src/emscripten/tests$ python runner.py clang_0_0.test_zlib
Running Emscripten tests...
test_zlib (__main__.clang_0_0) ... ./configure
Checking for shared library support...
Tested /opt/src/emscripten/tools/emmaken.py -w -c -m32 ztest689.c
emmaken.py:  /opt/src/emscripten/tools/emmaken.py -w -c -m32 ztest689.c
Running: /opt/src/llvm-2.9/bin/clang -w -c -m32 ztest689.c -m32 -U__i386__ -U__x86_64__ -U__SSE__ -UX87_DOUBLE_ROUNDING -UHAVE_GCC_ASM_FOR_X87 -m32 -emit-llvm -c
Tested cc -shared -m32 -o ztest689.so ztest689.o
ztest689.o: file not recognized: File format not recognized
collect2: ld returned 1 exit status
No shared library support; try without defining CC and CFLAGS
Building static library libz.a version 1.2.5 with /opt/src/emscripten/tools/emmaken.py.
Checking for off64_t... No.
Checking for fseeko... No.
Checking for unistd.h... No.
Checking whether to use vs[n]printf() or s[n]printf()... using s[n]printf().
Checking for snprintf() in stdio.h... No.
  WARNING: snprintf() not found, falling back to sprintf(). zlib
  can build but will be open to possible buffer-overflow security
  vulnerabilities.
Checking for return value of sprintf()... No.
  WARNING: apparently sprintf() does not return a value. zlib
  can build but will be open to possible string-format security
  vulnerabilities.

make libz.a
FAIL

======================================================================
FAIL: test_zlib (__main__.clang_0_0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "runner.py", line 1963, in test_zlib
    libraries=[self.get_library('zlib', os.path.join('libz.a'), make_args=['libz.a'])],
  File "runner.py", line 1918, in get_library
    self.do_link(map(lambda lib: os.path.join(project_dir, lib), generated_libs), bc_file)
  File "runner.py", line 155, in do_link
    assert output is None or 'Could not open input file' not in output, "Linking error: %s %s" % (output, cmd)
AssertionError: Linking error: /opt/src/llvm-2.9/bin/llvm-link: /dev/shm/tmp/building/zlib/libz.a: Could not open input file: No such file or directory
/opt/src/llvm-2.9/bin/llvm-link: error loading file '/dev/shm/tmp/building/zlib/libz.a'
 /opt/src/llvm-2.9/bin/llvm-link /dev/shm/tmp/building/zlib/libz.a -o /dev/shm/tmp/building/zlib/bc.bc

----------------------------------------------------------------------
Ran 1 test in 2.700s

FAILED (failures=1)

Investigate use of LLVM optimizations

pick_llvm_opts in tests/runner.py currently selects a subset of the LLVM optimizations, that appears to be safe and not generate nonportable code.

We might be missing some LLVM optimizations though. Also, in some cases LLVM optimizations help, in others they hurt. We should find out why.

Also, with typed arrays == 2, our memory model is much closer to the native one, and we should be able to use even nonportable optimizations.

OS X problems

Apparently we fail lots of tests (but not all) on OS X. I suspect header differences.

Errors have been reported in dlmalloc, libcxx, debug and freetype.

(I don't have an OS X machine to test on myself - need help with this.)

Translation failed with union

You can get a disassembled llvm from following link: http://desti.ruree.net/core_simple.o.ll
Original source code is here: http://dosbox.svn.sourceforge.net/viewvc/dosbox/dosbox/tags/RELEASE_0_74/src/cpu/core_simple.cpp?revision=3609&view=markup

I don't know why, but it failed after long execution, and a error message is following:

Assertion failed: What variable is this? |getelementptr|
Stack: Error
    at assertTrue (utility.js:60:23)
    at getVarImpl (jsifier.js:435:5)
    at jsifier.js:636:16
    at Object.processItem (jsifier.js:495:19)
    at Object.process (framework.js:155:26)
    at framework.js:104:25
    at Array.forEach (native)
    at Object.solve (framework.js:92:27)
    at JSify (jsifier.js:792:34)
    at JSify (jsifier.js:33:15)
utility.js:61: Assertion failed: What variable is this? |getelementptr|
    throw msg;
    ^

Global variables containing stubbed function not indexized

This is Python's _functools module compiled to LLVM bytecode, and this is it after being emscripted as a shared lib. On line 1721, inside run(), you can see:

HEAP[_partial_type+160]=_PyObject_GC_Del;

Note that _PyObject_GC_Del is an external function that exists only in the parent module. It appears that it is not indexized as it should be in the GlobalVariablePostSet pass.

Improve Code Readability and Aesthetics

Help with making the code prettier would be great. It is really easy to improve this, just changing function names would be a good start. Bigger refactorings are also possible of course.

Emscripten has tons of automatic tests, so we can make these changes with no risk of regressions.

The reason I don't do more of this myself, is I am too used to the code to know what stands out as unreadable and in need of cleanup.

Closure Compiler with the Python demo?

I've compiled the Python 2.7.1 source to a .ll file successfully and passed it through emscripten successfully to get out a JavaScript file. The problem I now have is that the Closure Compiler dies when I try to run it over the emscripten-produced JavaScript file. In your Python demo, the JS file looks like it has been Closure Compiled. I was wondering if there was anything you need to do in order to get this to work?

$ ./emscripten.py ~/downloads/Python-2.7.1/bin/pylibs/python.ll > input.js
$ java -Xmx2g -jar ../compiler/build/compiler.jar --js_output_file output.js --js input.js
java.lang.RuntimeException: java.lang.RuntimeException: INTERNAL COMPILER ERROR.
Please report this problem.
Analysis did not terminate after 200000 iterations
...
$

LLVM `blockaddress` node and `indirectbr` instruction are not supported

http://blog.llvm.org/2010/01/address-of-label-and-indirect-branches.html

Emscripten does not support the LLVM blockaddress node nor indirectbr instruction. These are rare instructions which are produced through using labels as values:

$ cat test.c
#include <stdio.h>

int
main(void) {
  const void *addr = &&FOO;
  goto *addr;

  printf("bad\n");
  return 1;
FOO:
  printf("good\n");
  return 0;
}

$ ~/downloads/llvm-gcc-4.2-2.9.source/cbuild/install/bin/llvm-gcc -Wall -W -emit-llvm -c -o test.o test.c
$ ~/downloads/llvm-2.9/cbuild/Release/bin/llvm-link -o=test.bc test.o
$ ~/downloads/llvm-2.9/cbuild/Release/bin/llvm-dis -show-annotations test.bc
$ ~/reposs/emscripten/emscripten.py test.ll 
intertyper.js:303: Invalid token, cannot triage: // {"tokens":[{"text":"indirectbr"},{"text":"i8*"},{"text":"%2"},{"text":","},{"tex
// t":"[label %FOO]","item":{"tokens":[{"text":"label"},{"text":"%FOO"}],"indent":0
// },"type":"["}],"indent":2,"lineNum":19,"__uid__":35}
        throw 'Invalid token, cannot triage: ' + dump(item);

And another example for blockaddress:

$ cat test.c
#include <stdio.h>

int
main(void) {
  const void *addr[] = {&&FOO, &&BAR};
  goto *addr[0];

  printf("bad\n");
  return 1;

FOO:
  printf("foo\n");
  return 0;

BAR:
  printf("bar\n");
  return 0;
}
$ ~/downloads/llvm-gcc-4.2-2.9.source/cbuild/install/bin/llvm-gcc -Wall -W -emit-llvm -c -o test.o test.c
$ ~/downloads/llvm-2.9/cbuild/Release/bin/llvm-link -o=test.bc test.o
$ ~/downloads/llvm-2.9/cbuild/Release/bin/llvm-dis -show-annotations test.bc
$ ~/reposs/emscripten/emscripten.py test.ll 
intertyper.js:347: Invalid segment: // [{"text":"i8*"},{"text":"blockaddress"},{"text":"(@main, %FOO)","item":{"tokens"
// :[{"text":"@main"},{"text":","},{"text":"%FOO"}],"indent":0},"type":"("}]
                throw 'Invalid segment: ' + dump(segment);
                                          ^

Asynchronous Javascript

To improve the performance of compiled code, asynchronous callbacks should be used wherever possible. The nimble library provides the functionality. It can be used without node.js.

http://caolan.github.com/nimble/

TypeError: Cannot read property 'tokens' of undefined

I get this output in the Javascript file:
// Processing item for llvm line 33124
intertyper.js:617: TypeError: Cannot read property 'tokens' of undefined
item.params = parseParamTokens(item.tokens[3].item.tokens);
^
TypeError: Cannot read property 'tokens' of undefined
at Object.processItem (intertyper.js:617:57)
at Object.process (framework.js:155:26)
at framework.js:104:25
at Array.forEach (native)
at Object.solve (framework.js:92:27)
at intertyper (intertyper.js:809:20)
at JSify (jsifier.js:33:30)
at /home/name/Dev/emscripten/src/compiler.js:79:1

This was the output at line 33124 in the .ll file:
invoke void bitcast (void (%"struct.std::__1::string", i32) @Z8toStringj to void (%"struct.std::basic_string<char,std::char_traits,std::allocator >", i32)_)(%"struct.std::basic_string<char,std::char_traits,std::allocator >"* noalias sret %3, i32 %99)
to label %invcont33 unwind label %lpad106

Here is the .ll file that was generated (plus the source file I think it may be caused by):
http://ge.tt/8wHRkn4

Cannot find settings file when compiling with v8

Affected version:
cca5278

To reproduce:
in ~/.emscripten
V8_ENGINE = ['/usr/bin/d8']
COMPILER_ENGINE=V8_ENGINE

Then, run emscripten.py anyfile.ll, leading to the following error (by the way, it would be better if errors were sent to stderr):
~/emscripten/src/compiler.js:30:Error loading file
var settings=JSON.parse(read(settings_file));^

To fix it:
tools/shared.py, line 47 in function run_js:

return timeout_run(Popen(engine + [filename] + (['--'] if 'v8' in engine[0] else []) + args,
stdout=stdout, stderr=stderr, cwd=cwd), 15*60 if check_timeout else None, 'Execution')

'v8' might not be in the pathname of the compiler. Changing it to 'd8' works in my case but it should be handled in a more general way.

Assertion failed: Type set to value: i64,i64

I'm trying to compile this LLVM:

  store i64 ptrtoint (i8* blockaddress(@vm_exec_core, %2882) to i64), i64* getelementptr inbounds ([1 x i64]* @finish_insn_seq, i32 0, i64 0), align 8

LLVM intrinsic unimplemented: llvm.uadd.with.overflow

Running LLVM opts sometimes produces calls to @llvm.uadd.with.overflow.i## intrinsics, where ## is 16/32/64. Emscripten does not have these implemented yet. Should be fairly simple to add.

Add profiler option

Basic idea is, add a PROFILING option to src/settings.js. When it is true, then entering and exiting a function will print to the console how much time was spent there. Then a commandline tool can parse that and generate profiling data, call stacks, etc.

test_indirectbr fails on clang

When I try runnig test_indirectbr using clang on the master branch, I get the following in src.cpp.o.js:

parseTools.js:499: TypeError: Cannot read property 'substr' of undefined
  if ((!type || type == 'double' || type == 'float') && (value.substr && value
                                                              ^
TypeError: Cannot read property 'substr' of undefined
    at parseNumerical (parseTools.js:499:63)
    at jsifier.js:166:61
    at Array.map (native)
    at parseConst (jsifier.js:166:36)
    at Object.processItem (jsifier.js:192:22)
    at Object.process (framework.js:155:26)
    at framework.js:104:25
    at Array.forEach (native)
    at Object.solve (framework.js:92:27)
    at JSify (jsifier.js:834:34)

Works fine on llvm-gcc though.

64-bit: Returning a multi-field structure by value does not work

When returning a structure from a function, only the first element is returned.

#include <stdio.h>

typedef struct {
    int x; int y;
} data;

data f(int x, int y) {
    data tmp;
    tmp.x = x;
    tmp.y = y;
    return tmp;
}

int main() {
    data z = f(5,6);
    printf("%d %d\n", z.x, z.y);
}

This program prints "5 0" instead of "5 6". If adding a third field to the structure, d8 dies with the following error:

ReferenceError: $ is not defined
    HEAP[$14]=$;; HEAP[$14+4]=1;; HEAP[$14+8]=5;

(llvm v2.9, v8 3.2.10.2, emscripten cca5278)

-- edit:
This only affects 64-bit compilation. Please compile with -m32 until 64-bit is properly supported.

Python raw_input

This isn't exactly an issue - more of a request.

I noticed the previous ticket in google code and I realise that there is probably no CommonJS way to do this, however I would love if it could be done.

I help out with http://ncss.edu.au (we teach python to high-school students) and we're looking for a way to get Python running in the browser so that students who don't have admin on their machine (ie at school computers etc) can write code and keep up with our challenge.

I'm not looking for someone to do all the work - I just need someone to point me in the direction of a solution. I see a few different possible ones:

Dive into the compiled Python JS and work out some way to make raw_input work
Hack at the emscripten compiler to see if there is a way to do it there

What do you think?

Problem with fixed size arrays in structs

The javascript generated by emscripten from the following C program prints "0 0", instead of "1.00000 1.00000". It appears the offset of fixed size arrays in structs is sometimes wrong (zero).



#include <stdio.h>
struct {   
    int x;
    double y[10];
    double z[10];
} commonblock;

int main(int argc, char **argv)
{
    int i;
    for (i = 0; i < 10; ++i) {
        commonblock.y[i] = 1;
        commonblock.z[i] = 1;
    }

    printf("%f %f\n", commonblock.y[0], commonblock.z[0]);
    return 0;
}

Code like this was obtained by converting a Fortran program to C. I worked around this issue by putting every fixed size array in its own common block.

Moonlight JS - C# / .NET Runtime for Javascript

Moonlight is the open source, client-side .NET runtime.

http://www.mono-project.com/Moonlight

Moonlight is currently being compiled by Xamarin @ http://xamarin.com (Miguel de lcaza) to iPhone, iPad, & Android devices.

However, the most important platform - is the browser. Our goal is to have a .NET runtime for Javascript. This would open the flood gates for rich-client business applications of all kinds to be deployed to the browser, to run on any operating system or device.

Ryan

PS. Mono compiler has full, built-in LLVM output support. http://www.mono-project.com/Mono_LLVM

Error in generated javascript (from InChI library)

The InChI library is a somewhat complex C library used in chemistry.

The generated javascript is available at http:.//www.redbrick.dcu.ie/~noel/tmp/tmp.js.gz

"v8/d8 tmp.js" gives a Javascript error as follows:

$ ../v8-repo/d8 tmp.js
tmp.js:417618: SyntaxError: Unexpected token {
IHEAP[cnMe+6] = {$2{i16_2__i8_16__i8_0__i8_0__i8_undef}__%2_{i16_4__i8

It put a carat "^" underneath the first "{".

I'd really appreciate some help with this. The InChI library is available from http://www.iupac.org/inchi/download/version1.03/INCHI-1-API.zip. The makefile in INCHI-1-API/INCHI_API/gcc_so_makefile should be run as follows:

LINKER=pathto/emmaken.py SHARED_LINK=pathto/emmaken.py C_COMPILER=pathto/emmaken.py make

after commenting out the section "=== InChI Main Link rule ================" (whose compilation causes an llvm error).

Automate building library components from C

We need a way to automate including library function that can be compiled from C. This would allow us to reduce the amount of hand-written JS code by quite a large margin. Two approaches have been proposed:

Compile the libraries from C, link using llvm-link, then pass to the emscripten compiler. This is what's currently used for dlmalloc.
Merge the libraries with C code then compile all together to LLVM bytecode and further to JS. This might require us to decompile provided bytecode to C first using the existing LLVM C backend.

Build Box2D and make Cool Demos

Assertion failed: Failed to find the # of uses of var: $0

I was compiling the cubescript example directly via the command line and encountered this:

Assertion failed: Failed to find the # of uses of var: $0
Stack: Error
at assertTrue (utility.js:60:23)
at analyzer.js:271:13
at Array.forEach (native)
at analyzer.js:260:20
at Array.forEach (native)
at Object.processItem (analyzer.js:238:22)
at Object.process (framework.js:155:26)
at framework.js:104:25
at Array.forEach (native)
at Object.solve (framework.js:92:27)
utility.js:61: Assertion failed: Failed to find the # of uses of var: $0
throw msg;
^

Emscripting Python with optimizations+assertions produces invalid code

Running emscripten on python.ll with OPTIMIZE = 1, RELOOP=1 and ASSERTIONS=1 produces many invalid blocks, of the general form:

if ($n) {
  ...
}
else {
  ...
}
else { throw "Bad multiple branching: " + __label__ + " : " + (new Error().stack); }

The extra else is added at jsifier.js:402. Removing these lines from the result produces a working file (which is about 10 times faster than the unoptimized version).

Emscripten crash on an optimized CPython 2.7.2 build

Here's python.opt.ll. Here's the stack trace:

Assertion failed: undefined
Stack: Error
    at assertTrue (utility.js:60:23)
    at toNiceIdent (parseTools.js:74:3)
    at getLabelId (jsifier.js:565:13)
    at makeBranch (jsifier.js:593:37)
    at jsifier.js:602:23
    at Object.processItem (jsifier.js:524:19)
    at Object.process (framework.js:155:26)
    at framework.js:104:25
    at Array.forEach (native)
    at Object.solve (framework.js:92:27)
utility.js:61: Assertion failed: undefined
    throw msg;
    ^

Running on an unoptimized version works fine.

emmaken.py needs to distinguish c from c++

With CMake its possible to compile .c files as C++. This is an unfortunate yet common practice in large, legacy codebases. emmaken.py uses the extension to figure out which compiler it needs to use, but this fails with .c files that are really c++.

For example, using helloworld.c:

#include <stdio.h>

// C++ headers
#include <cmath>
#include <limits>

int helloworld(void) {
   printf("helloworld \n");
   return 0;
}

and CMakeLists.txt:

cmake_minimum_required(VERSION 2.8.3)

# these must be set *before* project()
if (EMSCRIPTEN)
    set(EMSCRIPTEN_TOOLS ${EMSCRIPTEN_HOME}/tools)
    SET(CMAKE_C_COMPILER          "${EMSCRIPTEN_TOOLS}/emmaken.py")
    SET(CMAKE_CXX_COMPILER        "${EMSCRIPTEN_TOOLS}/emmaken.py")
    SET(CMAKE_LINKER              "${EMSCRIPTEN_TOOLS}/emmaken.py")
    SET(CMAKE_CXX_LINKER          "${EMSCRIPTEN_TOOLS}/emmaken.py")
    SET(CMAKE_C_LINK_EXECUTABLE   "${EMSCRIPTEN_TOOLS}/emmaken.py")
    SET(CMAKE_CXX_LINK_EXECUTABLE "${EMSCRIPTEN_TOOLS}/emmaken.py")
endif(EMSCRIPTEN)

project(helloworld)

# these must be set *after* project()
if (EMSCRIPTEN)
    SET(CMAKE_AR                  "${EMSCRIPTEN_TOOLS}/emmaken.py")
    SET(CMAKE_RANLIB              "${EMSCRIPTEN_TOOLS}/emmaken.py")
endif(EMSCRIPTEN)

# list of sources
set(SRCS helloworld.c)

# compile all files as C++, no matter their extension
set_source_files_properties( ${SRCS} PROPERTIES LANGUAGE CXX )

# make a lib
add_library(helloworld ${SRCS})

has this output:

$ cmake .. -DEMSCRIPTEN=1 -DEMSCRIPTEN_HOME=/home/vail/Development/emscripten; make VERBOSE=1
-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
-- Check for working C compiler: /home/vail/Development/emscripten/tools/emmaken.py
-- Check for working C compiler: /home/vail/Development/emscripten/tools/emmaken.py -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /home/vail/Development/emscripten/tools/emmaken.py
-- Check for working CXX compiler: /home/vail/Development/emscripten/tools/emmaken.py -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/vail/Development/helloworld/.build-js
/usr/bin/cmake -H/home/vail/Development/helloworld -B/home/vail/Development/helloworld/.build-js --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start /home/vail/Development/helloworld/.build-js/CMakeFiles /home/vail/Development/helloworld/.build-js/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory `/home/vail/Development/helloworld/.build-js'
make -f CMakeFiles/helloworld.dir/build.make CMakeFiles/helloworld.dir/depend
make[2]: Entering directory `/home/vail/Development/helloworld/.build-js'
cd /home/vail/Development/helloworld/.build-js && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/vail/Development/helloworld /home/vail/Development/helloworld /home/vail/Development/helloworld/.build-js /home/vail/Development/helloworld/.build-js /home/vail/Development/helloworld/.build-js/CMakeFiles/helloworld.dir/DependInfo.cmake --color=
Dependee "/home/vail/Development/helloworld/.build-js/CMakeFiles/helloworld.dir/DependInfo.cmake" is newer than depender "/home/vail/Development/helloworld/.build-js/CMakeFiles/helloworld.dir/depend.internal".
Dependee "/home/vail/Development/helloworld/.build-js/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "/home/vail/Development/helloworld/.build-js/CMakeFiles/helloworld.dir/depend.internal".
Scanning dependencies of target helloworld
make[2]: Leaving directory `/home/vail/Development/helloworld/.build-js'
make -f CMakeFiles/helloworld.dir/build.make CMakeFiles/helloworld.dir/build
make[2]: Entering directory `/home/vail/Development/helloworld/.build-js'
/usr/bin/cmake -E cmake_progress_report /home/vail/Development/helloworld/.build-js/CMakeFiles 1
[100%] Building CXX object CMakeFiles/helloworld.dir/helloworld.c.o
/home/vail/Development/emscripten/tools/emmaken.py     -o CMakeFiles/helloworld.dir/helloworld.c.o -c /home/vail/Development/helloworld/helloworld.c
emmaken.py:  /home/vail/Development/emscripten/tools/emmaken.py -o CMakeFiles/helloworld.dir/helloworld.c.o -c /home/vail/Development/helloworld/helloworld.c
emmaken.py:  /home/vail/Development/emscripten/tools/emmaken.py -o CMakeFiles/helloworld.dir/helloworld.c.o -c /home/vail/Development/helloworld/helloworld.c
Running: /home/vail/Development/llvm-gcc-4.2-2.9.source/cbuild/install/bin/llvm-gcc -o CMakeFiles/helloworld.dir/helloworld.c.o -c /home/vail/Development/helloworld/helloworld.c -m32 -U__i386__ -U__x86_64__ -U__SSE__ -UX87_DOUBLE_ROUNDING -UHAVE_GCC_ASM_FOR_X87 -emit-llvm -c
/home/vail/Development/helloworld/helloworld.c:1:17: error: cmath: No such file or directory
/home/vail/Development/helloworld/helloworld.c:2:18: error: limits: No such file or directory
make[2]: *** [CMakeFiles/helloworld.dir/helloworld.c.o] Error 1
make[2]: Leaving directory `/home/vail/Development/helloworld/.build-js'
make[1]: *** [CMakeFiles/helloworld.dir/all] Error 2
make[1]: Leaving directory `/home/vail/Development/helloworld/.build-js'
make: *** [all] Error 2

File system support

Currently emscripten stubs or fakes most filesystem access calls. This causes plenty of software to either not work at all or fail subtly due to incomplete functionality. I propose implementing a lightweight file system that would allow users of emscripted programs to set up a access to their real files with the paths that the emscripted program expects.

Basically, I plan to follow the same pre-configuration approach currently used for STDIO.prepare(). I imagine a main Module.FS object which would be used to set up all the paths before running the module. To create a file you could pass actual data or a relative URL that would be loaded lazily when/if requested. In non-browser contexts, the relative URL can be used as a real filesystem path.

A similar method could be used to create virtual folders or explicitly or mark leaf folders for lazy-loading, so any reads of file from inside the folder would be forwarded to a given URL.

In the interest of keeping it lightweight, I see no reason to support any of the following:

Separate read and execute permissions.
The sticky permission bit.
Multiple users/groups.
Separate created and modified timestamps.
Hard links.
Block devices.

Modifying anything (writing to files, deleting/creating folders, modifying permissions) by the emscripted program could either be handled by a user-specified callback, or happen in memory with the user being able to flush all the changes after the program is done running. The latter would be simpler, but the former significantly more flexible.

I believe this would allow emscripten to implement all of the Unix filesystem API at a minimum level.

In terms of actual implementation, I see the file system as a simple nested dictionary/Object. A straightforward but memory-wasteful approach would be to have something like this for a simple file system:

root = {
  read: true,
  write: false,
  isFolder: true,
  timestamp: <Date>,
  inodeNumber: 1,
  contents: {
    subfolder1: {
      read: true,
      write: true,
      isFolder: true,
      timestamp: <Date>,
      inodeNumber: 2,
      contents: {
        myfile: {
          read: true,
          write: true,
          isFolder: false,
          timestamp: <Date>,
          inodeNumber: 3,
          url: '/path/to/myfile/on/server'
        },
        myotherfile: {
          read: true,
          write: true,
          isFolder: false,
          timestamp: <Date>,
          inodeNumber: 4,
          contents: <index-into-HEAP>
        },
        mylink: {
          read: true,
          write: true,
          isFolder: false,
          timestamp: <Date>,
          inodeNumber: 5,
          link: '/subfolder1/myfile'
        },
        mydevice: {
          read: true,
          write: true,
          isFolder: false,
          timestamp: <Date>,
          inodeNumber: 6,
          read: readFromDevice,
          write: writeToDevice
        },
        subfolder3: {
          read: true,
          write: false,
          isFolder: true,
          timestamp: <Date>,
          inodeNumber: 7,
          url: '/getfileFromSubfolder?name=%s'
        }
      }
    },
    subfolder2: {
      read: false,
      write: false,
      isFolder: true
      timestamp: <Date>,
      inodeNumber: 8,
    }
  }
};

In the interest of memory conservation, the permissions read and write could be merged into a single flags field, and instead of an Object for each file, we can use a more flattened 5-tuple: permissions, inode, timestamp, type and content, where type implies isFolder. The above structure would look as follows using this approach:

root = [1, 1, {
  subfolder1: [3, 2, <Date>, 'contents', {
    myfile: [3, 3, <Date>, 'url', '/path/to/myfile/on/server'],
    myotherfile: [3, 4, <Date>, 'data', <index-into-HEAP>],
    mylink: [3, 5, <Date>, 'link', '/subfolder1/myfile'],
    mydevice: [3, 6, <Date>, 'device' [readFromDevice, writeToDevice]],
    subfolder3: [1, 7, <Date>, 'url_pattern', '/getfileFromSubfolder?name=%s']
  },
  subfolder2: [0, 8, <Date>, 'contents', {}]
}];

Parser bug: Invalid token, cannot triage

Input file is: http://desti.ruree.net/midi.o.ll

Output is following:

intertyper.js:303: Invalid token, cannot triage: // {"tokens":[{"text":"module"},{"text":"asm"},{"text":"\".symversnd_pcm_hw_params
// _get_access,snd_pcm_hw_params_get_access@ALSA_0.9\""}],"indent":0,"lineNum":5,"_
// _uid__":359}
    throw 'Invalid token, cannot triage: ' + dump(item);

GHC output: TypeError: Cannot read property 'tokens'

While running the following I get an error:

$ ./emscripten.py demo.ll /home/chris/v8-trunk/d8
intertyper.js:393: TypeError: Cannot read property 'tokens' of undefined
        params: parseParamTokens(item.tokens[2].item.tokens),
                                                ^

The full demo.ll file is here: https://gist.github.com/914021

Steps to reproduce (if desired). demo.hs:

main = putStrLn "Hello, World!"

Build:

/opt/ghc7/bin/ghc -fllvm -keep-llvm-files demo.hs -fforce-recomp  # outputs demo.ll

I suspect this is probably not a bug in emscripten, as GHC may use some novel extensions to support its calling conventions and such. I don't really know anything about LLVM or how GHC uses it. I suppose it's worth posting; if other Haskellers try this they will get the same problem, undoubtedly.

Error due to unhandled {} type

Input:

#include <cstdio>
#include <list>
#include <string>

using namespace std;

int main(void) {
        list<string> string_list;
        string_list.push_back("hello");

        printf("%d\n", string_list.size());

        return 0;
}

Output:

intertyper.js:137: TypeError: Cannot read property 'text' of undefined
  while (tokens[openBrace].text != '{') openBrace --;

Type info not available to library code

Currently the type info, in particular pre-calculated struct layouts, is not available to library code at either compile- or run-time. Since this info is already calculated from the LL, it would be nice to have it available directly, so one could call Runtime.generateStructInfo('foo') (or something similar) and get an object defining the size of the foo struct and the offsets of its members.

JS crash: |ReferenceError: _dlopen is not defined|.

The code: import os; print os.fork.__doc__ leads to:
JS crash: |ReferenceError: _dlopen is not defined|. Please let us know about this problem!

Windows: Problem with passing arguments by value

Running Emscripten with llvm 2.9 on my Windows system, test_raytrace fails with this output in src.cpp.o.js:

analyzer.js:285: TypeError: Cannot read property 'derivedTypes' of undefined
              if (!func.variables[item.value.ident].derivedTypes) {
                                                   ^

The problem seems to be related to Emscripten not understanding this llvm function signature:

define internal %struct.node_t* @_Z6createP6node_tii3v_tS1_d(%struct.node_t* %n, i32 %lvl, i32 %dist, %struct.v_t* byval align 4, %struct.v_t* byval align 4, double %r)

The complete llvm output can be found here (please note that I had to change iostream usage with printfs to make it compile):
http://pastebin.com/7s7vV5xJ

If I pass the v_t structs by const reference instead of by value in the create() function the test passes. In this case the llvm function signature looks like this:

define internal %struct.node_t* @_Z6createP6node_tiiRK3v_tS3_d(%struct.node_t* %n, i32 %lvl, i32 %dist, %struct.v_t* %c, %struct.v_t* %d, double %r)

If I insert these lines before line 285 in analyzer.js:

print(JSON.stringify(item.value.ident));
print(JSON.stringify(func.variables));

I get that item.value.ident is "$0", and then there is this oddly looking entry in func.variables:

"4":{"ident":"4","type":"%struct.v_t*byvalalign","origin":"funcparam","lineNum":216,"uses":null}

I'm guessing the "4" here comes from the parameter declaration:

%struct.v_t* byval align 4

Real malloc/free implementation

Right now malloc() simply extends the stack address and free() is a no-op. This means severe memory leaks if any serious emscripted program is used for any length of time. A real memory manager would be very useful indeed.

Python's print adds an extra newline

On the Python demo at http://syntensity.com/static/python.html type

print 'a', 'b'

Expected output:

a b

Actual output:

a
b

OpenGL Support

In theory it should be straightforward to support OpenGL ES, which is very close to WebGL. Some initial work is in src/library_gl.js There are some differences though, WebGL doesn't have clientside arrays for example.

It might also be possible to do full OpenGL, by emulating the function calls. This might end up being very slow. Might be interesting to look at ShaderGen, an open source tool to generate shader code for fixed-function code.

http://mew.cx/glsl/shadergen/
http://www.gp32x.com/board/index.php?/topic/55746-shadergen-for-linux/

Investigate Dataview usage for typed arrays == 2

If you use DataView, you are no longer required to divide the pointers (or bitshifting) with some constant but rather use [dataview].getInt32, getFloat64 and setInt32 etc. An other bonus is that you can support int8, int16, uints, float32 etc. and can provide better compability.
The question is of course of it is as fast as an array index at this moment, but likely will be in the future.

Lua string functions not fully supported

Apparently Lua string functions is not fully supported in the compiled Lua interpreter. I successfully tried

string.byte("ABCDE", 2)
string.char(65,66,67,68,69)
string.format("%.7f", math.pi)
string.gsub("Hello banana", "banana", "Lua user")
string.gsub("banana", "(an)", "%1-")
string.rep("Lua ",5)
string.reverse("lua")
string.sub("Hello Lua user", -8, 9)
string.upper("Hello, Lua user!")

but the following functions produces an (error object is not a string) which is not very helpful anyway:

string.find("Hello Lua user", "Lua")
string.gsub("Hello Lua user", "(%w+)", print)
for word in string.gmatch("Hello Lua user", "%a+") do print(word) end
string.match("I have 2 questions for you.", "%d+ %a+")

Since searching in strings is a very common task, the Lua interpreter is of little use with this limitations. I think the bug is in the compilation of lstrlib.c but I could not track down the error.

llvm/clang 3.0 support

Looks like the svn version of llvm produces things like this now:

%8 = call %2 @llvm.umul.with.overflow.i32(i32 %7, i32 4) ; [#uses=2 type={ i32, i1 }]

we crash on that. Also need to make sure we still parse the #uses info properly.

emmaken.py doesn't emulate ar correctly

The way that CMake uses the 'ar' tool doesn't seem to be compatible with the way that emmaken.py emulates it.