qut-ifn660-ecmascript / native-ecmascript

Dependencies

• gcc/g++ 5+ on ubuntu, on OSX 4.2.1 is sufficient (OSX 10.10+)
• flex 2.5+
• bison 2.3+ - 3.0 should work but there will be warnings when generating parser
• babel-cli from npm, installed with -g - if you want to run JS validations

Project Structure

|-- ast                    # contains AST node classes
|-- runtime                # contains classes
|-- scope                  # contains classes
|-- type                   # contains classes
|-- tests            
|   |-- parseable
|   |   |-- lexer-assert   # contains expected lexer output for parseable/test tests
|   |   |-- parser-assert  # contains expected AST output for parseable/test tests
|   |   |-- test           # tests in this folder must be
|   |                        valid javascript and passes all tests
|   |-- unparseable
|       |-- lexer-assert   # contains expected lexer output for unparseable/test tests
|       |-- test           # tests in this folder must be
|                            valid javascript, passes lexer tests,
|                            but does not need to be parseable
|-- .travis.yml            # Travis CI build configuration
|-- grammar.l              # lexer/flex grammar
|-- grammar.y              # bison/parser grammar
|-- Makefile               # GNU make configuration
|-- main.cpp               # main compiler program
|-- test_lex.c             # lexer test program
|-- test_parser.cpp        # parser test program
|-- utils.c                # helper utils
|-- y.tab.h                # helper headers
...

Build Commands

Just build the compiler

make

Build compiler, run tests, but without js file validations (faster)

make test

Build compiler, run tests, and js file validations

make test_all

Build compiler, and run lexer tests only

make test_lexer

Build compiler, and run parser tests only

make test_parser

Test compiler generates pseudo-assembly code from input file - Will generate inputFile.js.c

make
./compiler <inputFile.js>

Error Logs

Log	What's in it	What's it for
error.log	lexer error, parser error for /parseable/ tests	You need to fix these to have a successful build
error_parser.log	parser error for /unparseable/ tests	You want to fix these eventually
temperror.log	temporary error file	Ignore it
grammar.dot, grammar.output	Bison logs	Useful for fixing parser errors

native-ecmascript's Issues

Unary and shift expressions

I adding unary and shift expressions in section 12.5 and 12.8 like DELETE, VOID, TYPEOF, UNARY_ADD, UNARY_SUBTRACT, ADD, SUBTRACT, BITWISE_NOT, LOGICAL_NOT and LEFT_SHIFT, SIGNED_RIGHT_SHIFT, UNSIGNED_RIGHT_SHIFT

Make travis use bison >=3.0

update bison on travis
https://github.com/mbebenita/was/blob/master/.travis.yml

Fix failing tests

After porting code to c++ some tests are failing when lexing

language?

is there any requirement for this?

it seems we're going with c/c++ at the moment?
or will we go .net+c#?
do we have any guidelines or recommendations for this? (i was late, not sure if i missed anything about this part)

adding dump test for AST

create tests and expected outputs for lexical analyser

create tests (js files)

so that we can use these tests to do TDD

acceptance criteria:

1. please put your test files in tests/lex/test
2. expected output in tests/lex/assert
3. create at least 2 test cases
4. each test must include at least 2 not yet tested language features / tokens
5. 1 small test case (2-5 lines), 1 large test case (at least 10 lines)
6. test js files must be valid (try running in browser console)

extend grammar

extend grammar.l and grammar.y if needed, test_lex.c as well
so that it can parse your own tests

acceptance criteria:

1. main program must still build
2. test_lex must still build
3. tests must pass "make test"

create tests

from #21

handle jumping

no idea how to do it without big modification on emit

Clean up existing productions to match spec

A few of the rules don't match the ES6 spec, often in cases to reduce the number of conflicts. This may be something we want to address in the future (not urgent).

end of file shouldn't be END_OF_FILE ?

http://stackoverflow.com/questions/19997853/bison-end-of-file-gives-syntax-error

• Furthermore, flex always returns 0 when it finds the end of input.

check it is the case

or can END_OF_FILE set to 0?

that should solve the expected token $end but found END_OF_FILE error

create tests

from #21

Design and build an object-oriented class hierarchy to be able to represent the AST

• design
• build

setup tests for parser (bison)

setup unit testing structure for the parser part

Modify return statement test

When function declaration is complete, modify return statement.
return.js is in tests/unparseable directly.
Simply move it to /parseable directly to test it.

create tests

from #21

get d8 into Makefile to test the .js files

put d8 into make file to test the .js file first
so we know it is valid javascript file first before we do everything else

Multi-line strings should be able to be empty

It looks like modifying https://github.com/QUT-IFN660-ECMAScript/Native-ECMAScript/blob/develop/grammar.l#L151 to make the newline optional will fix the error people are having with multiline strings.

create tests

from #21

build tools / platforms

depends on #4

Week 5 deliverable

https://www.facebook.com/groups/1061342000595510/permalink/1072530986143278/

---

Could a representative from each team please email me the following by the end of this week:

1. a pdf file showing a derivation tree for the simple x=42; example in your language. Make sure you use the exact same grammar and names for non-terminal symbols as used in the official specification for your language.
2. your current lex file
3. your current yacc file
   I don't expect your scanner or parser to be complete, I'd rather a correct working subset of your language, than a more complete but wrong implementation. Again, you should very closely follow the lexical and context free grammar included in your language specification.
   This submission is not for assessment, it's just so I can check that you're all on the right path.
   Please don't attempt to build an Abstract Syntax Tree yet. We will start on that in the next Workshop session. Note - there will be no workshop next week due to public holiday.

13.3 Declarations and the Variable Statement

Shift/Reduce and Reduce/Reduce conflicts - problems implementing 13.3 of standard
Perhaps variabledeclaration and initialiser once implemented correctly (13.3.2 and 12.2.6 respectively) will remove conflict. Difficulty implementing standard......unsure

handle multiline string

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals

Port current code to c++

There will need to be some little changes here and the to get the current bison and yacc files to compile under c++

create AST for try_catch_finally statement

12.2.5 Array Initializer

add grammars

add some grammars in 12.2.6: ObjectLiteral, PropertyDefinitionList, PropertyDefinition, ComputedPropertyName, CoverInitializedName, Initializer

requirement specification

do we have any? or do we make one ourselves?

create tests

from #21

create tests

from #21

Add break statement test

class BreakStatement is added but test is not added yet.

extend lexical analyser and test program so that it can parse + pass the determined test

from #21

extend grammar.l and grammar.y if needed, test_lex.c as well
so that it can parse your own determined test

@AnthonyGough @farukbaig @HarjotKaur @jordanhennell @hscells @kojiagile @ksulinski @luantrongtran

separate lex only tests with lex and parser tests

so travis can pass

create tests

from #21

Makefile bugs

• get an error when I make: no matches found: grammar.tab.* I didn’t make before that so it can’t clean
• VAR gets created if tests fail, maybe its doing “something something > VAR, and gets interpreted by shell

Resolve shift/reduce & reduce/reduce conflicts

currently there are 12 s/r conflicts and 60 r/r

build broken

0b6b030 is throwing yyerror error

Identify shift/reduce conflicts

Try to address some of the reduce/reduce and shift/reduce conflicts present in the grammar. Generate bison output to enable the state to be more easily visualised
bison --verbose grammar.y

Will produce grammar.output to show unused/useless tokens, nonterminals but more imp the automaton state transitions - so currently the shift reduce conflicts can visualise the states which have conflicts
e.g.

• State 51 conflicts: 4 reduce/reduce
• State 60 conflicts: 4 reduce/reduce
• State 110 conflicts: 2 reduce/reduce

Sample grammar.output in branch anthony or generate yourself - added to .gitignore

create tests

from #21

extending the grammar

this is an epic

create tests

from #21

Remove all 'resolveNames()' methods from classes

:(

error messages with newlines

researched on it these few days, it's apparently very difficult, as gnumake simply doesn't play well with newlines.

• $(shell) strips out newlines
• $(error),$(warning) etc doesn't treat \n as newlines

there could be ways like reading the file in a way that $(foreach) can read it as multiple lines, then do echo for each line, but its just wasting time

may not look into it again, so marking it as enhancement

continuous integration environment

connection to travis ci

create travis.yml config file to make the ci platform do

• install appropriate build tools
• install / build tools
• run unit tests

what else?

setup tests

https://sourceforge.net/projects/cppunit/

do we need a framework or can we do unit tests without a framework?