danvegeto / pipelinescript Goto Github PK

we need better error-checking for both syntax and semantics errors

Implement the following method that executes an external algorithm file.

public static String call(String path, String[] args);

file is the relative path of an executable file that will be in the plugins folder (e.g. newspaper/scrape.py). PluginManager should detect the executable language based on the file extension (e.g. Python for .py files) and then construct the shell command to be executed with the given args. For example:

call("newspaper/scrape.py", new String[]{"http://www.nytimes.com"})

yields the following command

python plugins/newspaper/scrape.py http://www.nytimes.com

1. Newspaper article scraping + TF-IDF
2. Newspaper article scraping + Stanford NER
3. Newspaper article scraping + UMASS topic modeling + tSNE dimensionality reduction

• subset indexing
• row-wise/col-wise concatenation
• union, intersection, etc.

pls.sh and pls.py have been modified, testing system needs to be updated to fix it

see example: https://github.com/danvegeto/pipelinescript/blob/master/doc/examples.md#shell_args

see example: https://github.com/danvegeto/pipelinescript/blob/master/doc/examples.md#function_import

currently only raw text is working

for each example defined in the documentation:
https://github.com/danvegeto/pipelinescript/blob/master/doc/examples.md

create 3 files in the "tests" folder

1. test_name.pls (PipelineScript)
2. test_name.java (Java)
3. test_name.txt (Output)

create a copy of pls.sh called test_pls.sh that doesn't delete Output.java

then create a tester that loops of all the tests and does the following

1. run the PipelineScript code (test_pls.sh test_name.pls > output.txt)
2. compare Output.java to test_name.java
3. compare output.txt to test_name.txt

• subset indexing
• union, intersection, etc.

I think we should use semi-colon ";"

• variable type checking
• compile error messages
• operator overloading

The translator tries to lookup the text literal in the symbol table as a variable name. This causes text_concat.pls to fail.

This doesn't translate:

function f = !"script.py"
print f()
print f("arg1", "arg2", "arg3")

see examples:

https://github.com/danvegeto/pipelinescript/blob/master/doc/examples.md#file_read
https://github.com/danvegeto/pipelinescript/blob/master/doc/examples.md#file_write

for example:

num[][] x = {{1, 2}, {3, 4}}
text[][] y = {{"a", "b"}, {"c", "d"}}
table z = {{"a", 1}, {"b", 2}}