deedlefake / wdte

WDTE is a simple, functional-ish, embedded scripting language.

wdte

WDTE is a simple, functional-ish, embedded scripting language.

Why does this exist?

Good question. In fact, I found myself asking the same thing, hence the name.

I had a number of design goals in mind when I started working on this project:

• Extremely simple. Entire grammar is less than 20-30 lines of specification.
• Grammar is LL(1) parseable.
• Functional-ish, but not particularly strict about it.
• Designed primarily for embedding.
• Extremely easy to use from the binding side. In this case, that's primarily Go.

If you want to try the language yourself, feel free to take a look at the playground. It shows not only some of the features of the language in terms of actually writing code in it, but also how embeddable it is. The playground runs entirely in the browser on the client's end thanks to WebAssembly.

Example

package main

import (
	"fmt"
	"os"
	"strings"

	"github.com/DeedleFake/wdte"
	"github.com/DeedleFake/wdte/wdteutil"
)

const src = `
let i => import 'some/import/path/or/another';

i.print 3;
+ 5 2 -> i.print;
7 -> + 5 -> i.print;
`

func im(from string) (*wdte.Scope, error) {
	return wdte.S().Map(map[wdte.ID]wdte.Func{
		"print": wdteutil.Func("print", func(v interface{}) interface{} {
		fmt.Println(v)
		return v
	}),
	}), nil
}

func Sum(frame wdte.Frame, args ...wdte.Func) wdte.Func {
	frame = frame.Sub("+")

	if len(args) < 2 {
		return wdteutil.SaveArgs(wdte.GoFunc(Sum), args...)
	}

	var sum wdte.Number
	for _, arg := range args {
		sum += arg.(wdte.Number)
	}
	return sum
}

func main() {
	m, err := wdte.Parse(strings.NewReader(src), wdte.ImportFunc(im), nil)
	if err != nil {
		fmt.Fprintf(os.Stderr, "Error parsing script: %v\n", err)
		os.Exit(1)
	}

	scope := wdte.S().Add("+", wdte.GoFunc(Sum))

	r := m.Call(wdte.F().WithScope(scope))
	if err, ok := r.(error); ok {
		fmt.Fprintf(os.Stderr, "Error running script: %v\n", err)
		os.Exit(1)
	}
}

Output

3
7
12

Documentation

For an overview of the language's design and features, see the GitHub wiki.

Status

WDTE is in a pre-alpha state. It is filled with bugs and large amounts of stuff are subject to change without warning. That being said, if you're interested in anything, feel free to submit a pull request and get things fixed and/or implemented faster.

wdte's Issues

Return Newly Created Scope from `Compound.Collect()`

Scope bounds don't work. They're useful for a few very specific situations, but they don't provide enough flexibility.

Goal

A replacement should provide a way for items inside the scope to access things in parent scopes, while items in child scopes should only be able to access things in the scope above. For example, given the scope hierarchy a -> b -> c, b should be able to access a, but c should only be able to access b, not a. If an access of b from c accesses a, that's fine.

One possibility could be scope consolidation. Rather than attempting to insert special scopes into the hierarchy, it could be possible to consolidate scopes into a single scope such that items in scopes above that consolidated one are not accessible from below it.

Example

An example of the problem is demonstrated by the usage of collect to create a scope. The parent scope of that scope is accessible through that scope due to the way that the . operator works.

For example, the following will print Pi:

let io => import 'io';
let m => import 'math';

let s => collect (
  let t => 3;
);

io.stdout -> io.writeln s.m.pi;

Scopes as Objects

Somewhat accidentally, all of the pieces are now in place to use scopes as a type of immutable, hierarchical object implementation. This can be accomplished through the following changes:

• Add function, probably called something like objectify, to std that takes a single compound as its first argument and calls Collect(), rather than Call(), returning the *Scope instead of the final Func.
• Make Scope implement Atter.
• Add a function, probably called either add or sub, to std that takes a scope, an ID as a string, and a value and returns a subscope that has the value assigned to the ID.
• Have the Collect() method put bounds around the collected scope.

That's basically it. Viola. Accidental immutable objects. That'll make implementing things like FileInfo quite a bit simpler, as it'll be possible to simply return a *Scope that contains the information about the info.

#14 will probably become quite a bit more useful after this.

Rename `Expr` to `FuncCall`

This has been percolating for a bit, now. The name's just not accurate.

wdte: add syntactical support for array and string indexing

Using a.at and str.at is rather annoying in a lot of situations, especially for something that is so common. A syntax for indexing would be useful, although the existing functions should probably be left in for easier use in the chains. If the syntax can support slicing, that would also be useful.

Unfortunately, due to the way the parse works, using [] for indexing is not really possible. Possible alternatives:

• array[@i]. Possible, but a bit unwieldy. Definitely better than using the std functions, though.
• array.i. Ambigous, but it should be possible. Doesn't really work for slicing, though.
• array.[i]. Possible, but only by making .[ a single token. Probably the cleanest, though.

wdte: use `*Frame`

The current implementation uses Frame instead of *Frame, despite it being a linked list. I think the implementation of Scope is good evidence that this isn't necessary. An upside of using *Frame would be that a user could pass nil as the top-level frame.

It might make sense as well to allow the default frame to be overridden. This would allow the user to just use nil but still get access to, for example, std. Maybe something like a DefaultFrame global variable that gets used in the case of a nil frame.

Convert to Go Module

As of Go 1.11beta2, there is now official support for versioning in the form of 'modules'. This project should be updated to support the new system. The most awkward problem, of course, is that there are currently no version numbers of any kind in use with this, so that'll need to be done from now on. Version numbers will likely follow pull requests, such that, at least until version 1.0.0, all the patch will be incremented on every pull request, unless the pull request has major breaking changed, in which case the minor version will be incremented. The versioning scheme after v1.0.0 will be decided whenever that happens.

Calling Functions that Call Imported Functions in an Object Fails

Reproduction

let io => import 'io';
let ex => (
  let print val => io.writeln io.stdout val;
);
ex.print 3;

Expected

3

Got

Error: runtime error: invalid memory address or nil pointer dereference

all: better tests

The scanner has some tests, as well as a framework for adding more, but the other packages have extremely minimal testing. More tests must be added.

• Parser
• pgen
• WDTE
• std

Define Equality

Equality is currently undefined for a good number of the built-in function types. Most of them just panic. That's probably not good.

Alternative

Remove equality from function types. Introduce some type of boolean system instead. Have switches pass the condition to each case's left-hand side and accept it if it's 'true'. Add comparison functions to the standard library.

The Fibonacci definition would then look something like this:

fib n => switch n {
  == 0 => 0;
  == 1 => 1;
  default => + (fib (- n 1)) (fib (- n 2));
};

Referencing Function Arguments in an Expression Can Cause a Stack Overflow

Current System

Currently, function arguments are tracked by enclosing them in a special Arg type. This type references the number that they are in the current frame, which is passed around through function calls along with arguments to those specific functions. However, this breaks as soon as one function calls another one.

The problem comes from the fact that arguments are not evaluated until the Go code forces them to be. What this causes is that when an argument is passed to another function as part of an expression, the argument will be handed that functions call frame. When the argument is evaluated, it may find itself in the frame and call itself repeatedly, eventually cause a panic due to a stack overflow.

Potential Fixes

• Enclose all arguments to a DeclFunc in a new type that will track the calling frame. This would allow arguments to be evaluated in the context that they came from.

Language Overhaul

Replaces #28.

Under the current system, a module is broken into a list of imports and function declarations. This overhaul consists of several steps:

• Introduce let expressions. A let would look something like let x => 3. lets can only be used inside compounds. When used, it introduces the variable on the left into the scope for all subsequent statements, shadowing any existing variables with that name. It can also be used as an expression, simply returning the right-hand side when evaluated.
• Introduce an import keyword that takes a string as an argument. Alternatively, make it just a special built-in function. It simply imports and returns the requested module. The recent modules-as-Funcs overhaul should make this relatively straightforward.
• Allow compounds to take arguments. When a compound is evaluated, after each expression's evaluation the arguments given are passed to the return of that expression. On second thought, this'll probably break things. It would be neat, but isn't necessary. Maybe there can be a new syntax that works like this later.
• Replace modules with compounds. This means that expressions would become top-level syntactic items.
• Remove function declarations and replace them with lambdas. In other words, a function would be defined using let add => (@ self x y => + x y). Syntax changed. See comments.

This would clean up a large number of things, but it introduces one particular problem. Specifically, it breaks imports, as function declarations can't be referenced by name. There are some workarounds, however:

• Add a special method to compounds that iterates over a compound, collecting only the let expressions and returning them as a Scope.
• Collect top-level lets during translation, returning a special type that allows access to both those on their own and allows evaluation of the whole module as a compound.
• Use the JavaScript method and add an export statement that works like a let but doesn't introduce it into the scope, instead adding it to a top-level scope at translation-time. This top-level scope would then be available both to every scope in the module and would be accessible from outside. exports would only be available in top-level code. I'm leaning towards this one currently.

Scope Boundaries

Add the ability to mark boundaries in a Scope hierarchy. This would allow, for example, marking off the beginning and end of the scopes in the hierarchy that define the current function's arguments. To do this, add the following methods:

• Scope.UpperBound() *Scope: Marks the upper bound of a scope by inserting a special, marked scope into the hierarchy.
• Scope.LowerBound(name string) *Scope: Marks the lower bound in the same way, naming it.
• Scope.Latest(name string) []ID: Returns the IDs of all variables defined in the scope in between the latest upper and lower bound with the given name.

Once these are done, lambda calls should be changed to insert boundaries named args around the argument definitions. The self-reference that Lambda's are given should probably be a separate boundary.

This may, hopefully, make it possible to change Memo's implementation to not rely on being passed arguments directly.

Switches can Return `nil`

Switches currently return nil if none of the cases match. That shouldn't happen. They should probably return the test expression if that happens.

wdte: add support for variadic arguments

GoFuncs can already deal with their arguments however they want to, but functions declared in WDTE have no support for variadic arguments.

The plan, therefore, is to add support for a var, or maybe variadic, funcmod that indicates that the it takes any number of arguments. Any argument after the second to last is passed in place of the last argument as an array. In other words,

let a => import 'arrays';
let s => import 'stream';

let variadic example a b c => a.stream c -> s.reduce 0 (@ s p n => + a b p n);

example 1 2 3 4 5 -> io.writeln io.stdout;

would print 46.

Add REPL Package

Now that top-level expressions are allowed, a REPL implementation shouldn't be too difficult.

A repl package should provide a way to feed pieces of a script to a state machine that tracks the top-level scope automatically, starting with some given custom scope specified by the client.

Memoization is Broken for Recursive Functions

For some reason, #39 broke memoization of recursive functions. I've disabled the TestBasics/Fib/Memo test for now, but this needs fixing.

Add io Module

The plan is to provide simple reading and writing to and from io.Readers and io.Writers, as well as wrappers around os.Stdout, os.Stderr, and os.Stdin that could be used with that functionality.

For example:

'io' => io;

main => (
  # Write a string to stdout.
  io.stdout -> io.writeln 'This is an example.';

  # Open a file for reading, read the contents into a string, and print the string.
  io.stdout -> io.write (io.open 'file.txt' -> io.string);
);

Other possible functions:

• bytes: Read all of a reader into an array of bytes. WDTE, by default, doesn't have a concept of uint8s, or even integers at all, but there could be a special implementation of Func for bytes defined in the package. Alternatively, there could be a Func wrapper around []byte. It may make more sense to just not have this and only use strings. Or return a stream, rather than an array. Just using strings for now.
• create and append: Basically just wrappers around os.Create() and os.Append().
• combine: Takes either readers or writers and returns a new one of whichever it was that either reads from each reader until EOF in turn or writes to every writer in turn on each use.
• stringLines and bytesLines: Returns a stream that yields each line. Just using strings for now. lines is implemented.
• readString: Return a reader wrapper around a string.

std/stream: new module for dealing with streams of elements

Streams should be able to do all of the normal stream/iterator type things, not just map.

• filter. This will probably require booleans.
• concat.
• flatMap.
• anyMatch.
• allMatch.
• findFirst. (This might not be necessary, as it's kind of similar to anyMatch.)
• reduce.
• fold. Same thing as reduce but uses the first value of the stream as the initial value instead of getting one from the user. In other words, it's the same thing as -> s.reduce <next value of stream> r.
• sort. Not really necessary, thanks to arrays.sort. It would have to do a collect first anyways, so just do -> s.collect -> a.sort -> a.stream to do the same thing explicitly.
• unique. (This will require #4.)
• repeat. Repeats the entire stream infinitely. This will obviously require some buffering.
• limit. Stops the stream after the specified number of items. Might stop earlier if there are less items available. Combined with repeat could be used for, for example, repeating a string.
• join. Similar to collect, but creates a string instead of an array and separates things with a delimiter. Might want to remove strings.join since you could do the same thing with a.stream array -> s.join delimiter. Not exactly sure how it should work if the elements of the stream aren't strings, but it's probably fine to just assume they are and require the user to use a map to turn them into strings manually first. Of course, this, and collect for that matter, can also be done with reduce fairly easily.
• zip. Takes two or more streams and returns a new stream that returns arrays of each element. If one stream ends before the others then end will get inserted into the array where the next element would have been. In other words, zip s1 s2 will yield [<next element from s1>; <next element from s2>], and so on.
• inspect. Similar to map but doesn't change the value being yielded. It's kind of like sticking an ignored chain element into the middle of the stream, since you can't actually just ignore the map. That doesn't work.
• skip. Returns a stream that skips the specified number of elements from the underlying stream.
• step. Returns a stream that yields every n^th element from the underlying stream.
• reverse. Returns a stream that starts at the end of the given stream. Only works on streams that are reversible, which will also have to be implemented. Things like streams of array elements and some string-based streams should probably work with this.
• min and max. Takes a number and a comparison function and returns that many of either the minimum or maximum values yielded by the stream as an array, sorted in ascending or descending order, respectively.
• chain. Calls the elements of the stream as if they're a chain.
• new. Takes a function that produces stream values. Combining this with flatMap should make it possible to make middle streams, although map will probably be easier to use most of the time. Edit: This is significantly less useful than I expected due to the obvious fact that everything is immutable in this language. Right. Duh. It could still be useful for creating a stream over incoming data, however, but something like io.lines is more likely to be useful for that.
• end. A special value that indicates that a stream created with new is done. It's essentially the same as a Go implementation of Next() returning nil, false.

forEach is not necessary, as it's basically the same as a void map. Added drain to make it possible to use map as a foreach without doing the allocation that collect does.

wdte: error handling support

Currently, errors, especially ones that occur in a GoFunc, are unhandled. Panics will cause crashes, for example. This should not happen.

Plan

• Introduce a new Error Func type that also implements error. This type will be returned by anything that runs into an error.
• Put a wrapper in GoFunc.Call() that will recover from panics and convert them into Errors.
• Fix any potential panics in other built-in Func implementations. (There could still be other panics somewhere.)
• Mention Error in the language overview.
• Add Error support to the standard library.
• Add frames to errors. (Needs #7.)

Alternative

Panic errors all the way up. Only recover them in functions with a nil frame. This means that error handling doesn't need to be done manually, but it also makes it slightly more awkward to deal with an error if the client does want to actually handle it manually. It also might not work correctly, as the top function may be a custom type, thus bypassing the system.

Arguments Passed to Variables Are Ignored

For example:

let add => +;
add 3 7 -- io.writeln io.stdout; # Doesn't work. Just prints the address of `add`.

Implement Arbitrary Expression Evaluation

Currently, the parser is only capable of dealing with full scripts. This makes it impossible to implement, for example, a Quake-style terminal.

Allow Passing Arugments to Compounds

This sounds familiar, huh. It's a bit different than what was planned in #53, though.

Essentially, compounds completely ignore their arguments. This means that parenthesized expressions can sometimes do different things than it looks like they should. For example:

let example x y => + x y;
(example 3) 5 -- io.writeln io.stdout;

This looks like it should call example 3, and then call that return value with 5 as an argument, but instead it simply ignores the 5 and passes (example 3) to the second part of the chain.

The simplest solution is to just call the final value in a compound with any arguments that were passed to the compound. This could potentially cause problems with Go code, but any purely functional code should be fine with this. An alternative would be to only do this if arguments were actually given, preventing it from just being called twice in a lot of cases.

On a side not, lambdas may have a similar problem.

Subscope Scopes in Subscoped Expressions

There's an entry for the most confusing title contest...

let io => import 'io';
let print val => io.(writeln stdout val); # Doesn't work.
let print val => io.(writeln stdout) val; # Works.

This is because the inside of the compound doesn't have access to the surrounding scope. If the scope that it's executed in is subscoped to the surrounding scope using Scope.Sub(), it should work though, so... That should probably be done.

Allow Partial Expression Parsing in REPL

In most REPL implementations, such as Python's, entering a partial top-level statement results in a special prompt that allows you to continue the statement on the next line. For example, entering print( and pressing enter in the Python REPL, prints a new line with a ... prompt replacing the standard >>>. This will continue until the statement is actually finished with a ), at which point the EPL part of REPL will be run.

With the current implementation of the repl package, this is not possible. The current implementation reads from an input using bufio.Scanner, scanning line-by-line. When a line is entered, it immediately evaluates the line and prints the result. If the line only contained a partial expression, this will result in an error.

Unfortunately, solutions might not be so simple. There are two main possible solutions:

• Manually use scanner to determine depth into an expression, tracking parentheses and other matched operators, until a semicolon is detected at the top-level. Then evaluate.
• Do simple, rune-by-rune parsing to essentially do the same as the above.

The first solution is likely quite a bit more robust, so that's probably the better one.

The other question is how to let the client of the repl package know that this situation has happened to allow them to prompt cleanly:

• Return a special error value as the second return from Next() that can be checked for to see if it just wants more input.
• Add a third, boolean return value to Next() in between the other two that lets the user know that it needs more input.
• Make REPL act more like a scanner with the Next() method returning only bool and updating an internal state, and then providing a Ret() wdte.Func method and an Err() error method.

Personally, I'm leaning towards the first one.

Overhaul Frames

Again.

Frames have a number of major issues right now, but the biggest problem is that they were not designed to deal with scoping. The now-badly-named args field is being used to deal with not only actual function arguments but also with chain slots and lambda arguments. It's getting a bit messy, and it's causing all sorts of bugs. In fact, it's so messy that I had to use a workaround with the chain slots and wound up not using frames to store them.

The current plan is as follows:

1. Temporarily strip frames of everything that doesn't have to do with local variable storage. Probably just make WithID() a no-op for now to avoid having to strip it out of everything else.
2. Replace args with a new Scope struct which works similarly to context.Context's value system. Essentially, it'll be a map that maps variable names to values, but it'll also have a link to a parent Scope and a Get() method that'll search up the list for a variable recursively. This'll allow for clean shadowing, among other things.
3. Overhaul the translation system and every structural function type to use the new scoping system. This'll

The biggest problem will probably be chains. It may be necessary to reverse the way a chain is built so that each element of the chain can pass a new frame to the next one.

Find a Way to Deal with Closing Files

Currently, closing a file isn't always practical, syntax-wise. For example, to read a file line by line, you'd have to do something like file.open 'file.txt' -> io.lines. The problem is that after calling io.lines, there's no way to regain the reference to the file so that you can call io.close on it.

Possible solutions:

• End-slots. Basically, a place in a frame for expressions to store data that can then be passed to a function after the expression has returned. It's kind of like defer in Go, but for expressions, rather than function bodies. It could look something like this from the script's point-of-view: (file.open 'file.txt' -> io.lines) : io.close. In file.open, the file would be placed in an end-slot. At the end of the chain, io.close would then be called, and then the output of that would be passed the file.
• The simplest solution would be let expressions. I'm trying to avoid them, but I guess I'll add them if nothing else works.

Memoization Breaks in Certain Situations

Memoization currently doesn't work in some cases.

In the case of DeclFuncs, memoization works fine but breaks if a function is passed to something else with less arguments than it expects. For example:

memo example a b c =>
    # ...
    ;

other f => f 2 3;

main => other (example 1);

It also breaks in lambdas. It breaks for the same reason as DeclFuncs, but it also breaks because when a lambda places itself into a scope before calling the inner function, it doesn't wrap itself in the same memo that the outer one is in. It doesn't wrap it in any memo right now, actually.

Both of these can be fixed by inverting the memo again so that it wraps the inner expression's Func instead of wrapping the outer declaration. This will probably require a complete reworking of the Memo struct, unfortunately, since the inner Func doesn't get passed any arguments, so it never matches anything.

doc: add tour-style tutorial

Simple idea, but potentially annoying implementation. Essentially, add a tour-style tutorial, similar to the Go Tour, probably at https://deedlefake.github.io/wdte/tour.

Add Lambdas

Lambdas, although not entirely necessary, would be very useful for a number of different things, including more complex inline chain elements and callback specification. They might be quite a bit more useful if they're also closures.

Possible syntaxes:

• <expr>-based:
  • <funcmods> id <argdecls> => <expr> (Also add a possibly no-op lambda funcmod to make this work.)
  • <funcmods> @ id <argdecls> => <expr>
• <single>-based:
  • <funcmods> @ id <argdecls> { <expr> }

I'm leaning towards the second one.

Along with this, it may be a good idea to change the <chain> definition to -> <expr>.

wdte: add debugging support

Add some kind of debugging support. This would primarily involve some way to trigger breakpoints when certain functions are called. It's probable that this only needs to be done in Expr.Call().

ast: remove necessity for 'Ω's

It should not be necessary to specify where the EOF is expected in the grammar. This was originally a workaround, but it's now causing problems with expression evaluation. In particular, the expression parser is incapable of parsing compounds at the moment.

I've kind of started working on fixing this in the dev/omega branch, but I'm not sure what exactly I've done wrong. This might require a fairly heavy refactoring of the parser code, although I hope it doesn't.

std/arrays: new module for dealing with arrays

There should be an array module in the standard library for dealing with arrays in various ways. Specifically, it should have support for at least the following:

• Accessing a specific index. Dropped in favor of at in std. See #72.
• Getting the length of an array. Dropped in favor of len in std. See #72.
• Sorting.
• Run an array as if it was a compound.

Better Printing of Values in the CLI REPL

Currently, the REPL just prints complex value for structs and passes everything else to fmt.Println(). It should do custom printing of various kinds in a way that looks nice.

On top of this, built-in types should be handled in a custom way. For example, a Scope should print Scope.Known(), allowing you to easily see the list of functions provided by a module.

wdte: arrays don't keep track of frames and don't evaluate elements down to values

If an array is returned by a function and the elements of the array reference the arguments of that function, then the array is unusable. For example,

'stream' => s;
'io' => io;
example a => [a];
main => s.new (example 3) -> s.map (io.writeln io.stdout) -> s.drain;

will panic.

Similarly, an array containing unevaluated elements may not work as intended. For example,

'io' => io;
example a => [a];
main => io.write io.stdout (example 3);

will print [<address of a>], not [3].

Allow Referential Shadowing in Let Expressions

Currently, let expressions are translated into an ID and lambda pair. The lambda is given the same self-referential ID as the ID it's being bound to in the scope. This is fine for lambdas, but when using a no-argument let as a constant variable, this can cause problems when trying to both shadow and refer to variables of the same name. For example,

let x => 3;
let x => + x 1;

If x is accessed after the second line, it will go into infinite recursion as the x in the inner scope will refer to the function defined in the second line, not the first one.

Solutions:

• If a let has no arguments, don't use a lambda for the inside, meaning that the right-hand side of a let would have the exact same scope as the scope it was created in.
• Use a different ID for the self-reference inside of the lambda.

I'm leaning towards the first one, since self-reference inside a function with no arguments will always result in infinite recursion anyways.

std/io/file: new module for dealing with files

Move the functions for dealing with files out of the io package and into a new io/file package. The package should primarily have a set of functions for opening files, but can also include functions for, for example, reading directory entry names.

Similarly, add functions for seeking and other file-style stream functionality to the io package.

Better Documentation for Standard Library Functions

Currently, each standard library function attempts to explain how it works. This is sometimes complicated, especially since there's no way to cleanly see the what the function signature will be in actual WDTE code. Each function needs to have the function signature listed.

Possible formats:

• For std.Add(): + ...
• For std.Sub(): - x y OR (- y) x
• For io.Writeln: writeln w str OR writeln str w

Similarly, currently each package exports a S() function that yields a scope containing the functions in the package. It might make more sense to just make S a global variable containing the scope, as it would then be possible to see the function mapping in Godoc viewers. This could be a bit of a problem later, though, as some planned modules, such as exec, will likely use Custom(), rather than Map(). Might work anyways, though, depending on how it's done.

Context Support

• Add context to frames. This includes a ctx field and WithContext() and Context() methods.
• Make built-in functionality return errors when the context of the frame that their called with is canceled.

This makes it possible to put time-limits on scripts, as well as a few other, slightly fancier things. Functionality could be added, for example, for putting time-limits on any function call.

Make Modules Interfaces

Replace the current Module struct with a Module interface with one method:

type Module interface {
  Get(id ID) Func
}

This makes some things much more powerful, as it will be possible to dynamically look up methods, rather than just using a map. A default implementation, such as SimpleModule, can be provided, which basically just implements the method for a map[ID]Func. The standard module loading can be moved to that.

It might make sense to add several extension interfaces, too, such as the following:

type ModuleDefiner interface {
  // Define associates an id with a function in the module.
  Define(id ID, f Func)
}

type ModuleLister interface {
  // Funcs returns a list of all functions known by the module.
  Funcs() []ID
}

The default module implementation should implement all of these.

One possible usage of this would be an exec module that looks up requested functions by finding binaries in the path, probably using exec.LookPath. For example, running exec.echo 'This is an example.' would call the echo binary with 'This is an example.' as the first argument.

Let Expressions are Evaluating During Collection in Some Situations

let io => import 'io';
let example => io.(3 -> writeln stdout);

The right-hand side of argumentless let expressions are being evaluated during collection. This shouldn't happen.

Playground Layout is Completely Broken

Somehow, the playground's layout has become completely broken. This should be fixed.

std/strings: new module for dealing with strings

Contains functions for dealing with strings. For example:

• prefix and suffix. Checks for the existence of prefixes and suffixes in a string.
• contains.
• matches. Simple regular expression checker. Might want to put this in another module. Maybe string/re.
• len.
• upper and lower. Maybe title, too.
• repeat.
• split and join.
• replace. RE support might be useful, too.
• index.
• sub. slice?

Rework Frames

Frames are currently just the argument list to the function that started the frame. They should probably have a couple of extra features.

• Change frames into a custom struct type. This struct should contain the name of the function that generated the frame. The struct should be opaque, with methods for creating new frames from the old one. For example, there should be a Frame.WithID() which takes an ID and returns a new frame with that ID.
• Change errors to track frame information. Replace explicit construction of Errors with a function so that passing frame information will be necessary to create them.
• Add backtrace info to frames by having them track which frame they came from.
• Eventually, add position information to frames by having each type of expression keep track of where it came from in the code and add that to the frame during calls.

Allow Chaining Multiple Subexpressions

There are several steps to this:

• Add a Call method to Module, allowing it to be used as a Func.
• Move imports into the Funcs map and remove the Imports map. It may also make sense to just switch Module to be a map[wdte.ID]wdte.Func, rather than a struct.
• Allow arbitrary expressions on the left side of a ., allowing for expressions along the lines of (some expression that returns a module).function 'argument'. Probably should just allow <single>s. Unfortunately, this might cause problems in an LL(1) grammar. I'll have to experiment a bit.
• Allow an arbitrary number of . in a row. In other words, things like a.b.c 'argument'. This might be taken care of automatically if the previous one is done correctly.

Remove `default` Keyword?

The default keyword is technically unnecessary. Because of the way switchs work, using true as the left-hand side of a case will cause it to always trigger. Removing it will both simplify switch implementation and allow the word to be used as an ID.

std/child: new package for creating and controlling child processes

This involves two new modules, probably called child and child/exec, or something.

• child will contain functions such as spawn for creating child processes and stdout, stdin, and stderr for getting the various streams of the functions.
• child/exec will return a custom scope that spawns commands using the name of the function accessed. In other words, calling exec.ls '-a' will run ls -a.

This module will likely not be too useful until some type of concurrency is available.

Update Playground WebAssembly to Go 1.11

Waiting on gopherjs/gopherjs#705.

Genericize Scope Collection

• Add a Collector interface that has a method like Scope.Collect().
• Add a Collect() method to Chain?
• Add a Collect() method to ScopedFunc.
• Rename objectify to collect.
• Use Collector in collect.

This should fix #88.

Add Command-Line Interface

Probably at least partially dependant on #59.

A new command, probably under cmd/wdte, should give the ability to parse and execute scripts from both files and standard input. It should provide a custom importer that is able to load .wdte files from relative, and possibly absolute as well, paths. It would also be nice if it's able to load Go modules that have been compiled as plugins.

It should also provide an extra module or two for getting access to things that make sense on the command-line, such as arguments.

Most likely the command-line interface will be executed under std.F().

Return Expressions

There should be some way to return from a function early instead of having to run all the way to the end. It is probably simplest to do this by introducing a new expression type, denoted by return <expr>, which generates a new type, probably Return, and wraps the expression in that. That can then be returned up the stack. When called, the return expression simply calls the expression it wraps, but a compound, while looping, checks each return to detect instances of Return. If an expression in the compound returns a Return, then the inner value of the return gets returned from the compound immediately.

The following would then work:

example n => (
  print 'This is an example.';
  switch n {
    == 3 => return n;
  };
  print 'This does not get printed if n is 3.';
  n;
);

If a return manages to get up to a DeclFunc, then it should be unwrapped to prevent functions that called that one from returning unexpectedly.

TODO

• Figure out how this should affect chains. It seems likely that instances of Return should probably just be ignored and treated normally.
• Should compounds unwrap the Return? If so, then they'll cause a return from the current function. If not, then they'll only cause a return from the compound.