darrencruse / sugarlisp-core Goto Github PK

Although the idea of dialects acting as "mixins" that layer on top of other dialects makes for easy extensibility, there's an inherent fragility if dialects you depend on change underneath you.

This has proved especially true as I've been experimenting with different syntaxes and changes during initial development.

My fear is this may always be a challenge though. When typical npm modules you depend on change you may hit a bug that your tests can catch, but when a sugarlisp dialect you "#use" changes it can introduce ambiguities or conflicts that break your compile, or even worse just change your generated code such that your compile doesn't break, but your run time behavior changes and you don't even know why.

Luckily it seems like there's an easy fix(?!) Which is to promote the use of npm shrinkwrap to lock down the versions of the dialects you've developed and tested with so that they won't/can't change until/unless you explicitly retest with the newer versions of the dependent dialects.

i.e. precisely what npm shrinkwrap was meant for!

This issue is just a reminder to document both the problem and how to use npm shrinkwrap to deal with it.

In addition to the "atoms" retaining line/col info for source maps, there is also source for passing comments in the source out to the generated code.

Unfortunately I had trouble getting that to work cleanly though and disabled it - right now source comments do not transfer to the generated javascript code.

It seems like function docstrings would be a wonderful compromise though. i.e. being able to describe what functions do and what the inputs and outputs are in a way that can be extracted seems the most important kind of comment and s-expressions make that much wonderfully easy (i.e. if these are a standard option for function declaration s-expression forms they're much easier to deal with than arbitrary comments anywhere/everywhere which is what I'd been trying to support).

https://en.wikipedia.org/wiki/Docstring

This is the method chaining macro from LispyScript 1 as described here:
http://lispyscript.com/docs/#statements

It should still work but appears to be broken right now.

A quick test is to use the node server example from the link above:

(->
  (require "http")
  (.createServer
    (function (request response)
      (response.writeHead 200 {'Content-Type': 'text/plain'})
      (response.end "Hello World\n")))
  (.listen 3000 "localhost"))

But currently this transpiles to this(!)

(3000. listen(((function(request, response) {
  response.writeHead(200, {
    'Content-Type': 'text/plain'
  });
  return response.end("Hello World\n");
}.createServer)(require("http")))))

This is also breaking the promise macro in sugarlisp-async btw.

Esp. when using the #transpile "core" setting in atom preview to see e.g. sugarscript converted to the sugarlisp core s-expressions, sometimes the end parens are butted up against the left margin etc.

Immutable data types might be esp. of help for the html dialect for data binding in the spirit of React/Om where you can tell if something's changed very easily.

see e.g. http://jlongster.com/Using-Immutable-Data-Structures-in-JavaScript

Not that SugarLisp isn't trying to be a clojure or an Elm etc. so I think it's good this would be optional.

But possibly there's a nice way to support them as a dialect? Something like (in SugarScript):

#use "html"
#use "immutable"

val boundval;
var input = <h2>${boundval}</h2>;
boundval = if? (new Date().getHours() > 11) "good afternoon" else "good morning";

Note the keyword "val" for an immutable value.

The goal here would be to generate the code needed for one of these immutable libraries for get/set expressions using a "val" variable, but letting the sugarlisp programmer use normal javascript-y syntax.

(I need to look closer at these libraries but James Long's post above hints that most immutable libraries have special requirements i.e. something like "immutable.set(x, 10)" not just "x = 10")

A dialect in sugarlisp is made up of two parts: the "syntax" table and the "keywords" table.

The syntax table is similar to the read table of reader macros in a classic lisp. I intentionally chose to call it "syntax table" rather than "read table" because I didn't want to create a false impression that sugarlisp works exactly the same as other lisps that have read tables.

Then the "keywords" tables are what drive code generation and hold both the functions that generate code as well as (compiled) macros. Here the name was chosen partly because it's the name LispyScript had used and partly because I couldn't think of a better name. It was meant in the sense of reserved "keywords" in traditional languages (where basically "reserved words" really are the names used for lookups in the keywords tables).

But: "keywords" in Common Lisp and Clojure refer to :keywords (which currently sugarlisp does not have - mainly because there is no direct corollary in javascript and sugarlisp is trying above all to be easy for javascript programmers).

So maybe "keywords" table was an unfortunate/confusing choice of name for those coming from a lisp background.

I'm still unsure what a better name is though... the notion of "reserved keywords" really is close to what it holds - I can't decide what other name conveys that idea as well. It is the table that controls macro expansion and code generation - maybe I should simply call it the "translation" table. Or maybe it's the "transform" table?

Right now "sugar --version" and the ascii-art banner in the REPL etc. show hard coded version strings.

This is a reminder to show the actual version (including subversion number) set in package.json

Not only is the current code not showing any "subversion" in the version number but it's violated DRY in that there are several strings duplicating the version number that have to be manually changed on a version bump.

From googling getting the version number from package.json can be as easy as:

var pjson = require('./package.json');
console.log(pjson.version); 

The dot-property access with objects, commonjs modules, etc. works fine to avoid naming conflicts at run time. i.e. these work fine:

(console.log "hello")
(fs.readFileSync "/myfile.txt")
etc.

But right now we've got a flat namespace for the names found in first position in s-expressions i.e. "( arg1 arg2)" that are handled by the transpiler at compile time.

Meaning the names for macros and what we've termed the "keyword" handler functions (which generate the output javascript).

So far this hasn't proven a problem but there could (should?) be an option to namespace a #used dialect with a prefix to avoid any possibility of naming conflicts e.g.:

#use "html" as htm;
...
(htm.tag "div" "hello")

Note: I have toyed with a "#require" i.e. a normal require done at compile-time as opposed to run-time. But the point here is to truly insert such things in the keyword table and modify the transpiler to work the deeper nesting of names.