Coder Social home page Coder Social logo

kanaka / mal Goto Github PK

View Code? Open in Web Editor NEW
9.8K 184.0 2.5K 10.11 MB

mal - Make a Lisp

License: Other

Makefile 7.77% Shell 4.20% C 15.76% Clojure 1.93% CoffeeScript 1.65% C++ 4.71% C# 7.32% Forth 4.38% Go 3.15% Haskell 3.11% Java 20.51% JavaScript 5.18% Julia 1.71% Lua 2.46% MATLAB 2.97% Nim 2.09% OCaml 2.49% Perl 2.45% PHP 2.69% PostScript 3.47%
mal docker lisp lisp-interpreter c java javascript python c-plus-plus c-sharp

mal's Introduction

mal - Make a Lisp

Build Status

Description

1. Mal is a Clojure inspired Lisp interpreter

2. Mal is a learning tool

Each implementation of mal is separated into 11 incremental, self-contained (and testable) steps that demonstrate core concepts of Lisp. The last step is capable of self-hosting (running the mal implementation of mal). See the make-a-lisp process guide.

The make-a-lisp steps are:

Each make-a-lisp step has an associated architectural diagram. That elements that are new for that step are highlighted in red. Here is the final diagram for step A:

stepA_mal architecture

If you are interested in creating a mal implementation (or just interested in using mal for something), you are welcome to to join our Discord or join #mal on libera.chat. In addition to the make-a-lisp process guide there is also a mal/make-a-lisp FAQ where I attempt to answer some common questions.

3. Mal is implemented in 87 languages (93 different implementations and 115 runtime modes)

Language Creator
Ada Chris Moore
Ada #2 Nicolas Boulenguez
GNU Awk Miutsuru Kariya
Bash 4 Joel Martin
BASIC (C64 & QBasic) Joel Martin
BBC BASIC V Ben Harris
C Joel Martin
C #2 Duncan Watts
C++ Stephen Thirlwall
C# Joel Martin
ChucK Vasilij Schneidermann
Clojure (Clojure & ClojureScript) Joel Martin
CoffeeScript Joel Martin
Common Lisp Iqbal Ansari
Crystal Linda_pp
D Dov Murik
Dart Harry Terkelsen
Elixir Martin Ek
Elm Jos van Bakel
Emacs Lisp Vasilij Schneidermann
Erlang Nathan Fiedler
ES6 (ECMAScript 2015) Joel Martin
F# Peter Stephens
Factor Jordan Lewis
Fantom Dov Murik
Fennel sogaiu
Forth Chris Houser
GNU Guile Mu Lei
GNU Smalltalk Vasilij Schneidermann
Go Joel Martin
Groovy Joel Martin
Haskell Joel Martin
Haxe (Neko, Python, C++, & JS) Joel Martin
Hy Joel Martin
Io Dov Murik
Janet sogaiu
Java Joel Martin
Java (Truffle/GraalVM) Matt McGill
JavaScript (Demo) Joel Martin
jq Ali MohammadPur
Julia Joel Martin
Kotlin Javier Fernandez-Ivern
LiveScript Jos van Bakel
Logo Dov Murik
Lua Joel Martin
GNU Make Joel Martin
mal itself Joel Martin
MATLAB (GNU Octave & MATLAB) Joel Martin
miniMAL (Repo, Demo) Joel Martin
NASM Ben Dudson
Nim Dennis Felsing
Object Pascal Joel Martin
Objective C Joel Martin
OCaml Chris Houser
Perl Joel Martin
Perl 6 Hinrik Örn Sigurðsson
PHP Joel Martin
Picolisp Vasilij Schneidermann
Pike Dov Murik
PL/pgSQL (PostgreSQL) Joel Martin
PL/SQL (Oracle) Joel Martin
PostScript Joel Martin
PowerShell Joel Martin
Prolog Nicolas Boulenguez
PureScript mrsekut
Python (2.X & 3.X) Joel Martin
Python #2 (3.X) Gavin Lewis
RPython Joel Martin
R Joel Martin
Racket Joel Martin
Rexx Dov Murik
Ruby Joel Martin
Ruby #2 Ryan Cook
Rust Joel Martin
Scala Joel Martin
Scheme (R7RS) Vasilij Schneidermann
Skew Dov Murik
Standard ML Fabian Bergström
Swift 2 Keith Rollin
Swift 3 Joel Martin
Swift 4 陆遥
Swift 5 Oleg Montak
Tcl Dov Murik
TypeScript Masahiro Wakame
Vala Simon Tatham
VHDL Dov Murik
Vimscript Dov Murik
Visual Basic.NET Joel Martin
WebAssembly (wasm) Joel Martin
Wren Dov Murik
XSLT Ali MohammadPur
Yorick Dov Murik
Zig Josh Tobin

Presentations

Mal was presented publicly for the first time in a lightning talk at Clojure West 2014 (unfortunately there is no video). See examples/clojurewest2014.mal for the presentation that was given at the conference (yes, the presentation is a mal program).

At Midwest.io 2015, Joel Martin gave a presentation on Mal titled "Achievement Unlocked: A Better Path to Language Learning". Video, Slides.

More recently Joel gave a presentation on "Make Your Own Lisp Interpreter in 10 Incremental Steps" at LambdaConf 2016: Part 1, Part 2, Part 3, Part 4, Slides.

Building/running implementations

The simplest way to run any given implementation is to use docker. Every implementation has a docker image pre-built with language dependencies installed. You can launch the REPL using a convenient target in the top level Makefile (where IMPL is the implementation directory name and stepX is the step to run):

make DOCKERIZE=1 "repl^IMPL^stepX"
    # OR stepA is the default step:
make DOCKERIZE=1 "repl^IMPL"

External Implementations

The following implementations are maintained as separate projects:

HolyC

Rust

  • by Tim Morgan
  • by vi - using Pest grammar, not using typical Mal infrastructure (cargo-ized steps and built-in converted tests).

Q

  • by Ali Mohammad Pur - The Q implementation works fine but it requires a proprietary manual download that can't be Dockerized (or integrated into the mal CI pipeline) so for now it remains a separate project.

Other mal Projects

  • malc - Mal (Make A Lisp) compiler. Compiles a Mal program to LLVM assembly language, then binary.
  • malcc - malcc is an incremental compiler implementation for the Mal language. It uses the Tiny C Compiler as the compiler backend and has full support for the Mal language, including macros, tail-call elimination, and even run-time eval. "I Built a Lisp Compiler" post about the process.
  • frock - Clojure-flavoured PHP. Uses mal/php to run programs.
  • flk - A LISP that runs wherever Bash is
  • glisp - Self-bootstrapping graphic design tool on Lisp. Live Demo

Implementation Details

Ada

The Ada implementation was developed with GNAT 4.9 on debian. It also compiles unchanged on windows if you have windows versions of git, GNAT and (optionally) make. There are no external dependencies (readline not implemented).

cd impls/ada
make
./stepX_YYY

Ada.2

The second Ada implementation was developed with GNAT 8 and links with the GNU readline library.

cd impls/ada
make
./stepX_YYY

GNU awk

The GNU awk implementation of mal has been tested with GNU awk 4.1.1.

cd impls/gawk
gawk -O -f stepX_YYY.awk

Bash 4

cd impls/bash
bash stepX_YYY.sh

BASIC (C64 and QBasic)

The BASIC implementation uses a preprocessor that can generate BASIC code that is compatible with both C64 BASIC (CBM v2) and QBasic. The C64 mode has been tested with cbmbasic (the patched version is currently required to fix issues with line input) and the QBasic mode has been tested with qb64.

Generate C64 code and run it using cbmbasic:

cd impls/basic
make stepX_YYY.bas
STEP=stepX_YYY ./run

Generate QBasic code and load it into qb64:

cd impls/basic
make MODE=qbasic stepX_YYY.bas
./qb64 stepX_YYY.bas

Thanks to Steven Syrek for the original inspiration for this implementation.

BBC BASIC V

The BBC BASIC V implementation can run in the Brandy interpreter:

cd impls/bbc-basic
brandy -quit stepX_YYY.bbc

Or in ARM BBC BASIC V under RISC OS 3 or later:

*Dir bbc-basic.riscos
*Run setup
*Run stepX_YYY

C

The C implementation of mal requires the following libraries (lib and header packages): glib, libffi6, libgc, and either the libedit or GNU readline library.

cd impls/c
make
./stepX_YYY

C.2

The second C implementation of mal requires the following libraries (lib and header packages): libedit, libgc, libdl, and libffi.

cd impls/c.2
make
./stepX_YYY

C++

The C++ implementation of mal requires g++-4.9 or clang++-3.5 and a readline compatible library to build. See the cpp/README.md for more details:

cd impls/cpp
make
    # OR
make CXX=clang++-3.5
./stepX_YYY

C#

The C# implementation of mal has been tested on Linux using the Mono C# compiler (mcs) and the Mono runtime (version 2.10.8.1). Both are required to build and run the C# implementation.

cd impls/cs
make
mono ./stepX_YYY.exe

ChucK

The ChucK implementation has been tested with ChucK 1.3.5.2.

cd impls/chuck
./run

Clojure

For the most part the Clojure implementation requires Clojure 1.5, however, to pass all tests, Clojure 1.8.0-RC4 is required.

cd impls/clojure
lein with-profile +stepX trampoline run

CoffeeScript

sudo npm install -g coffee-script
cd impls/coffee
coffee ./stepX_YYY

Common Lisp

The implementation has been tested with SBCL, CCL, CMUCL, GNU CLISP, ECL and Allegro CL on Ubuntu 16.04 and Ubuntu 12.04, see the README for more details. Provided you have the dependencies mentioned installed, do the following to run the implementation

cd impls/common-lisp
make
./run

Crystal

The Crystal implementation of mal has been tested with Crystal 0.26.1.

cd impls/crystal
crystal run ./stepX_YYY.cr
    # OR
make   # needed to run tests
./stepX_YYY

D

The D implementation of mal was tested with GDC 4.8. It requires the GNU readline library.

cd impls/d
make
./stepX_YYY

Dart

The Dart implementation has been tested with Dart 1.20.

cd impls/dart
dart ./stepX_YYY

Emacs Lisp

The Emacs Lisp implementation of mal has been tested with Emacs 24.3 and 24.5. While there is very basic readline editing (<backspace> and C-d work, C-c cancels the process), it is recommended to use rlwrap.

cd impls/elisp
emacs -Q --batch --load stepX_YYY.el
# with full readline support
rlwrap emacs -Q --batch --load stepX_YYY.el

Elixir

The Elixir implementation of mal has been tested with Elixir 1.0.5.

cd impls/elixir
mix stepX_YYY
# Or with readline/line editing functionality:
iex -S mix stepX_YYY

Elm

The Elm implementation of mal has been tested with Elm 0.18.0

cd impls/elm
make stepX_YYY.js
STEP=stepX_YYY ./run

Erlang

The Erlang implementation of mal requires Erlang/OTP R17 and rebar to build.

cd impls/erlang
make
    # OR
MAL_STEP=stepX_YYY rebar compile escriptize # build individual step
./stepX_YYY

ES6 (ECMAScript 2015)

The ES6 / ECMAScript 2015 implementation uses the babel compiler to generate ES5 compatible JavaScript. The generated code has been tested with Node 0.12.4.

cd impls/es6
make
node build/stepX_YYY.js

F#

The F# implementation of mal has been tested on Linux using the Mono F# compiler (fsharpc) and the Mono runtime (version 3.12.1). The mono C# compiler (mcs) is also necessary to compile the readline dependency. All are required to build and run the F# implementation.

cd impls/fsharp
make
mono ./stepX_YYY.exe

Factor

The Factor implementation of mal has been tested with Factor 0.97 (factorcode.org).

cd impls/factor
FACTOR_ROOTS=. factor -run=stepX_YYY

Fantom

The Fantom implementation of mal has been tested with Fantom 1.0.70.

cd impls/fantom
make lib/fan/stepX_YYY.pod
STEP=stepX_YYY ./run

Fennel

The Fennel implementation of mal has been tested with Fennel version 0.9.1 on Lua 5.4.

cd impls/fennel
fennel ./stepX_YYY.fnl

Forth

cd impls/forth
gforth stepX_YYY.fs

GNU Guile 2.1+

cd impls/guile
guile -L ./ stepX_YYY.scm

GNU Smalltalk

The Smalltalk implementation of mal has been tested with GNU Smalltalk 3.2.91.

cd impls/gnu-smalltalk
./run

Go

The Go implementation of mal requires that go is installed on on the path. The implementation has been tested with Go 1.3.1.

cd impls/go
make
./stepX_YYY

Groovy

The Groovy implementation of mal requires Groovy to run and has been tested with Groovy 1.8.6.

cd impls/groovy
make
groovy ./stepX_YYY.groovy

Haskell

The Haskell implementation requires the ghc compiler version 7.10.1 or later and also the Haskell parsec and readline (or editline) packages.

cd impls/haskell
make
./stepX_YYY

Haxe (Neko, Python, C++ and JavaScript)

The Haxe implementation of mal requires Haxe version 3.2 to compile. Four different Haxe targets are supported: Neko, Python, C++, and JavaScript.

cd impls/haxe
# Neko
make all-neko
neko ./stepX_YYY.n
# Python
make all-python
python3 ./stepX_YYY.py
# C++
make all-cpp
./cpp/stepX_YYY
# JavaScript
make all-js
node ./stepX_YYY.js

Hy

The Hy implementation of mal has been tested with Hy 0.13.0.

cd impls/hy
./stepX_YYY.hy

Io

The Io implementation of mal has been tested with Io version 20110905.

cd impls/io
io ./stepX_YYY.io

Janet

The Janet implementation of mal has been tested with Janet version 1.12.2.

cd impls/janet
janet ./stepX_YYY.janet

Java 1.7

The Java implementation of mal requires maven2 to build.

cd impls/java
mvn compile
mvn -quiet exec:java -Dexec.mainClass=mal.stepX_YYY
    # OR
mvn -quiet exec:java -Dexec.mainClass=mal.stepX_YYY -Dexec.args="CMDLINE_ARGS"

Java, using Truffle for GraalVM

This Java implementation will run on OpenJDK, but can run as much as 30x faster on GraalVM thanks to the Truffle framework. It's been tested with OpenJDK 11, GraalVM CE 20.1.0, and GraalVM CE 21.1.0.

cd impls/java-truffle
./gradlew build
STEP=stepX_YYY ./run

JavaScript/Node

cd impls/js
npm install
node stepX_YYY.js

Julia

The Julia implementation of mal requires Julia 0.4.

cd impls/julia
julia stepX_YYY.jl

jq

Tested against version 1.6, with a lot of cheating in the IO department

cd impls/jq
STEP=stepA_YYY ./run
    # with Debug
DEBUG=true STEP=stepA_YYY ./run

Kotlin

The Kotlin implementation of mal has been tested with Kotlin 1.0.

cd impls/kotlin
make
java -jar stepX_YYY.jar

LiveScript

The LiveScript implementation of mal has been tested with LiveScript 1.5.

cd impls/livescript
make
node_modules/.bin/lsc stepX_YYY.ls

Logo

The Logo implementation of mal has been tested with UCBLogo 6.0.

cd impls/logo
logo stepX_YYY.lg

Lua

The Lua implementation of mal has been tested with Lua 5.3.5 The implementation requires luarocks to be installed.

cd impls/lua
make  # to build and link linenoise.so and rex_pcre.so
./stepX_YYY.lua

Mal

Running the mal implementation of mal involves running stepA of one of the other implementations and passing the mal step to run as a command line argument.

cd impls/IMPL
IMPL_STEPA_CMD ../mal/stepX_YYY.mal

GNU Make 3.81

cd impls/make
make -f stepX_YYY.mk

NASM

The NASM implementation of mal is written for x86-64 Linux, and has been tested with Linux 3.16.0-4-amd64 and NASM version 2.11.05.

cd impls/nasm
make
./stepX_YYY

Nim 1.0.4

The Nim implementation of mal has been tested with Nim 1.0.4.

cd impls/nim
make
  # OR
nimble build
./stepX_YYY

Object Pascal

The Object Pascal implementation of mal has been built and tested on Linux using the Free Pascal compiler version 2.6.2 and 2.6.4.

cd impls/objpascal
make
./stepX_YYY

Objective C

The Objective C implementation of mal has been built and tested on Linux using clang/LLVM 3.6. It has also been built and tested on OS X using XCode 7.

cd impls/objc
make
./stepX_YYY

OCaml 4.01.0

cd impls/ocaml
make
./stepX_YYY

MATLAB (GNU Octave and MATLAB)

The MatLab implementation has been tested with GNU Octave 4.2.1. It has also been tested with MATLAB version R2014a on Linux. Note that MATLAB is a commercial product.

cd impls/matlab
./stepX_YYY
octave -q --no-gui --no-history --eval "stepX_YYY();quit;"
matlab -nodisplay -nosplash -nodesktop -nojvm -r "stepX_YYY();quit;"
    # OR with command line arguments
octave -q --no-gui --no-history --eval "stepX_YYY('arg1','arg2');quit;"
matlab -nodisplay -nosplash -nodesktop -nojvm -r "stepX_YYY('arg1','arg2');quit;"

miniMAL

miniMAL is small Lisp interpreter implemented in less than 1024 bytes of JavaScript. To run the miniMAL implementation of mal you need to download/install the miniMAL interpreter (which requires Node.js).

cd impls/miniMAL
# Download miniMAL and dependencies
npm install
export PATH=`pwd`/node_modules/minimal-lisp/:$PATH
# Now run mal implementation in miniMAL
miniMAL ./stepX_YYY

Perl 5

The Perl 5 implementation should work with perl 5.19.3 and later.

For readline line editing support, install Term::ReadLine::Perl or Term::ReadLine::Gnu from CPAN.

cd impls/perl
perl stepX_YYY.pl

Perl 6

The Perl 6 implementation was tested on Rakudo Perl 6 2016.04.

cd impls/perl6
perl6 stepX_YYY.pl

PHP 5.3

The PHP implementation of mal requires the php command line interface to run.

cd impls/php
php stepX_YYY.php

Picolisp

The Picolisp implementation requires libreadline and Picolisp 3.1.11 or later.

cd impls/picolisp
./run

Pike

The Pike implementation was tested on Pike 8.0.

cd impls/pike
pike stepX_YYY.pike

PL/pgSQL (PostgreSQL SQL Procedural Language)

The PL/pgSQL implementation of mal requires a running PostgreSQL server (the "kanaka/mal-test-plpgsql" docker image automatically starts a PostgreSQL server). The implementation connects to the PostgreSQL server and create a database named "mal" to store tables and stored procedures. The wrapper script uses the psql command to connect to the server and defaults to the user "postgres" but this can be overridden with the PSQL_USER environment variable. A password can be specified using the PGPASSWORD environment variable. The implementation has been tested with PostgreSQL 9.4.

cd impls/plpgsql
./wrap.sh stepX_YYY.sql
    # OR
PSQL_USER=myuser PGPASSWORD=mypass ./wrap.sh stepX_YYY.sql

PL/SQL (Oracle SQL Procedural Language)

The PL/SQL implementation of mal requires a running Oracle DB server (the "kanaka/mal-test-plsql" docker image automatically starts an Oracle Express server). The implementation connects to the Oracle server to create types, tables and stored procedures. The default SQL*Plus logon value (username/password@connect_identifier) is "system/oracle" but this can be overridden with the ORACLE_LOGON environment variable. The implementation has been tested with Oracle Express Edition 11g Release 2. Note that any SQL*Plus connection warnings (user password expiration, etc) will interfere with the ability of the wrapper script to communicate with the DB.

cd impls/plsql
./wrap.sh stepX_YYY.sql
    # OR
ORACLE_LOGON=myuser/mypass@ORCL ./wrap.sh stepX_YYY.sql

PostScript Level 2/3

The PostScript implementation of mal requires Ghostscript to run. It has been tested with Ghostscript 9.10.

cd impls/ps
gs -q -dNODISPLAY -I./ stepX_YYY.ps

PowerShell

The PowerShell implementation of mal requires the PowerShell script language. It has been tested with PowerShell 6.0.0 Alpha 9 on Linux.

cd impls/powershell
powershell ./stepX_YYY.ps1

Prolog

The Prolog implementation uses some constructs specific to SWI-Prolog, includes readline support and has been tested on Debian GNU/Linux with version 8.2.1.

cd impls/prolog
swipl stepX_YYY

PureScript

The PureScript implementation requires the spago compiler version 0.20.2.

cd impls/purs
make
node ./stepX_YYY.js

Python (2.X and 3.X)

cd impls/python
python stepX_YYY.py

Python.2 (3.X)

The second Python implementation makes heavy use of type annotations and uses the Arpeggio parser library.

# Recommended: do these steps in a Python virtual environment.
pip3 install Arpeggio==1.9.0
python3 stepX_YYY.py

RPython

You must have rpython on your path (included with pypy).

cd impls/rpython
make        # this takes a very long time
./stepX_YYY

R

The R implementation of mal requires R (r-base-core) to run.

cd impls/r
make libs  # to download and build rdyncall
Rscript stepX_YYY.r

Racket (5.3)

The Racket implementation of mal requires the Racket compiler/interpreter to run.

cd impls/racket
./stepX_YYY.rkt

Rexx

The Rexx implementation of mal has been tested with Regina Rexx 3.6.

cd impls/rexx
make
rexx -a ./stepX_YYY.rexxpp

Ruby (1.9+)

cd impls/ruby
ruby stepX_YYY.rb

Ruby #2

A second Ruby implementation with the following goals:

  • No global variables
  • No modification (monkey-patching) of core Ruby classes
  • Modularized into the Mal module namespace
cd impls/ruby.2
ruby stepX_YYY.rb

Rust (1.38+)

The rust implementation of mal requires the rust compiler and build tool (cargo) to build.

cd impls/rust
cargo run --release --bin stepX_YYY

Scala

Install scala and sbt (http://www.scala-sbt.org/0.13/tutorial/Installing-sbt-on-Linux.html):

cd impls/scala
sbt 'run-main stepX_YYY'
    # OR
sbt compile
scala -classpath target/scala*/classes stepX_YYY

Scheme (R7RS)

The Scheme implementation of MAL has been tested with Chibi-Scheme 0.10, Kawa 3.1.1, Gauche 0.9.6, CHICKEN 5.1.0, Sagittarius 0.9.7, Cyclone 0.32.0 (Git version) and Foment 0.4 (Git version). You should be able to get it running on other conforming R7RS implementations after figuring out how libraries are loaded and adjusting the Makefile and run script accordingly.

cd impls/scheme
# chibi
scheme_MODE=chibi ./run
# kawa
make kawa
scheme_MODE=kawa ./run
# gauche
scheme_MODE=gauche ./run
# chicken
make chicken
scheme_MODE=chicken ./run
# sagittarius
scheme_MODE=sagittarius ./run
# cyclone
make cyclone
scheme_MODE=cyclone ./run
# foment
scheme_MODE=foment ./run

Skew

The Skew implementation of mal has been tested with Skew 0.7.42.

cd impls/skew
make
node stepX_YYY.js

Standard ML (Poly/ML, MLton, Moscow ML)

The Standard ML implementation of mal requires an SML97 implementation. The Makefile supports Poly/ML, MLton, Moscow ML, and has been tested with Poly/ML 5.8.1, MLton 20210117, and Moscow ML version 2.10.

cd impls/sml
# Poly/ML
make sml_MODE=polyml
./stepX_YYY
# MLton
make sml_MODE=mlton
./stepX_YYY
# Moscow ML
make sml_MODE=mosml
./stepX_YYY

Swift

The Swift implementation of mal requires the Swift 2.0 compiler (XCode 7.0) to build. Older versions will not work due to changes in the language and standard library.

cd impls/swift
make
./stepX_YYY

Swift 3

The Swift 3 implementation of mal requires the Swift 3.0 compiler. It has been tested with Swift 3 Preview 3.

cd impls/swift3
make
./stepX_YYY

Swift 4

The Swift 4 implementation of mal requires the Swift 4.0 compiler. It has been tested with Swift 4.2.3 release.

cd impls/swift4
make
./stepX_YYY

Swift 5

The Swift 5 implementation of mal requires the Swift 5.0 compiler. It has been tested with Swift 5.1.1 release.

cd impls/swift5
swift run stepX_YYY

Tcl 8.6

The Tcl implementation of mal requires Tcl 8.6 to run. For readline line editing support, install tclreadline.

cd impls/tcl
tclsh ./stepX_YYY.tcl

TypeScript

The TypeScript implementation of mal requires the TypeScript 2.2 compiler. It has been tested with Node.js v6.

cd impls/ts
make
node ./stepX_YYY.js

Vala

The Vala implementation of mal has been tested with the Vala 0.40.8 compiler. You will need to install valac and libreadline-dev or equivalent.

cd impls/vala
make
./stepX_YYY

VHDL

The VHDL implementation of mal has been tested with GHDL 0.29.

cd impls/vhdl
make
./run_vhdl.sh ./stepX_YYY

Vimscript

The Vimscript implementation of mal requires Vim 8.0 to run.

cd impls/vimscript
./run_vimscript.sh ./stepX_YYY.vim

Visual Basic.NET

The VB.NET implementation of mal has been tested on Linux using the Mono VB compiler (vbnc) and the Mono runtime (version 2.10.8.1). Both are required to build and run the VB.NET implementation.

cd impls/vb
make
mono ./stepX_YYY.exe

WebAssembly (wasm)

The WebAssembly implementation is written in Wam (WebAssembly Macro language) and runs under several different non-web embeddings (runtimes): node, wasmtime, wasmer, lucet, wax, wace, warpy.

cd impls/wasm
# node
make wasm_MODE=node
./run.js ./stepX_YYY.wasm
# wasmtime
make wasm_MODE=wasmtime
wasmtime --dir=./ --dir=../ --dir=/ ./stepX_YYY.wasm
# wasmer
make wasm_MODE=wasmer
wasmer run --dir=./ --dir=../ --dir=/ ./stepX_YYY.wasm
# lucet
make wasm_MODE=lucet
lucet-wasi --dir=./:./ --dir=../:../ --dir=/:/ ./stepX_YYY.so
# wax
make wasm_MODE=wax
wax ./stepX_YYY.wasm
# wace
make wasm_MODE=wace_libc
wace ./stepX_YYY.wasm
# warpy
make wasm_MODE=warpy
warpy --argv --memory-pages 256 ./stepX_YYY.wasm

XSLT

The XSLT implementation of mal is written with XSLT 3 and tested on Saxon 9.9.1.6 Home Edition.

cd impls/xslt
STEP=stepX_YY ./run

Wren

The Wren implementation of mal was tested on Wren 0.2.0.

cd impls/wren
wren ./stepX_YYY.wren

Yorick

The Yorick implementation of mal was tested on Yorick 2.2.04.

cd impls/yorick
yorick -batch ./stepX_YYY.i

Zig

The Zig implementation of mal was tested on Zig 0.5.

cd impls/zig
zig build stepX_YYY

Running tests

The top level Makefile has a number of useful targets to assist with implementation development and testing. The help target provides a list of the targets and options:

make help

Functional tests

The are almost 800 generic functional tests (for all implementations) in the tests/ directory. Each step has a corresponding test file containing tests specific to that step. The runtest.py test harness launches a Mal step implementation and then feeds the tests one at a time to the implementation and compares the output/return value to the expected output/return value.

  • To run all the tests across all implementations (be prepared to wait):
make test
  • To run all tests against a single implementation:
make "test^IMPL"

# e.g.
make "test^clojure"
make "test^js"
  • To run tests for a single step against all implementations:
make "test^stepX"

# e.g.
make "test^step2"
make "test^step7"
  • To run tests for a specific step against a single implementation:
make "test^IMPL^stepX"

# e.g
make "test^ruby^step3"
make "test^ps^step4"

Self-hosted functional tests

  • To run the functional tests in self-hosted mode, you specify mal as the test implementation and use the MAL_IMPL make variable to change the underlying host language (default is JavaScript):
make MAL_IMPL=IMPL "test^mal^step2"

# e.g.
make "test^mal^step2"   # js is default
make MAL_IMPL=ruby "test^mal^step2"
make MAL_IMPL=python "test^mal^step2"

Starting the REPL

  • To start the REPL of an implementation in a specific step:
make "repl^IMPL^stepX"

# e.g
make "repl^ruby^step3"
make "repl^ps^step4"
  • If you omit the step, then stepA is used:
make "repl^IMPL"

# e.g
make "repl^ruby"
make "repl^ps"
  • To start the REPL of the self-hosted implementation, specify mal as the REPL implementation and use the MAL_IMPL make variable to change the underlying host language (default is JavaScript):
make MAL_IMPL=IMPL "repl^mal^stepX"

# e.g.
make "repl^mal^step2"   # js is default
make MAL_IMPL=ruby "repl^mal^step2"
make MAL_IMPL=python "repl^mal"

Performance tests

Warning: These performance tests are neither statistically valid nor comprehensive; runtime performance is a not a primary goal of mal. If you draw any serious conclusions from these performance tests, then please contact me about some amazing oceanfront property in Kansas that I'm willing to sell you for cheap.

  • To run performance tests against a single implementation:
make "perf^IMPL"

# e.g.
make "perf^js"
  • To run performance tests against all implementations:
make "perf"

Generating language statistics

  • To report line and byte statistics for a single implementation:
make "stats^IMPL"

# e.g.
make "stats^js"

Dockerized testing

Every implementation directory contains a Dockerfile to create a docker image containing all the dependencies for that implementation. In addition, the top-level Makefile contains support for running the tests target (and perf, stats, repl, etc) within a docker container for that implementation by passing "DOCKERIZE=1" on the make command line. For example:

make DOCKERIZE=1 "test^js^step3"

Existing implementations already have docker images built and pushed to the docker registry. However, if you wish to build or rebuild a docker image locally, the toplevel Makefile provides a rule for building docker images:

make "docker-build^IMPL"

Notes:

  • Docker images are named "kanaka/mal-test-IMPL"
  • JVM-based language implementations (Groovy, Java, Clojure, Scala): you will probably need to run this command once manually first make DOCKERIZE=1 "repl^IMPL" before you can run tests because runtime dependencies need to be downloaded to avoid the tests timing out. These dependencies are downloaded to dot-files in the /mal directory so they will persist between runs.

License

Mal (make-a-lisp) is licensed under the MPL 2.0 (Mozilla Public License 2.0). See LICENSE.txt for more details.

mal's People

Contributors

alimpfard avatar asarhaddon avatar bendudson avatar bjh21 avatar chouser avatar chr15m avatar cookrn avatar def- avatar dubek avatar epylar avatar fabjan avatar harryterkelsen avatar iqbalansari avatar ivern avatar jig avatar jordanlewis avatar kanaka avatar kariya-mitsuru avatar keith-rollin avatar mmcgill avatar mrsekut avatar nalaginrut avatar nlfiedler avatar pstephens avatar rhysd avatar sdt avatar sgtatham avatar vvakame avatar wasamasa avatar zmower avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mal's Issues

Useful functions

I was wondering what useful functions people have written (in mal) to supplement the base mal language. I noticed some functions and macros in core.mal with some useful functions. I like the clojure-style threading macros. Very clever and shows what a capable language mal is. I didn't see filter in the git repository so I wrote my own the other day and was going to write 'range' next. Are there any other sources of supplemental functions/macros written in the core mal language that I am missing? Could anybody share what useful functions they have written? I'd be very interested in seeing what clever things people have written, what functions they have implemented, and how they implemented them.

@kanaka Joel, if it doesn't exist already, what do you think about a place for user contributed functions?

-Kerry

Fix string escaping/unescaping in Erlang implementation

@nlfiedler I recently added some tests that do better coverage of string escaping and unescaping. In particular, this allows ANSI escape codes to be used in output to support. For example, being able to run the mal/clojurewest2014.mal presentation correctly.

I was able to fix the escaping/unescaping for all the other languages, but I'm not familiar enough with Erlang to be able to fix it easily. I think the escaping in printer.erl is okay, but the unescaping in reader.erl is missing the "\n" case. At least, that appears to be the issue. Do you think you could take a look at it?

You can reproduce the failures with:

make test^erlang^step4

Implement native hash-map compare across all implementations

This is to track the remaining implementations that need native hash-map equality.

Here is current list of implementations which still need it native hash-map:

  • forth
  • guile
  • kotlin
  • matlab
  • miniMAL
  • ocaml
  • perl
  • r
  • racket
  • vb

@dubek, since we both seem to be working through them, let's claim them when we start working on them so that we don't duplicate work. So to start off, I'll do miniMAL.

Why a mal function needs the function in step 5 (TCO)?

It says:

The return value from the fn* special form will now become an object/structure
with attributes that allow the default invoke case of EVAL to do TCO
on mal functions. Those attributes are:

fn: the original function value return in step 4
...

But I don't see where that original function is used, anywhere.

Is that a mistake or is there something I'm overlooking?

Relevance of conj for self-hosting

guide.md speaks of conj being optional and not required for self-hosting. However the Mal implementation that is run in this step does contain conj in its core.mal. I cannot find any other usage of conj in its sources, so I suspect it's either been added with the intention of making use of it (and making conj mandatory to implement for self-hosting) or in error.

Keyword equals is false

Just noticed this in the CSharp version:

> (def! kw :abc)
:abc

> kw
:abc

> (= kw :abc)
false

> (= :abc :abc)
true

Add string manipulation capability

Right now there is no cross-implementation way to manipulate (index, slice, etc).

In Clojure, strings are treated as sequences of characters and so sequence functions operate on them. However, as a simple initial (or maybe final) step, simply adding a seq function that turns a string into a list of single character strings would allow manipulation using normal sequence functions (str will already concatenate a sequence of character strings into a single string).

For now I'm thinking this would be an optional stepA function.

So here is the list for adding a seq function:

  • awk
  • bash
  • c
  • d
  • clojure
  • coffee
  • cpp
  • crystal
  • cs
  • erlang
  • elixir
  • es6
  • factor
  • forth
  • fsharp
  • go
  • groovy
  • guile
  • haskell
  • haxe
  • java
  • julia
  • js
  • kotlin
  • lua
  • make
  • mal
  • ocaml
  • matlab
  • miniMAL
  • nim
  • perl
  • php
  • ps
  • python
  • r
  • racket
  • rpython
  • ruby
  • rust
  • scala
  • swift
  • swift3
  • tcl
  • vb
  • vimscript

Also:

  • Update: process/stepA_mal.txt and process/guide.md

TCO test not general enough

step5_tco.mal

TCO means 'Tail Call Optimization'. Not 'tail recursion'. TCO means that ALL tail calls are optimized, not only calls to a recursive function.

The test only tests the simple case of a self-recursive tail call. A test for the more general cases of TCO is missing.

; recursive, simple case

(define (foo a)
   (foo ...))  ; <- tail call

; mutually recursive. Will run in an endless loop, will not run out of stack space.

(define (bar a)
   (baz a))  ; <- tail call

(define (baz a)
  (bar a))  ; <- tail call

; general

(define (example-1 a)
  (+    ;  <-  tail call
     (* a a)
     (* a a a)))

; let's say you have a fantastillion functions and each calls the next one in a tail call
; -> will not run out of stack space 

TCO test in Guile-2.1+ implementation

I've encountered a happy trouble when I'm writing step5 in GNU Guile.
I think most of the languages will crash when calling (sum-to 10000) in non-tail-call way. But in Guile-2.1+, it's not true. It's because Guile-2.1+ has auto-grow-stack feature. So if the stack is out, there'll be new allocated stack, depends on your physical memory.
So I can get 50005000 after calling (sum-to 10000), which means rest1 will be set to this number rather than keep nil.
Some implementation, like C could undertake (sum-to 10000), but crash on (sum-to 100000).
But in Guile-2.1+, even (sum-to 1000000) is fine.

Now, how should we fix this properly...?

gensym

(Following discussion in #103)

Here's a pure mal gensym implementation, and an implementation of the or macro (called or2 below) with gensym:

(def! incr (fn* [x] (+ 1 x)))

(def! *gensym-counter* (atom 0))

(def! gensym
  (fn* []
    (symbol (str "G__" (swap! *gensym-counter* incr)))))

;;
;; test `or` implementation with `gensym` instead of `or_FIXME`
;;
(defmacro! or2
  (fn* (& xs)
    (if (empty? xs)
      nil
      (if (= 1 (count xs))
        (first xs)
        (let* (condvar (gensym))
          `(let* (~condvar ~(first xs))
             (if ~condvar ~condvar (or2 ~@(rest xs)))))))))

(prn (macroexpand (or2 nil "yes")))
(prn (or2 nil "yes"))

(prn (macroexpand (or2 "yes" false)))
(prn (or2 "yes" false))

Output:

(let* (G__1 nil) (if G__1 G__1 (or2 "yes")))
"yes"
(let* (G__3 "yes") (if G__3 G__3 (or2 false)))
"yes"

It relies on atom, so cannot be used in step 8 where we test the or and and macros (atom is added in step A).

Do you prefer a native impl in step 8, with all the quote special operators?

Testing commands fail in zsh

Love this project, finding it super pedagogical!

The guide mentions the shell command

make test^quux^stepX

to run tests.

This works fine in bash. It does not work in zsh, however. After an entire night and morning of thinking it was something wrong in my implementation code that prevented me from running the tests, I found out that zsh treats unquoted carets as globbing characters.

Perhaps you should change the guide to say

make "test^quux^stepX"

for sake of universality?

Or make a note that the args should be quoted, for zsh users?
Or make a note that the command should be prefixed with noglob, i.e.
noglob make test^quux^stepX, for zsh users?

Similar test failures in multiple languages

$ ./runtest.py  q.mal -- rust/target/release/step1_read_print
TEST: (+ 1 2) -> ['','(+ 1 2)'] -> FAIL (line 2):
    Expected : '(+ 1 2)\r\n(+ 1 2)'
    Got      : '(+ 1 2)\x08\x08\x08\x08\x08\x08\x08\x1b[C\x1b[C\x1b[C\x1b[C\x1b[C\x1b[C\x1b[C\r\n(+ 1 2)'
FAILURES: 1
$ ./runtest.py  q.mal -- python python/step1_read_print.py 
TEST: (+ 1 2) -> ['','(+ 1 2)'] -> FAIL (line 2):
    Expected : '(+ 1 2)\r\n(+ 1 2)'
    Got      : '(+ 1 2)\x08\x08\x08\x08\x08\x08\x08\x1b[C\x1b[C\x1b[C\x1b[C\x1b[C\x1b[C\x1b[C\r\n(+ 1 2)'
FAILURES: 1
$ ./runtest.py  q.mal -- perl perl/step1_read_print.pl 
TEST: (+ 1 2) -> ['','(+ 1 2)'] -> FAIL (line 2):
    Expected : '(+ 1 2)\r\n(+ 1 2)'
    Got      : '(+ 1 2)\x08\x08\x08\x08\x08\x08\x08\x1b[C\x1b[C\x1b[C\x1b[C\x1b[C\x1b[C\x1b[C\r\n(+ 1 2)'
FAILURES: 1

The programs does actually output those bytes:

$ rust/target/release/step1_read_print < q.mal  | hd
00000000  75 73 65 72 3e 20 28 2b  20 31 20 32 29 08 08 08  |user> (+ 1 2)...|
00000010  08 08 08 08 1b 5b 43 1b  5b 43 1b 5b 43 1b 5b 43  |.....[C.[C.[C.[C|
00000020  1b 5b 43 1b 5b 43 1b 5b  43 0a 28 2b 20 31 20 32  |.[C.[C.[C.(+ 1 2|
00000030  29 0a 75 73 65 72 3e 20  3b 3d 3e 28 2b 20 31 20  |).user> ;=>(+ 1 |
00000040  32 29 08 08 08 08 08 08  08 1b 5b 43 1b 5b 43 1b  |2)........[C.[C.|
00000050  5b 43 1b 5b 43 1b 5b 43  1b 5b 43 1b 5b 43 0a 75  |[C.[C.[C.[C.[C.u|
00000060  73 65 72 3e 20                                    |ser> |
00000065

Why does this happen? Is it some interference from readline?

Build Rust in release mode

cargo build --release

This enables compiler optimizations and will likely give it a big boost without touching the code.

Extending language using the host language

I'm using the javascript interpreter. What would be the best way to extend the interpreter by writing a new function in the host language (without modifying the original interpreter code)?

Trying to write filter function

Hi,

I'm trying to write a filter function. I've tried a couple attempts and I am getting some odd results. Would appreciate some feedback as to what I am doing wrong.

(def! zero? (fn* (n) (= 0 n)))

(def! car first)

(def! cdr rest)

(def! filter
  (fn* (pred xs)
    (if (> (count xs) 0)
      (if (pred (car xs)) 
        (cons (car xs) (filter pred (cdr xs)))
        (filter pred (cdr xs)) 
      '() 
    ))))

And then in the js repl I'm getting:

> (filter zero? '(0 1 2 3))
(0 nil)

Thanks

Move non-TCO step5 tests to implementation specific directories

About half the implementations do not fail the non-TCO test case: some crash, some pass due to the language already having automatic TCO or automatic stack growth. It would still be useful to test the TCO case. The non-TCO case should be moved to implementation specific directories and made specific to that implementation. For the few implementations that crash on stackoverflow, there would be no implementation specific case.

If macro expands to symbol it doesn't go through another EVAL

The behaviour I encounter is a bit strange, but maybe I'm missing something (not sure my issue description is correct).

Here is the problematic scenario:

user> (let* [foo 123] (or false foo))
foo

I expected 123 as the result here.

Or with a simpler macro:

Mal [clojure]
user> (defmacro! nothing (fn* [x] x))
#object[clojure.lang.AFunction$1 0x2a341864 "clojure.lang.AFunction$1@2a341864"]
user> (nothing a)
a

I expected an exception 'a' not found.

Note that if the macro expands to a list, then things seem OK:

user> (let* [foo 123] (or false (+ foo 0)))
123

I see the same behaviour in Clojure, Ruby, Python implementations.

Here is the fix status per implementation:

  • awk
  • bash
  • c
  • cpp
  • coffee
  • cs
  • clojure
  • crystal
  • d
  • elixir
  • erlang
  • es6
  • factor
  • forth
  • fsharp
  • go
  • groovy
  • guile
  • haskell
  • java
  • js
  • julia
  • kotlin
  • lua
  • make
  • mal
  • matlab
  • miniMAL
  • nim
  • ocaml
  • perl
  • php
  • ps
  • python
  • r
  • racket
  • rpython
  • ruby
  • rust
  • scala
  • swift
  • tcl
  • vb
  • vimscript

Question: Writing some macros for familiar lisp expressions

Hi,

I'd like to define some familiar lisp expressions in MAL. Lisp style defun, defmacro and creating aliases of functions/macros.

How would you write macros for lisp style defun and defmacro?

Also, I was wondering how you would write defalias in MAL.

Thanks,
Kerry

Threading macro question

I was trying out the -> operator that is in core.mal.

How would you use functions like map or filter with ->? Leaving off the argument doesn't automatically create a partially applied function and return another function. I could create an anonymous function or create a new function. Is there a more general way to do partial function application on map or filter so that I can use it in ->?

Thanks,
Kerry

Python: String comparsing fails when a unicode string is involved

(Discovered while working on #134)

Here it is with some debug prints I added (running python 2.6.6):

REPL implementation python, step file: python/step4_if_fn_do.py
Running: python ../python/step4_if_fn_do.py 
user> (= (str :abc) ":abc")
('DEBUG in _equal_Q: a=', u':abc', 'type(a)=', <type 'unicode'>)
('DEBUG in _equal_Q: b=', ':abc', 'type(b)=', <type 'str'>)
false

Somehow the str call creates a unicode string (probably due to the special char which is used to mark mal keywords). This fails in _equal_Q because the types are not identical.

Not sure what's the appropriate way in Python to convert str to unicode. Also I thought that unicode is only in Python 3 but it seems that it occurs here (Python 2.6) too.

Javascript interop: dot form

I noticed the dot form in the online repl docs. I was looking at the source but I'm not sure how it works. Does it work on the web version? Could you provide an example?
Thanks

Javascript interop

I was wondering what options are available for javascript (browser) interop in mal. Could you provide some examples of function execution and object property access?

Thanks!

Matlab code doesn't clone properly on case-insensitive file systems

Here's some fascinating collision behavior for you. On case-insensitive file systems, Git repositories that have two files differing only in case don't clone properly. The just-cloned repo thinks it already has modifications, and attempting checkout just toggles which file it believes was modified!

Since we're unlikely to change people's file systems, perhaps someone with more (any) Matlab knowledge would like to rejigger {R,r}eader.m.

src$ git clone https://github.com/kanaka/mal.git
Cloning into 'mal'...
remote: Counting objects: 4239, done.
remote: Total 4239 (delta 0), reused 0 (delta 0), pack-reused 4239
Receiving objects: 100% (4239/4239), 2.14 MiB | 1.76 MiB/s, done.
Resolving deltas: 100% (2994/2994), done.
Checking connectivity... done.
src$ cd mal
mal(master)$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   matlab/Reader.m

no changes added to commit (use "git add" and/or "git commit -a")
mal(master)$ git checkout matlab/Reader.m
mal(master)$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   matlab/reader.m

no changes added to commit (use "git add" and/or "git commit -a")
mal(master)$ git checkout matlab/reader.m
mal(master)$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   matlab/Reader.m

no changes added to commit (use "git add" and/or "git commit -a")

RFC: x64 Implementation Proposal

This is a slightly longer version of the proposal posted on Twitter: https://twitter.com/peterastephens/status/806710260815822849

Originally I was thinking the x64 implementation would need to emit binary opcodes, ELF headers, etc. After taking a look at @dubek's malc LLVM implementation I think a simpler approach can be taken: the compiler can be written in mal and bootstrap itself by emitting x64 assembler source code which can then be assembled using nasm and linked using ld. See this gist for a quick POC I did.

There are some challenges:
1. I'll need to implement a JIT compiler in order to execute (eval ...). Nasm could work here but binary address space manipulation would still be required to patch the running process. A more likely plan would be to create a higher level mal based AST which could be transformed to either assembly source or could be assembled in process using built-in binary support. The bootstrapping process would then be to build the compiler using the existing mal language as well as an existing mal implementation using AOT compilation. The produced binary would then implement a super set of mal capable of manipulating binary data and patching its running process.
2. GC - A garbage collector would be a nice to have but not an initial requirement. A fixed size heap could be used (1GB for instance) and then the process could be terminated when OOM was reached. This would likely pass most of the mal test suite but would likely fail with heavy recursion. Clearly a rudimentary GC would be something to look at down the road and even a stop-the-world single generational GC would be useful and educational.
3. Data structures - the heap could be tuned for the native mal data structures plus a few extras to support JIT compilation. Nice to have would be a native implementation of the persistent data structures as described by Chris Okasaki.
4. Exception handling - this would require keeping detailed metadata on how each function's opcodes were laid out and then to perform stack analysis when an exception was thrown. This will be an area of research.
5. Interop - not quite sure what form this would take. Maybe the ability to make unsafe syscalls? With GC this would require pinning, native pointers, and complexity.

This mal implementation could be written using the 10 steps and could pass the test suite when fully implemented.

Question: should this be built as part of the mal repo proper? Or as a separate repo as with @dubek's malc?

Comments welcome.

example from page 59 of TAPL not working quite as expected?

user> (def! test (fn* [l m n] l m n))

user> (def! tru (fn* [t f] t))

user> (def! fls (fn* [t f] f))

user> (test tru 3 4) should return 3?
user> (test fls 3 4) should return 4?

Instead I seem to get an unevaluated functions, either #<Function: (fn* [t f] t)> or #<Function: (fn* [t f] f)>

Thoughts?

.gitignore */mal?

Is there a reason you .gitignore the */mal path?

It makes it hard to add files under factor/mal without using git add -f...

Make test^js failed with a float precision error

TEST: (/ (- (+ 515 (* 222 311)) 300) 41) -> ['','1689'] -> FAIL (line 15):
    Expected : '(/ (- (+ 515 (* 222 311)) 300) 41)\r\n1689'
    Got      : '(/ (- (+ 515 (* 222 311)) 300) 41)\r\n1689.1951219512196'

Mal [c] … user () … Segmentation fault (core dumped)

The empty list causes the C version of Mal to fall over and die a screaming horrible death.

In the Ruby version a NoMethodError: undefined method []' for nil:NilClassexception is raised because inreturn f[*el.drop(1)]at the end ofEVALthe variablefisnil` (the non-existent first element of the list.

Is Mal meant to treat the empty list as both a List and Atom. If so, how?

Typing () into the online common Lisp interpreter http://learnlispthehardway.org/try-lisp/ gives the result NIL

haskell Type.hs ERROR

Env : opensuse 42.1 ghc 7.10.3
runhaskell step2,...,stepA
Types.hs:64:1:
Non type-variable argument
in the constraint: Control.Monad.Error.Class.MonadError MalError m
(Use FlexibleContexts to permit this)
When checking that ‘throwStr’ has the inferred type
throwStr :: forall (m :: * -> *) a.
Control.Monad.Error.Class.MonadError MalError m =>
String -> m a

Types.hs:65:1:
Non type-variable argument
in the constraint: Control.Monad.Error.Class.MonadError MalError m
(Use FlexibleContexts to permit this)
When checking that ‘throwMalVal’ has the inferred type
throwMalVal :: forall (m :: * -> *) a.
Control.Monad.Error.Class.MonadError MalError m =>
MalVal -> m a

Not able to run tests.

Just cloned the mal repo and I'm having problems running the tests. I use make test^ruby^step0 to run the ruby implementation, but it's the bash implementation that gets run, and it even breaks, same for other languages I tried.

Fix nested equality tests and then make them mandatory

The following implementations fail the soft tests for nested equality. Sequence nested tests in step4 and hash-map nested steps in step9:

  • bash (hash-map in hash-map)
  • elixir (seq in seq)
  • erlang (seq in seq)
  • factor (seq in seq)
  • forth (no native hash-map equality, see #116)
  • guile
  • haskell (seq in hash-map)
  • julia
  • kotlin (no native hash-map equality, see #116)
  • php (seq and hash-map in hash-map)
  • scala

Once those are all fixed we should make those tests mandatory.

Tests do not work under Windows

I'm trying to run the unit tests for step 0 under Windows, using cygwin. But there seems to be an issue with the line endings. I'm doing what is described in step 0 - I write the prompt, read the input, write the input out and repeat.

Here's the output of the test call:

$ make test^quux^step0
----------------------------------------------
Testing test^quux^step0, step file: quux/step0_repl.exe, test file: tests/step0_repl.mal
Running: ../runtest.py  ../tests/step0_repl.mal -- ../quux/step0_repl.exe
 -> ['','hello world\r'] -> FAIL (line 2):
    Expected : ['hello world\r\r\nhello world\r', 'hello world\r\r\nhello world\r\r\nhello world\r']
    Got      : 'hello world\r\n\r\nhello world\r'
 -> ['','abcABC123\r'] -> FAIL (line 5):
    Expected : ['abcABC123\r\r\nabcABC123\r', 'abcABC123\r\r\nabcABC123\r\r\nabcABC123\r']
    Got      : 'user> abcABC123\r\n\r\nabcABC123\r'
 -> ['',';:() []{}"\'*\r'] -> FAIL (line 8):
    Expected : [';:() []{}"\'*\r\r\n;:() []{}"\'*\r', ';:() []{}"\'*\r\r\n;:() []{}"\'*\r\r\n;:() []{}"\'*\r']
    Got      : 'user> ;:() []{}"\'*\r\n\r\n;:() []{}"\'*\r'
 -> ['','hello world abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ 0123456789 (;:() []{}"\'* ;:() []{}"\'* ;:() []{}"\'*)\r'] -> FAIL (line 13):
    Expected : ['hello world abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ 0123456789 (;:() []{}"\'* ;:() []{}"\'* ;:() []{}"\'*)\r\r\nhello world abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ 0123456789 (;:() []{}"\'* ;:() []{}"\'* ;:() []{}"\'*)\r', 'hello world abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ 0123456789 (;:() []{}"\'* ;:() []{}"\'* ;:() []{}"\'*)\r\r\nhello world abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ 0123456789 (;:() []{}"\'* ;:() []{}"\'* ;:() []{}"\'*)\r\r\nhello world abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ 0123456789 (;:() []{}"\'* ;:() []{}"\'* ;:() []{}"\'*)\r']
    Got      : 'user> hello world abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ 0123456789 (;:() []{}"\'* ;:() []{}"\'* ;:() []{}"\'*)\r\n\r\nhello world abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ 0123456789 (;:() []{}"\'* ;:() []{}"\'* ;:() []{}"\'*)\r'
FAILURES: 4
Makefile:156: recipe for target 'test^quux^step0' failed
make: *** [test^quux^step0] Error 2

Got any idea on how to solve this?

Objective-C with xcodebuild doesn't integrate with top-level Makefile

Root Makefile Diff:

diff --git a/Makefile b/Makefile
index 29ac249..37b370f 100644
--- a/Makefile
+++ b/Makefile
@@ -18,7 +18,8 @@ mal_TEST_OPTS = --start-timeout 60 --test-timeout 120

 IMPLS = awk bash c clojure coffee cpp crystal cs erlang elixir es6 factor forth fsharp go groovy \
        guile haskell java julia js kotlin lua make mal ocaml matlab miniMAL nim \
-       perl php ps python r racket rpython ruby rust scala swift vb vimscript
+       perl php ps python r racket rpython ruby rust scala swift vb vimscript \
+       objc

 step0 = step0_repl
 step1 = step1_read_print
@@ -104,6 +105,7 @@ swift_STEP_TO_PROG =   swift/$($(1))
 vb_STEP_TO_PROG =      vb/$($(1)).exe
 vimscript_STEP_TO_PROG = vimscript/$($(1)).vim
 guile_STEP_TO_PROG =   guile/$($(1)).scm
+objc_STEP_TO_PROG =    objc/DerivedData/mal/Build/Products/Debug/$($(1))

 # Needed some argument munging
 COMMA = ,
@@ -155,6 +157,7 @@ vb_RUNSTEP =      mono ../$(2) --raw $(3)
 vimscript_RUNSTEP = ./run_vimscript.sh ../$(2) $(3)
 # needs TERM=dumb to work with readline
 guile_RUNSTEP =   guile --no-auto-compile -L ../guile ../$(2) $(3)
+objc_RUNSTEP =    ../$(2) $(3)


 vimscript_TEST_OPTS = --test-timeout 30

objc/Makefile:

DerivedData/mal/Build/Products/Debug/step0_repl: step0_repl/main.m
    xcodebuild -scheme step0_repl -configuration Debug -derivedDataPath DerivedData/mal

I get the following output from make test^objc^step0

$ make test^objc^step0
/Applications/Xcode7.1.app/Contents/Developer/usr/bin/make -C objc/DerivedData/mal/Build/Products/Debug/ step0_repl
make: *** objc/DerivedData/mal/Build/Products/Debug/: No such file or directory.  Stop.
make: *** [objc/DerivedData/mal/Build/Products/Debug/step0_repl] Error 2

If I built with Xcode (which creates the same binary as make would), I then get the following output from make test^objc^step0

$ make test^objc^step0
/Applications/Xcode7.1.app/Contents/Developer/usr/bin/make -C objc/DerivedData/mal/Build/Products/Debug/ step0_repl
make: *** objc/DerivedData/mal/Build/Products/Debug/: No such file or directory.  Stop.
make: *** [objc/DerivedData/mal/Build/Products/Debug/step0_repl] Error 2
LT-A8-120617:mal heath.borders$ make test^objc^step0
----------------------------------------------
Testing test^objc^step0, step file: objc/DerivedData/mal/Build/Products/Debug/step0_repl, test file: tests/step0_repl.mal
Running: ../runtest.py   ../tests/step0_repl.mal -- ../objc/DerivedData/mal/Build/Products/Debug/step0_repl 
Testing basic string
TEST: abcABC123 -> ['',abcABC123] -> SUCCESS
Testing string containing spaces
TEST: hello mal world -> ['',hello mal world] -> SUCCESS
Testing string containing symbols
TEST: []{}"'* ;:() -> ['',[]{}"'* ;:()] -> SUCCESS
Test long string
TEST: hello world abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ 0123456789 (;:() []{}"'* ;:() []{}"'* ;:() []{}"'*) -> ['',hello world abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ 0123456789 (;:() []{}"'* ;:() []{}"'* ;:() []{}"'*)] -> SUCCESS
TEST RESULTS (for ../tests/step0_repl.mal):
    0: soft failing tests
    0: failing tests
    4: passing tests
    4: total tests

I'm not blocked because I can build from Xcode and then run the tests from make, but I won't be able to issue a pull request until I can build entirely within the make infrastructure.

Mal Fails to Compile on Windows 64 bit, Go 1.4.2

# readline src\readline\readline.go:12:31: fatal error: readline/readline.h: No such file or directory #include <readline/readline.h> ^ compilation terminated. make: *** [step0_repl] Error 2

Missing regression testing

During the development of the Tcl implementation I found it important in some steps to verify that I haven't broken something that was working OK earlier. In my case, while implementing TCO in step 5 I broke let* by mistake, but no tests covered it. However, running the tests/step4_if_fn_do.mal test file against my step5_tco.tcl program quickly showed that let* is broken.

I wrote a small regression tester bash script (for Tcl - see code below) that runs all the test cases from previous steps (except step0 and step1) against the given step implementation. For example, if you run the script with step6 it'll concatenate the tests files for step2, step3, step4, step5 and step6 and feed the combined test file into the step6 program.

A better implementation would be in the Makefile with all the info about RUNSTEP and STEP_TO_PROG available there. We can also improve runtest.py to accept several test files and execute them one after the other (reporting total scores of success/failed tests at the end).

regtest.sh

#!/bin/bash

step=$1

if [ -z $step ] ; then
  echo "Usage: $0 step"
  echo "Example: $0 step9"
  exit 1
fi

step0_regs="step0"
step1_regs="step1"
step2_regs="step2"
step3_regs="$step2_regs step3"
step4_regs="$step3_regs step4"
step5_regs="$step4_regs step5"
step6_regs="$step5_regs step6"
step7_regs="$step6_regs step7"
step8_regs="$step7_regs step8"
step9_regs="$step8_regs step9"
stepA_regs="$step9_regs stepA"

projectdir=$(dirname $0)
cd $projectdir/tcl

testfiles=""
eval "regsteps=\$${step}_regs"
for s in $regsteps ; do
  testfile=$(echo ../tests/$s*.mal)
  testfiles="$testfiles $testfile"
done

../runtest.py <(cat $testfiles) -- tclsh ../tcl/${step}_*

runtest.py needs the ability to test expected failures

I am currently implementing mal in D and I've almost reached the end of step 1.

At the moment the only types I've implemented are symbols, lists, and vectors, but despite this I am passing the "string error" test for step 1.

Actual ./mal Output:

user> [1 2
expected ], got EOF
user> "abc
abc

Observed Test Output:

TEST: (1 2 -> ["expected ')', got EOF\r\n",''] -> SUCCESS
TEST: [1 2 -> ["expected ']', got EOF\r\n",'
'] -> SUCCESS
TEST: "abc -> ['expected '"', got EOF\r\n','*'] -> SUCCESS

This issue is related to #89.

Node / Js / es6 step 0

I've noticed the node-gyp rebuild fails when I npm --install in the js or es6 implementations.
Found out why here had to bump ffi to 2.2.0

That is not however the issue per se. My issue is that the tests fail (I would assume unless I use the ffi readline)

I was trying to rework it myself from scratch and started on my own implementation.

Currently I'm using the following for step 0

const readline = require('readline')
const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout
})
rl.setPrompt('user> ')

function READ(inp) {return inp}
function EVAL(inp) {return inp}
function PRINT(inp) {return inp}

function rep(inp) {
    return PRINT(EVAL(READ(inp)))
}


rl.on('line', (inp) => {
    process.stdout.write(`${inp}\n`)
    rl.prompt()
})
rl.on('close',()=>{
    process.exit()
})

rl.prompt()

And all tests fail like this:

FAILED TEST (line 3): abcABC123 -> ['',abcABC123]:
    Expected : 'abcABC123\r\nabcABC123'
    Got      : None

however when i run this myself and enter input etc it works as expected. What is it that the python test is doing which prevents this from working?

Regexp problem with unterminated strings

[\s,]*(~@|[\[\]{}()'`~^@]|"(?:\\.|[^\\"])*"|;.*|[^\s\[\]{}('"`,;)]*)

The regular expression provided in the guide is really impressive, however, when the input string is "123, it will filter out the first double quote. So how can we detect the unmatched " then?

I found several implementations, which is using regular expressions to do lexical analysis, have this problem, including C, Go and OCaml.

Mal [c]
user> "123
123

Finish/improve guide (steps 9, A, keyword, vector, hash-map, atom)

The prompt has to be "user>" or "mal-user>", the guide doesn't mention that until you run the test suite and it complains about that.

"distringuishing" is not a word :P

Generally there is a lot of handwaving going on I feel. For example "Add support for the other mal types: keyword, vector, hash-map, and atom." Not even examples for syntax are given! Same with the optional step of implementing quoting for example.

I'm just at end of step 1. Might fill in more in this ticket when I continue,

Filename typo in step 3

In step three of the process guide, when renaming the files for implementing step 3, the new file name is printed as step2_env.qx, but it should be step3_env.qx.

Create a spec

Note: This has been brought up by @Benaiah on IRC.

It's wonderful that MAL has a guide, loads of tests and some example code, however some parts of the guide are less than clear. One example of this is the number type (there is no mention whether you need to support signed numbers, what range they must have, whether they may be floats, etc.), another one would be the specific semantics of hash maps (how is key equality defined, would it be possible to compare the values only instead of having to keep track of the key type?). There is no need to specify everything, simply describing the minimum requirements in great detail and marking the unspecified parts would be enough.

Pure mal equal implementation

After implementing a few of those hash-equality fixes, I have an idea: how about we implement scalar-equal? with a native implementation, and implement the generic = in mal itself (like we implement not). The mal implementation will use sequential?, hash_map?, contains?, get, first, rest (and scalar-equal?) to recursively check whether the two data structures are equal.

Of course changing it now is a bit late, because = is already implemented natively in all languages (and few still got hash-equality wrong).

Here is the "pure" mal implementation of = (called here new-equal?):

(def! scalar-equal? =)

;; need the `and` macro
(load-file "../core.mal")

(def! sequential-equal?
  (fn* [a b]
    (if (scalar-equal? (count a) (count b))
      (if (scalar-equal? 0 (count a))
        true
        (if (new-equal? (first a) (first b))
          (sequential-equal? (rest a) (rest b))
          false))
    false)))

(def! hash-vals-equal?
  (fn* [a b map-keys]
    (if (scalar-equal? 0 (count map-keys))
      true
      (let* [key (first map-keys)]
        (if (and (contains? a key) (contains? b key))
          (if (new-equal? (get a key) (get b key))
            (hash-vals-equal? a b (rest map-keys))
            false)
          false)))))

(def! hash-map-equal?
  (fn* [a b]
    (if (scalar-equal? (count (keys a)) (count (keys b)))
      (hash-vals-equal? a b (keys a))
      false)))

;; This implements = in pure mal (using only scalar-equal? as native impl)
(def! new-equal?
  (fn* [a b]
    (cond
      (and (sequential? a) (sequential? b)) (sequential-equal? a b)
      (and (map? a) (map? b)) (hash-map-equal? a b)
      true (scalar-equal? a b))))

Pretty printing

Hi,

I just wrote some code in MAL and got back a pretty long s-expression. Has anybody looked at pretty printing in MAL? Would it be difficult to write a pprint function in MAL?

Kerry

Quasiquote within quasiquote appears to be broken

Here's a simplified example in Common Lisp of what I'm trying to do.

* (let ((foo 2)) (eval `(let ((foo 1)) (eval `(print ,foo)))))

1
1
* (let ((foo 2)) (eval `(let ((foo 1)) (eval `(print ,,foo)))))

2
2

The equivalent in mal:

user> (let* [foo 2] (eval `(let* [foo 1] (eval `(prn ~foo)))))
2
nil
user> (let* [foo 2] (eval `(let* [foo 1] (eval `(prn ~~foo)))))
[long traceback omitted]
Exception: 'unquote' not found

The first case gets the wrong value at the wrong time. And the second case just fails altogether. Nested quasiquotes are not unusual in macros, but I can't seem to make them work in mal. Is there a different syntax for this I'm not seeing?

I've tested this in the python implementation, but since the code seems to follow the guide, I have to assume it's broken in the other implementations too.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.