Greetings,

I am new to XL. Looks very interesting. I am running on a modern X86-64 Linux machine. I have run into the following two issues:

1. "configure" should be mode 777. I know it's not really used but making it mode 777 will make the system just build.
2. When I run make, I get:

blake@i9-tower:~/Backup/xl.git$ make
Makefile:43: make-it-quick/rules.mk: No such file or directory
git submodule update --init --recursive
Submodule 'make-it-quick' (https://github.com/c3d/make-it-quick) registered for path 'make-it-quick'
Submodule 'recorder' (https://github.com/c3d/recorder) registered for path 'recorder'
Cloning into '/home/blake/Backup/xl.git/make-it-quick'...
Cloning into '/home/blake/Backup/xl.git/recorder'...
Submodule path 'make-it-quick': checked out 'df4c924ff6192c91185984132b68b578b4290a4e'
Submodule path 'recorder': checked out '563ab67c1c9733b4f2f11381602630683617ee8e'
Submodule 'make-it-quick' (https://github.com/c3d/make-it-quick) registered for path 'recorder/make-it-quick'
Cloning into '/home/blake/Backup/xl.git/recorder/make-it-quick'...
Submodule path 'recorder/make-it-quick': checked out 'df4c924ff6192c91185984132b68b578b4290a4e'
make[1]: Entering directory '/home/blake/Backup/xl.git'
[BEGIN] opt linux in [top]
make[2]: Entering directory '/home/blake/Backup/xl.git/recorder'
[CONFIG] regex NO
[BEGIN] opt linux in [top]recorder/
[CONFIG] setlinebuf OK
[CONFIG] drand48 OK
[CONFIG] sys/mman OK
[CONFIG] regex OK
[CONFIG] sigaction OK
[BEGIN] opt linux in [top]recorder/
[GENERATE] config.h
[COMPILE 1/2] recorder_ring.c
[COMPILE 2/2] recorder.c
[LINK] librecorder.so
[END] opt linux in [top]recorder/
make[2]: Leaving directory '/home/blake/Backup/xl.git/recorder'
make[2]: Entering directory '/home/blake/Backup/xl.git/src'
[CONFIG] regex NO
/bin/bash: llvm-config: command not found
[INFO] Building with LLVM version
[BEGIN] opt linux in [top]src/
[CONFIG] longlong OK
[CONFIG] ulong OK
[CONFIG] uint OK
[CONFIG] ushort OK
[CONFIG] uchar NO
[CONFIG] glob OK
[CONFIG] drand48 OK
[CONFIG] sys/socket OK
[CONFIG] sys/mman OK
[CONFIG] regex OK
[CONFIG] mingw_aligned_malloc NO
[CONFIG] posix_memalign OK
[CONFIG] sbrk OK
[CONFIG] struct_stat OK
[CONFIG] sigaction OK
/bin/bash: llvm-config: command not found
[INFO] Building with LLVM version
[BEGIN] opt linux in [top]src/
[VARIANT] lib
make[3]: Entering directory '/home/blake/Backup/xl.git/src'
/bin/bash: llvm-config: command not found
[INFO] Building with LLVM version
[BEGIN] opt linux in [top]src/[lib]
[GENERATE] basics_module.h
[GENERATE] io_module.h
[GENERATE] math_module.h
[GENERATE] text_module.h
[GENERATE] remote_module.h
[GENERATE] time_functions_module.h
[GENERATE] temperature_module.h
[GENERATE] config.h
/bin/bash: llvm-config: command not found
/bin/bash: llvm-config: command not found
/bin/bash: llvm-config: command not found
[COMPILE 1/39] compiler-expr.cpp
In file included from compiler.h:43:0,
from compiler-expr.h:40,
from compiler-expr.cpp:38:
llvm-crap.h:67:20: error: operator '<' has no left operand
#elif LLVM_VERSION < 370
^
In file included from llvm-crap.h:74:0,
from compiler.h:43,
from compiler-expr.h:40,
from compiler-expr.cpp:38:
llvm-crap.h:406:19: error: operator '>=' has no left operand

if LLVM_VERSION >= 350 && LLVM_VERSION < 360

               ^~

llvm-crap.h:414:19: error: operator '>=' has no left operand

if LLVM_VERSION >= 370 && LLVM_VERSION < 380

               ^~

llvm-crap.h:419:19: error: operator '>=' has no left operand

if LLVM_VERSION >= 400

               ^~

In file included from compiler.h:43:0,
from compiler-expr.h:40,
from compiler-expr.cpp:38:
llvm-crap.h:78:10: fatal error: llvm/IR/Type.h: No such file or directory
#include <llvm/IR/Type.h>
^~~~~~~~~~~~~~~~
compilation terminated.
../make-it-quick/rules.mk:466: recipe for target '/home/blake/Backup/xl.git/.build/linux/opt/src/compiler-expr.cpp.o' failed
make[3]: *** [/home/blake/Backup/xl.git/.build/linux/opt/src/compiler-expr.cpp.o] Error 1
make[3]: Leaving directory '/home/blake/Backup/xl.git/src'
../make-it-quick/rules.mk:365: recipe for target 'lib.variant' failed
make[2]: *** [lib.variant] Error 2
make[2]: Leaving directory '/home/blake/Backup/xl.git/src'
make-it-quick/rules.mk:362: recipe for target 'src.recurse' failed
make[1]: *** [src.recurse] Error 2
make[1]: Leaving directory '/home/blake/Backup/xl.git'

real 0m1.830s
user 0m1.470s
sys 0m0.437s
5 Errors, 0 Warnings in /home/blake/Backup/xl.git/.logs/build-linux-opt-20210725-095243.log
make-it-quick/rules.mk:204: recipe for target 'opt' failed
make: *** [opt] Error 2

This post will be written with XL being conceptual language in mind. At some places it might sound differently, but I've chosen to use that terminology to be clear about the intent. The concepts I'm presenting are though generic and apply as well to XL as a conceptual language.

I believe everybody is familiar with the "billion dollar mistake" (as Tony Hoare calls it). I saw there is a bunch of recent commits (c3dde14 , 68b258d , 3da52b4 , 11ed0bd ) leveraging nil functionality.

I'd like to oppose using nil value. The question is though "how to deal with such interfaces"?

My answer is "use ephemeral optionals". First "optionals" is a well known concept (Haskell's Maybe, Rust's Option, V's ?, Zig's ...). Despite all the pros (being dead-simple to reason about even in highly parallel systems, no stack unwinding like with exceptions, universal - can be used for errors or alternative values of any kind etc.), it brings some serious usability issues.

Namely it "reifies" the fact whether it's a value or error and thus feels like a "primed bomb" you're tossing around. So what does the ephemeral mean? That means just one thing - the primed bomb can't be assigned to anything. In other words it can be only returned and directly at the caller site has to be dealt with.

Sounds still too inflexible? Well, not anymore if you provide some sugar covering the most common use case - namely propagate (return) the optional as I don't want to handle it here yet. V lang does this perfectly - you just need to write ? after a function call and it'll propagate it. And to handle the ephemeral optional you just append or { print( err ) } to the function call to "catch" the optional and work with the data the callee has put into the optional (which is under the hoods a struct holding arbitrary data).

All in all I'd like XL to get rid of a "lonely" nil value completely and instead use this ephemeral optionals trick together with e.g. sum types (aka tagged unions aka variants). Note there should definitely exist a nil type (a type can't be assigned to anything in run time because it's a type 😉 and thus doesn't appear in run time) - e.g. for tree structures - but no lonely nil value. So a function returning sum type T | nil (the value is either of type T or of type nil - note here nil is not lonely but "bound" to T) will be something totally different than ?T (aka "optional T" - the value is either of type T or an ephemeral error).

Note one can easily have ?(T | nil). E.g. a function which shall return the leftmost child of the leftmost tree node at level 5 - if the tree depth is 4 or less, it'll return an error otherwise it'll return the value of type "sum type of T and nil" which can be assigned to a variable unlike a lonely nil (this assumes it's impossible to define a sum type with less than two types - otherwise one could again create the generic nil bomb by defining a sumtype consisting of only nil type). In other words nil as value actually exists under the hood, but always as a "special" case of some more important value thus providing a compile time explicit guarantee that all such nil values under the hood will be handled and will not leak anywhere.

A bit related is the fact, that nil value should never be used to designate an all-encompassing "alternative value". Not having nil value at all (but only a nil type) is a simple measure to ensure this. Thus e.g. all the I/O functions in one of the commits I refer to above should return some optional instead of nil.

Of course this ephemeral optionals mechanism is totally agnostic from any exception-like or effect-like or Dylan-like or any other mechanism for dealing with alternative computational branches. So no need to do anything on that side.

Any thoughts on this?

The bytecode interpreter (-O1 option) is entirely broken.

Choose one of the following options:

1. Get rid of it entirely (which is a problem because of opcodes, which contains al the automation for bytecode
2. Fix it, make it type-safe and faster than the interpreter.
3. Redesign it to emit opcodes as text, much like the old XL2 bytecode

Provide an easy-to-use way to generate the foreign function interface (FFI) to call into C code.

The objective is make it very quick to "export" a C function in such a way that it can be called by XL code. Ideally, the whole process would be automated by the C++ compiler using meta-programming techniques.

I couldn't find much background about the considerations underpinning the earlier decision to swap out -> for is as the definition operator, which will potentially be very relevant in what I am aware might sound like I'm framing to be a simple Find/Replace... job. But, for now, I'm providing the following comments within the context of an ideal world:

Comment

The definition operator ought not to be is

Justification

1. It is not easy-to-read

As a single entity and a word in the English language, "is" is undeniably easy-to-read for a lot of people. Placed within lines or paragraphs of code, a lot of which will be comprised of other latin-based strings, it suffers from being paradoxically too easy-to-read (and instinctively not processed during a skim-read), thus making it more difficult to read quickly and efficiently, and forcing one to engage more cognitively to dissociate from linguistic context to a non-linguistic context but one in which it's intentionally mimicking a linguistic approximation in a jarring way. Linguistics aside, as initially alluded to, it is easy to lose among a group of other strings that camouflage its occurrence.

2. It's a misappropriation

It is the singularly-defined syntactic form for an operator that takes on the appearance of an English word in a language that, otherwise, has no other hard-coded entities we would be required to invoke that parallel a meaningful relationship between a coding language one would otherwise be free to cultivate esoterically dissociated syntax divorced from a spurious linguistic connotation hinted at by such a specific choice of entity. Somewhat frustratingly, it uses that word which, in English contextual prose, seldomly crops up in its copulative (relational) state: "is" is most often used intransitively, which underpins the previous argument; here, it copulates one clause to bond it relationally to another, which is, indeed, what would be valid grammatically when speaking, most typically in mathematical (and computer science) discussions as a loose-and-free way to infer equality, but never in formal usage. The overall effect is one that has borrowed a well-recognised object commonly used in one way, to crowbar it into an unrelated situation where its less-common form serves a proxy for a muddy-sounding description of what purpose it is intended to serve.

3. It's language-biased

"is" is specifically taken from English. Aside from those on the team likely having English as a common language between them, there's no need or reason to impose that on those who'd not want that. Other programming languages do this, rightly-or-wrongly, because of the industry that is driven by English-language developers, necessitating anyone wishing to develop professional basically have to deal with that. But that's part of what what XL appears to free itself from the burden of formally pre-defined syntax of procedural code where things have names and names ended up English-like. However, imagine how really weird it would seem to have a programming language where everything semantically or syntactically was curated by the coder, except for the one appearance of the Mandarin word 為 (wèi). We'd live, and I would probably use it to define a synonym to serve as a definition operator, if that is possible (and, if so, it still undesirable).

Suggested Solution

Use a symbol that is representative, without being language-specific. It might be additional effort to implement an accessible way to type special symbols from selected unicode blocks, but that could be a worthwhile endeavour in the long-run for a language untethered from what one conventionally sees on their keyboard's keys.

1. The most obvious would be to revert back to the symbolic arrow ->.

2. To be distinct, whilst remaining familiar, another unicode arrow form could be used. Given the nature of the operator is effectively (in mathematical terms) a functional mapping of syntax onto an operation, the most representatively authentic arrow form might be ↦ (the maths symbol for maps onto, e.g. 𝑓﹕𝑧 ↦ 𝑧² + 𝑐) but doesn't print well in some fonts.

   Other symbol suggestions:

   • ⇒ ("implies"), or the keyboard-friendly => which we all know from JS
   • ≔ (a familiar definition operator used in other languages) or the easier-to-type :=
   • ⧴ (no idea)
   • ⁑ a symbolic, decorative colon
   • ≡ ("is equivalent to", but would prevent it being defined for use as a mathematical operator)
   • ➔ an arrow that prints consistently well, which I have on shortcut

3. Create your own symbol and stick it in a reserved block of the unicode table the compiler can access.

Steps:

cd xl2/samples/HelloWorld
cp ../../xl.syntax .
cp ../../native/library/runtime/C/xl.bytecode .
../../bxl HelloWorld.xl > HelloWorld_native.cpp
g++ -I../../ HelloWorld_native.cpp -o HelloWorld_native

gives:

/usr/bin/ld: /tmp/ccIKhQRM.o: in function `main':
HelloWorld_native.cpp:(.text+0x112): undefined reference to `xl::ui::console::arguments[abi:cxx11]'
collect2: error: ld returned 1 exit status

Is this supposed to work now? Or is more work needed?

I'm fully aware XL undergoes a big "merging" of all the diverse efforts emerging throughout the last 10-15 years and that it also has some sharp edges.

It's though already very capable and it definitely makes sense to keep an eye also on the performance side (both speed of debug build compilation /assuming unoptimized debug build by default for subsecond compilation and production build upon explicit request/ and the running speed of the resulting binary).

Currently the most curated and maintained "generic" benchmark is https://github.com/kostya/benchmarks/ . It'd be great to see XL there as well (the production build of course).

Maybe if you have time you could explain how the compiler works (or is supposed to work), specifically for the two basic examples of factorial (or fibonacci) and hello world in bxl (which, if I understand correctly is abandoned because of typing issues) and your newer fast branch?

I have simply cloned this repo, did cd xl/xl2 and then make:

make[1]: Entering directory '/home/pp/git/xl/xl2'
Compiling main.cpp
Generating dependencies in obj/linux/debug/Makefile.depend
Compiling main.cpp
Compiling main.cpp
Compiling main.cpp
make[2]: Entering directory '/home/pp/git/xl/xl2'
Compiling predepend
Compiling predepend
make[2]: Leaving directory '/home/pp/git/xl/xl2'
----------- Making debug in /home/pp/git/xl/xl2 ------------
Dependencies for debug done
Compiling main.cpp
Compiling scanner.cpp
Compiling tree.cpp
Compiling context.cpp
Compiling parser.cpp
Compiling ctrans.cpp
Compiling options.cpp
Compiling errors.cpp
Building xl
cd bootstrap && make test
make[2]: Entering directory '/home/pp/git/xl/xl2/bootstrap'
Compiling compiler.xl
Generating dependencies in ../obj/linux/debug/bootstrap/Makefile.depend
Compiling compiler.xl
Compiling compiler.xl
Compiling compiler.xl
make[3]: Entering directory '/home/pp/git/xl/xl2/bootstrap'
No dependency for %
Compiling predepend
Compiling predepend
make[3]: Leaving directory '/home/pp/git/xl/xl2/bootstrap'
make[3]: Entering directory '/home/pp/git/xl/xl2/bootstrap'
----------- Making debug in /home/pp/git/xl/xl2/bootstrap ------------
Dependencies for debug done
cp ../xl_lib.h .
cp ../xl.syntax .
Compiling compiler.xl
Compiling ../obj/linux/debug/bootstrap/compiler.xl.C
../obj/linux/debug/bootstrap/compiler.xl.C: In function ‘void xl::translator::xl2cargs(xl::parser::tree::tree)’:
../obj/linux/debug/bootstrap/compiler.xl.C:2046:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debug/bootstrap/compiler.xl.C: In function ‘void xl::translator::xl2c(xl::parser::tree::tree)’:
../obj/linux/debug/bootstrap/compiler.xl.C:2658:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debug/bootstrap/compiler.xl.C:3492:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debug/bootstrap/compiler.xl.C: In function ‘bool xl::translator::xlnamespacescope(xl::parser::tree::tree)’:
../obj/linux/debug/bootstrap/compiler.xl.C:3822:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
Building bxl
---------- Done with debug in /home/pp/git/xl/xl2/bootstrap ----------
make[3]: Leaving directory '/home/pp/git/xl/xl2/bootstrap'
make PRODUCT=bbxl XL=./bxl OBJROOT="../obj/linux/debugbbxl"
make[3]: Entering directory '/home/pp/git/xl/xl2/bootstrap'
Compiling compiler.xl
Generating dependencies in ../obj/linux/debugbbxl/bootstrap/Makefile.depend
Compiling compiler.xl
Compiling compiler.xl
Compiling compiler.xl
make[4]: Entering directory '/home/pp/git/xl/xl2/bootstrap'
No dependency for %
Compiling predepend
Compiling predepend
make[4]: Leaving directory '/home/pp/git/xl/xl2/bootstrap'
make[4]: Entering directory '/home/pp/git/xl/xl2/bootstrap'
----------- Making debug in /home/pp/git/xl/xl2/bootstrap ------------
Dependencies for debug done
Compiling compiler.xl
Compiling ../obj/linux/debugbbxl/bootstrap/compiler.xl.C
../obj/linux/debugbbxl/bootstrap/compiler.xl.C: In function ‘void xl::translator::xl2cargs(xl::parser::tree::tree)’:
../obj/linux/debugbbxl/bootstrap/compiler.xl.C:3608:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debugbbxl/bootstrap/compiler.xl.C: In function ‘void xl::translator::xl2c(xl::parser::tree::tree)’:
../obj/linux/debugbbxl/bootstrap/compiler.xl.C:4304:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debugbbxl/bootstrap/compiler.xl.C:5246:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debugbbxl/bootstrap/compiler.xl.C: In function ‘bool xl::translator::xlnamespacescope(xl::parser::tree::tree)’:
../obj/linux/debugbbxl/bootstrap/compiler.xl.C:5744:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
Building bbxl
---------- Done with debug in /home/pp/git/xl/xl2/bootstrap ----------
make[4]: Leaving directory '/home/pp/git/xl/xl2/bootstrap'
make[3]: Leaving directory '/home/pp/git/xl/xl2/bootstrap'
make PRODUCT=bbbxl XL=./bbxl OBJROOT="../obj/linux/debugbbxl_test"
make[3]: Entering directory '/home/pp/git/xl/xl2/bootstrap'
Compiling compiler.xl
Generating dependencies in ../obj/linux/debugbbxl_test/bootstrap/Makefile.depend
Compiling compiler.xl
Compiling compiler.xl
Compiling compiler.xl
make[4]: Entering directory '/home/pp/git/xl/xl2/bootstrap'
No dependency for %
Compiling predepend
Compiling predepend
make[4]: Leaving directory '/home/pp/git/xl/xl2/bootstrap'
make[4]: Entering directory '/home/pp/git/xl/xl2/bootstrap'
----------- Making debug in /home/pp/git/xl/xl2/bootstrap ------------
Dependencies for debug done
Compiling compiler.xl
Compiling ../obj/linux/debugbbxl_test/bootstrap/compiler.xl.C
../obj/linux/debugbbxl_test/bootstrap/compiler.xl.C: In function ‘void xl::translator::xl2cargs(xl::parser::tree::tree)’:
../obj/linux/debugbbxl_test/bootstrap/compiler.xl.C:3608:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debugbbxl_test/bootstrap/compiler.xl.C: In function ‘void xl::translator::xl2c(xl::parser::tree::tree)’:
../obj/linux/debugbbxl_test/bootstrap/compiler.xl.C:4304:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debugbbxl_test/bootstrap/compiler.xl.C:5246:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debugbbxl_test/bootstrap/compiler.xl.C: In function ‘bool xl::translator::xlnamespacescope(xl::parser::tree::tree)’:
../obj/linux/debugbbxl_test/bootstrap/compiler.xl.C:5744:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
Building bbbxl
---------- Done with debug in /home/pp/git/xl/xl2/bootstrap ----------
make[4]: Leaving directory '/home/pp/git/xl/xl2/bootstrap'
make[3]: Leaving directory '/home/pp/git/xl/xl2/bootstrap'
make PRODUCT=bbbbxl XL=./bbbxl OBJROOT="../obj/linux/debugbbxl_stability"
make[3]: Entering directory '/home/pp/git/xl/xl2/bootstrap'
Compiling compiler.xl
Generating dependencies in ../obj/linux/debugbbxl_stability/bootstrap/Makefile.depend
Compiling compiler.xl
Compiling compiler.xl
Compiling compiler.xl
make[4]: Entering directory '/home/pp/git/xl/xl2/bootstrap'
No dependency for %
Compiling predepend
Compiling predepend
make[4]: Leaving directory '/home/pp/git/xl/xl2/bootstrap'
make[4]: Entering directory '/home/pp/git/xl/xl2/bootstrap'
----------- Making debug in /home/pp/git/xl/xl2/bootstrap ------------
Dependencies for debug done
Compiling compiler.xl
Compiling ../obj/linux/debugbbxl_stability/bootstrap/compiler.xl.C
../obj/linux/debugbbxl_stability/bootstrap/compiler.xl.C: In function ‘void xl::translator::xl2cargs(xl::parser::tree::tree)’:
../obj/linux/debugbbxl_stability/bootstrap/compiler.xl.C:3608:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debugbbxl_stability/bootstrap/compiler.xl.C: In function ‘void xl::translator::xl2c(xl::parser::tree::tree)’:
../obj/linux/debugbbxl_stability/bootstrap/compiler.xl.C:4304:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debugbbxl_stability/bootstrap/compiler.xl.C:5246:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debugbbxl_stability/bootstrap/compiler.xl.C: In function ‘bool xl::translator::xlnamespacescope(xl::parser::tree::tree)’:
../obj/linux/debugbbxl_stability/bootstrap/compiler.xl.C:5744:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
Building bbbbxl
---------- Done with debug in /home/pp/git/xl/xl2/bootstrap ----------
make[4]: Leaving directory '/home/pp/git/xl/xl2/bootstrap'
make[3]: Leaving directory '/home/pp/git/xl/xl2/bootstrap'
echo ------ Differences: -----------------------------------------
------ Differences: -----------------------------------------
diff ../obj/linux/debugbbxl_test/bootstrap/compiler.xl.C \
     ../obj/linux/debugbbxl_stability/bootstrap/compiler.xl.C
echo ------ End of Differences -----------------------------------
------ End of Differences -----------------------------------
make[2]: Leaving directory '/home/pp/git/xl/xl2/bootstrap'
cd native && make test
make[2]: Entering directory '/home/pp/git/xl/xl2/native'
Compiling compiler.xl
Generating dependencies in ../obj/linux/debug/native/Makefile.depend
Compiling compiler.xl
Compiling compiler.xl
Compiling compiler.xl
make[3]: Entering directory '/home/pp/git/xl/xl2/native'
No dependency for %
Compiling predepend
Compiling predepend
make[3]: Leaving directory '/home/pp/git/xl/xl2/native'
make[3]: Entering directory '/home/pp/git/xl/xl2/native'
----------- Making debug in /home/pp/git/xl/xl2/native ------------
Dependencies for debug done
cp ../xl_lib.h .
cp ../xl.syntax .
Compiling compiler.xl
Compiling ../obj/linux/debug/native/compiler.xl.C
../obj/linux/debug/native/compiler.xl.C: In function ‘void xl::codegenerator::xl2cargs(xl::bytecode::bytecode)’:
../obj/linux/debug/native/compiler.xl.C:4295:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debug/native/compiler.xl.C: In function ‘void xl::codegenerator::xl2c(xl::bytecode::bytecode)’:
../obj/linux/debug/native/compiler.xl.C:6033:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debug/native/compiler.xl.C: In function ‘bool xl::codegenerator::xlnamespacescope(xl::bytecode::bytecode)’:
../obj/linux/debug/native/compiler.xl.C:6640:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debug/native/compiler.xl.C: In function ‘xl::parser::tree::treenode* xl::errors::errortree(xl::parser::tree::tree)’:
../obj/linux/debug/native/compiler.xl.C:9008:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debug/native/compiler.xl.C: In function ‘xl::parser::tree::treenode* xl::plugin::listing::translator_220(xl::parser::tree::tree, xl::parser::tree::tree, xl::symbols::rewrite, xl::parser::tree::treemap&)’:
../obj/linux/debug/native/compiler.xl.C:13427:6: warning: variable ‘XLtranslateDone’ set but not used [-Wunused-but-set-variable]
 bool XLtranslateDone = false;
      ^~~~~~~~~~~~~~~
../obj/linux/debug/native/compiler.xl.C: In function ‘xl::parser::tree::treenode* xl::plugin::xl2c::convert(xl::parser::tree::tree)’:
../obj/linux/debug/native/compiler.xl.C:13654:33: error: cannot convert ‘bool’ to ‘xl::parser::tree::tree {aka xl::parser::tree::treenode*}’ in initialization
 xl::parser::tree::tree fnargs = false;
                                 ^~~~~
../obj/linux/debug/native/compiler.xl.C: In function ‘xl::parser::tree::treenode* xl::semantics::declarations::enterdeclaration(xl::parser::tree::tree, xl::parser::tree::tree, xl::parser::tree::tree, bool)’:
../obj/linux/debug/native/compiler.xl.C:17313:31: error: expected primary-expression before ‘decltype’
 xl::semantics::types::anytype decltype = xl::semantics::types::evaluatetype (type);
                               ^~~~~~~~
../obj/linux/debug/native/compiler.xl.C:17314:39: error: expected primary-expression before ‘decltype’
 if (xl::semantics::types::istypetype (decltype)) {
                                       ^~~~~~~~
../obj/linux/debug/native/compiler.xl.C:17321:83: error: expected primary-expression before ‘decltype’
 ::types::anytype declbasetype = xl::semantics::types::nonsourcetype (decltype);
                                                                      ^~~~~~~~
../obj/linux/debug/native/compiler.xl.C:17336:58: error: expected primary-expression before ‘decltype’
 (result = xl::semantics::iterators::enteriterator (name, decltype, value, name));
                                                          ^~~~~~~~
../obj/linux/debug/native/compiler.xl.C:17341:58: error: expected primary-expression before ‘decltype’
 (result = xl::semantics::functions::enterfunction (name, decltype, value, name));
                                                          ^~~~~~~~
../obj/linux/debug/native/compiler.xl.C:17361:2: error: expected primary-expression before ‘decltype’
 (decltype = xl::semantics::types::evaluatetype (type));
  ^~~~~~~~
../obj/linux/debug/native/compiler.xl.C:17361:2: error: expected ‘)’ before ‘decltype’
../obj/linux/debug/native/compiler.xl.C:17362:115: error: expected primary-expression before ‘decltype’
 type2 = dynamic_cast< xl::semantics::types::generics::generictype > (decltype);
                                                                      ^~~~~~~~
../obj/linux/debug/native/compiler.xl.C:17362:115: error: expected ‘)’ before ‘decltype’
In file included from /usr/include/c++/7/cassert:44:0,
                 from ./xl_lib.h:39,
                 from ../obj/linux/debug/native/compiler.xl.C:2:
../obj/linux/debug/native/compiler.xl.C: In function ‘bool xl::semantics::generics::instantiatedtypematch(xl::semantics::types::anytype, xl::semantics::types::anytype)’:
../obj/linux/debug/native/compiler.xl.C:23137:32: warning: the address of ‘bool xl::semantics::generics::instantiatedtypematch(xl::semantics::types::anytype, xl::semantics::types::anytype)’ will never be NULL [-Waddress]
 assert ((instantiatedtypematch == 0));
          ~~~~~~~~~~~~~~~~~~~~~~^~~~
Makefile:138: recipe for target '../obj/linux/debug/native/compiler.xl.o' failed
make[3]: *** [../obj/linux/debug/native/compiler.xl.o] Error 1
make[3]: Leaving directory '/home/pp/git/xl/xl2/native'
../Makefile.rules:62: recipe for target 'debug' failed
make[2]: *** [debug] Error 2
make[2]: Leaving directory '/home/pp/git/xl/xl2/native'
Makefile:49: recipe for target 'native_compiler' failed
make[1]: *** [native_compiler] Error 2
make[1]: Leaving directory '/home/pp/git/xl/xl2'
Makefile.rules:62: recipe for target 'debug' failed
make: *** [debug] Error 2

What can I do to build xl2 and xlr? I am very keen to try it out.

The current documentation now defines XL scoping rules in more details. Basically:

• A map is a definition with only definitions in it, but no statement.
• Two scoping operations exist, scope.value and scope value (often conventionally written as scope[value], although the block does not matter. The first one is called scoping and the second one is called indexing or applying. Scoping restricts the lookup while evaluating value to only what is in scope. Indexing injects the scope into the current evaluation context.

For example:

digit_spelling is
    0 is "zero"
    1 is "one"
    2 is "two"
    3 is "three"
    4 is "four"
    5 is "five"
    6 is "six"
    7 is "seven"
    8 is "eight"
    9 is "nine"

A is 3
digit_spelling.0   // "zero"
digit_spelling[0] // "zero"
digit_spelling.A  // Error: no "A" in scope
digit_spelling[A] // "three"
digit_spelling A  // "three", the block is unnecessary in that case
digit_spelling A+3 // "six" - This is a statement, so this parses as digit_spelling(A+3)
(digit_spelling A+3) // Error - This is an expression, so this parses as digit_spelling(A)+3
digit_spelling[A+3] // "six"
(digit_spelling[A+3]) // "six"

These scoping rules are partially implemented in the interpreter, see for example this test for maps, this test for anonymous maps and this test for the scoping operation. Note that all these tests mistakenly talk about "array", when it should be "map".

They are probably not working at all in the compiler at the moment.

XL modules have existed in two implementations so far:

• The XL2 implementation which is hierarchical, but has no dynamic loading, no version numbers nor much self-desccription / introspection.
• The Tao3D implementation has ABI and version checking, can load modules dynamically, has support for module description, but is not hierarchical and, paradoxically, does not deal too well with static modules.

Neither implementation covers syntax statements in modules correctly.

The new implementation should be:

• Hierarchical
• Cover static modules well
• Support dynamic modules
• Support version checks, ideally using semantic versioning (which depends on issue #20)

http://c3d.github.io/xl/#data-inheritance

I don't understand the difference between the two examples presented.

Also, what is data inheritance (or am I looking too deep)?

Since Tao3D relies heavily on the old syntax -> for what is now is, it would be neat to have an option to automatically convert A ->B into A is B.

There are clear limitations to this approach. Notably, X is 0 is now a constant, whereas X -> 0 in Tao3D would create a variable.

The Tao3D compiler used a relatively straightforward compilation strategy, which has been re-imported in the compiler-fast.cpp file:

• Keeps the parse trees in their original normal representation. For example, when you compute 2+3, you generate a parse tree for the result of the addition, which is an Integer tree with value 5.
• The only real type-checking occurs against built-in types, by implementing a "tag test"
• The code generated for all candidates checks if the type matches, if so keeps the computation, and the whole thing otherwise falls back to a runtime error.

This implementation is not particularly useful, except maybe in re-connecting with Tao3D (issue #32)

Issue #29 documents a foreign-function interface, but that currently only works with the compiler.

The same mechanism should also export functions for use by the interpreter, i.e. the evaluate function should be visible from the interpreter just because it's been tagged as XL_NATIVE.

Currently, a version number like 1.20.3 is parsed as a a real number, 1.2 followed by an infix . operator, followed by integer value 3. This makes it indistinguishable from 1.2.3.

This could be implemented through issue #27.

Historically, XL implemented a rule that in patterns, any name that was already visible was seen as a constant. This allowed the following definition of if statements to work as long as true and false were in scope:

if true then X else Y is X
if false then X else Y is Y

The three main problems with this approach were that:

• There was a difference between true and X, one being a constant, the other being a formal parameter, that was not obvious by reading the code and required the whole context to understand.
• The code above would be broken if someone defined X in the same scope, since now X would become a constant instead of a formal parameter.
• It forced the programmer to introduce named constants for expressions such as sqrt 2.

These three problems are solved in the documentation by introducing the notion of metabox. A metabox is written as [[Expr]] and evaluates to Expr in all contexts. With the metabox notation, the proper definition for if becomes:

if [[true]] then X else Y is X
if [[false]] then X else Y is Y

The previous definition will indeed make true a formal parameter.

The metabox notation can also be used in normal evaluation context in cases where evaluation of an expression must be forced. This is the case with the definition of the for loop:

for N:name in R:[range of discrete] loop Body is 
    loop_context is 
        [[N]] : R.type := R.first 
    LoopVar is loop_context.[[N]] 
    while LoopVar <= R.last loop
        (loop_context) (Body) 
        ++LoopVar 

Here, N is a name parameter, which may contain for example I when the pattern matches

for I in 1..5 loop
    print "I=", I

If loop_context was written as:

    loop_context is 
        N : R.type := R.first 

N would not be evaluated in a type annotation, but instead would create a local variable named N.

Similarly, the expression loop_context.N would not evaluate N but look it up in loop_context, where it does not exist since the variable there is called I. Using the metabox forces N to be evaluated, so that this transforms into loop_context.I, which will find I.

Metabox are a recent addition to the language are are not implemented neither in the compiler nor the interpreter.

There are a number of XL native modules that currently use the opcode.h interface.

It would be useful to see what can be converted to XL_NATIVE (or at least have a single mechanism to invoke native functions), see issue #29.

With the existence of XL_NATIVE (issue #29), it should now be possible to provide a strategy to transfer the 1500 or so native function interfaces in Tao3D.

Lifetime is a new concept in XL, inspired by the Rust "borrow checker", and destined to make it possible in XL to implement a wider class of borrow-checks from the library.

A lifetime value is generated for the expression lifetime X, which returns a compile-time constant that can be compared to other lifetime constants. For instance, a reference type can be made from an owning value with the following condition:

type ref T
type own T

Source:own T as Target:ref T when lifetime Target < lifetime Source is ...
Source:own T as Target:ref T when not (lifetime Target < lifetime Source) is
    compile_error "The reference %1 might outlive the source %2", Target, Source

The documentation now describes various forms of syntactic sugar which, incidentally, make most of the code written at the time of XL2 acceptable again.

Most syntactic sugar transform a prefix notation using some particular word into an infix type annotation with possibly some additions. The words being considered include: type, class, module, function, method, procedure, to, operation, data, in, out, inout, io,
constant, variable, macro, generic, polymorphic, fast, small, global, thread and static.

The transformation are generally similar to the following, and might be implementable fully in the library:

{ type T is Impl } is { T as type is Impl }
{ type T with Interface } is { T as type with Interface }
{ type T inherits Base } is { T like Base }
{ type T like Base } is { T like Base }
{ T inherits Base } is { T like Base }
{ module M is Impl } is { M as module is Impl }
{ module M with Interface } is { M as module with Interface }  
{ procedure Proc is Impl } is { Proc as mayfail is Impl }

There are quite a few more, but the pattern is relatively general and simple.

More direct compiler analysis might be needed for a few cases, but that remains to be confirmed.

The README shows the following code for a program that compute the factorial of 1 to 5:

0! is 1
N! is N * (N-1)!

for I in 1..5 loop
    print "The factorial of ", I, " is ", I!

I expect this to print:

The factorial of 1 is 1
The factorial of 2 is 2
The factorial of 3 is 6
The factorial of 4 is 24
The factorial of 5 is 120

However, the following is printed instead:

The factorial of I is N * (N - 1)!
The factorial of I is N * (N - 1)!
The factorial of I is N * (N - 1)!
The factorial of I is N * (N - 1)!
false

Environment: gcc:latest Docker image (Debian)
Occurs in both interpreted (xl -i file.xl) and compiled (xl file.xl) forms.

The current implementation reads its input in Unicode UTF-8 format, and makes crude attempts at accepting Unicode.

This was good enough for Tao3D to deal with multi-lingual text, including in languages such as Hebrew or Arabic. However, that implementation is a bit naive with respect to distinguishing Unicode letters from non-letter characters.

For example, 𝝿_2 or étalon are valid XL names, and this is intentional, but ⇒A2 is presently a valid XL name, and this can easily be considered a bug.

Implement the XL ownership model including the own and ref types.

The own T is the "safe" way to do a dynamic allocation in XL. The ref T type is the safe way to pass a reference around.

The in T type optimises type T for passing as an input parameter.
The out T type optimises type T for passing as an output parameter.
The inout T type (or io T optimises type T for passing as an input/output parameter.

This relies on binding binding rules, and selecting passing by copy or by reference depending on what is more efficient.

The main building stone on prolog systems is a "fact", same as in XL each time a new definition is given with the keyword is.
How does one differentiate from each other in conceptual level? How is the evaluator planned? AOT or lazy as in red/rebol/wolfram alpha?

I like the ideas behind XL, as I myself have as well theorized with some of them in my free time, and the whole definition of the concepts within the standard library, if well combined with literate programming can become simply game changing.

But isn't it missing a spot not looking deeper into prolog? For me it seems like implementing a more complete logic system for the definitions (concepts/facts) would definetely frame XL as a prolog descendent with a whole new set of features.

By logic programing I meant implementing a system with Horn Clauses and a facts database.

https://en.wikipedia.org/wiki/Horn_clause

Dynamic dispatch is currently somewhat implemented both in the compiler and interpreter.

However, given recent evolutions of the language, notably with respect to pattern matching, this code needs to be seriously revisited.

Key changes:

• Support for type inheritance / implicit conversions (two passes)
• Support for metaboxes (issue #10) in patterns
• Smarter dispatch in maps for common cases, e.g. contiguous values (ideally, turn that to "switch" statements in LLVM IR and let the switch optimizer do its magic).

A syntax statement within an imported file will not currently change the syntax in the importing file, only locally.

In order for the syntax extensions to propagate to the importing file, we need the scanner and parser to recognize that there is a syntax statement, mark it so that import will be aware of it. That, in turn, may require import to be processed early, e.g. during scanning.

Example:

// imported.xl
module IMPORTED with
    syntax { POSTFIX 130 cm mm km dm m }
    type distance is matching (D:real m)
    D:real km as distance is (D * 1000) m
    D:real mm as distance is (D * 0.001) m

// Importing file
use IMPORTED

D : distance := 3.2km

After building and sudo make install the shared library files for xl and recorder are installed (in /usr/local/lib) but not found by the xl binary. What is the recommended way to do this? I think usually on installation the path gets embedded by r(un)path.

Any plans for a chat room?

I am trying to read through and test things from the large docs, but sometimes there is just some niggling details that would be nice to bounce off other people.

Options:

• IRC
• Gitter
• matrix

The language could be made more extensible if you could add arbitrary terminal node types defined for example by a regular expression. This could be used to address issue #20 or issue #26.

In the syntax, it could be something like:

TEXT
        "<<" ">>"
        "HTML" "END_HTML"

CUSTOM
        version_number "[0-9]+\(\.[0-9]+\)+"
        bits "bits 16#[0-9a-fA-F_]+"

Note that the renderer should be updated to correctly render this kind of node. Also think about colorization (issue #23 and issue #24).

There is a very old xl2/xl.el Emacs Lisp file to offer some colorization, indentation, etc.

It's totally outdated and is badly in need of some repair.

Currently, the symbol table in the compiler is implemented as an XL prefix, renamed as Scope, containing definitions that are stored as infix.

The addition of a symbol is done by inserting them in a somewhat nonsensical name for a rewrite and its children. This means that printing a symbol table as a regular tree gives a somewhat unexpected result.

(lldb) xl symbols
nil tell host:text, code:tree as integer is builtin tell
( invoke host:text, code:tree as tree is builtin invoke
nil; reply code:tree as integer is builtin reply
( listen_forking as integer is builtin listen_forking
nil; nil); listen_hook hook:tree as tree is builtin listen_hook
( listen_received is listen_received
nil; nil); nil); ask host:text, code:tree as tree is builtin ask
( listen_on port:integer as integer is builtin listen_on
nil; nil); listen as integer is builtin listen
nil; nil

This is for an entry that really contains the following definitions:

xl symbols.pointer
tell host:text, code:tree as integer is builtin tell
invoke host:text, code:tree as tree is builtin invoke
reply code:tree as integer is builtin reply
listen_forking as integer is builtin listen_forking
listen_hook hook:tree as tree is builtin listen_hook
listen_received is listen_received
ask host:text, code:tree as tree is builtin ask
listen_on port:integer as integer is builtin listen_on
listen as integer is builtin listen

The lookup is O(log(N)) by selecting which branch to follow using a tree hash.

I believe that it's possible to store the entries without the nil, simply balancing between left and right as you need.

The XL documentation now gives a description of the error type and how they are processed.

Implementing this involves:

• Defining the error type.
• Recognizing the error type during evaluation of statements, and propagating it out.
• Recognizing compile_error and emit them at compile time.
• Implementing error parameter passing, and from this, try...catch

Implement type expressions as documented.

The most important types are:

• T1 or T2, e.g. for T or error (aka mayfail T)
• T1 and T2, mostly for traits, e.g. number and ordered
• not T, mostly for rules that exclude a given set of type, e.g. ln X:not positive is error "Log of negative"

Internally, the compiler currently generates very basic union types using the | operator.

Ideally, we would be able to implement all this in the library only, once the type interface (issue #11) is properly defined. For example:

type T1 or T2 is
    contains Value is T1.contains Value or T2.contains Value

There is basic support for colorization of Tao3D-style XL in Highlight.js.

The recent language updates have not been incorporated into this file, notably:

• is as the rewrite operator
• procedure, to, function, type, etc. as syntactic sugar keywords
• The XL modules are not known. But should we colorize the modules based on their name, or rather on the fact that they follow an import or use?
• Defining what we colorize as "keywords", what we colorize as "modules", etc.

In XL, type inheritance is defined by the existence of an implicit conversion.

This has some impact on dynamic dispatch (issue #13).

Also, automate the generation of the implicit conversion in the case of data inheritance.

Some functions have been already removed from the runtime, but it would be interesting to revisit how many of them can be removed completely.

Have you seen https://www.youtube.com/watch?v=A3r0cYRwrSs?

Do you think this would help you any in your quest to get a working implementation?

Define the precise interface of a type, and how it interacts with the compiler.

There is the beginning of a very crude definition of what a type is. It needs to be fleshed out, and to interface correctly with both the interpreter and the compiler.

There will be some "meta" connections here, since type is itself a type, so I'm afraid the full definition cannot be given solely in XL without resorting to a builtin trick along the way.

There are only a handful of runtime functions tagged using XL_NATIVE at the moment. More functions should be tagged, which might expose the need to add a few more types in the native interface.

The documentation used to refer to a pattern-based type definition using the type Pattern notation. For example, complex would be defined as

complex is type complex (Re:real, Im:real)

This notation is a bit confusing since it does not explain why something is a type. Also, I would like to be able to use type (Expression) to return the type of an expression (although that might be typeof)

The documentation now uses matching for that usage. With syntactic sugar, the recommended way to describe the complex type above would now be:

type complex is matching complex(Re:real, Im:real)

The interpreter and compiler should be updated to match.

With LLVM 9.0.1, there is the following warning:

llvm-crap.cpp: In constructor ‘XL::JITPrivate::JITPrivate(int, char**)’:
llvm-crap.cpp:560:20: warning: ‘llvm::orc::LegacyRTDyldObjectLinkingLayer::LegacyRTDyldObjectLinkingLayer(llvm::orc::ExecutionSession&, llvm::orc::LegacyRTDyldObjectLinkingLayer::ResourcesGetter, llvm::orc::LegacyRTDyldObjectLinkingLayer::NotifyLoadedFtor, llvm::orc::LegacyRTDyldObjectLinkingLayer::NotifyFinalizedFtor, llvm::orc::LegacyRTDyldObjectLinkingLayer::NotifyFreedFtor)’ is deprecated [-Wdeprecated-declarations]
  560 |       moduleHandle()
      |                    ^
In file included from /usr/include/llvm/ADT/APInt.h:18,
                 from /usr/include/llvm/ADT/APFloat.h:19,
                 from /usr/include/llvm/IR/Type.h:17,
                 from llvm-crap.h:55,
                 from llvm-crap.cpp:40:
/usr/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:356:3: note: declared here
  356 |   LLVM_ATTRIBUTE_DEPRECATED(
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~
llvm-crap.cpp:560:20: warning: ‘llvm::orc::LegacyIRCompileLayer<BaseLayerT, CompileFtor>::LegacyIRCompileLayer(BaseLayerT&, CompileFtor, llvm::orc::LegacyIRCompileLayer<BaseLayerT, CompileFtor>::NotifyCompiledCallback) [with BaseLayerT = llvm::orc::LegacyRTDyldObjectLinkingLayer; CompileFtor = llvm::orc::SimpleCompiler; llvm::orc::LegacyIRCompileLayer<BaseLayerT, CompileFtor>::NotifyCompiledCallback = std::function<void(long unsigned int, std::unique_ptr<llvm::Module>)>]’ is deprecated [-Wdeprecated-declarations]
  560 |       moduleHandle()
      |                    ^
In file included from llvm-crap.cpp:168:
/usr/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h:136:1: note: declared here
  136 | LegacyIRCompileLayer<BaseLayerT, CompileFtor>::LegacyIRCompileLayer(
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
llvm-crap.cpp:560:20: warning: ‘llvm::orc::LegacyIRTransformLayer<BaseLayerT, TransformFtor>::LegacyIRTransformLayer(BaseLayerT&, TransformFtor) [with BaseLayerT = llvm::orc::LegacyIRCompileLayer<llvm::orc::LegacyRTDyldObjectLinkingLayer, llvm::orc::SimpleCompiler>; TransformFtor = std::function<std::unique_ptr<llvm::Module>(std::unique_ptr<llvm::Module>)>]’ is deprecated [-Wdeprecated-declarations]
  560 |       moduleHandle()
      |                    ^
In file included from llvm-crap.cpp:169:
/usr/include/llvm/ExecutionEngine/Orc/IRTransformLayer.h:120:1: note: declared here
  120 | LegacyIRTransformLayer<BaseLayerT, TransformFtor>::LegacyIRTransformLayer(
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The test suite is currently somewhat weak with respect to testing error cases and validating the resulting error messages.

The test suite should probably move tests that check error cases into a separate directory.

It might be interesting to have a binary tree to hold arbitrary (possibly typed) binary data in the parse tree. This could also be represented as a very large integer, which could use the base-16 or base-64 notation.

Example:

some_data is bits 16#FFFF_FFFE_FFFD_FFFC_FFFB__FFFA_FFF9_FFF8_FFF7_FFF6_FFF5_FFF4_FFF3_FFF2_FFF1_FFF0

Currently, the above does not work because the integer value is too big. There is probably a need for specific prefixes (e.g. bits above), which could also serve as an elementary type for the value.

This could be implemented through issue #27.

The documentation now describes scoping rules which are currently, for the most part, not implemented.

The Tao3D compiler was notoriously bad at scoping, sometimes leaking definitions in the enclosing scope in order to workaround limitations with the LLVM code generation used at the time. Nesting should now be possible, although some special care must be taken with respect to how variables from the enclosing context are accessed by the inner context. I believe that LLVM should be supporting that since the dialect of C / C++ that GNU support does support nested functions, but I have not studied yet how this is represented at the LLVM IR level yet.

Symbol lookup is presently relatively well scoped, but additional testing is needed. Nested scope might even be working correctly in the interpreter.

Because XL is fundamentally kind of functional, it seems uneasy to simulate continue from imperative loops. Same goes for break (one could use return instead, but that'd work only without nesting I suppose which renders it completely unusable).

Any plans to provide break is builtin jmp_right_right_after_nearest_loop and continue is builtin jmp_to_the_nearest_loop_label?

The current for loop definition is broken with recent evolutions of the language. This is the root cause for issue #4 and many other similar failures. The bottom line is that in the current implementation, the loop variable is unusable.

There was no way to properly inject a name in the body scope, so the loop was really "hacked" into the compiler in the historical Tao3D compiler, and did a number of things that are not as easy to do with the optimizing compiler, like having C++ code modify the values of variables directly.

The correct definition is now described in the documentation:

for N:name in R:[range of discrete] loop Body is 
    loop_context is 
        [[N]] : R.type := R.first 
    LoopVar is loop_context.[[N]] 
    while LoopVar <= R.last loop
        (loop_context) (Body) 
        ++LoopVar 

It uses a number of things that are not yet implemented and may need further clarification:

• The N:name notation to match a name in the parse tree should work even with the new type system, but was only tested in the old one.
• The [range of discrete] notation uses the range and discrete types, neither of which is presently implemented.
• The creation of a context like loop_context is only partially supported, and may need rework of the symbol table to fulfil its potential.
• The notation LoopVar is loop_context.[[N]] remains to be checked for feasibility. This usage of such an alias declaration would imply to be able to write into the alias. But so far, a lot of the documentation assumes that is can generally be implemented by generating a function. It's unclear if allowing an is definition to be used to write by default is a good idea. Maybe variable LoopVar is loop_context.[[N]] would be useful here.
• The (loop_context)(Body) notation injects a scope into Body, and is used to clarify that we are not looking for a loop_context prefix, but for a scope. Things should work without parenthese. Such use of declaration-only bodies as maps is documented but not implemented yet.
• The ++LoopVar notation uses a general increment operator for all discrete types that is not implemented yet.

With decimal float literals finally making it into C23 (implemented 2021 and formally adopted in 2022) and seeing XL making significant changes in its basics incl. float literals, I'd like to propose making decimal float literals the default in XL. C++ already has decimal floats as an optional extension since 2011 if I'm not mistaken.

To use binary float literals (e.g. to double the performance and unpredictably lose precision) one can easily cast a decimal float literal to a binary one. Note, it's impossible to do the other way around (cast a decimal float lit. to a binary float lit.) if we wanted to maintain the precision the user wrote the float literal with.

Implement creation and destruction.

This involves automated calls to

• create calls
• delete calls