elm-in-elm / compiler Goto Github PK
View Code? Open in Web Editor NEWElm compiler written in Elm
Home Page: https://elm-in-elm.github.io/compiler/
License: BSD 3-Clause "New" or "Revised" License
Elm compiler written in Elm
Home Page: https://elm-in-elm.github.io/compiler/
License: BSD 3-Clause "New" or "Revised" License
cool project!!
in the readme, it mentions this project unlocks new kinds of Elm applications, like Slack bots. i don't understand!
wouldn't it already be feasible to write a Slack bot in Elm, compile it with the standard compiler, and run the JS in node? what am i missing?
Should be similar to how Plus
is done. Keep it a separate case in the Frontend.Expr
- later we can try to think of a way to abstract all binary operators into something dynamic, read from source instead of baked into the compiler.
myStuff :: [1, 2]
should become [myStuff, 1, 2]
[1,2,3]
do something like List$fromJSArray([1,2,3])
?(::)
should be just a call of Elm function with those two params(EDITED after comments from Jakub)
Ideally we'd like to create an elm-in-elm/core
package that does everything elm/core
does but in pure Elm (without kernel modules). That won't be entirely possible but we can try and minimize the amount of "magic" Elm needs the underlying platform to support.
Go through all the kernel modules in elm/core
and create a table, say in notes/elm-core.md
or xls
or whatever, documenting which of those can be done in pure Elm. ("Don't know" is OK too!)
In case of any doubts, ask here / in your PR / on Discord!
A compiler needs to read source files and write output files. Practical compilers will also read and write a number of intermediate files, caching build artifacts for incremental compilation reasons.
Elm code cannot interact with the file system; we have to use ports for that. However, the elm code does have to tell the javascript subscribed to the port where to save the file to.
We need to be able to join paths (i.e. add '/' or '' depending on unix/windows), check if files are the same, get the file name from a path, get paths relative to other paths. I think the best way to achieve this is to write an elm-in-elm/path
package.
This means:
InferTypesTest.elm
in the test project. (You can skip that if it's already done. #39 might do that)typedExprFuzzer : Type -> Fuzzer Expr
function. This should generate a random expression that conforms to the type.fuzzInference : Type -> Test
function. This should use typedExprFuzzer
to test that the random expressions have the input type inferred.fuzzInference someTypeToCheck
. You can put them in a nested describe
for grouping.The idea is that generating expressions that fit a type should be easier than inferring the type of an expression.
If you get stuck on anything, ask here or on Discord! We'll help you ๐
Kernel functions are defined in JavaScript (no static typing) but can be called from elm (static typing) and thus have some sort of implicit type signature.
For example the kernel function
var _Basics_remainderBy = F2(function(b, a) { return a % b; });
is called in elm
https://github.com/elm/core/blob/665624859a7a432107059411737e16d1d5cb6373/src/Basics.elm#L533-L535
remainderBy : Int -> Int -> Int
remainderBy =
Elm.Kernel.Basics.remainderBy
and so we deduce that a
and b
must have type Int
and the function returns an Int
*.
However, other functions have much stranger signatures. For example, what is the type signature for stepperBuilder
here?
function _Platform_initialize(flagDecoder, args, init, update, subscriptions, stepperBuilder)
It is far from clear. Looking at the use of stepperBuilder
var stepper = stepperBuilder(sendToApp, model);
stepperBuilder is function that takes two arguments (which is not allowed in elm) but what are the types of those arguments? What is the return type?
If elm-in-elm is able to implement kernel code for targets other than javascript, or move logic from "kernel space" into "elm space" then we first need to solidly document all these implicit type signatures.
I am proposing an effort to compliment the documentation written as part of #57 with type signatures.
String
, List
etc to this function. There is no checking that the javascript function actually returns an Int
. This is why folk see 96 \= 0
having a type of Bool
and a value of 0
(elm/core#1022).Order of optimizations matters.
It should be possible to look at the list of optimizations in Stage.Optimize.optimizeExpr
and either experimentally or in some more scientific way find a better permutation of them.
This will probably need a corpus of programs or an expression fuzzer to evaluate the different orders on.
The result of this issue should be something reproducible, runnable on different inputs and with different set of optimizations to choose from. (ie. if we add a new optimization pass, we should be able to run this to find a new optimal order.)
This Reddit comment in particular seems like a great place to read up on various ways to do this, should you want to.
let
x = 1
in
2 + x
can become
2 + 1
(which the plus
optimization will then pick up and create 3
from it).
Let's keep it at let
inlinings, not top-level inlinings just yet.
It makes sense to me to optimize piece-wise first, then try to inline the optimized stuff, then try optimizing again after inlining. That probably can't be done with what we have now - we'll have to split optimization stage into three: "piece-wise before inlining", "inlining", "piece-wise after inlining".
The workflow deploy-to-gh-pages.yml is referencing action JamesIves/github-pages-deploy-action using references 3.4.9. However this reference is missing the commit 661e33871bce8107c9af083f7415176608dce40e which may contain fix to the some vulnerability.
The vulnerability fix that is missing by actions version could be related to:
(1) CVE fix
(2) upgrade of vulnerable dependency
(3) fix to secret leak and others.
Please consider to update the reference to the action.
Technically an argument is what you pass in, and the parameter defines the var in the function definition. I know people use these as synonyms, but it might make some code cleaner when you can distinguish parameter names (i.e. Lambda) from args (i.e. Call).
Records shouldn't support multiple fields of the same name:
{ a = 123
, a = ()
}
Official Elm compiler says:
This record has multiple `a` fields. One here:
12| { a = 123
^
And another one here:
13| , a = ()
^
How can I know which one you want? Rename one of them!
Currently we don't raise any error and silently drop all but the last field when we transform the List into Dict. Return a DesugarError instead!
Currently if you try to put
module Main exposing (main)
main =
let
x = 2
in
1 + x
into example-project/src/Main.elm
and run it with make
, it complains about x
not being visible in the body. Make it so!
Depending on where the fix is, it might be a good idea to also add a regression unit test somewhere.
Feel free to discuss/ask how to solve this on our Discord!
Add parser tests for custom types:
type Foo
= Bar
| Baz Int
| Quux (Foo, Foo)
This will most likely not go through Frontend.Expr
(as those aren't expressions), but somewhere alongside top-level definitions. Maybe TopLevelDefinition needs to distinguish between expressions and custom types / aliases / port definitions / ...?
This means:
Blocked by #27
Hi, I'm looking forward to use parts of your libraries (AST, type infer, optimize) to implement my elmsl compiler (elm serverless), which will be implemented in rust and create rust code.
This means:
1.2
? 1.
? scientific notation like 2e10
? maybe stuff like NaN
, Infinity
etc? (probably not))... go see official Elm's parser to be sure!When we try to infer the type for the list [ [ "" ], [ "" ] ]
we would expect to get List (List String)
. Instead we get an error complaining about not bein able to unify two type variables:
โ InferTypesTest
โ Stage.InferType
โ fuzz exprInfer
โ List List String
Given List [List [Literal (String "")],List [Literal (String "")]]
Err (TypeMismatch (List (Var 7)) (List (Var 6)))
โท
โ Expect.equal
โต
Ok (List (List String))
It seems like we might be missing some equations. Try looking in the list case inside Stage.InferTypes.GenerateEquations.generateEquations
.
Fix the string parser implementation such that the string tests pass.
It probably will be a good idea to separate (with P.oneOf
) the string parser into normal "..."
and multiline """..."""
string parsers.
Also, use the unicode and escaping logic that's already in the Char parser!
Elm.Compiler.parseModule
produces a parse error on the following valid Elm code:
module Target exposing (..)
something =
{ a = 1
, b = 2
}
The error being:
Err
(ParseError
(ParseProblem
[ { col = 1
, contextStack = []
, problem = ExpectingModuleKeyword
, row = 1
}
, { col = 1, contextStack = [], problem = ExpectingPortKeyword, row = 1 }
, { col = 1, contextStack = [], problem = ExpectingEffectKeyword, row = 1 }
]
)
)
SSCCE:
module Main exposing (main)
import Elm.Compiler
import Html
code =
"""
module Target exposing (..)
something =
{ a = 1
, b = 2
}
"""
main =
{ filePath = "Target.elm"
, sourceCode = code
}
|> Elm.Compiler.parseModule
|> Debug.toString
|> Html.text
We'd like to have some performance tracking over time, similar to https://arewefastyet.com/
Probably don't bother with setting up a webpage that will hold the historic data and add to them - as a first iteration, let's just have a script (possibly in a different repo) that:
terezka/line-charts
/ elm-plot
/ ...? Feel free to choose the easiest solution) or at least a table with the numbers.We'll need:
elm-in-elm
on each input - possibly this:
The output files should also have some metadata about the machine they were run on (Node version, OS, CPU, memory, etc?)
We can hook you up with a new repository (say elm-in-elm/benchmarks
) when needed.
This means:
List (List Expr)
constructor Frontend.Expr
type... solve any compiler errors that arise as a result (emitExpr
etc)[]
, [ ]
[1]
, [ 1 ]
[1,2]
, [ 1, 2 ]
[A,B,C]
do equals typeA typeB, equals typeA typeC
. Don't have to do all the combinations. This will leave us to do the type of the actual list itself in a later issue after we implement type parameters.[ 1, 2, 3 ]
โก๏ธ JavaScript's [1, 2, 3]
[]
โก๏ธ []
If you get stuck on anything, ask here or on Discord! We'll help you ๐
(::)
operator(++)
operatorThis means:
"""abc"""
)"{\"foo\":1}"
vs """{"foo":1}"""
)Some of the tests should fail - not implemented yet.
Fixing the parser is not needed in this ticket (can be in a separate one)
See notes/AssignIdsAbstraction.elm for some notes on a possible solution.
Essentially this function is giving us trouble - in case of recursive calls it nests Result.andThen
s and it doesn't look very nice. That's a consequence of doing the monadic style with no do
notation in the language (which would essentially make it not indent further and further to the right).
notes/AssignIdsAbstraction.elm is an attempt to imagine a better API than Result.andThen
. I'm not yet sure if something like this is possible, but we can try! ๐ช
For some more context, the underlying problem is turning stuff like this (ignore the nonsense AST)
If
{ test = Plus (Literal (Int 1)) (Literal (Int 2))
, then_ = Unit
, else_ = Literal (Bool True)
}
into
( If
{ test =
( Plus
( Literal (Int 1)
, Var 0
)
( Literal (Int 2)
, Var 1
)
, Var 2
)
, then_ = ( Unit, Var 3 )
, else_ = ( Literal (Bool True), Var 4 )
}
, Var 5
)
ie. giving every subexpression an unique ID.
This module here parses pretty much all types already. Why not re-use the code?
Should be similar to how Plus
is done. Keep it a separate case in the Frontend.Expr
- later we can try to think of a way to abstract all binary operators into something dynamic, read from source instead of baked into the compiler.
[1, 2] ++ [3, 4]
should become [1, 2, 3, 4]
[1] ++ [2]
should become [1].concat([2])
stuff ++ [1]
should become stuff.concat([1])
[1] ++ stuff
should become [1].concat(stuff)
(++)
for stringsRight now type variables get emitted this way:
Type.Var 123
-> "t123"
It would be great to print them like a
and b
, like we're used to from the official Elm compiler.
Note that toString
for one Type
is not enough. In the error messages you sometimes compare two types together, so something like toStringHelper : Dict Int String -> Type -> (String, Dict Int String)
might be needed to allow you to thread names from one type into the other.
Eg. if the first Type is Function (Var 1) (Function (Var 2) Int)
and the second one is Function (Var 2) Bool
, they should be consistent with each other: a -> b -> Int
and b -> Bool
(not a -> Bool
, which would probably happen if you'd just call toString
without context and without threading the Dict through)
Hey. Cool project!
I wanted to play with it but wanted to know if you could add emit test. It would be great if it just feeds the elm code string and asserted that result matched expected.
I currently don't have a clear idea on how to do this. I'd start with this:
number
type to Type
number
s instead of straight-up Int
snumber
and Int
/Float
behaviour (maybe in the unification phase?)
Int
s can be used in place of number
sFloat
s can be used in place of number
sThis means:
\u{...}
? \x{...}
? straight-up emoji in the source code? something else?)Implement parser for tuples and 3-tuples so that the tests created in #34 are green! ๐
I created a fresh Nix installation, and attempted to build the project, getting an elm version mismatch error. Am I missing something?
[nix-shell:~/dev/compiler]$ make
rm -rf build/elm.js elm-stuff
cd cli && npx elm make Main.elm --output ../build/elm.js
-- ELM VERSION MISMATCH ----------------------------------------------- elm.json
Your elm.json says this application needs a different version of Elm.
It requires 0.19.1, but you are using 0.19.0 right now.
make: *** [Makefile:26: build] Error 1
[nix-shell:~/dev/compiler]$
()
should be a valid Elm syntax. What we emit it as is probably not that important right now (we will find out later!) - let's emit as {"type": "unit"}
That means we need to implement:
No optimizations needed (or at least I can't think of any), no desugaring needed either.
The compiler errors will probably guide you ๐ Ask here or on Discord if anything is unclear!
Jakt is an imperative language (from the SerenityOS project) that compiles to C++ (and thus to native binaries), with a few charaacteristics that make it nice to use from Elm backend perspective:
malloc
and free
needed)Here's a translation example of an ADT:
type Foo a
= Bar
| Baz a
| Quux
{ x : String
, y : Int
}
| Quz Int a
โฌ๏ธ
enum Foo<A> {
Bar
Baz(A)
Quux (
x: String
y: i64
)
Quz(a1: i64, a2: A)
}
I am writing this up as an issue just because it seemed like a good place to write up something kind of long form. It's sort of an enhancement request, but I don't even want it to be as strongly worded as "request"; it's more just a data point.
So meta-elm is my project, and it's an expression evaluator for an AST that is similar in nature to elm-in-elm's AST. There's a pretty close-to-1:1 mapping between the two.
I have another project called mini-elm that uses elm-in-elm to parser Elm expressions and then hands off to meta-elm to parse them.
Right now meta-elm is strong in these areas:
It's also missing lots of very fundamental stuff like strings, records, custom types, etc. Also, it never intends to do things like parsing or deep type checking. For that it wants to play nice with elm-in-elm!
I know that elm-in-elm's evolution is gonna be driven 99.99% of the time by its own concerns, and not meta-elm. And of course that's reasonable. But to the extent I can influence things, here are my would-be-nice things:
|>
): I would really like this! It just makes it a lot easier to write sample code.For meta-elm the only thing I care about at the moment is front-end parsing. So if I were king of the world, I'd say just parse |>
and don't worry about de-sugaring it. (It might be good to have a dev process for that kinda stuff, anyway--be able to write parsers and have downstream stages just gracefully deal with "unknown" things--that probably already sorta happens?).
I am happy to elaborate on any of this in Discord; I just thought it's good to write it up here to avoid repeating myself.
For what should be in them: don't check that it "does nothing". Check the interesting stuff that happens in the desugarExpr
function. Eg.:
\a b -> c
becomes \a -> \b -> c
With Pratt parsing, a common bug is that you can successfully parse f a b c
and x < 5
, but you can't parse f a b c < 5
. We should try adding tests that put somewhat complicated expressions on either sides of the operators. I quickly skimmed the tests, and I don't believe the tests do much coverage of edge cases.
If you add tests and they pass, then great, leave them in, and we'll have more robust coverage.
If tests fail, either try to make a quick fix (if possible) or open an issue and/or discuss on Discord.
Would a PR adding elm format tooling be accepted?
I guess with the current tooling that should mean:
nix-shell
inside the project and get a shell with elm
, elm-format
, elm-test
, node
(v10+) and make
ready to use. (Bonus points for elm-analyse
)If you know more about Nix (I know awfully little about it) and think it should / can do more, I'm all ears!
We have a whole lot of types like:
Elm.Data.Exposing.Exposing
Elm.Data.ModuleName.ModuleName
where there is a module fundamentally there to define a type and some helper functions. My idea would be to rename the type defined by this module to T
. Therefore we would have:
Elm.Data.Exposing.T
Elm.Data.ModuleName.T
and we would not have to write import Elm.Data.Exposing exposing (Exposing)
but could instead use Elm.Data.Exposing.T
directly. Definitely, cons to this idea but I thought worth throwing it out.
We have multiple elm.json
files and the versions of the dependencies are not the same.
Find a solution that complains in CI when they get misaligned.
(At work we have a Haskell script that checks the various versions and tells you the discrepancies; I've talked to @jhrcek about open-sourcing it ๐ I'll update this issue if/when that happens.)
This means:
Tuple (Expr, Expr)
and Triple (Expr, Expr, Expr)
constructors to Frontend.Expr
type... solve any compiler errors that arise as a result (emitExpr
etc)(1,2)
, ( 1, 2 )
(1,2,3)
, ( 1, 2, 3 )
(A, B)
. (Yeah, you'll have to create new Type
constructors to account for this)(A, B, C)
(1, 2)
โก๏ธ JavaScript's [1, 2]
, similarly with the 3-tuplesemitExpr
) to pass the emit tests aboveIf you get stuck on anything, ask here or on Discord! We'll help you ๐
Triple
Running make
for the first time in the root directory of this repo:
Compiled in DEV mode. Follow the advice at https://elm-lang.org/0.19.0/optimize for better performance and smaller assets.
---------------------------
-- STARTING THE COMPILER --
---------------------------
(node:4519) ExperimentalWarning: The fs.promises API is experimental
Main.main: Let { bindings = [{ body = Literal (Int 1), name = VarName "x" }], body = Var { name = VarName "x", qualifier = Nothing } }
---------------------------
-- COMPILER ERROR ---------
---------------------------
Can't find the variable `x` in the module `Main`. Have you imported it?
I'm using node v10.15.3
and elm 0.19.0
.
Standard elm uses a syntax borrowed from Haskell to define new operators.
I propose that elm-in-elm should hard code operators into the compiler. The main advantage of this is approach is that we avoid having to define a kernel function for each arithmetic operator: instead the compiler can generate code directly*.
Only standard packages can define operators and they do so very rarely so I think it would be feasible to handle all elm operators in the compiler.
*: The standard compiler also generates code directly as an optimsation.
Given
module Meh exposing (..)
meh = \a b -> 3 + a
someFunction = meh 5
The correct inferred type for someFunction
should be a -> Int
but instead is a
.
ie, the parsed AST declarations are:
[ { body =
Value
( Lambda
{ argument = "a"
, body = ( Lambda { argument = "b", body = ( Plus ( Int 3, Int ) ( Argument "a", Int ), Int ) }, Function (Var 6) Int )
}
, Function Int (Function (Var 6) Int)
)
, module_ = "Meh"
, name = "meh"
}
, { body =
Value
( Call
{ argument =
( Int 5
, Int
)
, fn = ( Var { module_ = "Meh", name = "meh" }, Function Int (Var 2) )
}
, Var 2
)
, module_ = "Meh"
, name = "someFunction"
}
]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.