The metalua programming language
Home Page: http://metalua.luaforge.net
Metalua is a Lua code analysis tool, as well as a compiler for a
superset of Lua 5.1 supporting Compile-Time Meta-Programming. It's
separated into two LuaRocks, metalua-parser and
metalua-compiler. The documentation of each rock can be found in
README-parser.md and README-compiler.md.
All the code in Metalua is released under dual lincenses:
Hi,
Trailing coma or semi-colon are no longer handled by Metalua.
M> local ast = mlc.luastring_to_ast('local t = {0,}') local status, error = mlc.check_ast(ast) table.print(ast, 'nohash', 1)
{ `Local{ { `Id "t" },
{ `Table{ `Number "0",
`Error "line 1, char 14: An expression was expected, and `}' can't start an expression\n>>> local t = {0,}\n>>> ^" } } } }
There are still supported by Lua
$ lua
Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio
> local t = {0,}
>
Hi,
I use Metalua to explore a Lua AST and I would like to traverse every node.
Unfortunately I do not manage to traverse `Pairs nodes.
I've written the following code to explain.
local traverse = function(node, ...)
-- Just print node's type
if node and node.tag then print(node.tag) end
end
local cfg = {
binder = traverse,
block = { up = traverse },
expr = { up = traverse },
stat = { up = traverse },
Pair = traverse
}
require 'metalua.compiler'
local ast = mlc.luastring_to_ast("table={p=1}")
require 'metalua.walk'
walk.block(cfg, ast)
Here is my output
Id
String
Number
Table
Set
I'm on branch LuaEclipse
Is it a bug? Am I doing it the wrong way?
Regards
The website for metalua has been non-responsive for months now...
For example, I can't run with the example in quick tour:
---------------------------------------------------------
-- the "-{...}" means that we're going to do compile-time
-- stuff (here, syntax extension)
---------------------------------------------------------
-{ block:
------------------------------------------------------
-- Register the additional keywords in mlp.lexer
------------------------------------------------------
mlp.lexer:add{ "let", "in" }
------------------------------------------------------
-- Extend the expression parser; code generation is
-- delegated to the function let_in_builder() below.
------------------------------------------------------
mlp.expr:add{
"let", mlp.id, "=", mlp.expr, "in", mlp.expr,
builder = let_in_builder }
------------------------------------------------------
-- This creates the code returned by the macro.
-- Notice the holes-in-quote-in-splice.
------------------------------------------------------
local function let_in_builder (x)
local variable, value, expr = unpack (x)
return +{
function (-{variable})
return -{expr}
end (-{value}) }
end
} -- back to "normal" code
a, b, c = 1, 1, -2
roots = let sqrt_delta = (b^2-4*a*c)^0.5 in
{ (sqrt_delta-b)/(2*a), (-sqrt_delta-b)/(2*a) } It fails with these error messages:
C:\Program Files (x86)\LuaRocks\lua5.1: splice:9: attempt to index global 'mlp' (a nil value)
stack traceback:
splice:9: in function 'f'
...stree/share/lua/5.1/metalua\compiler\parser\meta.lua:41: in function <...stree/share/lua/5.1/metalua\compiler\parser\meta.lua:37>
(tail call): ?
(tail call): ?
...\systree/share/lua/5.1/metalua\grammar\generator.lua:113: in function 'raw_parse_sequence'
...\systree/share/lua/5.1/metalua\grammar\generator.lua:216: in function <...\systree/share/lua/5.1/metalua\grammar\generator.lua:212>
(tail call): ?
(tail call): ?
...\systree/share/lua/5.1/metalua\grammar\generator.lua:332: in function <...\systree/share/lua/5.1/metalua\grammar\generator.lua:330>
(tail call): ?
...
(tail call): ?
...stree/share/lua/5.1/metalua\compiler\parser\misc.lua:164: in function 'e'
...\systree/share/lua/5.1/metalua\grammar\generator.lua:113: in function 'raw_parse_sequence'
...\systree/share/lua/5.1/metalua\grammar\generator.lua:216: in function <...\systree/share/lua/5.1/metalua\grammar\generator.lua:212>
(tail call): ?
...\LuaRocks\systree/share/lua/5.1/metalua\compiler.lua:104: in function 'f'
...\LuaRocks\systree/share/lua/5.1/metalua\compiler.lua:153: in function 'srcfile_to_ast'
...e\lib\luarocks\rocks-5.1\metalua\0.7.3-1\bin\metalua:162: in function <...e\lib\luarocks\rocks-5.1\metalua\0.7.3-1\bin\metalua:105>
(tail call): ?
[C]: ?
Lots of examples shown in the website fail, I think it's because of the older documents. mlp, clist do not exist for now.
Running the following I get:
M> ast = mlc.luastring_to_ast([[local s ='\ ']])
Evaluation error:
lexer.lua:284: Unknown escape sequence '\ '
stack traceback:
[C]: in function 'error'
lexer.lua:284: in function <lexer.lua:279>
[C]: in function 'gsub'
lexer.lua:290: in function 'unescape_string'
lexer.lua:410: in function '?'
lexer.lua:340: in function 'extract'
lexer.lua:508: in function 'peek'
lexer.lua:584: in function 'lineinfo_right'
gg.lua:604: in function 'parse'
gg.lua:51: in function <gg.lua:50>
(tail call): ?
...
gg.lua:51: in function 'chunk'
compiler/mlc.mlua:133: in function 'f'
compiler/mlc.mlua:165: in function 'luastring_to_ast'
stdin:1: in main chunk
[C]: in function 'xpcall'
/d/metalua/build/lib/metalua/metaloop.mlua:56: in function 'run'
compiler/metalua.mlua:251: in function 'main'
compiler/metalua.mlua:258: in main chunk
(tail call): ?
[C]: ?Whereas, lua interpreter accepts it.
Hi,
With the following source:
(This is blank line )
#!/usr/bin/lua
returnRunning lua I get:
$ lua /tmp/main.lua
lua: /tmp/main.lua:2: unexpected symbol near '#'
I know provided code is not valid, but it would be nice to terminate more nicely.But running Metalua, I get:
$ metalua /tmp/main.lua
Cannot compile `File "/tmp/main.lua":
mlp_misc.lua:172: attempt to perform arithmetic on field 'line' (a nil value)When searching for variable declarations with bindings, I get an loop error when ther are "..." in the parsed source
require 'bind'
require 'metalua.walk.bindings'
local declared, leftovers = bindings( mlc.luastring_to_ast("sample = function( ... ) end") )
.../Bureau/metalua/build/lib/metalua/walk/bindings.mlua:17: table index is nil
stack traceback:
.../Bureau/metalua/build/lib/metalua/walk/bindings.mlua:17: in function 'f'
...e/kkinfoo/Bureau/metalua/build/lib/metalua/walk.mlua:292: in function <...e/kkinfoo/Bureau/metalua/build/lib/metalua/walk.mlua:290>
(tail call): ?
...e/kkinfoo/Bureau/metalua/build/lib/metalua/walk.mlua:226: in function 'traverse'
...e/kkinfoo/Bureau/metalua/build/lib/metalua/walk.mlua:275: in function 'expr'
...e/kkinfoo/Bureau/metalua/build/lib/metalua/walk.mlua:249: in function 'traverse'
...e/kkinfoo/Bureau/metalua/build/lib/metalua/walk.mlua:275: in function <...e/kkinfoo/Bureau/metalua/build/lib/metalua/walk.mlua:259>
(tail call): ?
...e/kkinfoo/Bureau/metalua/build/lib/metalua/walk.mlua:193: in function 'traverse'
...e/kkinfoo/Bureau/metalua/build/lib/metalua/walk.mlua:275: in function 'stat'
...e/kkinfoo/Bureau/metalua/build/lib/metalua/walk.mlua:243: in function 'traverse'
...e/kkinfoo/Bureau/metalua/build/lib/metalua/walk.mlua:275: in function <...e/kkinfoo/Bureau/metalua/build/lib/metalua/walk.mlua:259>
(tail call): ?
.../Bureau/metalua/build/lib/metalua/walk/bindings.mlua:39: in function 'bindings'
stdin:1: in main chunk
[C]: ?
Hi,
I'm on branch konekildt. I have notice that ASTs from Lua code which end with an error have an error node with faulty offset.
The offset of the error node is source length + 1.
M> source = "l = "
M> print( #source )
4
M> ast = mlc.luastring_to_ast( source )
M> table.print(ast, 1)
{ lineinfo = <?|L1|C1-5|K1-5>, `Set{ lineinfo = <?|L1|C1-5|K1-5>, { lineinfo = <?|L1|C1|K1>, `Id "l" },
{ lineinfo = <?|L1|C5|K5>, `Error{ lineinfo = <?|L1|C5|K5>,
error = true, "line 1, char 5: End of file reached when an expression was expected\n>>> l = \n>>> ^" } } } }
We can see that last column offset and last node offset are 5 but according to Lua file contains only 4 characters.
Cheers
Hi,
I'm working on luaeclipse branch.
When trying to compile a plain lua file I get:
M> =mlc.luafile_to_ast("/d/runs/spaces/spaces/src/errornode.lua")
Evaluation error:
Parsing error in @/d/runs/spaces/spaces/src/errornode.lua line 10, column 29, char 316:
interrupted!
stack traceback:
[C]: in function 'error'
compiler/mlc.mlua:142: in function <compiler/mlc.mlua:103>
(tail call): ?
(tail call): ?
[C]: in function 'xpcall'
/d/metalua/build/lib/metalua/metaloop.mlua:56: in function 'run'
compiler/metalua.mlua:251: in function 'main'
compiler/metalua.mlua:258: in main chunk
(tail call): ?
[C]: ?
Here is the file
local Q = require 'metalua.treequery'
local sources = {
notypo = "function() end",
onetypo = "function() typo end",
twotypos = "function() typo retypo end"
}
for sample, source in pairs(sources) do
local generated, ast = pcall(mlc.luastring_to_ast, source)
if generated then x
local status, message = pcall(mlc.check_ast, ast)
if not status then
print ('On sample: '..sample)
print (message)
else
local errornode = Q(ast):filter('Error'):first()
if not errornode then
print 'No error node in ast'
else
print 'Error node found'
table.print(errornode, 'nohash', 1)
end
end
else
print('No AST generated for '..sample)
end
end
Am I doing something wrong or did I find a bug?
with this code :
local toto
for var =1, 0 do
var = 2
end
for _ in toto() do
var()
var = var()
end
bindings returns an empty list for the unbound variables
Hi,
I'm on branch luaeclipse and I use the new getast function from errnode.lua.
Due to some constraints, I'm force to call Metalua code from a lua context.
And after loading my metalua code, the getast function seems unable to deal with any reference to variable match.
Here is my sample:
kkinfoo@tourterelle:/d/LuaEclipse/plugins/org.eclipse.koneki.ldt.metalua.32bits/lib$ lua
Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio
> package.path = [[?.lua;?.luac]]
> require 'metalua.compiler'
> loadf = mlc.luafile_to_function('/d/LuaEclipse/plugins/org.eclipse.koneki.ldt.parser/scriptMetalua/dltk_ast_builder.mlua')
> loadf()
> require 'errnode'
> ast = getast('local match')
> table.print( ast, 80 )
`Error{ lineinfo = { last = { "1", "7", "7",
"Parsing error in <nofilename> line 1, column 5, char 5" },
first = { "1", "7", "7",
"Parsing error in <nofilename> line 1, column 5, char 5" } }, "Identifier expected" }
> ast = getast('local m')
> table.print( ast, 'nohash',80 )
{ `Local{ { `Id "m" }, { } } }
Before you ask for it, here is the Metalua file I load which may corrupt something. But there are no reference to match in it
require 'metalua.compiler'
require 'metalua.walk'
-{ extension 'match' }
-- Just redefining classic print, as there is a flush problem calling it from Java
local print = function (string) print(string) io.flush() end
---
-- Simple type printing visitor
--
local ind = 0
local sDown = function(node, ...) ind=ind+1; print(string.rep('| ', ind)..'Down on '..(node.tag or 'chunk')) end
local sUp = function(node, ...) print(string.rep('| ', ind)..'Up on '..(node.tag or 'chunk')); ind=ind-1 end
local simple={
binder = function(node, ...) sUp(node) sDown(node)end,
block = {up = sUp, down = sDown},
expr = {up = sUp, down = sDown},
stat = {up = sUp, down = sDown}
}
--
-- The real work
--
local module = {}
---
-- Initialize cache for Lua ast objects
--
-- While processing this table will contain
-- key: Metalua AST Nodes
-- values: Children key node
local hash = {}
---
-- Initialize cache for Java Objects
--
-- During process this table will associate a Metalua AST Node to a Java one
-- key: Metalua AST Node
-- value: Java DLTK Node
local object= {}
---
-- Contains Declaration Metalua nodes
--
-- Use to backpatch nodes which had not full descents at construction
-- key: Random numbers
-- value: Metalua AST Node
local declaration = {}
---
-- Provides Metalua AST Nodes offset
--
-- @param node Metalua AST Node
-- @return number Node's start offset
-- @return number Node's end offset
local function offsets(node)
local start = node.lineinfo and node.lineinfo.first[3] - 1 or 0
local _end = node.lineinfo and node.lineinfo.last[3] or 0
return start, _end
end
---
-- Converts a Metalua node list in a DLTK Chunk
--
-- Append list's Metalua nodes in a Java Chunk object using association available in <strong>object</strong> table.
-- @param list Table containing nodes at integer fields
-- @return Filled Java DLTK Chunk representing given <strong>list</strong> content
local function statListToChunk(list)
local chunk = DLTK.Chunk(0, 0)
for k, node in ipairs(list) do
DLTK.appendStatementToChunk(chunk, object[node])
end
return chunk
end
---
-- Converts a list of Metalua AST Nodes in Java DLTK CallArgumentsList
--
-- @param list Table gathering Metalua AST Nodes on integer values
-- @return Filled Java DLTK CallArgumentsList representing nodes in <strong>list</strong>
local function statListToCallArgList(list)
local clist = DLTK.CallArgumentsList(0, 0)
for k, node in ipairs(list) do
DLTK.appendNodeToCallArgumentList(clist, object[node])
end
return clist
end
---
-- Indexes nodes and relates then to their parents
--
-- <strong>This function is intended to be called from a visitor parsing an AST down.</strong>.
-- Fills <code>hash</code> table with Metalua AST Nodes as key and back patch parent node adding children to build a direct hirarchy between nodes.
--
-- @param node Processed Metalua AST Node
-- @param parent Metalua AST Node, parent of given node, allow to back patch parent children
-- @param parent ... Metalua AST Node, reprensenting grand-parent, grand-grand-parent and so on
local function down(node,parent, ...)
if not hash[node] then
hash[node] = {}
end
if parent then
table.insert(hash[parent], node)
end
end
---
-- Visitor that enables building DLTK ASTs from visting Metalua AST ones
--
-- The method is simple. Firstly, visit down the Metalua AST brearing in mind parenthood between nodes.
-- In order to do so, parse down fill <code>hash</code> using <code>down(node, parent, ...)</code> function.<br/>
-- Secondly, while parsing up, visitor intaciates Java DLTK object, storing them in <code>object</code> table and using <code>hash</code> to ensure
-- parenthood consistency.
local visitor = {
block = {down=down},
expr = {down=down},
stat = {down=down}
}
---
-- Common processing for nodes which are Expressions and Statements
--
-- While parsing up and creating Java objects, some node's types are both expressions and and statements. So, they could be processed identically in
-- <code>visitor.expr.up</code> and <code>visitor.stat.up</code>. Therefore, they are dealt with here, in order to factorise treatment.
-- <strong>This function uses pattern matching, it is intented to be called while visiting an AST.</strong>
--
-- @param node Metalua AST node
local function apply(node, ...)
local first, last = offsets(node)
match node with
| `Call {...} ->
-- Load param list
local clist = DLTK.CallArgumentsList(0, 0)
for child = 2,#node do
DLTK.appendNodeToCallArgumentList(clist, object[node[child]] )
end
object[ node ] = DLTK.Call(first, last, object[node[1]], clist )
| `Invoke {expr, string } ->
object[ node ] = DLTK.Invoke(first, last, object[expr], object[string])
| `Invoke {expr, string, ...} ->
local clist = DLTK.CallArgumentsList(0, 0)
for child = 3,#node do
DLTK.appendNodeToCallArgumentList(clist, object[node[child]])
end
object[ node ] = DLTK.Invoke(first, last, object[expr], object[string], clist)
end
end
---
-- Registers given node in <code>hash</code> and creates appropriate object in <code>object</table>
--
-- @param node Metalua AST Node to process
-- @param parent Given <code>node</code> parent
-- @param ... grand-parent, grand-grand-parent and so on
visitor.binder = function(node, parent, ...)
local first, last = offsets(node)
match node with
| `Id{name} ->
-- Use general indexation
down(node, parent, ...)
--Process as a Declaration
if mark.is_declaration( node ) then
-- Create appropriate declaration type
local type = mark.declaration_type (node) or 'undef'
local init = mark.declaration_initialization( node )
local name, iStart, iEnd = node[1]
if init then iStart, iEnd = offsets(init) end
if type == 'Function' then
-- Function Declaration
object[ node ] = DLTK.FunctionDeclaration(name, first, last, iStart, iEnd, mark.declaration_scope(node))
elseif type == 'Table' then
-- Table Declaration
object[ node ] = DLTK.TableDeclaration(name, first, last, iStart, iEnd, mark.declaration_scope(node))
elseif type == 'Nil' or type == 'Number' or type == 'String' or type == 'Boolean' then
-- Scalar Declaration
object[ node ] = DLTK.ScalarVariableDeclaration(name, first, last, iStart, iEnd, mark.declaration_scope(node))
else
-- Default Declaration
object[ node ] = DLTK.VariableDeclaration(name, first, last, mark.declaration_scope(node))
end
-- Register node for backpatch as Java object for parameters and body are still not available
table.insert(declaration, node)
else
-- Create Java Identifier
object[ node ] = DLTK.Identifier(first, last, name)
end
| `Dots ->
object[ node ] = DLTK.Dots(first, last)
| _ ->
-- print('In binder missing type `'..(node.tag))
-- table.print(node, 'nohash',1)
end
end
visitor.Dots = visitor.binder
visitor.Id = visitor.binder
---
-- Instanciates a Java object in <code>object</code> from the given node.
--
-- @param node Metalua node which will have its Java Object image stored in <code>object</code>
-- @param parent Given node parent
-- @param ... grand-parent, grand-grand-parent and so on
visitor.block.up =function(node, parent, ...)
local first, last = offsets(node)
match node with
| { tag=nil, ... } -> -- Dealing with Chunks
local chunk = DLTK.Chunk(first, last)
-- Append childern Java objects
if hash[node] then
for index, child in pairs(hash[node]) do
DLTK.appendStatementToChunk(chunk, object[child])
end
end
object[ node ] = chunk
| `Do { ... } ->
object[ node ] = DLTK.Do(first, last, statListToChunk(node))
| _ ->
-- print('In block missing type `'..(node.tag))
-- table.print(node, 'nohash',1)
end
end
---
-- Instanciates a Java object in <code>object</code> from the given node.
--
-- @param node Metalua node which will have its Java Object image stored in <code>object</code>
-- @param ... parent, grand-parent, grand-grand-parent and so on
visitor.stat.up = function (node, ...)
local first, last = offsets(node)
match node with
| `Set {left, right} ->
object[ node ] = DLTK.Set(first, last, statListToChunk(left), statListToChunk(right))
| `While {expr, block} ->
object[ node ] = DLTK.While(first, last, object[expr], object[block])
| `Repeat {block, expr} ->
object[ node ] = DLTK.Repeat(first, last, object[block], object[expr])
| `If { expr , block} ->
object[ node ] = DLTK.If(first, last, object[expr], object[block])
| `If { expr , block, alt} ->
object[ node ] = DLTK.If(first, last, object[expr], object[block], object[alt])
| `If { expr , block, ... } ->
local nodeSize = select("#", ...)
if (nodeSize % 2) == 0 then
object[ node ] = DLTK.ElseIf(first, last, object[expr], object[block])
else
local elseBlock = select(nodeSize, node)
object[ node ] = DLTK.ElseIf(first, last, object[expr], object[block], object[elseBlock])
end
for k =2,nodeSize,2 do
local cond, chunk = select(k-1, ...)
DLTK.addExpressionAndRelatedChunk(object[ node ], object[cond], object[chunk])
end
| `Fornum {identifier, min, max, range, block} ->
object[ node ] = DLTK.ForNumeric(first, last, object[identifier], object[min], object[max], object[range], object[block])
| `Fornum {identifier, min, max, block} ->
object[ node ] = DLTK.ForNumeric(first, last, object[identifier], object[min], object[max], object[block])
| `Forin {identifiers, exprs, block} ->
object[ node ] = DLTK.ForInPair(first, last, statListToChunk(identifiers), statListToChunk(exprs), object[block])
| `Local {identifiers} ->
object[ node ] = DLTK.Local(first, last, statListToChunk(identifiers))
| `Local {identifiers, inits} ->
object[ node ] = DLTK.Local(first, last, statListToChunk(identifiers), statListToChunk(inits))
| `Localrec {identifiers} ->
object[ node ] = DLTK.LocalRec(first, last, statListToChunk(identifiers))
| `Localrec {identifiers, inits} ->
object[ node ] = DLTK.LocalRec(first, last, statListToChunk(identifiers), statListToChunk(inits))
| `Break ->
object[ node ] = DLTK.Break(first, last)
| `Return {...} ->
object[ node ] = DLTK.Return(first, last)
-- Back patch children
for k,child in ipairs( node ) do
DLTK.addReturnValue(object[ node ], object[child] )
end
| `Call{...} | `Invoke{...} ->
apply(node, ...)
| _ ->
-- print('In stat.up missing type `'..(node.tag))
-- table.print(node, 'nohash',1)
end
end
---
-- Instanciates a Java object in <code>object</code> from the given node.
--
-- @param node Metalua node which will have its Java Object image stored in <code>object</code>
-- @param ... parent, grand-parent, grand-grand-parent and so on
visitor.expr.up = function (node, ...)
local first, last = offsets(node)
match node with
| `Function {param, block} ->
object[ node ] = DLTK.Function(first, last, statListToChunk( param ), object[block])
| `Nil ->
object[ node ] = DLTK.Nil(first, last)
| `True | `False ->
object[ node ] = DLTK.Boolean(first, last, node.tag == "True")
| `Number{number} ->
object[ node ] = DLTK.Number(first, last, number)
| `String{string} ->
object[ node ] = DLTK.String(first, last, string)
| `Pair { expr, sexpr } ->
object[ node ] = DLTK.Pair(first, last, object[expr], object[sexpr])
| `Table { ... } ->
object[ node ] = DLTK.Table(first, last)
-- Backpatch table content at initialisation
for k, child in ipairs( node ) do
DLTK.addStatement(object[ node ], object[child])
end
| `Op { operator, left, right} ->
object[ node ] = DLTK.BinaryExpression(first, last,object[left], operator, object[right])
| `Op { operator, expr } ->
object[ node ] = DLTK.UnaryExpression(first, last, operator, object[expr])
| `Paren{ expr } ->
object[ node ] = DLTK.Parenthesis(first, last, object[expr])
| `Id { name } ->
object[ node ] = DLTK.Identifier(first, last, name)
| `Index { expr, sexpr } ->
object[ node ] = DLTK.Index(object[expr], object[sexpr])
| `Call{...} | `Invoke{...} ->
apply(node, ...)
| `Id {name} ->
visitor.binder(node, ...)
| `Dots ->
visitor.binder(node, ...)
| _ ->
-- print('In expr.up missing type `'..(node.tag))
-- table.print(node, 'nohash',1)
end
end
---
-- Back patch node missing Java objects at their construction, uses global table <code>declaration</code>
local function backpatch()
for k, node in ipairs(declaration) do
local itype = type
local type = mark.declaration_type(node)
local init = mark.declaration_initialization(node)
local occurrences
if type == 'Function' then
local params, body = init[1], init[2]
-- Define function's parameters
DLTK.acceptArguments(object[node], statListToChunk(params))
-- Define function's body
DLTK.acceptBody(object[node], object[ body ])
elseif type == 'Nil' or type == 'Number' or type == 'String' or type == 'Boolean' then
DLTK.setInitialization(object[node], object[init])
end
--
-- Match node with its occurrences
--
local occ = mark.declaration_occurrences( node )
for i, occurrence in ipairs( occ ) do
-- Link occurrence to declaration
DLTK.setDeclaration(object[occurrence], object[node])
-- Add occurrence to declaration
DLTK.addOccurrence(object[node], object[occurrence])
end
end
end
---
-- Sets parent of DLTK ASTNode
--
-- @param node Metalua node
-- @param parent Metalua parent node
-- @param ... other nodes, not used
local parentMatcher = function ( node, parent , ... )
if parent then
DLTK.setParent( object[node], object[parent] )
end
end
---
-- This visitor enable to link child node to their parents
--
local parenthood = {
binder = parentMatcher,
block = { up = parentMatcher },
expr = { up = parentMatcher },
stat = { up = parentMatcher },
Id = parentMatcher,
Dots = parentMatcher
}
---
-- Build a Metalua AST from source code
--
-- @param source Code to parse
-- @return LuaModuleDeclaration, DLTK node, root of DLTK AST
module.ast_builder = function(source)
-- Build AST
require 'errnode'
local ast = getast( source )
local root = DLTK.LuaModuleDeclaration(#source, true)
if ast and ast.tag == 'Error' then
local line, column, offset = ast.lineinfo.first[1], ast.lineinfo.first[2], ast.lineinfo.first[3]
local errorMessage = ast[1] or 'Unable to determine error'
DLTK.setProblem(root, line, column, offset, errorMessage )
return root
end
-- Mark intersting nodes such as declaration
ast = mark.declaration( ast )
-- Walk through AST
walk.block(visitor, ast)
-- Backpatch partial nodes
backpatch()
-- Link nodes to their parents
walk.block(parenthood, ast)
-- Achieve DLTK Java Objects AST
DLTK.addStatementToModuleDeclaration(root, object[ast])
return root
end
return module
I have written an email to @fab13n but got no reply.
I have been exploring the project for quite a few days and I really want to move this project forward, if this issue is not responded, I might go ahead and make a new repo and go from there? Not sure. But I would really like to get a response and just do it from here.
The manual is written as if there is a metalua executable, but I could not find one. Linking metalua.lua to the $PATH appears to be insufficient since a) there is no shebang and b) it does not set up LUA_PATH correctly. I've created my own script that evals luarocks path before execing lua, but even this only works when it's run from the metalua checkout.
The Lua community overall prefers continuing the legacy of C-style snake_case to CamelCase, for filenames, variables, and function names. Could metalua print a warning when it encounters CamelCase?
Hi,
I'm on branch konekildt.
I created an AST containing an error, while traversing it, I got a function instead of an ``Error` node. Here is the code to reproduce.
require 'metalua.compiler'
local walk = require 'metalua.treequery.walk'['block']
local ast = mlc.luastring_to_ast( 'f = function()x end')
local errorhandling = function(node, ...)
table.print(node, 'nohash', 1)
end
local cfg = { error = errorhandling }
walk(cfg, ast)Here is the output
$ metalua functionerror.lua
`Function{ { },
{ `Error "line 1, char 15: This expression (\"x\") is an identifier; a statement was expected, and only function and method call expressions can be used as statements\n>>> f = function()x end\n>>> ^" },
`Error "A keyword was expected, probably `end'." }I browsed the code to find where this bug may come from walk.mlua in M.traverse.expr. In the match statement, the faulty function node does not match the guard ``Function{ params, body }` and is treated like an error node.
match x with
| `Paren{ e } -> E(e)
| `Call{...} | `Invoke{...} -> EL(x)
| `Index{ a, b } -> E(a); E(b)
| `Op{ opid, ... } -> E(x[2]); if #x==3 then E(x[3]) end
| `Function{ params, body } -> OS(body); IL(params); B(body); CS(body)
| `Stat{ b, e } -> OS(body); B(b); E(e); CS(body)
| `Id{ name } -> M.occurrence(cfg, x, unpack(ancestors))
| `Table{ ... } ->
for i = 1, #x do match x[i] with
| `Pair{ k, v } -> E(k); E(v)
| v -> E(v)
end end
| `Nil|`Dots|`True|`False|`Number{_}|`String{_} -> -- terminal node
| { tag=tag, ...} if M.tags.expr[tag]-> M.malformed (cfg, x, unpack (ancestors))
| _ -> M.unknown (cfg, x, unpack (ancestors))
endI think the cause of its malformation comes from funcdef_builder in mlp_stat.lua, where we can see a Function{ {}, {}, {} }` is created in error cases, instead of a Function{ {}, {} }aswalk.traverse.expr` expects it.
local function funcdef_builder(x)
local name = x[1] or gg.earlier_error()
local method = x[2]
local func = x[3] or gg.earlier_error()
if method then
name = { tag="Index", name, method, lineinfo = {
first = name.lineinfo.first,
last = method.lineinfo.last } }
_G.table.insert (func[1], 1, {tag="Id", "self"})
end
local r = { tag="Set", {name}, {func} }
r[1].lineinfo = name.lineinfo
r[2].lineinfo = func.lineinfo
return r
end So I think I get the bug, but I am not sure of the clean way to fix it, should I:
is created infuncdef_builder`?in the error node handling inM.traverse.expr`?Do you have a better idea?
For Lua 5.2 compatibility, remove getfenv in base.lua.
Trace:
$ luadist install metalua
Downloading repository information...
Finding out available versions of metalua...
Getting metalua-0.5 (binary)...
- trying another candidate due to: Error getting dependency of 'metalua-0.5': Package 'lua ~>5.1' needed, but installed at version '5.2.3'.
Getting metalua-scm (source)...
Cannot install package 'metalua': Error getting dependency of 'metalua-0.5': Package 'lua ~>5.1' needed, but installed at version '5.2.3'.
$ luac -v
Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio
I've love to be able to use metalua with Lua 5.3;
One of the more interesting applications would be compiling 5.3 => 5.1; changing (e.g.) bitwise operators for function calls.
I'm trying to compile metalua, but first it gave me an error with pluto.so saying I should recompile with -fPIC, so I added that flag there, and it worked, but now it's giving me the same error for rings.so. Here is my metalua src, i'm posting it since i edited some config files http://i.shibe.ml/QmUGavQcdsh13kKi9iDwTxfrNAYq97eRMjL9LKsy1TBtF7.tar
The error: http://hastebin.com/wivokikoha
The run.mlua test in the 5.0-rc2 tgz does not work as such because it calls os.getenv"OS": match, where $OS does not exist (at least not in Ubuntu Karmic). Below is a patch which changes OS detection to trial-and-error.
Greetings,
Jan-Pieter
6c6,14
< ls = io.popen (os.getenv "OS" :match "^Windows" and "dir /b" or "ls")
---
>
> if io.popen("ls"):read("_a") ~= "" then -- on Linux
> ls=io.popen("ls")
> elseif io.popen("dir /b"):read("_a") ~= "" then -- on Windows
> ls=io.popen("dir /b")
> else
> error("Don't know which OS you're on")
> end
>
this code is accepted by metalua :
for var, var2 =1, 0 do
var = 2
var2 = 2
end
Hi,
I'm on branch lua eclipse. I have noticed that incomplete if and elseif statements are smoothly accepted by Metalua but not handled.
Running the following code:
require 'metalua.compiler'
local codesamples = {
'if end',
'if true then elseif end'
}
for _, code in ipairs( codesamples ) do
local status, ast = pcall(mlc.luastring_to_ast, code)
if not status then
error ( ast )
end
local status, errormessage = pcall(mlc.check_ast, ast)
if not status then
error( errormessage)
end
print(string.format('No error encountered for "%s".', code))
table.print(ast, 1)
end
You get the following output:
$ metalua wrongcode.lua
No error encountered for "if end".
{ lineinfo = <?|L1|C1-6|K1-6>, `If{ lineinfo = <?|L1|C1-6|K1-6> } }
No error encountered for "if true then elseif end".
{ lineinfo = <?|L1|C1-23|K1-23>, `If{ lineinfo = <?|L1|C1-23|K1-23>, `True{ lineinfo = <?|L1|C4-7|K4-7> },
{ lineinfo = <?|L1|C14-12|K14-12> } } }
I fixed some very small bugs for the sake of which I do not want to do fork. Please, accept them.
Sorry my bad english.
There is fixed version:
-- iforeach(f[[, first], last], t1[, t2[,...]])
function table.iforeach(f, ...)
-- assert (type (f) == "function") [wouldn't allow metamethod __call]
local nargs = select("#", ...)
if nargs==1 then -- Quick iforeach (most common case), just one table arg
local t = ...
assert (type (t) == "table")
for i = 1, #t do
local result = f (t[i])
-- If the function returns non-false, stop iteration
if result then return result end
end
else -- advanced case: boundaries and/or multiple tables
-- 1 - find boundaries if any
local args, fargs, first, last, arg1 = {...}, { }
if type(args[1]) ~= "number" then first, arg1 = 1, 1
elseif type(args[2]) ~= "number" then first, last, arg1 = 1, args[1], 2
else first, last, arg1 = args[1], args[2], 3 end
assert (nargs >= arg1)
-- 2 - determine upper boundary if not given
if not last then for i = arg1, nargs do
assert (type (args[i]) == "table")
last = max (#args[i], last)
end end
-- 3 - perform the iteration
for i = first, last do
for j = arg1, nargs do fargs[j-arg1+1] = args[j][i] end -- build args list
local result = f (unpack (fargs)) -- here is the call
-- If the function returns non-false, stop iteration
if result then return result end
end
end
end
If I follow the document http://metalua.luaforge.net/manual004.html#toc9 to write an ast walker, such as
local cfg = { expr = { },
stat = { },
block = { } }
cfg.expr.down = function(x) ... end
walk.guess(cfg, term)The walker doesn't work. The function returned by walker_builder() in walk.mlua cannot get the correct cfg up/down functions from the input cfg.
The below changes can fix it.
208c208,209
< local down, up = cfg.down, cfg.up
---
> local cfgt = cfg[traverse]
> local down, up = cfgt and cfgt.down, cfgt and cfgt.up
However, I found the current treequery.mlua will use cfg's first level methods. It seems the issue is still caused by some refactoring inconsistence.
Could the tabs be updated? Download should point to an example luadist install metalua command, and mailing list should be replaced with a link to fab13n/metalua GitHub issues.
Hi,
I'm on origin/luaeclipse, when generating an AST on faulty code. mlc.check_ast is not able to check it.
M> local ast = mlc.luastring_to_ast('local =') local status, error = mlc.check_ast(ast)
Evaluation error:
lib/metalua/treequery/walk.mlua:154: Invalid binders list
stack traceback:
[C]: in function 'assert'
lib/metalua/treequery/walk.mlua:154: in function <lib/metalua/treequery/walk.mlua:148>
(tail call): ?
lib/metalua/treequery/walk.mlua:72: in function '?'
lib/metalua/treequery/walk.mlua:171: in function 'stat'
lib/metalua/treequery/walk.mlua:121: in function '?'
lib/metalua/treequery/walk.mlua:171: in function 'f'
compiler/mlc.mlua:69: in function <compiler/mlc.mlua:44>
(tail call): ?
stdin:1: in main chunk
[C]: in function 'xpcall'
/d/metalua/build/lib/metalua/metaloop.mlua:56: in function 'run'
compiler/metalua.mlua:251: in function 'main'
compiler/metalua.mlua:258: in main chunk
(tail call): ?
[C]: ?
The unm operator is ignored during the ast_to_src transformation.
The source code bug.lua
a = -1The test code test.lua
require 'metalua.loader'
mlc = require 'metalua.compiler'.new()
ast = mlc:srcfile_to_ast("bug.lua")
pp = require 'metalua.pprint'
local cfg = { line_max=1, fix_indent=2, metalua_tag=1, hide_hash=1 }
pp.print(ast, cfg)
str = mlc:ast_to_src(ast)
print(str)The result
$ lua test.lua
{ `Set{
{ `Id "a" },
{ `Op{
"unm",
`Number "1" } } } }
a = 1I just committed three sets of changes to my copy of metalua; the last two (LICENSE UTF-8 conversion and DESTDIR support) are definitely safe; the first one (documentation pruning) might be a bit iffy. Would it be possible to get them committed (and metalua 0.5 released soon)?
I'm getting metalua packaged for Fedora Linux, and the DESTDIR change is particularly needed (the LICENSE fix would save us having to use iconv at build time to fix the encoding as well).
Run with lua ./metalua.lua --file helloworld.lua --print-ast
And it throws exceptions.
I believe there some bugs introduced by inconsistent versions of code.
Hi,
I'm on branch origin/luaeclipse, using my installed Metalua, I parse a lua file that Metalua seems to not understand.
$ lua win32.lua
$ metalua win32.lua
lua: compiler/mlc.mlua:106: Parsing error in @win32.lua line 18, column 17, char 593:
lexer.lua:91: Unknown escape sequence '\.'
- (l.18, c.17, k.593) in parser ( ... )
- (l.18, c.17, k.593) in parser function argument(s)
- (l.18, c.8, k.584) in parser expression
- (l.18, c.8, k.584) in parser <anonymous>
- (l.18, c.8, k.584) in parser list
- (l.18, c.5, k.581) in parser if ... end
- (l.18, c.5, k.581) in parser statement
- (l.16, c.4, k.519) in parser statements block
- (l.15, c.11, k.510) in parser <anonymous>
- (l.15, c.1, k.500) in parser function ...
- (l.15, c.1, k.500) in parser statement
- (l.15, c.1, k.500) in parser statements block
- (l.15, c.1, k.500) in parser <anonymous>
stack traceback:
[C]: in function 'error'
compiler/mlc.mlua:106: in function <compiler/mlc.mlua:69>
(tail call): ?
...ome/kkinfoo/.bin/local/lib/lua/metalua/mlc_xcall.lua:13: in function 'server'
(command line):1: in main chunk
[C]: ?
/usr/bin/lua: ...ome/kkinfoo/.bin/local/lib/lua/metalua/mlc_xcall.lua:31: xcall failure. FIXME: transmit failure and backtrace
stack traceback:
[C]: in function 'error'
...ome/kkinfoo/.bin/local/lib/lua/metalua/mlc_xcall.lua:31: in function 'client_file'
compiler/metalua.mlua:142: in function 'main'
compiler/metalua.mlua:256: in main chunk
(tail call): ?
[C]: ?
Here is the file that generates the error:
--- Windows implementation of filesystem and platform abstractions.
-- Download http://unxutils.sourceforge.net/ for Windows GNU utilities
-- used by this module.
--[[module("luarocks.fs.win32", package.seeall)
local fs = require("luarocks.fs")
local cfg = require("luarocks.cfg")
local dir = require("luarocks.dir")
]]
--- Quote argument for shell processing. Fixes paths on Windows.
-- Adds single quotes and escapes.
-- @param arg string: Unquoted argument.
-- @return string: Quoted argument.
function Q(arg)
assert(type(arg) == "string")
-- Quote DIR for Windows
if arg:match("^[\.a-zA-Z]?:?[\\/]") then
return '"' .. arg:gsub("/", "\\"):gsub('"', '\\"') .. '"'
end
-- URLs and anything else
return '"' .. arg:gsub('"', '\\"') .. '"'
end
--- Return an absolute pathname from a potentially relative one.
-- @param pathname string: pathname to convert.
-- @param relative_to string or nil: path to prepend when making
-- pathname absolute, or the current dir in the dir stack if
-- not given.
-- @return string: The pathname converted to absolute.
function absolute_name(pathname, relative_to)
assert(type(pathname) == "string")
assert(type(relative_to) == "string" or not relative_to)
relative_to = relative_to or fs.current_dir()
if pathname:match("^[\.a-zA-Z]?:?[\\/]") then
return pathname
else
return relative_to .. "/" .. pathname
end
end
--- Create a wrapper to make a script executable from the command-line.
-- @param file string: Pathname of script to be made executable.
-- @param dest string: Directory where to put the wrapper.
-- @return boolean or (nil, string): True if succeeded, or nil and
-- an error message.
function wrap_script(file, dest)
assert(type(file) == "string")
assert(type(dest) == "string")
local base = dir.base_name(file)
local wrapname = fs.is_dir(dest) and dest.."/"..base or dest
wrapname = wrapname..".bat"
local wrapper = io.open(wrapname, "w")
if not wrapper then
return nil, "Could not open "..wrapname.." for writing."
end
wrapper:write("@echo off\n")
wrapper:write("setlocal\n")
wrapper:write('set LUA_PATH='..package.path..";%LUA_PATH%\n")
wrapper:write('set LUA_CPATH='..package.cpath..";%LUA_CPATH%\n")
wrapper:write('"'..dir.path(cfg.variables["LUA_BINDIR"], cfg.lua_interpreter)..'" -lluarocks.loader "'..file..'" %*\n')
wrapper:write("endlocal\n")
wrapper:close()
return true
end
function is_actual_binary(name)
name = name:lower()
if name:match("%.bat$") or name:match("%.exe$") then
return true
end
return false
end
function copy_binary(filename, dest)
local ok, err = fs.copy(filename, dest)
if not ok then
return nil, err
end
local exe_pattern = "%.[Ee][Xx][Ee]$"
local base = dir.base_name(filename)
local dest = dir.dir_name(dest)
if base:match(exe_pattern) then
base = base:gsub(exe_pattern, ".lua")
local helpname = dest.."/"..base
local helper = io.open(helpname, "w")
if not helper then
return nil, "Could not open "..helpname.." for writing."
end
helper:write('package.path=\"'..package.path:gsub("\\","\\\\")..';\"..package.path\n')
helper:write('package.cpath=\"'..package.path:gsub("\\","\\\\")..';\"..package.cpath\n')
helper:close()
end
return true
end
function chmod(filename, mode)
return true
end
function get_permissions(filename)
return ""
endLuaJIT is much faster than ordinary Lua, but it uses a different (and incompatible) bytecode format and opcode set. LuaJIT also offers an easy-to-use FFI, which means that many libraries require it.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.