euclidianace / ltreesitter

Standalone tree sitter bindings for the Lua language

ltreesitter

Tree sitter bindings for Lua

Documentation

Documentation can be found at this repo's github pages (Which are autogenerated using the Teal script scripts/gen.tl)

Installation

ltreesitter is avaliable on luarocks

luarocks install ltreesitter

or the latest main branch

luarocks install --dev ltreesitter

Examples

Looking for a quick start? These snippets should be descriptive enough to get you started. If you need more detail, take a look at the documentation

Setup

Loading parsers you have installed on your system

local ltreesitter = require("ltreesitter")

ltreesitter.require

Assuming you have a compiled c parser named c.so (or c.dll on windows) in ~/.tree-sitter/bin/ or package.cpath

local c_parser = ltreesitter.require("c")

You have a parser.so (or .dll) with the symbol tree_sitter_lua to load the language

local lua_parser = ltreesitter.require("parser", "lua")

ltreesitter.load

load will just directly load from the filename given.

local local_c_parser = ltreesitter.load("./c-parser.so", "c")

Using a path without a path separator may have unintended consequences, so when in doubt, include a leading ./ or use an absolute path.

For more information, look into how dlopen and LoadLibrary find paths.

Basic parsing and usage of trees and nodes

local source_code = [[
#include <stdio.h>

// a function that does stuff
static void stuff_doer(void) {
    printf("I'm doing stuff! :D\n");
}

int main(int argc, char **argv) {
    stuff_doer();
    return 0;
}
]]

local tree = c_parser:parse_string(source_code)
print(tree) -- tostring (which print calls automatically) will return the string of s-expressions of trees and nodes

for child in tree:root():named_children() do -- some iterators over nodes' children are provided
   print(child)
end

Queries

Using the above c_parser and tree

-- Grab the names of all functions
local my_query = c_parser:query[[
    (translation_unit
      (function_definition
        declarator: (function_declarator
                      declarator: (identifier) @name))) ]]

for capture, capture_name in my_query:capture(tree:root()) do -- iterate over captured nodes without caring about order
   -- Node:source() gives the source code that the node comes from
   print(capture:source(), capture_name) -- => "stuff_doer", "name" and "main", "name"
end

for match in my_query:match(tree:root()) do
   print(match.captures["name"]:source()) -- => "stuff_doer" and "main"
end

ltreesitter's Issues

Rubbish values from `Node.source` after editing tree

After editing a certain range of the tree using Tree.edit / Tree.edit_s, some calls to Node.source will return rubbish values.

This seems to only apply to nodes that were not affected by the edit. Nodes that did experience changes will return their source text correctly, while unaffected nodes return what I can only assume is junk.

Scoping of predicates

The when a predicate is defined, it is applied globally. Take the following query:

((item) @a (#eq? @a @a))
(thing) @b

One would expect that the predicate only applies to the pattern it is declared in, but it would instead be executed for all other patterns within the query.
In the example provided, @a would be matched as per normal, while the pattern @b would not be matched at all.

attempt to index a userdata value on subsequent calls to require

i noticed while trying to add treesitter support to lite xl, any calls after the 1st to ltreesitter.require causes an attempt to index a userdata value error. it seems that on the call after 1st it returns a plain userdata value (somehow) instead of an ltreesitter parser as testing in lua repl shows:

Function-like predicates being ran when conditional predicates return false

The current behaviour (stated in title) seems undesirable. If a conditional predicate prevents a capture from being returned, I think that the function-like predicates should be omitted as well. This would allow predicates such as #set! used by nvim to be implemented much more easily.

Seems like an obvious change to make to me, but it is breaking, so I would like to ask for an opinion first before trying to implement it.

Unable to load parser for c help

Hello I wanted to experiment with lua's bindings for treesitter but the demo in the readme does not work. I don't know where to find or generate the C shared object file. Any help would be appreciated. Please see attached screenshoot.

Write a test suite

query:capture has a bug: returns wrong capture @name

In file ltreesitter/csrc/query.c on line 295 name is incorrectly calculated.
The correct line should read:
const char *name = ts_query_capture_name_for_id(q->query, m.captures[capture_index].index, &len);

add a way to customize `ltreesitter.require` lookup paths

changing package.path and package.cpath for this feels hacky, so we should provide a nicer way to do this

Syntax highlighting demo?

Would you mind adding some demos, specifically like syntax highlighting with escape codes

Many objects get garbage collected when they shouldn't

Ideally this should be compatible for 5.1-5.4+JIT so no uservalues to hold a reference to the required objects.

Minimal example:
iterating over the matches of a query done 'inline' to a for loop causes it to be garbage collected

for match in parser:query[[ ... ]]:match(node) do
   collectgarbage()
end

will segfault often, whereas

local q = parser:query[[ ... ]]
for match in q:match(node) do
   collectgarbage()
end

wont, due to the alive reference to q

ltreesitter/dynamiclib.h missing

error: csrc/dynamiclib.c:4:10: fatal error: 'ltreesitter/dynamiclib.h' file not found
#include <ltreesitter/dynamiclib.h>

error when requiring ltreesitter

Hello, I've installed ltreesitter on my new laptop so I can use my Evergreen plugin on Lite XL and I get the following error with it:

error loading module 'ltreesitter' from file '/home/sammy/.luarocks/lib64/lua/5.4/ltreesitter.so':
	/home/sammy/.luarocks/lib64/lua/5.4/ltreesitter.so: undefined symbol: ltreesitter_create_source_text_metatable
stack traceback:
	[C]: in ?
	[C]: in function 'require'
	stdin:1: in main chunk
	[C]: in ?

(this is when I require via Lua REPL)

Any idea/fix? I installed it via luarocks install ltreesitter --local --dev

segfault when using query with predicate

hi, i'm working on evergreen for treesitter highlighting in lite xl and while trying to add support for zig i noticed that it would just cleanly exit with no error. after that, i removed predicates that i didnt have (since i was copying them from nvim-treesitter) and it still had a problem with predicates that were stated to be builtin (like eq?).

i did some minimal testing outside lite xl and noticed that if was any predicate at all in the query it would be an issue.

here's the code i used to check: https://safe.kashima.moe/y4wq3yrbdzxq.lua
if you remove the last part that queries for function builtins it will run successfully.

License?

Hey there, which license is this code released under?

Fix failing tests for basically every non 5.3 version

Now that github actions are set up, there are a decent amount of failures in the "not-lua-5.3 department"

tree-sitter has changed some of the installation paths

Parsers used to be put in ~/.tree-sitter/bin/ but these paths have changed and this needs to be reflected in ltreesitter.require