tree-sitter-grammars / tree-sitter-hcl

HCL grammar for tree-sitter

tree-sitter-hcl

tree-sitter grammar for the HCL language

Try It Out

Try the parser in the playground

Example

Highlighting example/example.hcl:

Developing

It is recommended to use nix to fulfill all development dependencies. To activate the development environment simply run nix-shell in the project root.

Running Tests

To run tests simply run nix-shell --run 'tree-sitter test'.

Compliance

The directory example/real_world_stuff contains a corpus of hcl files that I found with the github query language:HCL for users coreos, hashicorp, oracle and terraform-community-modules.

tree-sitter parse --quiet --stat example/real_world_stuff/*/*

Total parses: 1892; successful parses: 1892; failed parses: 0; success percentage: 100.00%

Fuzzing

See the fuzzing repo for this parser

Attributions

Pages were copied from https://github.com/m-novikov/tree-sitter-sql

tree-sitter-hcl's Issues

Parentheses in a heredoc break highlighting

Description
Starting from a valid Terraform file containing heredocs that have parentheses inside blocks:

• if there are no nested blocks, then it works
• if the parentheses are after a nested block, then it works
• if the parentheses are around or before a nested block, then the rest of the file's highlighting breaks

To Reproduce
Steps to reproduce the behavior:

1. In the example file, tpl5, add parentheses around the string "foo", everything works normally.
   
   

2. In tpl6, add parentheses around or after the nested if block, the highlight will break.
   
   
   
   

Expected behavior
Parentheses around and after nested blocks in a heredoc are valid and correctly highlighted.

Additional information
Also verified with nested for loops.

terraform.so: undefined symbol: tree_sitter_terraform

Describe the bug
Hi, I'm trying to install this parser, using https://github.com/nvim-treesitter/nvim-treesitter#adding-parsers directions. The installation seems to be ok, but I get this error when opening a terraform file:

Error detected while processing CursorHold Autocommands for "<buffer=1>":
E5108: Error executing lua Failed to load parser: uv_dlsym: /home/cova/.local/share/lunarvim/site/pack/packer/start/nvim-treesitter/parser/terraform.so: undefined symbol: tree_sitter_terraform
stack traceback:
        [C]: in function '_ts_add_language'
        /usr/share/nvim/runtime/lua/vim/treesitter/language.lua:33: in function 'require_language'
        /usr/share/nvim/runtime/lua/vim/treesitter.lua:38: in function '_create_parser'
        /usr/share/nvim/runtime/lua/vim/treesitter.lua:93: in function 'get_parser'
        ...ext-commentstring/lua/ts_context_commentstring/utils.lua:124: in function 'get_node_at_cursor_start_of_line'
        ...-commentstring/lua/ts_context_commentstring/internal.lua:116: in function 'calculate_commentstring'
        ...-commentstring/lua/ts_context_commentstring/internal.lua:145: in function 'update_commentstring'
        [string ":lua"]:1: in main chunk

To Reproduce
I just followed the directions described above. My configuration for this plugin:

local parser_config = require "nvim-treesitter.parsers".get_parser_configs()
parser_config.terraform = {
  install_info = {
    url = "/home/cova/repos/github/utils/tree-sitter-hcl/", -- local path or git repo
    files = {
      "src/parser.c",
      "src/scanner.cc",
    }
  },
    filetype = "terraform", -- if filetype does not agrees with parser name
}

Then I just ran TSInstall terraform (also tried from grammar)

I'm pretty sure that I'm doing something wrong, but can't see what :)
I'm running lunarvim (master) with neovim 0.6.1

Any suggestions where to look?

Thanks!

block parsing fails when block string_lit is empty string

Describe the bug
The parser fails to parse consul global-management policy that is used by consul when bootstrapping.

❯ tree-sitter parse admin.hcl
(config_file [0, 0] - [36, 0]
  (body [0, 0] - [0, 13]
    (attribute [0, 0] - [0, 13]
      (identifier [0, 0] - [0, 3])
      (expression [0, 6] - [0, 13]
        (literal_value [0, 6] - [0, 13]
          (string_lit [0, 6] - [0, 13]
            (quoted_template_start [0, 6] - [0, 7])
            (template_literal [0, 7] - [0, 12])
            (quoted_template_end [0, 12] - [0, 13]))))))
  (ERROR [2, 0] - [35, 1]
    (identifier [2, 0] - [2, 12])
    (string_lit [2, 13] - [3, 11]
      (quoted_template_start [2, 13] - [2, 14])
      (template_literal [2, 14] - [3, 9])
      (quoted_template_end [3, 10] - [3, 11]))
    (identifier [3, 11] - [3, 16])
    (string_lit [3, 16] - [6, 14]
      (quoted_template_start [3, 16] - [3, 17])
      (template_literal [4, 0] - [6, 12])
      (quoted_template_end [6, 13] - [6, 14]))
    (string_lit [6, 14] - [7, 11]
      (quoted_template_start [6, 14] - [6, 15])
      (template_literal [6, 16] - [7, 9])
      (quoted_template_end [7, 10] - [7, 11]))
    (identifier [7, 11] - [7, 16])
    (string_lit [7, 16] - [10, 12]
      (quoted_template_start [7, 16] - [7, 17])
      (template_literal [8, 0] - [10, 10])
      (quoted_template_end [10, 11] - [10, 12]))
    (string_lit [10, 12] - [11, 11]
      (quoted_template_start [10, 12] - [10, 13])
      (template_literal [10, 14] - [11, 9])
      (quoted_template_end [11, 10] - [11, 11]))
    (identifier [11, 11] - [11, 16])
    (string_lit [11, 16] - [14, 11]
      (quoted_template_start [11, 16] - [11, 17])
      (template_literal [12, 0] - [14, 9])
      (quoted_template_end [14, 10] - [14, 11]))
    (identifier [14, 11] - [14, 16])
    (string_lit [14, 16] - [16, 13]
      (quoted_template_start [14, 16] - [14, 17])
      (template_literal [16, 0] - [16, 11])
      (quoted_template_end [16, 12] - [16, 13]))
    (string_lit [16, 13] - [17, 11]
      (quoted_template_start [16, 13] - [16, 14])
      (template_literal [16, 15] - [17, 9])
      (quoted_template_end [17, 10] - [17, 11]))
    (identifier [17, 11] - [17, 16])
    (string_lit [17, 16] - [20, 12]
      (quoted_template_start [17, 16] - [17, 17])
      (template_literal [18, 0] - [20, 10])
      (quoted_template_end [20, 11] - [20, 12]))
    (identifier [20, 12] - [20, 17])
    (string_lit [20, 17] - [22, 8]
      (quoted_template_start [20, 17] - [20, 18])
      (template_literal [22, 0] - [22, 6])
      (quoted_template_end [22, 7] - [22, 8]))
    (identifier [22, 8] - [22, 13])
    (string_lit [22, 13] - [24, 14]
      (quoted_template_start [22, 13] - [22, 14])
      (template_literal [24, 0] - [24, 12])
      (quoted_template_end [24, 13] - [24, 14]))
    (string_lit [24, 14] - [25, 11]
      (quoted_template_start [24, 14] - [24, 15])
      (template_literal [24, 16] - [25, 9])
      (quoted_template_end [25, 10] - [25, 11]))
    (identifier [25, 11] - [25, 16])
    (string_lit [25, 16] - [28, 16]
      (quoted_template_start [25, 16] - [25, 17])
      (template_literal [26, 0] - [28, 14])
      (quoted_template_end [28, 15] - [28, 16]))
    (string_lit [28, 16] - [29, 11]
      (quoted_template_start [28, 16] - [28, 17])
      (template_literal [28, 18] - [29, 9])
      (quoted_template_end [29, 10] - [29, 11]))
    (identifier [29, 11] - [29, 16])
    (string_lit [29, 16] - [30, 15]
      (quoted_template_start [29, 16] - [29, 17])
      (template_literal [30, 1] - [30, 13])
      (quoted_template_end [30, 14] - [30, 15]))
    (identifier [30, 15] - [30, 20])
    (string_lit [30, 20] - [33, 16]
      (quoted_template_start [30, 20] - [30, 21])
      (template_literal [31, 0] - [33, 14])
      (quoted_template_end [33, 15] - [33, 16]))
    (string_lit [33, 16] - [34, 11]
      (quoted_template_start [33, 16] - [33, 17])
      (template_literal [33, 18] - [34, 9])
      (quoted_template_end [34, 10] - [34, 11]))
    (identifier [34, 11] - [34, 16])
    (quoted_template_start [34, 16] - [34, 17])))
admin.hcl	0 ms	(ERROR [2, 0] - [35, 1])

To Reproduce
Steps to reproduce the behavior:

Put this into admin.hcl:

acl = "write"
agent_prefix "" {
	policy = "write"
}
event_prefix "" {
	policy = "write"
}
key_prefix "" {
	policy = "write"
}
keyring = "write"
node_prefix "" {
	policy = "write"
}
operator = "write"
mesh = "write"
query_prefix "" {
	policy = "write"
}
service_prefix "" {
	policy = "write"
	intentions = "write"
}
session_prefix "" {
	policy = "write"
}

And run

tree-sitter parse admin.hcl

Expected behavior
The parser parses the provided hcl file without ERROR.

Additional context
The hcl above is the global-management policy for consul, which is the default policy to bootstrap consul.
One might want to write a similar policy that references empty prefixes, but the tree-sitter parser fails to parse it correctly.

I'm using the parser together with nvim-treesitter and nvim-tree-sitter-textobjects to get syntax highlighting and improved hcl editing in nvim.

Emacs crashes due to SIGABRT when using new scanner

When building the grammar from HEAD (currently b553906) and attempting to use the grammar to highlight an HCL file in Emacs 29.1, the grammar causes Emacs to crash with a munmap_chunk(): invalid pointer error.

I am unsure of whether this bug resides in the grammar or within Emacs. It only occurs when Emacs is configured with the flag --with-pgtk, which tends to be finicky. However, it does not occur with the v1.1.0 release from this repo, and so may be a problem with the rewritten scanner. I have also submitted a report to the GNU Emacs bug tracker.

I've written up a full reproduction of the crash here: https://github.com/erik-overdahl/emacs-29-pgtk-ts-crash-bugreport. Please let me know if you need more information.

A comment at the beginning of a resource ends-up outside of the `body` block instead of inside

Describe the bug
I believe there is an issue with the position of a comment node in the tree when the comment is right at the beginning of a block.

To Reproduce
Steps to reproduce the behavior:

resource "foo" "bar" {
  # comment
  baz = "qux"
  aaa = "www"
}

will produce the following tree

config_file
    └── body
        └── block
            └── identifier (resource)
            └── string_lit
                └── quoted_template_start (")
                └── template_literal (foo)
                └── quoted_template_end (")
            └── string_lit
                └── quoted_template_start (")
                └── template_literal (bar)
                └── quoted_template_end (")
            └── block_start ({)
            └── comment (# comment)
            └── body
                └── attribute
                    └── identifier (baz)
                    └── expression
                        └── literal_value
                            └── string_lit
                                └── quoted_template_start (")
                                └── template_literal (qux)
                                └── quoted_template_end (")
                └── attribute
                    └── identifier (aaa)
                    └── expression
                        └── literal_value
                            └── string_lit
                                └── quoted_template_start (")
                                └── template_literal (www)
                                └── quoted_template_end (")
            └── block_end (})

Notice the comment node being after the block_start node as it should, but before the body node, and it seems weird to me.

If we move the comment line later in the resource like this :

resource "foo" "bar" {
  baz = "qux"
  # comment
  aaa = "www"
}

the `comment` node ends-up at the same level as the siblings `attribute` inside the `body` node.

config_file
    └── body
        └── block
            └── identifier (resource)
            └── string_lit
                └── quoted_template_start (")
                └── template_literal (foo)
                └── quoted_template_end (")
            └── string_lit
                └── quoted_template_start (")
                └── template_literal (bar)
                └── quoted_template_end (")
            └── block_start ({)
            └── body
                └── attribute
                    └── identifier (baz)
                    └── expression
                        └── literal_value
                            └── string_lit
                                └── quoted_template_start (")
                                └── template_literal (qux)
                                └── quoted_template_end (")
                └── comment (# comment)
                └── attribute
                    └── identifier (aaa)
                    └── expression
                        └── literal_value
                            └── string_lit
                                └── quoted_template_start (")
                                └── template_literal (www)
                                └── quoted_template_end (")
            └── block_end (})

Unless am I mistaken ?

The two examples can be reproduced in the playground too.

Expected behavior
The comment node should be inside the body node, as are the attribute, and not right before.

Screenshots
n/a

Additional context
n/a

Thank you very much,

Add terraform filetype

Is your feature request related to a problem? Please describe.
I've noticed recently neovim opens tf files as filetype=terraform, I'm not sure when this change was made. As a result tree-sitter doesn't pick up this parser unless I set syntax=tf.

Describe the solution you'd like
Would it be possible to add terraform as a supported filetype in package.json?

Auto indent outside blocks

Describe the bug
When adding new line auto indent works great when inside a block, but when outside of a block it also auto indent when new line is added.

To Reproduce
Steps to reproduce the behavior:

1. I stock use AstroVim with :TSInstall hcl
2. Write any Terraform block and try inserting a new line after that block. And see it auto indent.

Expected behavior
I expect auto indent not to happen out side Terraform blocks.

Screenshots
Hard to show on screenshots. But here is an attempt. See it on line 5.

Additional context
Also disabling indent makes it work outside Terraform blocks, but breaks indentation inside blocks.

Compile warning with missing-field-initializers

Describe the bug

I'm trying to integrate tree-sitter-hcl with difftastic

But meeting the following compile warning:

warning: vendor/tree-sitter-hcl-src/scanner.cc: In member function ‘bool {anonymous}::Scanner::scan(TSLexer*, const bool*)’:
warning: vendor/tree-sitter-hcl-src/scanner.cc:99:39: warning: missing initializer for member ‘{anonymous}::Context::heredoc_identifier’ [-Wmissing-field-initializers]
warning:    99 |       Context ctx = { QUOTED_TEMPLATE };
warning:       |                                       ^
warning: vendor/tree-sitter-hcl-src/scanner.cc:117:48: warning: missing initializer for member ‘{anonymous}::Context::heredoc_identifier’ [-Wmissing-field-initializers]
warning:   117 |         Context ctx = { TEMPLATE_INTERPOLATION };
warning:       |                                                ^
warning: vendor/tree-sitter-hcl-src/scanner.cc:145:44: warning: missing initializer for member ‘{anonymous}::Context::heredoc_identifier’ [-Wmissing-field-initializers]
warning:   145 |         Context ctx = { TEMPLATE_DIRECTIVE };
warning:       |                                            ^

To Reproduce
Steps to reproduce the behavior:

1. Build with clang

Expected behavior
No warning

Screenshots

---

I'm glad to send a PR if fixing the warning looks good to you 😄

provider keyword is highlighted even as a property of a block

Describe the bug
The provider keyword is highlighted as a keyword even though it is part of a configuration block of a resource.

To Reproduce
Steps to reproduce the behavior:

1. Configure an action block nested in a stage block of a aws_codepipeline resource
2. Define a provider argument in the action block
3. The provider argument is highlighted as a keyword

Expected behavior
provider should be a plain argument, not a keyword.

Screenshots

Additional context
I use the latest version provided by nvim-treesitter/nvim-treesitter.

New release?

Hi Micha,

would it be possible to tag a new release? I'm packaging this project for Alpine Linux and it would be nice to include the fixes since the v1.0.0 release.

Thanks!

Consider moving the repo to `tree-sitter-grammars`

Hey @MichaHoffmann, I hope you've been well. We at nvim-treesitter (and potentially more folks from other teams interested in an ecosystem-wide effort to maintain grammars) have decided to move all of our own grammars collectively into one org, to make it easier for us all to fix bugs or keep things up to date (scanner crashes, regenerating with the latest abi, etc.) without having to rely on one sole maintainer (you).

This is 100% optional, and please don't feel pressured or feel like you have an obligation of moving the grammar to the org, but it will help everyone a lot, us at nvim-treesitter, other users downstream, to know that a grammar won't ever die should a maintainer call it quits or go MIA. We also have org-wide actions that we maintain (out of sight, out of mind for you) and a grammar template (soon for newbies) in place to ensure authors have a better experience with working on and fixing tree-sitter grammars. It also enables anyone looking to find grammars for a specific language to only have to search in one spot

Our org is here: https://github.com/tree-sitter-grammars

If you want to participate (and don't worry, you'll still have full write perms to your repo, just no admin access), that'd be great! All you have to do is transfer the repo to me in the repository settings and I'll handle it from there. Thanks a lot, and again no worries if the answer is no!

Detect .nomad files as filetypes to parse

Hashicorp's Nomad orchestrator commonly uses the filetype .nomad for its files. While the file extension is named differently, these files are in fact HCL files, and it would be great if this plugin detected and parsed .nomad files as well, which it doesn't seem to do now. I think this has already been done for .tf terraform files in this plugin, I get syntax highlighting for those. .tf files are in the exact same position as .nomad files - not named .hcl, but the contents are always HCL.