Nested terragrunt file not working on Windows about terragrunt

[suriyanto]

Ok, let me modify it there and send a PR.

[brikis98]

Could certainly be a bug. Our CI environment is Linux.

What version of Terragrunt? Could you re-run it with the TERRAGRUNT_DEBUG environment variable set to true?

[suriyanto]

Yevgeniy,

Here's the output with debug on.

[terragrunt] 2016/12/29 08:07:58 Reading Terragrunt config file at C:\Users\bongsos\dev\trep-cloud-deploy\envs\perf\vpc\.terragrunt
[terragrunt] 2016/12/29 08:07:58 *parser.PosError Error parsing Terragrunt config file C:\Users\bongsos\dev\trep-cloud-deploy\envs\perf\vpc\.terragrunt: At 2:20: illegal char escap
e
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/config/config.go:93 (0x4d7f3f)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/config/config.go:64 (0x4d7c0e)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/config/config.go:52 (0x4d7b4e)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/cli/cli_app.go:118 (0x4663b6)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/cli/cli_app.go:111 (0x4662e6)
/usr/local/go/src/runtime/asm_amd64.s:479 (0x456323)
/usr/local/go/src/reflect/value.go:434 (0x52b09f)
/usr/local/go/src/reflect/value.go:302 (0x52a99b)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/vendor/github.com/urfave/cli/app.go:480 (0x4c7ee7)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/vendor/github.com/urfave/cli/app.go:241 (0x4c5bcd)
/home/ubuntu/.go_workspace/src/github.com/gruntwork-io/terragrunt/main.go:19 (0x4010de)
/usr/local/go/src/runtime/proc.go:183 (0x42c720)
/usr/local/go/src/runtime/asm_amd64.s:2086 (0x458ec1)

[brikis98]

Hmm, odd. Perhaps it's some Windows character issue that I can't see.

Out of curiosity, if you run terragrunt plan in the root folder, what happens?

[suriyanto]

No error when I ran it on the root folder. All initialization succeeded and terraform.tfstate file is created in S3.

It gives the error Error loading config: No Terraform configuration files found in directory: ..., which is expected as there's no terraform file in the root directory.

Looks like the issue is isolated to ${find_in_parent_folders()} then?

[brikis98]

Does seem like a bug in find_in_parent_folders, or at least how it handles Windows. What version of Terragrunt are you using?

[suriyanto]

Latest version. Terragrunt 0.7.2 with Terraform 0.8.2.

[brikis98]

Roger. I don't have a Windows system to test with, so it's going to be a bit tricky to fix this one. Any interest in doing some debugging?

The error happens from this line of code that is trying to parse HCL of the parent .terragrunt file. If we had just a single Printf to show us the contents of the resolvedConfigString variable, we could see what's going wrong with the parsing...

[suriyanto]

Of course. I have not touch go, so let me setup my environment and try it out. Will let you know soon. Thanks.

[brikis98]

Great, thanks! Go is fairly easy to get started with. Let me know if you have questions.

[suriyanto]

Ok, got it.

The print statement (fmt.Printf("resolvedConfigString = %s\n", resolvedConfigString) returns:

resolvedConfigString = include = {
  path = "..\..\..\.terragrunt"   
}

Looks like hcl.Decode does not like the \ character as it is treated as escape character.

I modified line 92 to double the \ before passing to hcl.Decode as below:

if err := hcl.Decode(terragruntConfigFromFileSecondPass, strings.Replace(resolvedConfigString, "\\", "\\\\", -1)); err != nil {

This results in "..\\..\\..\\.terragrunt" and everything works perfectly. However I figured that it is not a clean solution.

While I have the code open, if you have suggestion on how I should proceed, I can make the changes and create a PR.

[brikis98]

Ohhh, good find!

I guess where things go wrong is in executeTerragruntHelperFunction. The findInParentFolders and pathRelativeToInclude functions should probably escape the paths they return so that they are valid HCL.

[KashifSaadat]

Thanks for looking into this and working on a fix!

Another issue I noticed: Across Windows and Linux terragrunt will complain that the directory structure is different (backslashes on windows, forward slashes on linux). I made a hacky workaround (shown below, added to remote_state.go in 'shouldOverrideExistingRemoteState' fn) to keep the state file consistent but we may need a more elegant solution for this:

remoteStateFromTerragruntConfig.Config["key"] = strings.Replace(remoteStateFromTerragruntConfig.Config["key"], "\\", "/", -1)

[cstavro]

Did a PR for this actually get made? I can't seem to find it and I'm running in to the same issue.

[brikis98]

@cstavro I have not see an PR for it yet. But I'm willing to accept one from anyone that has time and access to a Windows box to test it :)

[suriyanto]

Sorry for the delay. I'm getting a lot more error in the tests on Windows, especially in the file_test.go. There are quite a few differences between path handling in Windows vs Linux.

[cstavro]

So I think there are 2 problems here.
First is as already identified the escaping problem.

Second, there looks like there's an issue with the detection of the absolute file paths on Windows. The tests are using /root/.... which in Linux is absolute, but in Windows if it's not prefixed with a drive letter, it's never going to be an absolute path. So this guy gets tripped up and ends up concatenating the strings when it shouldn't be.

I'm not entirely sure offhand how to resolve this second issue without resorting to a check to see if the path contains '..' anywhere inside of it.

Thoughts?

[brikis98]

The tests are using /root/.... which in Linux is absolute, but in Windows if it's not prefixed with a drive letter, it's never going to be an absolute path. So this guy gets tripped up and ends up concatenating the strings when it shouldn't be.

Ah, that's a good catch.

Probably the solution is to use Go build tags. For example, you could create two files: test_constants_windows.go and test_constants_unix.go.

test_constants_windows.go would contain:

// +build windows

package config

var rootFolder = "C:/"

test_constants_unix.go would contain:

// +build !windows

package config

var rootFolder = "/root"

Then, in all the test code that currently uses /root, or any other path that is supposed to look like an absolute path, you use the rootFolder variable.

[cstavro]

@brikis98 I was able to implement your recommendation almost verbatim, and I learned something new in the process! Thanks!