cloud-custodian / tfparse Goto Github PK

python extension for terraform hcl parsing

License: Apache License 2.0

Go 39.61% Python 36.69% HCL 23.05% Shell 0.66%

terraform

tfparse's Introduction

Cloud Custodian (c7n)

Cloud Custodian, also known as c7n, is a rules engine for managing public cloud accounts and resources. It allows users to define policies to enable a well managed cloud infrastructure, that's both secure and cost optimized. It consolidates many of the adhoc scripts organizations have into a lightweight and flexible tool, with unified metrics and reporting.

Custodian can be used to manage AWS, Azure, and GCP environments by ensuring real time compliance to security policies (like encryption and access requirements), tag policies, and cost management via garbage collection of unused resources and off-hours resource management.

Custodian also supports running policies on infrastructure as code assets to provide feedback directly on developer workstations or within CI pipelines.

Custodian policies are written in simple YAML configuration files that enable users to specify policies on a resource type (EC2, ASG, Redshift, CosmosDB, PubSub Topic) and are constructed from a vocabulary of filters and actions.

It integrates with the cloud native serverless capabilities of each provider to provide for real time enforcement of policies with builtin provisioning. Or it can be run as a simple cron job on a server to execute against large existing fleets.

Cloud Custodian is a CNCF Incubating project, lead by a community of hundreds of contributors.

Features

Comprehensive support for public cloud services and resources with a rich library of actions and filters to build policies with.
Run policies on infrastructure as code (terraform, etc) assets.
Supports arbitrary filtering on resources with nested boolean conditions.
Dry run any policy to see what it would do.
Automatically provisions serverless functions and event sources ( AWS CloudWatchEvents, AWS Config Rules, Azure EventGrid, GCP AuditLog & Pub/Sub, etc)
Cloud provider native metrics outputs on resources that matched a policy
Structured outputs into cloud native object storage of which resources matched a policy.
Intelligent cache usage to minimize api calls.
Supports multi-account/subscription/project usage.
Battle-tested - in production on some very large cloud environments.

Quick Install

Custodian is published on pypi as a series of packages with the c7n prefix, its also available as a docker image.

$ python3 -m venv custodian
$ source custodian/bin/activate
(custodian) $ pip install c7n

Usage

The first step to using Cloud Custodian (c7n) is writing a YAML file containing the policies that you want to run. Each policy specifies the resource type that the policy will run on, a set of filters which control resources will be affected by this policy, actions which the policy with take on the matched resources, and a mode which controls which how the policy will execute.

The best getting started guides are the cloud provider specific tutorials.

As a quick walk through, below are some sample policies for AWS resources.

will enforce that no S3 buckets have cross-account access enabled.
will terminate any newly launched EC2 instance that do not have an encrypted EBS volume.
will tag any EC2 instance that does not have the follow tags "Environment", "AppId", and either "OwnerContact" or "DeptID" to be stopped in four days.

policies:
 - name: s3-cross-account
   description: |
     Checks S3 for buckets with cross-account access and
     removes the cross-account access.
   resource: aws.s3
   region: us-east-1
   filters:
     - type: cross-account
   actions:
     - type: remove-statements
       statement_ids: matched

 - name: ec2-require-non-public-and-encrypted-volumes
   resource: aws.ec2
   description: |
    Provision a lambda and cloud watch event target
    that looks at all new instances and terminates those with
    unencrypted volumes.
   mode:
    type: cloudtrail
    role: CloudCustodian-QuickStart
    events:
      - RunInstances
   filters:
    - type: ebs
      key: Encrypted
      value: false
   actions:
    - terminate

 - name: tag-compliance
   resource: aws.ec2
   description: |
     Schedule a resource that does not meet tag compliance policies to be stopped in four days. Note a separate policy using the`marked-for-op` filter is required to actually stop the instances after four days.
   filters:
    - State.Name: running
    - "tag:Environment": absent
    - "tag:AppId": absent
    - or:
      - "tag:OwnerContact": absent
      - "tag:DeptID": absent
   actions:
    - type: mark-for-op
      op: stop
      days: 4

You can validate, test, and run Cloud Custodian with the example policy with these commands:

# Validate the configuration (note this happens by default on run)
$ custodian validate policy.yml

# Dryrun on the policies (no actions executed) to see what resources
# match each policy.
$ custodian run --dryrun -s out policy.yml

# Run the policy
$ custodian run -s out policy.yml

You can run Cloud Custodian via Docker as well:

# Download the image
$ docker pull cloudcustodian/c7n
$ mkdir output

# Run the policy
#
# This will run the policy using only the environment variables for authentication
$ docker run -it \
  -v $(pwd)/output:/home/custodian/output \
  -v $(pwd)/policy.yml:/home/custodian/policy.yml \
  --env-file <(env | grep "^AWS\|^AZURE\|^GOOGLE") \
  cloudcustodian/c7n run -v -s /home/custodian/output /home/custodian/policy.yml

# Run the policy (using AWS's generated credentials from STS)
#
# NOTE: We mount the ``.aws/credentials`` and ``.aws/config`` directories to
# the docker container to support authentication to AWS using the same credentials
# credentials that are available to the local user if authenticating with STS.

$ docker run -it \
  -v $(pwd)/output:/home/custodian/output \
  -v $(pwd)/policy.yml:/home/custodian/policy.yml \
  -v $(cd ~ && pwd)/.aws/credentials:/home/custodian/.aws/credentials \
  -v $(cd ~ && pwd)/.aws/config:/home/custodian/.aws/config \
  --env-file <(env | grep "^AWS") \
  cloudcustodian/c7n run -v -s /home/custodian/output /home/custodian/policy.yml

The custodian cask tool is a go binary that provides a transparent front end to docker that mirors the regular custodian cli, but automatically takes care of mounting volumes.

Consult the documentation for additional information, or reach out on gitter.

Cloud Provider Specific Help

For specific instructions for AWS, Azure, and GCP, visit the relevant getting started page.

Get Involved

GitHub - (This page)
Slack - Real time chat if you're looking for help or interested in contributing to Custodian!
- Gitter - (Older real time chat, we're likely migrating away from this)
Linen.dev - Follow our discussions on Linen
Mailing List - Our project mailing list, subscribe here for important project announcements, feel free to ask questions
Reddit - Our subreddit
StackOverflow - Q&A site for developers, we keep an eye on the cloudcustodian tag
YouTube Channel - We're working on adding tutorials and other useful information, as well as meeting videos

Community Resources

We have a regular community meeting that is open to all users and developers of every skill level. Joining the mailing list will automatically send you a meeting invite. See the notes below for more technical information on joining the meeting.

Community Meeting Videos
Community Meeting Notes Archive
Upcoming Community Events
Cloud Custodian Annual Report 2021 - Annual health check provided to the CNCF outlining the health of the project

Additional Tools

The Custodian project also develops and maintains a suite of additional tools here https://github.com/cloud-custodian/cloud-custodian/tree/master/tools:

Org: Multi-account policy execution.
ShiftLeft: Shift Left ~ run policies against Infrastructure as Code assets like terraform.
PolicyStream: Git history as stream of logical policy changes.
Salactus: Scale out s3 scanning.
Mailer: A reference implementation of sending messages to users to notify them.
Trail Creator: Retroactive tagging of resources creators from CloudTrail
TrailDB: Cloudtrail indexing and time series generation for dashboarding.
LogExporter: Cloud watch log exporting to s3
Cask: Easy custodian exec via docker
Guardian: Automated multi-account Guard Duty setup
Omni SSM: EC2 Systems Manager Automation
Mugc: A utility used to clean up Cloud Custodian Lambda policies that are deployed in an AWS environment.

Contributing

See https://cloudcustodian.io/docs/contribute.html

Security

If you've found a security related issue, a vulnerability, or a potential vulnerability in Cloud Custodian please let the Cloud Custodian Security Team know with the details of the vulnerability. We'll send a confirmation email to acknowledge your report, and we'll send an additional email when we've identified the issue positively or negatively.

Code of Conduct

This project adheres to the CNCF Code of Conduct

By participating, you are expected to honor this code.

tfparse's People

Contributors

Stargazers

Watchers

Forkers

sontek djeebus kapilt pratmis howbazaar thugdoubt abhibms06 fwereade christokur

tfparse's Issues

handling module inputs from other module outputs

tfparse doesn't seem to handle passing outputs from module A as an input to module B.

the evaluation of the input within module B seems to only handle values known at module's B evaluation time.

This is due to behavior in the underlying golang library. Separately that golang library has been refactored
recently into a separate package (defsec -> trivy-iac). The current source for that evaluation is

https://github.com/aquasecurity/trivy-iac/blob/v0.7.1/pkg/scanners/terraform/parser/evaluator.go#L121

the location/version used by tfparse is defsec @ 0.90.1
https://github.com/aquasecurity/defsec/blob/988b9e9444c616a573dbbb670e65a05253e3ab2d/pkg/scanners/terraform/parser/evaluator.go#L118

wrt to resolution, it could be a question of fixing evaluateStep to consider module input changes. I think we need to validate
as well that the input variable outside the module is the same identity as the input variable inside the module, ie. there is a
common linkage for when the variable outside the module is resolved.

at the moment looking at the structure generated by tfparse

/root
  - mod-a output -> correct
  - mod-b input var -> correct
  /mod-b
      - input var -> unknown

Setting allow_downloads to false still allows downloads

Hello. Perhaps I am misunderstanding the purpose of the allow_downloads flag- my expectation is that setting this to False would prevent tfparse from downloading any external modules. This does not appear to be the case! A demonstration:

~/Work/misc/tfparse-wtf/test $ pyenv activate tfparse-test
pyenv-virtualenv: prompt changing will be removed from future release. configure `export PYENV_VIRTUALENV_DISABLE_PROMPT=1' to simulate the behavior.
(tfparse-test) [2:43PM][zzzzz@xx-m-zzzzz]~/Work/misc/tfparse-wtf/test $ python3 --version
Python 3.10.11
(tfparse-test) [2:43PM][zzzzz@xx-m-zzzzz]~/Work/misc/tfparse-wtf/test $ pip3 install tfparse
Collecting tfparse
  Using cached tfparse-0.6.2-cp310-cp310-macosx_10_9_x86_64.whl (8.8 MB)
Collecting cffi>=1.0.0
  Using cached cffi-1.16.0-cp310-cp310-macosx_10_9_x86_64.whl (182 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Installing collected packages: pycparser, cffi, tfparse
Successfully installed cffi-1.16.0 pycparser-2.21 tfparse-0.6.2

[notice] A new release of pip is available: 23.0.1 -> 23.3.1
[notice] To update, run: python3.10 -m pip install --upgrade pip
(tfparse-test) ~/Work/misc/tfparse-wtf/test $ ls
parse.py* test.tf
(tfparse-test) ~/Work/misc/tfparse-wtf/test $ cat parse.py
#!/usr/bin/env python3

from tfparse import load_from_path

parsed = load_from_path('.', allow_downloads=False, debug=True)
(tfparse-test) ~/Work/misc/tfparse-wtf/test $ cat test.tf
locals {
  vpc_id    = "<vpc_id>"
  subnet_id = "<subnet_id>"
  tags = {
    service     = "myservice"
    application = "myapplication"
    oncall      = "ME"
  }
}

module "my_instance" {
  source             = "[email protected]:iac/terraform-aws-ec2.git//instance"
  hostname           = "myhostname"
  service_key        = "myservice"
  security_group_ids = [aws_security_group.my_sg.id]
  subnet_id          = "mysubnet"
  tags               = local.tags
}

resource "aws_security_group" "my_sg" {
  name   = "myservice-sg"
  vpc_id = local.vpc_id

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["10.0.0.0/8"]
  }
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = local.tags
}
(tfparse-test) ~/Work/misc/tfparse-wtf/test $ ./parse.py
51:50.555307000 terraform.parser.<root>          Setting project/module root to '.'
51:50.555803000 terraform.parser.<root>          Parsing FS from '.'
51:50.556035000 terraform.parser.<root>          Parsing 'test.tf'...
51:50.556599000 terraform.parser.<root>          Added file test.tf.
51:50.556631000 terraform.parser.<root>          Evaluating module...
51:50.556844000 terraform.parser.<root>          Read 3 block(s) and 0 ignore(s) for module 'root' (1 file[s])...
51:50.556870000 terraform.parser.<root>          Added 0 variables from tfvars.
51:50.556892000 terraform.parser.<root>          Error loading module metadata: open .terraform/modules/modules.json: no such file or directory.
51:50.556913000 terraform.parser.<root>          Working directory for module evaluation is '/Users/zzzzz/Work/misc/tfparse-wtf/test'
51:50.556973000 terraform.parser.<root>.evaluator Filesystem key is '4a38c2413d8b6e6df90c8e8f87e44e0e96fabb95e3ee33d3e6d7253583c5e1dd'
51:50.556982000 terraform.parser.<root>.evaluator Starting module evaluation...
51:50.557148000 terraform.parser.<root>.evaluator Starting submodule evaluation...
51:50.557159000 terraform.parser.<root>.evaluator locating non-initialised module '[email protected]:iac/terraform-aws-ec2.git//instance'...
51:50.557167000 terraform.parser.<root>.evaluator.resolver Resolving module 'module.my_instance' with source: '[email protected]:iac/terraform-aws-ec2.git//instance'...
51:50.557397000 terraform.parser.<root>.evaluator.resolver Trying to resolve: 6930244acd24452ca855ba1695a1b415
51:50.557463000 terraform.parser.<root>.evaluator.resolver Module 'module.my_instance' resolving via cache...
51:50.557612000 terraform.parser.<root>.evaluator.resolver Module path is .
51:50.557624000 terraform.parser.<root>.evaluator Module 'module.my_instance' resolved to path '.' in filesystem '/var/folders/3t/btv872q92n3c69blx6f70q8h0000gp/T/.aqua/cache/6930244acd24452ca855ba1695a1b415' with prefix '[email protected]:iac/terraform-aws-ec2.git/instance'
51:50.557637000 terraform.parser.<my_instance>   Parsing FS from '.'
51:50.557833000 terraform.parser.<my_instance>   Parsing 'data.tf'...
51:50.558244000 terraform.parser.<my_instance>   Added file data.tf.
51:50.558269000 terraform.parser.<my_instance>   Parsing 'main.tf'...
51:50.559748000 terraform.parser.<my_instance>   Added file main.tf.
51:50.559780000 terraform.parser.<my_instance>   Parsing 'outputs.tf'...
51:50.560012000 terraform.parser.<my_instance>   Added file outputs.tf.
51:50.560035000 terraform.parser.<my_instance>   Parsing 'variables.tf'...
51:50.561406000 terraform.parser.<my_instance>   Added file variables.tf.
51:50.561436000 terraform.parser.<root>.evaluator Loaded module 'my_instance' from '.'.
51:50.561441000 terraform.parser.<my_instance>   Evaluating module...
51:50.565829000 terraform.parser.<my_instance>   Read 58 block(s) and 0 ignore(s) for module 'my_instance' (4 file[s])...
51:50.565871000 terraform.parser.<my_instance>   Added 7 input variables from module definition.
51:50.565973000 terraform.parser.<my_instance>   Error loading module metadata: open /var/folders/3t/btv872q92n3c69blx6f70q8h0000gp/T/.aqua/cache/6930244acd24452ca855ba1695a1b415/.terraform/modules/modules.json: no such file or directory.
51:50.566000000 terraform.parser.<my_instance>   Working directory for module evaluation is '/Users/zzzzz/Work/misc/tfparse-wtf/test'
51:50.566050000 terraform.parser.<my_instance>.evaluator Filesystem key is 'fdf7f0dbccf648dfcfdc8b6e11736de2f3cd032ea9cf129a81ff98cfb9d26896'
51:50.566056000 terraform.parser.<my_instance>.evaluator Starting module evaluation...
51:50.568457000 terraform.parser.<my_instance>.evaluator Expanded block 'aws_ebs_volume.add_volumes' into 0 clones via 'count' attribute.
51:50.568488000 terraform.parser.<my_instance>.evaluator Expanded block 'aws_network_interface.localnic' into 0 clones via 'count' attribute.
51:50.568507000 terraform.parser.<my_instance>.evaluator Expanded block 'aws_volume_attachment.volume_attachment' into 0 clones via 'count' attribute.
51:50.568520000 terraform.parser.<my_instance>.evaluator Expanded block 'module.fqdn' into 0 clones via 'count' attribute.
51:50.568669000 terraform.parser.<my_instance>.evaluator Expanded block 'module.hostname' into 1 clones via 'count' attribute.
51:50.568766000 terraform.parser.<my_instance>.evaluator Expanded block 'module.idm_enroll' into 1 clones via 'count' attribute.
51:50.568793000 terraform.parser.<my_instance>.evaluator Expanded block 'module.outpost' into 0 clones via 'count' attribute.
51:50.568924000 terraform.parser.<my_instance>.evaluator Expanded block 'module.the_instance_rr' into 1 clones via 'count' attribute.
51:50.569040000 terraform.parser.<my_instance>.evaluator Starting submodule evaluation...
51:50.569057000 terraform.parser.<my_instance>.evaluator locating non-initialised module '../_modules/asa'...
51:50.569067000 terraform.parser.<my_instance>.evaluator.resolver Resolving module 'module.my_instance.module.asa' with source: '../_modules/asa'...
51:50.569316000 terraform.parser.<my_instance>.evaluator.resolver Trying to resolve: b9e5cc718cbbafa1ced42bb35c2187e8
51:50.569392000 terraform.parser.<my_instance>.evaluator.resolver Module 'module.my_instance.module.asa' resolved locally to ../_modules/asa
51:50.569400000 terraform.parser.<my_instance>.evaluator.resolver Module path is ../_modules/asa
51:50.569408000 terraform.parser.<my_instance>.evaluator Module 'module.my_instance.module.asa' resolved to path '../_modules/asa' in filesystem '/var/folders/3t/btv872q92n3c69blx6f70q8h0000gp/T/.aqua/cache/6930244acd24452ca855ba1695a1b415' with prefix '[email protected]:iac/terraform-aws-ec2.git/instance'
51:50.569422000 terraform.parser.<my_instance>.evaluator Failed to load module 'stat ../_modules/asa: invalid argument'. Maybe try 'terraform init'?
51:50.569430000 terraform.parser.<my_instance>.evaluator locating non-initialised module '../_modules/dns'...
51:50.569435000 terraform.parser.<my_instance>.evaluator.resolver Resolving module 'module.my_instance.module.dns' with source: '../_modules/dns'...
51:50.569655000 terraform.parser.<my_instance>.evaluator.resolver Trying to resolve: 8e5be0f5c65f05b59591ccc200a431ff
51:50.569773000 terraform.parser.<my_instance>.evaluator.resolver Module 'module.my_instance.module.dns' resolved locally to ../_modules/dns
51:50.569784000 terraform.parser.<my_instance>.evaluator.resolver Module path is ../_modules/dns
51:50.569791000 terraform.parser.<my_instance>.evaluator Module 'module.my_instance.module.dns' resolved to path '../_modules/dns' in filesystem '/var/folders/3t/btv872q92n3c69blx6f70q8h0000gp/T/.aqua/cache/6930244acd24452ca855ba1695a1b415' with prefix '[email protected]:iac/terraform-aws-ec2.git/instance'
51:50.569800000 terraform.parser.<my_instance>.evaluator Failed to load module 'stat ../_modules/dns: invalid argument'. Maybe try 'terraform init'?
51:50.569807000 terraform.parser.<my_instance>.evaluator locating non-initialised module '../_modules/finalize'...
51:50.569812000 terraform.parser.<my_instance>.evaluator.resolver Resolving module 'module.my_instance.module.finalize' with source: '../_modules/finalize'...
51:50.570054000 terraform.parser.<my_instance>.evaluator.resolver Trying to resolve: 53b6d0992a103f84b742bcb9ce1ffb15
51:50.570133000 terraform.parser.<my_instance>.evaluator.resolver Module 'module.my_instance.module.finalize' resolved locally to ../_modules/finalize
51:50.570141000 terraform.parser.<my_instance>.evaluator.resolver Module path is ../_modules/finalize
51:50.570148000 terraform.parser.<my_instance>.evaluator Module 'module.my_instance.module.finalize' resolved to path '../_modules/finalize' in filesystem '/var/folders/3t/btv872q92n3c69blx6f70q8h0000gp/T/.aqua/cache/6930244acd24452ca855ba1695a1b415' with prefix '[email protected]:iac/terraform-aws-ec2.git/instance'
51:50.570156000 terraform.parser.<my_instance>.evaluator Failed to load module 'stat ../_modules/finalize: invalid argument'. Maybe try 'terraform init'?
51:50.570163000 terraform.parser.<my_instance>.evaluator locating non-initialised module '../_modules/initialize'...
51:50.570169000 terraform.parser.<my_instance>.evaluator.resolver Resolving module 'module.my_instance.module.initialize' with source: '../_modules/initialize'...
51:50.570315000 terraform.parser.<my_instance>.evaluator.resolver Trying to resolve: 8231b14d2e0327179ad85efbf82a26a3
51:50.570374000 terraform.parser.<my_instance>.evaluator.resolver Module 'module.my_instance.module.initialize' resolved locally to ../_modules/initialize
51:50.570381000 terraform.parser.<my_instance>.evaluator.resolver Module path is ../_modules/initialize
51:50.570387000 terraform.parser.<my_instance>.evaluator Module 'module.my_instance.module.initialize' resolved to path '../_modules/initialize' in filesystem '/var/folders/3t/btv872q92n3c69blx6f70q8h0000gp/T/.aqua/cache/6930244acd24452ca855ba1695a1b415' with prefix '[email protected]:iac/terraform-aws-ec2.git/instance'
51:50.570396000 terraform.parser.<my_instance>.evaluator Failed to load module 'stat ../_modules/initialize: invalid argument'. Maybe try 'terraform init'?
51:50.570402000 terraform.parser.<my_instance>.evaluator locating non-initialised module '../_modules/nessusagent'...
51:50.570407000 terraform.parser.<my_instance>.evaluator.resolver Resolving module 'module.my_instance.module.nessusagent' with source: '../_modules/nessusagent'...
51:50.570604000 terraform.parser.<my_instance>.evaluator.resolver Trying to resolve: 95cfc8137c2f8a2c99ff43683d3dd323
51:50.570684000 terraform.parser.<my_instance>.evaluator.resolver Module 'module.my_instance.module.nessusagent' resolved locally to ../_modules/nessusagent
51:50.570691000 terraform.parser.<my_instance>.evaluator.resolver Module path is ../_modules/nessusagent
51:50.570698000 terraform.parser.<my_instance>.evaluator Module 'module.my_instance.module.nessusagent' resolved to path '../_modules/nessusagent' in filesystem '/var/folders/3t/btv872q92n3c69blx6f70q8h0000gp/T/.aqua/cache/6930244acd24452ca855ba1695a1b415' with prefix '[email protected]:iac/terraform-aws-ec2.git/instance'
51:50.570706000 terraform.parser.<my_instance>.evaluator Failed to load module 'stat ../_modules/nessusagent: invalid argument'. Maybe try 'terraform init'?
51:50.570721000 terraform.parser.<my_instance>.evaluator locating non-initialised module '../_modules/hostname'...
51:50.570727000 terraform.parser.<my_instance>.evaluator.resolver Resolving module 'module.my_instance.module.hostname[0]' with source: '../_modules/hostname'...
51:50.570933000 terraform.parser.<my_instance>.evaluator.resolver Trying to resolve: 477caa982fb73ac8995bc2b848b47375
51:50.570992000 terraform.parser.<my_instance>.evaluator.resolver Module 'module.my_instance.module.hostname[0]' resolved locally to ../_modules/hostname
51:50.571000000 terraform.parser.<my_instance>.evaluator.resolver Module path is ../_modules/hostname
51:50.571008000 terraform.parser.<my_instance>.evaluator Module 'module.my_instance.module.hostname[0]' resolved to path '../_modules/hostname' in filesystem '/var/folders/3t/btv872q92n3c69blx6f70q8h0000gp/T/.aqua/cache/6930244acd24452ca855ba1695a1b415' with prefix '[email protected]:iac/terraform-aws-ec2.git/instance'
51:50.571015000 terraform.parser.<my_instance>.evaluator Failed to load module 'stat ../_modules/hostname: invalid argument'. Maybe try 'terraform init'?
51:50.571022000 terraform.parser.<my_instance>.evaluator locating non-initialised module '../_modules/idm-enroll'...
51:50.571027000 terraform.parser.<my_instance>.evaluator.resolver Resolving module 'module.my_instance.module.idm_enroll[0]' with source: '../_modules/idm-enroll'...
51:50.571154000 terraform.parser.<my_instance>.evaluator.resolver Trying to resolve: feb2fb368a76cc3973acce945be93719
51:50.571202000 terraform.parser.<my_instance>.evaluator.resolver Module 'module.my_instance.module.idm_enroll[0]' resolved locally to ../_modules/idm-enroll
51:50.571208000 terraform.parser.<my_instance>.evaluator.resolver Module path is ../_modules/idm-enroll
51:50.571215000 terraform.parser.<my_instance>.evaluator Module 'module.my_instance.module.idm_enroll[0]' resolved to path '../_modules/idm-enroll' in filesystem '/var/folders/3t/btv872q92n3c69blx6f70q8h0000gp/T/.aqua/cache/6930244acd24452ca855ba1695a1b415' with prefix '[email protected]:iac/terraform-aws-ec2.git/instance'
51:50.571220000 terraform.parser.<my_instance>.evaluator Failed to load module 'stat ../_modules/idm-enroll: invalid argument'. Maybe try 'terraform init'?
51:50.571227000 terraform.parser.<my_instance>.evaluator locating non-initialised module '../_modules/rr'...
51:50.571232000 terraform.parser.<my_instance>.evaluator.resolver Resolving module 'module.my_instance.module.the_instance_rr[0]' with source: '../_modules/rr'...
51:50.571363000 terraform.parser.<my_instance>.evaluator.resolver Trying to resolve: 8d0386dccb4799df85b869f6e012cd0d
51:50.571409000 terraform.parser.<my_instance>.evaluator.resolver Module 'module.my_instance.module.the_instance_rr[0]' resolved locally to ../_modules/rr
51:50.571415000 terraform.parser.<my_instance>.evaluator.resolver Module path is ../_modules/rr
51:50.571421000 terraform.parser.<my_instance>.evaluator Module 'module.my_instance.module.the_instance_rr[0]' resolved to path '../_modules/rr' in filesystem '/var/folders/3t/btv872q92n3c69blx6f70q8h0000gp/T/.aqua/cache/6930244acd24452ca855ba1695a1b415' with prefix '[email protected]:iac/terraform-aws-ec2.git/instance'
51:50.571426000 terraform.parser.<my_instance>.evaluator Failed to load module 'stat ../_modules/rr: invalid argument'. Maybe try 'terraform init'?
51:50.571429000 terraform.parser.<my_instance>.evaluator Finished processing 0 submodule(s).
51:50.571432000 terraform.parser.<my_instance>.evaluator Starting post-submodule evaluation...
51:50.572522000 terraform.parser.<my_instance>.evaluator Module evaluation complete.
51:50.572540000 terraform.parser.<my_instance>   Finished parsing module 'my_instance'.
51:50.572547000 terraform.parser.<my_instance>.evaluator Added module output fqdn=cty.NilVal.
51:50.572600000 terraform.parser.<my_instance>.evaluator Added module output instance=cty.ObjectVal(map[string]cty.Value{"ami":cty.NilVal, "arn":cty.StringVal("eac6e51a-69f4-4449-8282-7cae1fd39ee1"), "associate_public_ip_address":cty.False, "availability_zone":cty.NilVal, "iam_instance_profile":cty.NilVal, "id":cty.StringVal("eac6e51a-69f4-4449-8282-7cae1fd39ee1"), "instance_type":cty.StringVal("t3.small"), "key_name":cty.NilVal, "subnet_id":cty.NilVal, "tags":cty.ObjectVal(map[string]cty.Value{"Name":cty.NilVal, "ami":cty.StringVal("mq_base_amzn2_x86_64"), "application":cty.StringVal("myapplication"), "oncall":cty.StringVal("ME"), "service":cty.StringVal("myservice")}), "user_data":cty.NilVal, "vpc_security_group_ids":cty.TupleVal([]cty.Value{cty.StringVal("c077ad44-d95d-424f-9dad-5bc47959e7be")})}).
51:50.572625000 terraform.parser.<my_instance>.evaluator Added module output instance_profile=cty.ObjectVal(map[string]cty.Value{"arn":cty.StringVal("ce6e1755-fd5f-45f8-9ce4-cf9d1237cf60"), "id":cty.StringVal("ce6e1755-fd5f-45f8-9ce4-cf9d1237cf60"), "role":cty.NilVal, "tags":cty.ObjectVal(map[string]cty.Value{"application":cty.StringVal("myapplication"), "oncall":cty.StringVal("ME"), "service":cty.StringVal("myservice")})}).
51:50.572643000 terraform.parser.<my_instance>.evaluator Added module output private_dns=cty.NilVal.
51:50.572648000 terraform.parser.<my_instance>.evaluator Added module output private_ip=cty.NilVal.
51:50.572658000 terraform.parser.<my_instance>.evaluator Added module output route53_record=cty.NilVal.
51:50.572672000 terraform.parser.<root>.evaluator Finished processing 1 submodule(s).
51:50.572676000 terraform.parser.<root>.evaluator Starting post-submodule evaluation...
51:50.572827000 terraform.parser.<root>.evaluator Module evaluation complete.
51:50.572833000 terraform.parser.<root>          Finished parsing module 'root'.
(tfparse-test) ~/Work/misc/tfparse-wtf/test $

Document local dev and add some basic scripts for working locally (makefile/justfile/whatever)

It'd be nice to have some docs on:

How to build the go/python to make sure its working
How to setup your vscode so that it can debug the go
How to run tests and all that

If we have a makefile that does this it'd be great, but docs are good enough.

Module file paths are incorrect

If you write a policy against a resource that matches a module it comes up with a path:

terraform/staging-new/terraform-aws-modules/acm/aws/.terraform/modules/approval_staging_com_acm/main.tf

It should not have terraform-aws-modules/acm/aws/ on there. The file path should be:

terraform/staging-new/.terraform/modules/approval_staging_com_acm/main.tf

We'll need to pre-process it, so for example if they say:

source  = "terraform-aws-modules/notify-slack/aws"

thats just saying "pull from the terraform-aws-modules group in the registry" but on disk they don't include the group so we need to strip it.

This causes problems if you run c7n-left because its going to try to show you the source code but not be able to open the file:

Traceback (most recent call last):
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-nOD_wM_f-py3.10/bin/c7n-left", line 6, in <module>
    sys.exit(cli())
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-nOD_wM_f-py3.10/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-nOD_wM_f-py3.10/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-nOD_wM_f-py3.10/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-nOD_wM_f-py3.10/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-nOD_wM_f-py3.10/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/john/code/cloud-custodian/tools/c7n_left/c7n_left/cli.py", line 43, in run
    runner.run()
  File "/Users/john/code/cloud-custodian/tools/c7n_left/c7n_left/core.py", line 63, in run
    self.reporter.on_results(result_set)
  File "/Users/john/code/cloud-custodian/tools/c7n_left/c7n_left/output.py", line 96, in on_results
    self.console.print(RichResult(r))
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-nOD_wM_f-py3.10/lib/python3.10/site-packages/rich/console.py", line 1673, in print
    extend(render(renderable, render_options))
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-nOD_wM_f-py3.10/lib/python3.10/site-packages/rich/console.py", line 1305, in render
    for render_output in iter_render:
  File "/Users/john/code/cloud-custodian/tools/c7n_left/c7n_left/output.py", line 111, in __rich_console__
    lines = resource.get_source_lines()
  File "/Users/john/code/cloud-custodian/tools/c7n_left/c7n_left/providers/terraform/resource.py", line 38, in get_source_lines
    lines = (self.src_dir / self.filename).read_text().split("\n")
  File "/Users/john/.asdf/installs/python/3.10.5/lib/python3.10/pathlib.py", line 1134, in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
  File "/Users/john/.asdf/installs/python/3.10.5/lib/python3.10/pathlib.py", line 1119, in open
    return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: '/Users/john/code/terraform/staging-new/terraform-aws-modules/acm/aws/.terraform/modules/approval_staging_com_acm/main.tf'

Update Golang version in 'ci-setup-golang.sh' to 1.21.5

In https://github.com/cloud-custodian/tfparse/blob/main/ci-setup-golang.sh script we are using the go1.19.2 version. Apparently go1.19.2 has vulnerabilities CVE-2023-24538, CVE-2023-24540, CVE-2023-45285.

I would like to check if we can update the golang version to >= 1.21.5 to remediate above mentioned vulnerabilities?

Hit an exception when running against this that uses `toset`

I hit this exception:

Traceback (most recent call last):
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-left-m2D9Q8mD-py3.10/bin/c7n-left", line 6, in <module>
    sys.exit(cli())
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-left-m2D9Q8mD-py3.10/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-left-m2D9Q8mD-py3.10/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-left-m2D9Q8mD-py3.10/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-left-m2D9Q8mD-py3.10/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/john/Library/Caches/pypoetry/virtualenvs/c7n-left-m2D9Q8mD-py3.10/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/john/code/stacklet/cloud-custodian/tools/c7n_left/c7n_left/cli.py", line 41, in run
    runner.run()
  File "/Users/john/code/stacklet/cloud-custodian/tools/c7n_left/c7n_left/provider.py", line 61, in run
    for rtype, resources in graph.get_resources_by_type():
  File "/Users/john/code/stacklet/cloud-custodian/tools/c7n_left/c7n_left/provider.py", line 284, in get_resources_by_type
    resources.append(self.as_resource(name, data))
  File "/Users/john/code/stacklet/cloud-custodian/tools/c7n_left/c7n_left/provider.py", line 288, in as_resource
    if isinstance(data["__tfmeta"], list):
KeyError: '__tfmeta'

when running against this TF File:

locals {
  cluster_endpoints = toset(["us-east-1-eks-01", "us-east-1-eks-02"])
  pages_to_monitor  = toset([{ path = "/pricing", name = "pricing" }, { path = "/about/", name = "about" }])
  cluster_pages_monitor = distinct(flatten([
    for ep in local.cluster_endpoints : [
      for page in local.pages_to_monitor : {
        page    = page
        cluster = ep
      }
    ]
  ]))
}
# Healthcheck per cluster and page
resource "aws_route53_health_check" "cluster_pages" {
  for_each          = { for entry in local.cluster_pages_monitor : "${entry.cluster}.${entry.page.name}" => entry }
  fqdn              = "${each.value.cluster}.${var.domain}"
  port              = 443
  type              = "HTTPS"
  resource_path     = each.value.page.path
  failure_threshold = "2"
  request_interval  = "10"

  tags = {
    Name    = "${var.domain}-${each.value.cluster}-${each.value.page.name}"
    Cluster = each.value.cluster
  }
}

resource "aws_route53_health_check" "cluster_aggregate" {
  for_each               = local.cluster_endpoints
  type                   = "CALCULATED"
  child_health_threshold = length([for s in aws_route53_health_check.cluster_pages : s.id if s.tags_all["Cluster"] == each.key])
  child_healthchecks     = [for s in aws_route53_health_check.cluster_pages : s.id if s.tags_all["Cluster"] == each.key]

  tags = {
    Name    = "${var.domain}-${each.key}-agg"
    Cluster = each.key
  }
}
resource "aws_sns_topic" "failed_dns_healthchecks" {
  name = "${var.environment}-dns-healthcheck-failures"
}
resource "aws_sns_topic_subscription" "monitoring_pagerduty" {
  topic_arn              = aws_sns_topic.failed_dns_healthchecks.arn
  protocol               = "https"
  endpoint               = "https://events.pagerduty.com/integration/c0cac4d374f04806c0d7029726a7059e/enqueue"
  endpoint_auto_confirms = true
}
resource "aws_cloudwatch_metric_alarm" "cluster_agg_healthcheck_failed" {
  for_each            = local.cluster_endpoints
  alarm_name          = "${var.environment}_${each.key}_agg_healthcheck_failed"
  namespace           = "AWS/Route53"
  metric_name         = "HealthCheckStatus"
  comparison_operator = "LessThanThreshold"
  evaluation_periods  = "1"
  period              = "60"
  statistic           = "Minimum"
  threshold           = "1"
  unit                = "None"
  dimensions = {
    HealthCheckId = [for s in aws_route53_health_check.cluster_aggregate : s.id if s.tags_all["Cluster"] == each.key][0]
  }
  alarm_description  = "This metric monitors whether the cluster aggregate healthcheck is down or not."
  alarm_actions      = [aws_sns_topic.failed_dns_healthchecks.arn]
  ok_actions         = [aws_sns_topic.failed_dns_healthchecks.arn]
  treat_missing_data = "breaching"
  depends_on         = ["aws_route53_health_check.cluster_aggregate"]

  tags = {
    Cluster = each.key
  }
}

The data looked like this:

{'about"]': {'__tfmeta': {'filename': 'dns_healthchecks.tf',
                          'line_end': 27,
                          'line_start': 14},
             'failure_threshold': '2',
             'for_each': {'us-east-1-eks-01.about': {'cluster': 'us-east-1-eks-01',
                                                     'page': {'name': 'about',
                                                              'path': '/about/'}},
                          'us-east-1-eks-01.pricing': {'cluster': 'us-east-1-eks-01',
                                                       'page': {'name': 'pricing',
                                                                'path': '/pricing'}},
                          'us-east-1-eks-02.about': {'cluster': 'us-east-1-eks-02',
                                                     'page': {'name': 'about',
                                                              'path': '/about/'}},
                          'us-east-1-eks-02.pricing': {'cluster': 'us-east-1-eks-02',
                                                       'page': {'name': 'pricing',
                                                                'path': '/pricing'}}},
             'fqdn': 'us-east-1-eks-01.staging.com',
             'id': '98434e8a-1a52-4e94-a603-4e503168aab6',
             'port': 443,
             'request_interval': '10',
             'resource_path': '/about/',
             'tags': {'Cluster': 'us-east-1-eks-01',
                      'Name': 'staging.com-us-east-1-eks-01-about'},
             'type': 'HTTPS'},
 'pricing"]': {'__tfmeta': {'filename': 'dns_healthchecks.tf',
                            'line_end': 27,
                            'line_start': 14},
               'failure_threshold': '2',
               'for_each': {'us-east-1-eks-01.about': {'cluster': 'us-east-1-eks-01',
                                                       'page': {'name': 'about',
                                                                'path': '/about/'}},
                            'us-east-1-eks-01.pricing': {'cluster': 'us-east-1-eks-01',
                                                         'page': {'name': 'pricing',
                                                                  'path': '/pricing'}},
                            'us-east-1-eks-02.about': {'cluster': 'us-east-1-eks-02',
                                                       'page': {'name': 'about',
                                                                'path': '/about/'}},
                            'us-east-1-eks-02.pricing': {'cluster': 'us-east-1-eks-02',
                                                         'page': {'name': 'pricing',
                                                                  'path': '/pricing'}}},
               'fqdn': 'us-east-1-eks-01.staging.com',
               'id': 'f95f4895-f142-4942-a74a-0a1b2043fe8c',
               'port': 443,
               'request_interval': '10',
               'resource_path': '/pricing',
               'tags': {'Cluster': 'us-east-1-eks-01',
                        'Name': 'staging.com-us-east-1-eks-01-pricing'},
               'type': 'HTTPS'}}

unit tests

embedded block gets recorded twice

resource "aws_apprunner_service" "example" {
  service_name = "example"

  source_configuration {
    image_repository {
      image_configuration {
        port = "8000"
      }
      image_identifier      = "public.ecr.aws/aws-containers/hello-app-runner:latest"
      image_repository_type = "ECR_PUBLIC"
    }
    auto_deployments_enabled = false
  }

  tags = {
    Name = "example-apprunner-service"
  }
}

parsed it comes out to

{
  "aws_apprunner_service": {
    "example": {
      "__tfmeta": {
        "filename": "main.tf",
        "line_start": 1,
        "line_end": 18
      },
      "id": "233bfdbe-87e0-4c68-b1f8-eeed134bfbc2",
      "service_name": "example",
      "source_configuration": [
        {
          "__tfmeta": {
            "filename": "main.tf",
            "line_end": 13,
            "line_start": 4
          },
          "auto_deployments_enabled": false,
          "id": "59b3a77a-7bd7-441a-8052-92e7918c9902"
        },
        {
          "image_repository": [
            {
              "__tfmeta": {
                "filename": "main.tf",
                "line_end": 11,
                "line_start": 5
              },
              "id": "97120d4b-6757-4e22-8bb2-214cb220b8a9",
              "image_identifier": "public.ecr.aws/aws-containers/hello-app-runner:latest",
              "image_repository_type": "ECR_PUBLIC"
            }
          ]
        },
        {
          "image_repository": {
            "image_configuration": [
              {
                "__tfmeta": {
                  "filename": "main.tf",
                  "line_end": 8,
                  "line_start": 6
                },
                "id": "6abfb151-96d9-4027-85fb-9a7707967098",
                "port": "8000"
              }
            ]
          }
        }
      ],
      "tags": {
        "Name": "example-apprunner-service"
      }
    }
  }
}

image_repository.configuration got appended to a sibling block instead of as a child under image_repository block.

CI isn't pushing a source build or binary builds for other python versions

We should ship python 3.9, 3.10, and 3.11 binaries along with a source tarball for those who are using older versions

server side encryption configuration and their s3 bucket need to be linked together

Right now if write this terraform:

resource "aws_s3_bucket" "unencrypted-bucket" {
    bucket = "my-unencrypted-bucket"
}

resource "aws_s3_bucket_server_side_encryption_configuration" "encrypt-bucket" {
  bucket = aws_s3_bucket.unencrypted-bucket.bucket

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = "AES256"
    }
  }
}

There is no link that says "This configuration is on this bucket" for the policy to compare against. Ideally we'd be able to write a policy that checks for the absence of a server_side_encryption_configuration and fail the build. This is what the tfparse data looks like right now:

{
        "aws_s3_bucket": [
                {
                        "__tfmeta": {
                                "filename": "main.tf",
                                "line_end": 3,
                                "line_start": 1,
                                "path": "aws_s3_bucket.unencrypted-bucket"
                        },
                        "bucket": "my-unencrypted-bucket",
                        "id": "69e3aa81-ce65-4cfb-ab39-2bca0026af8f"
                }
        ],
        "aws_s3_bucket_server_side_encryption_configuration": [
                {
                        "__tfmeta": {
                                "filename": "main.tf",
                                "line_end": 13,
                                "line_start": 5,
                                "path": "aws_s3_bucket_server_side_encryption_configuration.encrypt-bucket"
                        },
                        "bucket": "69e3aa81-ce65-4cfb-ab39-2bca0026af8f",
                        "id": "3ce4c2f0-4947-4eb3-8a48-ad6af49f6e38",
                        "rule": {
                                "__tfmeta": {
                                        "filename": "main.tf",
                                        "line_end": 12,
                                        "line_start": 8
                                },
                                "apply_server_side_encryption_by_default": {
                                        "__tfmeta": {
                                                "filename": "main.tf",
                                                "line_end": 11,
                                                "line_start": 9
                                        },
                                        "id": "c1a30d4d-5edb-4dfa-90a4-d8659640540d",
                                        "sse_algorithm": "AES256"
                                },
                                "id": "75b4b3de-ca9c-4829-adeb-84256602239a"
                        }
                }
        ]
}

pass variables to load_from_path

alternatively we could try to inject auto variables files into directory.

Resource references don't seem to work when combined with `for_each`

In https://github.com/orgs/cloud-custodian/discussions/9181 there is a question about checking S3 server-side encryption settings that are defined in separate resources. That works fine for individual buckets:

resource "aws_s3_bucket" "sse_enabled_separately" {
  bucket = "sse_enabled_separately"
}

resource "aws_s3_bucket_server_side_encryption_configuration" "sse_enabled_separately" {
  bucket = aws_s3_bucket.sse_enabled_separately.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

But if we add a for_each the resulting graph doesn't seem to include references, so the c7n-left traverse filter can't do its magic.

locals {
  sse_separate = toset(["mybucket"])
}

resource "aws_s3_bucket" "sse_separate_for_each" {
  for_each            = local.sse_separate
  bucket              = each.key
  object_lock_enabled = true
}

resource "aws_s3_bucket_server_side_encryption_configuration" "sse_separate_for_each" {
  for_each = aws_s3_bucket.sse_separate_for_each
  bucket   = local.sse_separate

  # References are also missing for these variations:
  # bucket = each.key
  # bucket = each.value.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

support python 3.12

certain blocks end up with combined line numbers instead of separate

locals and provider for example

>>> pprint.pprint(x['locals'])
{'__tfmeta': [{'filename': 'db.tf', 'line_end': 6, 'line_start': 1},
              {'filename': 'ecs_task.tf', 'line_end': 29, 'line_start': 2},
              {'filename': 'locals.tf', 'line_end': 69, 'line_start': 23}],

Missing submodule parsed data

modules.tf file content:

module "utils_naming_degradation" {
  source    = "../utils/naming_degradation"
  providers = {}
  env_name  = var.env_name
  names = {
    name = {
      components = [var.name]
      max_length = -1
    }
    validate_certificate_name = {
      components = [var.name]
      max_length = 60
    }
  }
}

module "validate_certificate_files" {
  source = "../utils/local_script"
  providers = {
    aws   = aws
    shell = shell
  }
  count    = var.custom_certificate == null ? 0 : 1
  env_name = var.env_name
  name     = module.utils_naming_degradation.names.validate_certificate_name[1]
  create = {
    script_file = "${path.module}/script/validate_certificate_files/validate_certificate_files.py"
    arguments = [
      aws_route53_zone.zone.name,
      var.custom_certificate.private_key,
      local.custom_certificate_body,
      local.custom_certificate_chain
    ]
  }
}

validate_certificate_files isn't even parsed for some reason.

Parsed data:

{
  "aws_acm_certificate": [
    {
      "__tfmeta": {
        "filename": "main.tf",
        "label": "aws_acm_certificate",
        "line_end": 51,
        "line_start": 37,
        "path": "aws_acm_certificate.certificate[0]",
        "type": "resource"
      },
      "count": 1,
      "domain_name": "2d43a436-905e-4efd-8cf7-ec68c45d4972",
      "id": "14934d07-8eee-49c7-8d1e-696d72297e19",
      "options": {
        "__tfmeta": {
          "filename": "main.tf",
          "line_end": 46,
          "line_start": 44
        },
        "certificate_transparency_logging_preference": "ENABLED",
        "id": "866be1e4-431a-42cf-83ad-4a5a82de6c1a"
      },
      "subject_alternative_names": "2d43a436-905e-4efd-8cf7-ec68c45d4972",
      "tags": null,
      "validation_method": "DNS"
    },
    {
      "__tfmeta": {
        "filename": "main.tf",
        "label": "aws_acm_certificate",
        "line_end": 35,
        "line_start": 23,
        "path": "aws_acm_certificate.certificate_secondary[0]",
        "type": "resource"
      },
      "count": 1,
      "domain_name": "2d43a436-905e-4efd-8cf7-ec68c45d4972",
      "id": "f98f45e6-3c90-4367-9ab1-81379c7cd486",
      "options": {
        "__tfmeta": {
          "filename": "main.tf",
          "line_end": 33,
          "line_start": 31
        },
        "certificate_transparency_logging_preference": "ENABLED",
        "id": "44ec91c7-e7c7-4a9a-95df-26d287529df0"
      },
      "provider": null,
      "subject_alternative_names": "2d43a436-905e-4efd-8cf7-ec68c45d4972",
      "tags": null,
      "validation_method": "DNS"
    }
  ],
  "aws_acm_certificate_validation": [
    {
      "__tfmeta": {
        "filename": "main.tf",
        "label": "aws_acm_certificate_validation",
        "line_end": 83,
        "line_start": 77,
        "path": "aws_acm_certificate_validation.certificate[0]",
        "type": "resource"
      },
      "certificate_arn": "14934d07-8eee-49c7-8d1e-696d72297e19",
      "count": 1,
      "id": "24825a1f-5a6c-4ca5-91ab-9616c6c87a46",
      "validation_record_fqdns": "f8489c22-e4dc-4113-8b61-4d0fc405342e"
    },
    {
      "__tfmeta": {
        "filename": "main.tf",
        "label": "aws_acm_certificate_validation",
        "line_end": 75,
        "line_start": 68,
        "path": "aws_acm_certificate_validation.certificate_secondary[0]",
        "type": "resource"
      },
      "certificate_arn": "f98f45e6-3c90-4367-9ab1-81379c7cd486",
      "count": 1,
      "id": "a73cc630-ef64-4c55-abac-bac1eb73dabc",
      "provider": null,
      "validation_record_fqdns": "f8489c22-e4dc-4113-8b61-4d0fc405342e"
    }
  ],
  "aws_partition": [
    {
      "__tfmeta": {
        "filename": "data.tf",
        "label": "aws_partition",
        "line_end": 7,
        "line_start": 7,
        "path": "data.aws_partition.current",
        "type": "data"
      },
      "id": "16284287-610d-4cd5-a117-94ef9b910011"
    }
  ],
  "aws_route53_record": [
    {
      "__tfmeta": {
        "filename": "main.tf",
        "label": "aws_route53_record",
        "line_end": 101,
        "line_start": 85,
        "path": "aws_route53_record.caa",
        "type": "resource"
      },
      "allow_overwrite": true,
      "health_check_id": null,
      "id": "99e0cacc-9700-4c48-adcb-2b69f500dc53",
      "multivalue_answer_routing_policy": false,
      "name": "",
      "records": [
        "0 issue \"amazon.com\"",
        "0 issuewild \"amazon.com\""
      ],
      "set_identifier": null,
      "ttl": 300,
      "type": "CAA",
      "zone_id": "2d43a436-905e-4efd-8cf7-ec68c45d4972"
    },
    {
      "__tfmeta": {
        "filename": "main.tf",
        "label": "aws_route53_record",
        "line_end": 66,
        "line_start": 53,
        "path": "aws_route53_record.certificate[0]",
        "type": "resource"
      },
      "allow_overwrite": true,
      "count": 1,
      "health_check_id": null,
      "id": "f8489c22-e4dc-4113-8b61-4d0fc405342e",
      "multivalue_answer_routing_policy": false,
      "name": null,
      "records": [
        null
      ],
      "set_identifier": null,
      "ttl": 300,
      "type": null,
      "zone_id": "2d43a436-905e-4efd-8cf7-ec68c45d4972"
    }
  ],
  "aws_route53_zone": [
    {
      "__tfmeta": {
        "filename": "main.tf",
        "label": "aws_route53_zone",
        "line_end": 7,
        "line_start": 1,
        "path": "aws_route53_zone.zone",
        "type": "resource"
      },
      "comment": null,
      "delegation_set_id": null,
      "force_destroy": true,
      "id": "2d43a436-905e-4efd-8cf7-ec68c45d4972",
      "name": null,
      "tags": null
    },
    {
      "__tfmeta": {
        "filename": "data.tf",
        "label": "aws_route53_zone",
        "line_end": 5,
        "line_start": 1,
        "path": "data.aws_route53_zone.parent",
        "type": "data"
      },
      "count": 0,
      "id": "5a837eaa-999d-4028-96f2-441c3c6305fe",
      "provider": null,
      "zone_id": ""
    }
  ],
  "locals": [
    {
      "__tfmeta": {
        "filename": "locals.tf",
        "line_end": 51,
        "line_start": 45,
        "path": "locals"
      },
      "custom_certificate_chain": null,
      "id": "446b0d3b-5ffa-4a76-88ca-669d383403a1"
    },
    {
      "__tfmeta": {
        "filename": "locals.tf",
        "line_end": 10,
        "line_start": 1,
        "path": "locals"
      },
      "default_tags": null,
      "id": "e77addb4-9e2d-44d4-907e-c445b0d2a331"
    },
    {
      "__tfmeta": {
        "filename": "locals.tf",
        "line_end": 29,
        "line_start": 27,
        "path": "locals"
      },
      "arn_prefix": "16284287-610d-4cd5-a117-94ef9b910011",
      "id": "12eae71a-ec63-42b2-ae03-ae5a19471cea"
    },
    {
      "__tfmeta": {
        "filename": "locals.tf",
        "line_end": 39,
        "line_start": 31,
        "path": "locals"
      },
      "id": "95be1065-db1e-45be-9746-5d2b9dfca67a",
      "split_custom_certificates": [

      ]
    },
    {
      "__tfmeta": {
        "filename": "locals.tf",
        "line_end": 43,
        "line_start": 41,
        "path": "locals"
      },
      "custom_certificate_body": null,
      "id": "b9972b08-d1f8-4c99-a9fa-4e1fb3dfdf12"
    },
    {
      "__tfmeta": {
        "filename": "locals.tf",
        "line_end": 18,
        "line_start": 12,
        "path": "locals"
      },
      "domain_validation_options": null,
      "id": "27be0dd3-f657-4d12-b9ee-3e6c7c9e1ebf"
    },
    {
      "__tfmeta": {
        "filename": "locals.tf",
        "line_end": 25,
        "line_start": 20,
        "path": "locals"
      },
      "certificate_authorities": null,
      "id": "5ad63428-6a79-4b50-86c9-730b084f01a2"
    }
  ],
  "module": [
    {
      "__tfmeta": {
        "filename": "modules.tf",
        "label": "utils_naming_degradation",
        "line_end": 15,
        "line_start": 1,
        "path": "module.utils_naming_degradation"
      },
      "env_name": "",
      "id": "696b6778-81d6-42e9-85bb-97c421321d9a",
      "names": {
        "name": {
          "components": [
            null
          ],
          "max_length": -1
        },
        "validate_certificate_name": {
          "components": [
            null
          ],
          "max_length": 60
        }
      },
      "providers": {

      },
      "source": "../utils/naming_degradation"
    }
  ],
  "output": [
    {
      "__tfmeta": {
        "filename": "outputs.tf",
        "label": "arn",
        "line_end": 7,
        "line_start": 5,
        "path": "output.arn"
      },
      "id": "e5683869-7f69-4b68-abd9-aa4bf2d035db",
      "value": "2d43a436-905e-4efd-8cf7-ec68c45d4972"
    },
    {
      "__tfmeta": {
        "filename": "outputs.tf",
        "label": "certificate_arn",
        "line_end": 19,
        "line_start": 17,
        "path": "output.certificate_arn"
      },
      "id": "fb811ab5-4804-41a5-b1d3-6ea145833909",
      "value": "24825a1f-5a6c-4ca5-91ab-9616c6c87a46"
    },
    {
      "__tfmeta": {
        "filename": "outputs.tf",
        "label": "certificate_arn_secondary",
        "line_end": 23,
        "line_start": 21,
        "path": "output.certificate_arn_secondary"
      },
      "id": "c2494cb1-2bac-4e1c-a51a-d74491070a4a",
      "value": "a73cc630-ef64-4c55-abac-bac1eb73dabc"
    },
    {
      "__tfmeta": {
        "filename": "outputs.tf",
        "label": "custom_certificate_arn",
        "line_end": 27,
        "line_start": 25,
        "path": "output.custom_certificate_arn"
      },
      "id": "881cd140-9529-43fe-944b-9762046f33bb",
      "value": ""
    },
    {
      "__tfmeta": {
        "filename": "outputs.tf",
        "label": "domain_name",
        "line_end": 15,
        "line_start": 13,
        "path": "output.domain_name"
      },
      "id": "116c3ac7-57b8-426a-a88c-a7643c978a34",
      "value": "2d43a436-905e-4efd-8cf7-ec68c45d4972"
    },
    {
      "__tfmeta": {
        "filename": "outputs.tf",
        "label": "id",
        "line_end": 3,
        "line_start": 1,
        "path": "output.id"
      },
      "id": "1a1284f6-5248-4a4c-b8d0-949865d9910e",
      "value": "2d43a436-905e-4efd-8cf7-ec68c45d4972"
    },
    {
      "__tfmeta": {
        "filename": "outputs.tf",
        "label": "name_servers",
        "line_end": 11,
        "line_start": 9,
        "path": "output.name_servers"
      },
      "id": "bdee3354-cdda-4d8a-b681-6cfd446fe2cb",
      "value": "2d43a436-905e-4efd-8cf7-ec68c45d4972"
    }
  ],
  "terraform": [
    {
      "__tfmeta": {
        "filename": "providers.tf",
        "line_end": 14,
        "line_start": 1,
        "path": "terraform"
      },
      "id": "c2db8a21-cc60-4433-9920-a1b59ff42869",
      "required_providers": {
        "__tfmeta": {
          "filename": "providers.tf",
          "line_end": 13,
          "line_start": 2
        },
        "aws": {
          "configuration_aliases": [
            null,
            null
          ],
          "source": "hashicorp/aws"
        },
        "id": "aa27a0ed-1914-4480-a2c6-167dcdbcddbc",
        "shell": {
          "source": "scottwinkler/shell"
        }
      }
    }
  ],
  "variable": [
    {
      "__tfmeta": {
        "filename": "variables.tf",
        "label": "certificate_authorities",
        "line_end": 50,
        "line_start": 42,
        "path": "variable.certificate_authorities"
      },
      "default": [

      ],
      "description": "Zone certificate authorities (CAA record)",
      "id": "595c9152-8cc9-4105-8b60-5e72b6f22483",
      "type": null,
      "validation": {
        "__tfmeta": {
          "filename": "variables.tf",
          "line_end": 48,
          "line_start": 45
        },
        "condition": true,
        "error_message": "1. Zone certificate authorities (CAA record) should be valid.",
        "id": "c65eed04-63fd-4f9f-9a05-fdc3db8e52dd"
      }
    },
    {
      "__tfmeta": {
        "filename": "variables.tf",
        "label": "custom_certificate",
        "line_end": 69,
        "line_start": 52,
        "path": "variable.custom_certificate"
      },
      "default": null,
      "description": "Zone custom certificate",
      "id": "04336740-1a5a-415a-a97c-41f13d2e853a",
      "sensitive": true,
      "type": null,
      "validation": {
        "__tfmeta": {
          "filename": "variables.tf",
          "line_end": 66,
          "line_start": 59
        },
        "condition": null,
        "error_message": "1. Zone custom certificate files content have to be valid.",
        "id": "d47ff5a1-ac45-4999-a5a7-a7331eda6c0e"
      }
    },
    {
      "__tfmeta": {
        "filename": "variables.tf",
        "label": "env_name",
        "line_end": 9,
        "line_start": 1,
        "path": "variable.env_name"
      },
      "default": "",
      "description": "Environment name",
      "id": "e7171c5e-0b61-4c3f-8e3b-1843e2c7f47c",
      "type": null,
      "validation": {
        "__tfmeta": {
          "filename": "variables.tf",
          "line_end": 7,
          "line_start": 4
        },
        "condition": true,
        "error_message": "1. Environment name length should be between 1 and 15 characters.\n2. Environment name should consist only of alphanumeric characters, dots, spaces and dashes.",
        "id": "67007bef-90e6-4936-bc4b-07a88812f381"
      }
    },
    {
      "__tfmeta": {
        "filename": "variables.tf",
        "label": "generate_primary_certificate",
        "line_end": 34,
        "line_start": 30,
        "path": "variable.generate_primary_certificate"
      },
      "default": true,
      "description": "Generate primary certificate",
      "id": "a07fa03b-d166-4935-8355-197f3a4dc392",
      "type": null
    },
    {
      "__tfmeta": {
        "filename": "variables.tf",
        "label": "generate_secondary_certificate",
        "line_end": 40,
        "line_start": 36,
        "path": "variable.generate_secondary_certificate"
      },
      "default": true,
      "description": "Generate secondary certificate",
      "id": "94042d76-4390-482a-aade-b9e99f4f4e82",
      "type": null
    },
    {
      "__tfmeta": {
        "filename": "variables.tf",
        "label": "name",
        "line_end": 18,
        "line_start": 11,
        "path": "variable.name"
      },
      "description": "Zone domain name",
      "id": "e7fb6d5f-dcb1-4894-adf3-26ea5de42439",
      "type": null,
      "validation": {
        "__tfmeta": {
          "filename": "variables.tf",
          "line_end": 17,
          "line_start": 14
        },
        "condition": false,
        "error_message": "1. Zone domain name should be valid.",
        "id": "b4d647ad-59b5-4704-b2c7-c4646d7ab146"
      }
    },
    {
      "__tfmeta": {
        "filename": "variables.tf",
        "label": "parent_id",
        "line_end": 28,
        "line_start": 20,
        "path": "variable.parent_id"
      },
      "default": "",
      "description": "Parent zone ID",
      "id": "b0640952-982f-4136-9196-0a35713381d5",
      "type": null,
      "validation": {
        "__tfmeta": {
          "filename": "variables.tf",
          "line_end": 26,
          "line_start": 23
        },
        "condition": true,
        "error_message": "1. Parent zone ID should be between 2 and 32 characters in length, start with 'Z' and consist only of uppercase alphanumeric characters.",
        "id": "573b8ea7-2c05-4a20-88a3-ff18218a2a4e"
      }
    }
  ]
}

support binary wheels on windows

Currently we build (aarch64, amd64) * (linux, macOS) we should add windows (amd64) to the matrix.

github actions for building

Dynamic fields don't expand properly

dynamic fields in terraform are effectively a template and so they should be expanded out and the blocks they generate should be included for evaluation but not the dynamic blocks themselves.

Example:

variable "ephemeral_storage_size" {
  default = 100
}

variable "create_function" {
  default = true
}

variable "create_layer" {
  default = false
}

variable "function_name" {
  default = "one function"
}

locals {
  create = true
}

variable "create_role" {
  default = true
}

variable "lambda_role" {
  default = "lambda-role"
}

variable "package_type" {
  default = "some-package"
}

variable "handler" {
  default = "some-handler"
}

variable "lambda_at_edge" {
  default = true
}

variable "timeout" {
  default = 20
}

variable "publish" {
  default = true
}

resource "aws_lambda_function" "this" {
  count = local.create && var.create_function && !var.create_layer ? 1 : 0

  function_name                  = var.function_name
  role                           = var.create_role ? "some-arn" : var.lambda_role
  handler                        = var.package_type != "Zip" ? null : var.handler
  timeout                        = var.lambda_at_edge ? min(var.timeout, 30) : var.timeout
  publish                        = var.lambda_at_edge ? true : var.publish

  /* ephemeral_storage is not supported in gov-cloud region, so it should be set to `null` */
  dynamic "ephemeral_storage" {
    for_each = var.ephemeral_storage_size == null ? [] : [true]

    content {
      size = var.ephemeral_storage_size
    }
  }
}

ModuleNotFoundError first thing out of the box?

I updated to Python 3.11, did a pip install of tfparse, and get: "ModuleNotFoundError: No module named 'tfparse._tfparse'"

Having the same issue on Linux and Windows. Should I install an older version or something? Thanks!

allow passing in a pathlib.Path or str, don't require bytes

github actions - cache golang modules for faster ci/build

It is not handling `moved` resources

Example:

moved {
  from = aws_cloudwatch_log_group.task
  to   = aws_cloudwatch_log_group.task_upgrade_database
}

Error:

panic: unexpected resource type: moved

goroutine 17 [running, locked to thread]:
github.com/cloud-custodian/tfparse/gotfparse/pkg/converter.(*terraformConverter).visitBlock(0x105d74108?, 0x14000bc1500?, {0x0?, 0x0?}, 0x0?)
        /private/var/folders/0x/t100ywm11373bgv2zwxx7g7w0000gn/T/tmpt8qckrlx/src/github.com/cloud-custodian/tfparse/gotfparse/gotfparse/pkg/converter/converter.go:64 +0x288
github.com/cloud-custodian/tfparse/gotfparse/pkg/converter.(*terraformConverter).visitModule(0x11133ea20?, 0x14000d621e0, 0x0?)
        /private/var/folders/0x/t100ywm11373bgv2zwxx7g7w0000gn/T/tmpt8qckrlx/src/github.com/cloud-custodian/tfparse/gotfparse/gotfparse/pkg/converter/converter.go:45 +0x74
github.com/cloud-custodian/tfparse/gotfparse/pkg/converter.(*terraformConverter).VisitJSON(0x1400018f450)
        /private/var/folders/0x/t100ywm11373bgv2zwxx7g7w0000gn/T/tmpt8qckrlx/src/github.com/cloud-custodian/tfparse/gotfparse/gotfparse/pkg/converter/converter.go:34 +0xa4
main.Parse(0x14000190000?, 0x6000034043e8, 0x6000034042f8)
        /private/var/folders/0x/t100ywm11373bgv2zwxx7g7w0000gn/T/tmpt8qckrlx/src/github.com/cloud-custodian/tfparse/gotfparse/gotfparse/cmd/tfparse/main.go:35 +0x1b8
zsh: abort      c7n-left run --policy-dir $HOME/code/sontek/k8s/policies/terraform/ -d

Mac m1 binaries from github action builds appear to be wrong arch

they build fine locally, but the GitHub action powered build appears to be building with the wrong arch

~ on ☁️  (us-east-1) 
❯ pyenv virtualenv 3.10.6 test
pyenv⏎                                                                                                                                                                                                                                                                                                                                                                                                                                    
~ on ☁️  (us-east-1) took 2s 
❯ pyenv activate test
pyenv-virtualenv: prompt changing not working for fish.

~ via 🐍 v3.10.6 (test) on ☁️  (us-east-1) 
❯ pip install tfparse
Collecting tfparse
  Using cached tfparse-0.1.3-cp310-cp310-macosx_11_0_arm64.whl (7.3 MB)
Collecting cffi>=1.0.0
  Using cached cffi-1.15.1-cp310-cp310-macosx_11_0_arm64.whl (174 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Installing collected packages: pycparser, cffi, tfparse
Successfully installed cffi-1.15.1 pycparser-2.21 tfparse-0.1.3

[notice] A new release of pip available: 22.2.1 -> 22.2.2
[notice] To update, run: python3.10 -m pip install --upgrade pip

~ via 🐍 v3.10.6 (test) on ☁️  (us-east-1) 
❯ python
Python 3.10.6 (main, Sep 28 2022, 11:00:32) [Clang 14.0.0 (clang-1400.0.29.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tfparse
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/kapilt/.pyenv/versions/test/lib/python3.10/site-packages/tfparse/__init__.py", line 18, in <module>
    lib = load_lib()
  File "/Users/kapilt/.pyenv/versions/test/lib/python3.10/site-packages/tfparse/__init__.py", line 15, in load_lib
    return ffi.dlopen(str(libpath))
OSError: cannot load library '/Users/kapilt/.pyenv/versions/test/lib/python3.10/site-packages/tfparse.cpython-310-darwin.so': dlopen(/Users/kapilt/.pyenv/versions/test/lib/python3.10/site-packages/tfparse.cpython-310-darwin.so, 0x0002): tried: '/Users/kapilt/.pyenv/versions/test/lib/python3.10/site-packages/tfparse.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e))), '/Users/kapilt/.pyenv/versions/3.10.6/envs/test/lib/python3.10/site-packages/tfparse.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e)))
>>>

multiple var files issue

support for multiple var files in #136 didn't actually test multiple var files, and is incorrectly calling the parser options, which takes an array of strings, instead it was passed a single string in a loop, stomping over the previous values.

separate module loader function

in cases of static analysis we won't have the modules checked in, consider if we can export a function that performs module downloading.

expand __tfmeta data

Expand __tfmeta to include block type and id

block type
id

load_from_path on non existent directory errors

>       load_from_path(bytes(tmp_path / "xyz"))

test_tfparse.py:30: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

filePath = b'/private/var/folders/z3/8p9wm6ds2815k1mvx_k3f7740000gn/T/pytest-of-kapilt/pytest-48/test_parse_no_dir0/xyz'
stop_on_hcl_error = False, debug = False

    def load_from_path(
        filePath: bytes, stop_on_hcl_error: bool = False, debug: bool = False
    ) -> tp.Dict:
        s = ffi.new("char[]", filePath)
        e1 = ffi.new("int*", 1 if stop_on_hcl_error else 0)
        e2 = ffi.new("int*", 1 if debug else 0)
        ret = lib.Parse(s, e1, e2)
>       ret_json = ffi.string(ret.json)
E       RuntimeError: cannot use string() on <cdata 'char *' NULL>

module input / output with nested modules

#185 attempted to address this but the solution doesn't work for nested modules

ie. root module contains module a and module b, feeding output of a to b, but module b has a two child modules c and d, that it tries to pass through outputs from module a as inputs to c and d. currently it appears those inputs in module c and d appear blank.

reference building doesn't handle multiple refs on the same block

resource "aws_db_subnet_group" "subnetGroup" {
  name       = "main"
  subnet_ids = [aws_subnet.frontend.id, aws_subnet.backend.id]

  tags = {
    Name = "My DB subnet group"
  }
}

gets serialized out with a single __ref__ when there are in fact two refs.

                          'subnet_ids': {'__attributes__': ['aws_subnet.frontend.id',
                                                            'aws_subnet.backend.id'],
                                         '__name__': 'frontend',
                                         '__ref__': '3a23e2cd-b8ce-405f-87dc-2aca548ed72f',
                                         '__type__': 'aws_subnet'},

variable block is missing type

at the moment the type value on on all variables is empty.

we need to grab the hclattribute Expr, and simplify back to types. Its either a scope traverse or a function call traverse afaics. we need this for default variable injection by type to have the correct types. expanded validation (function call parameters) is probably not something we need at the moment.

several terraform functions with issues

func-check.zip

while testing a few functions it became apparent that there are a few functions that don't work as expected

locals {

  check_mod_path = path.module

  check_toset_int = toset([1, 2, 3])
  check_toset_str = toset(["a", "b", "c"])

  check_tomap  = tomap({ "a" = 1, "b" = 2 })
  check_tolist = tolist(["a", "b", "c"])

  check_fileexists = fileexists("${path.module}/readme.md")
  check_file       = file("${path.module}/readme.md")

  check_fileset_abs_path      = fileset("/etc/", "*")
  check_fileset_rel_path      = fileset("files", "*.py")
  check_fileset_mod_path      = fileset("${path.module}/files", "*")
  check_fileset_wild_rel_path = fileset("${path.module}", "files/*.py")

  check_trimprefix = trimprefix("abc/def", "abc")

  modules_list = toset([for lambda_main in fileset("${path.module}/modules/*/", "main.tf") : trimsuffix(trimprefix(lambda_main, "../"), "/main.tf")])
  lambdas_list = toset([for lambda_main in fileset("${path.module}/../lambdas/*/", "main.go") : trimsuffix(trimprefix(lambda_main, "../"), "/main.go")])
}

parsing this results in a few abnormalities

{
  "input_vars": {},
  "graph": {
    "locals": [
      {
        "__tfmeta": {
          "filename": "main.tf",
          "line_end": 24,
          "line_start": 2,
          "path": "locals"
        },
        "check_file": "test\n\n",
        "check_fileexists": true,
        "check_fileset_abs_path": null,
        "check_fileset_mod_path": null,
        "check_fileset_rel_path": null,
        "check_fileset_wild_rel_path": null,
        "check_mod_path": ".",
        "check_tolist": [
          "a",
          "b",
          "c"
        ],
        "check_tomap": {
          "a": 1,
          "b": 2
        },
        "check_toset_int": null,
        "check_toset_str": null,
        "check_trimprefix": "/def",
        "id": "98668bdb-5773-41a8-93cc-b6effffd5a7d",
        "lambdas_list": null,
        "modules_list": null
      }
    ]
  }
}

the toset function and fileset function both seem to have some issues.