Error Opening Certificates for Provider - file name too long about terraform-provider-kafka

[Mongey]

👋 Hey @joshbranham , what version of the provider are you using? Prior to v0.2.3 the ca_cert etc took a path, which, as you said it looks like it's attempting to do that.

If you're not using the latest version, I think you'll need something like this....

provider "kafka" {
  version           = "~> 0.2"
  bootstrap_servers = ["production-kafka.<redacted>:9093"]
  ca_cert           = "/home/ca.crt"
  client_cert       = "/home/client.crt"
  client_key        = "/home/private.key"
}

If you are using the 0.2.3 version, and this is happening, it's a bug

[joshbranham]

@Mongey I am using 0.2.3

[joshbranham]

also fwiw I am just using certificate files on disk (running in Atlantis) and I tried moving the files to shorter paths with no luck so I am pretty sure its doing something funky like trying to load the contents of the file as the path or something?

I also tried using 0.2.2 but then Terraform does not seem to find the provider called kafka anymore..

Initializing provider plugins...
- Checking for available provider plugins...

Provider "kafka" not available for installation.

A provider named "kafka" could not be found in the Terraform Registry.

This may result from mistyping the provider name, or the given provider may
be a third-party provider that cannot be installed automatically.

[MattPumphrey]

We installed the Kafka provider into our atlantis docker container to be able to execute this. I hope that helps with that.

[joshbranham]

We installed the Kafka provider into our atlantis docker container to be able to execute this. I hope that helps with that.

Yeah I have that part down, its more the SSL error. What version are you using? Did 0.2.2 work for you?

[Mongey]

also fwiw I am just using certificate files on disk (running in Atlantis) and I tried moving the files to shorter paths with no luck so I am pretty sure its doing something funky like trying to load the contents of the file as the path or something?

We do have some tests that are loading in certs with files, so I'm a bit surprised

terraform-provider-kafka/kafka/resource_kafka_acl_test.go

Lines 147 to 152 in 0f186dd

	provider "kafka" {
	bootstrap_servers = ["localhost:9092"]
	ca_cert = file("../secrets/snakeoil-ca-1.crt")
	client_cert = file("../secrets/kafkacat-ca1-signed.pem")
	client_key = file("../secrets/kafkacat-raw-private-key.pem")
	}

@joshbranham can you use terraform console to make sure the files are being loaded correctly?

Also debug logs might help track down some issues. TF_LOG=debug terraform plan

[joshbranham]

I found one issue with my private key being formatted wrong and fixed that..sorry for the confusion. Now looking at Error connecting to kafka kafka: client has run out of available brokers to talk to (Is your cluster reachable?) which could be a transport issue.

It was weird that the error above about opening the file was not my private key but the client cert. That library seems to be giving an odd error back?

[joshbranham]

Closing this, the issue was we are using a ELB in TCP mode that points to all kafka brokers in order to easily discover the brokers. For instance production-kafka.dns.com points to ELB that then forwards to all the brokers on the same port. For some reason that is not working, I will dig some more into that but it is probably something in https://github.com/shopify/sarama

Thanks for your time @Mongey