provider_installation seems not working, is there a workaround you aware of about emr-eks-terraform

[dacort]

Hm, what's the output of your terraform version?

My guess is you're running <0.14 per https://www.terraform.io/docs/cli/config/config-file.html#development-overrides-for-provider-developers

[sahotay]

usmlt103687:eks rsahota$ terraform version
There are some problems with the CLI configuration:

Error: Invalid provider_installation method block

Unknown provider installation method "dev_overrides" at 2:3.

As a result of the above problems, Terraform may not behave as intended.


Terraform v0.13.6
+ provider registry.terraform.io/cyrilgdn/postgresql v1.14.0
+ provider registry.terraform.io/hashicorp/aws v3.58.0
+ provider registry.terraform.io/hashicorp/cloudinit v2.2.0
+ provider registry.terraform.io/hashicorp/helm v1.3.2
+ provider registry.terraform.io/hashicorp/kubernetes v2.5.0
+ provider registry.terraform.io/hashicorp/kubernetes-alpha v0.6.0
+ provider registry.terraform.io/hashicorp/local v2.1.0
+ provider registry.terraform.io/hashicorp/null v3.1.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/template v2.2.0
+ provider registry.terraform.io/hashicorp/tls v3.1.0

Your version of Terraform is out of date! The latest version
is 1.0.6. You can update by downloading from https://www.terraform.io/downloads.html
usmlt103687:eks rsahota$ 

[dacort]

Terraform v0.13.6

OK, you'll need to upgrade to at least v0.14 or later.

I also need to update the README. :)

[sahotay]

@dacort probably missing something, is there a secret that neds to have some access to EMR ?

module.eks_on_emr.aws_emrcontainers_virtual_cluster.cluster: Creating...

Error: error creating EMR containers virtual cluster: ValidationException: Unauthorized to perform read namespace on emr-jobs

  on ../../../../modules/aws/eks_on_emr/main.tf line 125, in resource "aws_emrcontainers_virtual_cluster" "cluster":
 125: resource "aws_emrcontainers_virtual_cluster" "cluster" {

[dacort]

That should be part of the terraform plan. Can you try running apply again and seeing if that helps? And if not, is it possible to paste the full log with any sensitive info redacted? (If not, feel free to email github at dcortesi.com)

[sahotay]

sent you an email @dacort , please have a look

[sahotay]

unrelated to this but this is not the correct namespace?
https://github.com/dacort/emr-eks-terraform/blob/main/kubernetes_roles.tf#L65

[dacort]

Ah, good catch. The key should be api_group not namespace. Just pushed a fix.

[sahotay]

@dacort Yay, I got the EMR on EKS now :)

Should we commit to PR 20003 and help to get it merged?

Thanks for your big help!!

[dacort]

Sweet! Was it the error in the kubernetes role or something else?

I believe the original author of 20003 is going to add an example set of requirements, so currently waiting on that. We need somebody from the terraform provider to help review it/get it merged.

[sahotay]

the only error I saw is the api_group instead of namespace, And what I was missing is cluster map roles

{
      rolearn  = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:role/AWSServiceRoleForAmazonEMRContainers"
      username = "emr-containers"
      groups   = ["emr-jobs-admins"]
 },

[dacort]

OK, perfect, thank you!