project_creation - add to (don't override) iam-policy about deploymentmanager-samples

[krwenholz]

Hi Davide,

I'm not sure what you mean in step 2. Could you include an example config for the IAM policies you're trying to change?

Our general recommendation is to use the project creation template to create the project and any service accounts. Then you should use a different deployment in the created project to manage IAM permissions and resources in that project.

[davidebelloni]

Hi,
like in your "project_creation" example (on this repo) this deployment manage also the IAM policy on the new project. I can use another deployment on the new project to manage IAM policies, but the problem about the requirement to manage google default service accounts permissions remain. And on top I think the two deployment can overwrite each other the IAM settings.

The example config you ask for may evolve from this (on project creation):

imports:
- path: project.py

resources:
# The "name" property below will be the name of the new project
- name: <my-project>
  type: project.py
  properties:
    ...
    iam-policy:
      bindings:
      - role: roles/iam.securityReviewer
        members:
        - group:[email protected]

to this (on project update):

imports:
- path: project.py

resources:
# The "name" property below will be the name of the new project
- name: <my-project>
  type: project.py
  properties:
    ...
    iam-policy:
      bindings:
      - role: roles/iam.securityReviewer
        members:
        - group:[email protected]
        - group:[email protected]

With the update action the google default service accounts permission vanish because I don't configure explicitly the IAM policies about them.

What I would like to do is not have to handle the google default service accounts permissions on the project, i.e. permissions on the project about this accounts:

• <project_number>[email protected]
• service-<project_number>@container-engine-robot.iam.gserviceaccount.com
• service-<project_number>@containerregistry.iam.gserviceaccount.com
• ...

Thank you

[krwenholz]

Ah that is clear now. Thanks for providing a config example.

The iam-policy sections define the entirety of the policy. Unfortunately, we aren't able to support partial updates to this property. We're working on a better story around this, but don't have anything to report yet.

I'm going to change the title of this issue so other users can find it, and we'll update back here once we have something more concrete.

[davidebelloni]

Ok,
thank you for the fast reply!

I hope to hear you soon

[aljim]

Added support for add and not override IAM policies, please check #47 For a sample of making one service account owner

[davidebelloni]

Hi @aljim ,
to me this code return:

  message: '{"ResourceType":"gcp-types/cloudresourcemanager-v1:cloudresourcemanager.projects.setIamPolicy","ResourceErrorCode":"409","ResourceErrorMessage":{"code":409,"message":"There
    were concurrent policy changes. Please retry the whole read-modify-write with
    exponential backoff.","status":"ABORTED","statusMessage":"Conflict","requestPath":"https://cloudresourcemanager.googleapis.com/v1/projects/xxx:setIamPolicy","httpMethod":"POST"}}'

What can be? And how to solve this problem?

Thanks

[mihnjong]

I also see the same issue. Seems it's a bug?

[krwenholz]

Reopenenig so our on-call can take a look. Sorry nobody saw this sooner Davide, we've been changing a lot about how we monitor this repo and it looks like your comment was before those changes.

[mihnjong]

I found the work-around for this issue. I'll send a PR soon.

[mihnjong]

#71 is out for review

[davidebelloni]

Hi @krwenholz @mihnjong ,
I've tried the fix but I've te following error:

$ gcloud deployment-manager deployments update "$DMPROJECT" --config project.yaml 
The fingerprint of the deployment is Cd7LCauKuMOv3hC-xk_cEg==
Waiting for update [operation-1512987810798-5600ded0319b0-980c8251-2af424f3]...failed.                                                                                                                                                                                                                                    
ERROR: (gcloud.deployment-manager.deployments.update) Error in Operation [operation-1512987810798-5600ded0319b0-980c8251-2af424f3]: errors:
- code: CYCLIC_REFERENCES
  message: |
    A cycle was found during reference analysis:
    Cycles detected:
    deploymentmanager.googleapis.com <- get-iam-policy-1512987812 <- patch-iam-policy <- get-iam-policy-1512987537

I've tried also to delete, recreate and update the deployment. But the update step fails always with a CYCLIC_REFERENCES error.

Thanks

[likeulb]

@davidebelloni
The error indicate there is cycle between the resources mentioned in the error message.
It seems you have the resource dependencies like deploymentmanager.googleapis.com <- get-iam-policy- <- patch-iam-policy
After you creating the deployment and try to do the update, can you try to update the config file to remove the dependency between deploymentmanager.googleapis.com and get-iam-policy-? Does this work for the update?

resources.extend([{
'name': get_iam_policy_name,
'action': 'gcp-types/cloudresourcemanager-v1:cloudresourcemanager.projects.getIamPolicy',
'properties': {
'resource': project_id,
},
''' remove below:
'metadata': {
'dependsOn': [ApiResourceName(project_id, 'deploymentmanager.googleapis.com')]
}
'''
},

[davidebelloni]

Hi @likeulb ,
after removing "deploymentmanager.googleapis.com" the update task works.
This dependency is also in your code, why it has been necessary to remove it? I can't see the cyclic reference between the resources pointed out and I think the dependency between deploymentmanager.googleapis.com and get-iam-policy are correct.

Can you explain me?
Thanks

[likeulb]

Hi @davidebelloni

DM will ensure the dependencies between resources to make sure the order when creating/updating/deleting them based on the "ref" or "dependsOn" specified in the config file. The cycle error due to the current processing logic of DM. But we have to admit this is some rare cases that DM cannot handle perfectly. We will have more investigation about this.
On the other side, we will have a coming fix that you will not need to attach "timestamp" to force a new "get-iam-policy" action. This will address most of the problems discussed here. And we will update the project creation sample accordingly.

Thanks

[davidebelloni]

Ok,
I think the problem now is on deployment/project creation, that return:

ERROR: (gcloud.deployment-manager.deployments.create) Error in Operation [operation-1513351309840-560628f3e4e81-5a62a677-4ed1f747]: errors:
- code: RESOURCE_ERROR
  location: /deployments/dtdd-appb2c-customers-test/resources/get-iam-policy-1513351314
  message: '{"ResourceType":"gcp-types/cloudresourcemanager-v1:cloudresourcemanager.projects.getIamPolicy","ResourceErrorCode":"403","ResourceErrorMessage":{"code":403,"message":"The
    caller does not have permission","status":"PERMISSION_DENIED","statusMessage":"Forbidden","requestPath":"https://cloudresourcemanager.googleapis.com/v1/projects/dtdd-appb2c-customers-test:getIamPolicy","httpMethod":"POST"}}'

[likeulb]

Hi @davidebelloni
Does this happen when you remove the dependency?
You may need the "dependsOn" to make sure the deployment manager API has been activated before doing getIamPolicy. You may remove it for a temporary fix if you need to update the deployment.
Also, can you double check if the service account [project_number]@cloudservices.gserviceaccount.com has "project owner" role?

[davidebelloni]

Hi @likeulb ,
yes the problem derives from the removed dependency.
I think that manually remove the dependency for the update task is not suitable for an enterprise service, in particular for code shared with a template (project.py) between different projects.
Is there a forecast for a complete fix that solve these problems about GCP projects management?

Thanks

[likeulb]

Hi, @davidebelloni
We will probably do some testing later this weeks, and will update the github sample according. Thanks.

[davidebelloni]

Hi @likeulb ,
I've solved the issue with the code " 'runtimePolicy': ['UPDATE_ALWAYS']" (see #52)

I hope to see this features more documented in the next future.

Thanks for the help

[likeulb]

Hi @davidebelloni
Thanks for the update! There will be updated documentation about Actions in January.
Thanks.

[likeulb]

#74 Is updating the sample

[likeulb]

The sample has been updated. Closing the issue now.