jimgbritt / azurepolicy Goto Github PK

Invoke-AzRestMethod was added to the Az PowerShell some time ago and should deal with all the authentication for you
Similarly I've triggered policy evaluation whilst using Invoke-AzResourceAction before Invoke-RestMethod was available.

This may help to simplify your scripts

This policy doesn't allow for customizing the type of diagnostic information to send to a sink like Log Analytics Workspace. It assumes that metrics & logs will always be forwarded. Unfortunately, my customers don't push metrics to LAW due to ingestion latency and cost implications. We leverage Azure Monitor Metrics for all metrics.

Recommendation

Add two parameters: LogsEnabled and MetricsEnabled so that we can set the correct type of data to be forwarded during assignment. Assignment can be either of the policy or the initiative that uses this policy.

This policy is used by initiatives such as Canada Federal PBMM and HITRUST/HIPAA.

Current Policy Definition

{
  "properties": {
    "displayName": "Audit diagnostic setting",
    "policyType": "BuiltIn",
    "mode": "All",
    "description": "Audit diagnostic setting for selected resource types",
    "metadata": {
      "version": "1.0.0",
      "category": "Monitoring"
    },
    "parameters": {
      "listOfResourceTypes": {
        "type": "Array",
        "metadata": {
          "displayName": "Resource Types",
          "strongType": "resourceTypes"
        }
      }
    },
    "policyRule": {
      "if": {
        "field": "type",
        "in": "[parameters('listOfResourceTypes')]"
      },
      "then": {
        "effect": "AuditIfNotExists",
        "details": {
          "type": "Microsoft.Insights/diagnosticSettings",
          "existenceCondition": {
            "allOf": [
              {
                "field": "Microsoft.Insights/diagnosticSettings/logs.enabled",
                "equals": "true"
              },
              {
                "field": "Microsoft.Insights/diagnosticSettings/metrics.enabled",
                "equals": "true"
              }
            ]
          }
        }
      }
    }
  },
  "id": "/providers/Microsoft.Authorization/policyDefinitions/7f89b1eb-583c-429a-8828-af049802c1d9",
  "type": "Microsoft.Authorization/policyDefinitions",
  "name": "7f89b1eb-583c-429a-8828-af049802c1d9"
}

When trying to run the script Trigger-PolicyInitiativeRemediation.ps1 in management group mode, both interactively or by specifying parameters, the script asks:

Selecting Azure Policy Initiative: ...

Remediate "" Initiative?
Create a set of remediation tasks for Policy Initiative "". Continue?
[Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"):

I am afraid to say Yes because I don't know what Initiative it's trying to run.

Full output of both methods:

PS /home/david/AzurePolicy/AzureMonitor/Scripts> ./Trigger-PolicyInitiativeRemediation.ps1 -ManagementGroup
Authenticating to Azure...
Please select a Management Group from the list below

 # Name                    DisplayName             Id
 - ----                    -----------             --
 1 [...] [...] /providers/Microsoft.Management/managementGroups/[...]
 2 [...]    [...]    /providers/Microsoft.Management/managementGroups/[...]
 3 [...]   [...]   /providers/Microsoft.Management/managementGroups/[...]
 4 [...]         [...]         /providers/Microsoft.Management/managementGroups/[...]
 5 [...]        [...]        /providers/Microsoft.Management/managementGroups/[...]
 6 [...]           [...]           /providers/Microsoft.Management/managementGroups/[...]
 7 [...]          [...]          /providers/Microsoft.Management/managementGroups/[...]
 8 [...]  [...]  /providers/Microsoft.Management/managementGroups/[...]
 9 [...]     [...]     /providers/Microsoft.Management/managementGroups/[...]
10 [...]   [...]   /providers/Microsoft.Management/managementGroups/[...]
11 [...]      [...]      /providers/Microsoft.Management/managementGroups/[...]
12 [...]                [...]                /providers/Microsoft.Management/managementGroups/[...]
13 [...]           [...]           /providers/Microsoft.Management/managementGroups/[...]
14 [...]        [...]        /providers/Microsoft.Management/managementGroups/[...]
15 [...]           [...]           /providers/Microsoft.Management/managementGroups/[...]
16 [...]    [...]    /providers/Microsoft.Management/managementGroups/[...]
17 [...]       [...]       /providers/Microsoft.Management/managementGroups/[...]
18 [...]          [...]          /providers/Microsoft.Management/managementGroups/[...]
19 [...]          [...]          /providers/Microsoft.Management/managementGroups/[...]
20 [...]             [...]             /providers/Microsoft.Management/managementGroups/[...]

If you don't see your ManagementGroupID try using the parameter -ManagementGroupID
Please enter a selection from 1 to 20: 12
Selecting Management Group: [...] ...
Selecting Azure Policy Initiative: ...

Remediate "" Initiative?
Create a set of remediation tasks for Policy Initiative "". Continue?
[Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"): n
You have cancelled the remediation request for
Setting Context back to initial subscription [...]
Complete

Script execution time: 0 minutes and 28 seconds.

PS /home/david/AzurePolicy/AzureMonitor/Scripts> ./Trigger-PolicyInitiativeRemediation.ps1 -ManagementGroup -ManagementGroupID "[...]" -PolicyAssignmentId "/providers/Microsoft.Management/managementGroups/[...]/providers/Microsoft.Authorization/policyAssignments/[...]"
Authenticating to Azure...
Selecting Azure Policy Initiative: ...

Remediate "" Initiative?
Create a set of remediation tasks for Policy Initiative "". Continue?
[Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"): n
You have cancelled the remediation request for
Setting Context back to initial subscription [...]
Complete

Script execution time: 0 minutes and 35 seconds.

Not sure if the error I get is due to the fact that I did not follow exaclty the process described or because I am applying the iniatives to a RG and my LAW is in a different RG but I am getting the following error when applying the remediation task.

Checking Monitor roles under https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#monitor it looks like in order to access the sharedKeys, we need "Monitoring Contributor" role but the Policy seems to be configured with Log Analytics Contributor in "roleDefinitionIds".

Did I miss something ?

Detailed error.
{
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
{
"code": "Forbidden",
"message": "{\r\n "error": {\r\n "code": "LinkedAuthorizationFailed",\r\n "message": "The client '11fcfc2f-4467-4d38-b9b7-f8f1e808bfdd' with object id '11fcfc2f-4467-4d38-b9b7-f8f1e808bfdd' has permission to perform action 'Microsoft.Insights/diagnosticSettings/write' on scope '/subscriptions/91bb19d8-d81c-47fc-ad7d-0b67d426dc39/resourcegroups/IP-Law-Policy-Test/providers/Microsoft.Storage/storageAccounts/storepolicytest01/providers/Microsoft.Insights/diagnosticSettings/cbezinitiatives - LAW LabSecMonitoring'; however, it does not have permission to perform action 'Microsoft.OperationalInsights/workspaces/sharedKeys/action' on the linked scope(s) '/subscriptions/91bb19d8-d81c-47fc-ad7d-0b67d426dc39/resourcegroups/labsec/providers/microsoft.operationalinsights/workspaces/labsecmonitoring' or the linked scope(s) are invalid."\r\n }\r\n}"
}
]
}

Ran the script, created an initiative that works for most resource types, but not for microsoft.web/staticsites:

"errorMessage": "The attempt to retrieve resources of type 'Microsoft.Insights/diagnosticSettings' for the 'DeployIfNotExists' policy resulted in a BadRequest response with error '{"code":"ResourceTypeNotSupported","message":"The resource type 'microsoft.web/staticsites' does not support diagnostic settings."}

From what I read the script should auto-detect all resource types that support diags. Is there an option planned to easily exclude certain resource types from being exported, e.g. a parameter ExcludeResourceTypes (for example to skip preview resources)?

Thanks for the script BTW! We have been struggling with configuring diags in a robust and consistent manner, and the automation runbook we setup 3 years ago doesn't handle 1000 subscriptions and 50 resource types very well any more. I was hoping for years that diags settings would show up in Resource Graph at a certain point so we could run a query instead of having to check individual resources, but I guess that's not going to happen. :)

Exported ARM templates do not support deployment on management groups scope. Resource IDs used in dependsOn and policyDefinitions -> policyDefinitionId must be changed for this. The name of the MG must be known to build the ResourceId of the policy. I propose to create a variable into which we will insert a management group in which the scan is carried out.

I am happy to make these changes. @JimGBritt What do you think?

Thanks for your awesome script, it really helps us out!

While testing the various resources for monitoring, I found that the Azure VM policy shows up as compliant, but when visiting the VM Diagnostic settings blade it states that guest-level monitoring still needs to be enabled:

Also the Insights tab is not configured (as it requires a lot more parameters).

Is that something that still needs to be done manually/scripted, for example configuring the performance counters/event logs/sinks/agents of the VM? And the policy provided remediation is the basic diagnostics?

When remediating storage accounts, the diagnostic settings blade displays that the diagnostics for the storage account itself is enabled, but not for the blob/queue/table/file:

Is that something that can be added? If so, what needs to be done?

Hello All,

Just used this code and I presumed that the newly created storage account will be with the diagnostic login, but it does not as per screenshot:

Then I created a ticket to MS asking why the policy does not work as should.

They recommended me to use a build in policy named "Configure diagnostic settings for storage accounts to Log Analytics workspace"

Perhaps anyone had this "issue" or I misunderstood the functionality?
I thought by creating policy and assigning a scope (creating assignment) the new storage account would have diagnostic logging as does build in policy?

I am rather new to policies so sorry if all/any part of the description is not valid. I am just looking for answers.

Hello Jim,

I have looked at the policy script for creating/generating the diagnostic settings config to Event Hub.
I have noticed that all Policies were showing Non-compliant, after some analysis, I found that the Current and Target values were not matching. The Current was empty.
I've changed the Template manually under deployIfNotExists/existenceCondition field: from eventHubName to eventHubAuthorizationRuleId, and then the policy gets compliant.

Thanks,
-Charbel

Hi Jim,

I used the script to create an initiative at management group level and it creates the initiative fine.
However when I try to apply the initiative using the generated ARM template it errors out
New-AzManagementGroupDeployment : 16:10:44 - The deployment 'SentinelDiagInitiative' failed with error(s). Showing 3 out of 27 error(s).
Status Message: The existing policy has '5' parameter(s) which is greater than the count of parameter(s) '4' in the policy being added. Policy parameters cannot be removed during policy update.
(Code:InvalidPolicyParameterUpdate)
Status Message: The existing policy has '5' parameter(s) which is greater than the count of parameter(s) '4' in the policy being added. Policy parameters cannot be removed during policy update.
(Code:InvalidPolicyParameterUpdate)
Status Message: The existing policy has '5' parameter(s) which is greater than the count of parameter(s) '4' in the policy being added. Policy parameters cannot be removed during policy update.
(Code:InvalidPolicyParameterUpdate)
CorrelationId: a10cf882-b3b8-4c78-b9ab-d32a2bbee0a1

When I open the ARM template it does look like a parameter as "name" is extra in the remediation section of the policies ( within that initiative).

Any advise please.

regards,
Ravindra

I noticed that diags setting gets enabled on v2 storage:

Is it somehow possible to include preview diags for Blob (e.g. Microsoft.Storage/storageAccounts/blobServices) from the exported template as well, or will this automatically become available once it gets GA?

The added functionality would ensure that the generated ARM Template would support the initiative assignment and permission assignment for MSI.

As in the previous issue, I would like to develop it.

I am looking to use this utility to create a subscription level Diagnostic Setting for the Activity Logs for the subscription itself. However, based on my results, it does not look like that's supported. Is that correct?

PS C:\Users\coopem67\Documents\WindowsPowerShell\Scripts> Create-AzDiagPolicy.ps1 -SubscriptionID <REDACTED> -ResourceType Microsoft.Insights/diagnosticSettings -ExportStorage
Authenticating to Azure...
Selecting Azure Subscription: Disney Azure Managed Logging - Policy ...
No diagnostic capable resources of type Microsoft.Insights/diagnosticSettings available in selected subscription <REDACTED>
ScriptHalted
At C:\Users\coopem67\Documents\WindowsPowerShell\Scripts\Create-AzDiagPolicy.ps1:2137 char:13
+             Throw write-host "No diagnostic capable resources of type ...
+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (:) [], RuntimeException
    + FullyQualifiedErrorId : ScriptHalted