dannybrody / terraform-aws-cost-anomaly-detection

Terraform module to configure cost anomaly monitor that sends notifications to SNS and then to slack

Cost anomaly detection and alerting

This module leverages AWS Cost Anomaly Detector to identify unusual cost patterns in AWS and notify them immediately. It creates a Cost Anomaly Monitor, a Cost Anomaly Subscription, a SNS topic, and optionally a slack channel configuration on AWS ChatBot. It also will optionally deploy Lambda function that will run weekly and will report the current forecasted cost of the account, last month's cost and the variation percent. This lambda is set by default to run every Monday at 9:00 AM ET. However it can be configured by either using cron or rate sintax.

AWS Cost Anomaly Monitor Monitors the AWS account for unexpected costs. This module uses AWS' recommended configuration to evaluate each of the services you use individually, allowing smaller anomalies to be detected. Anomaly thresholds are automatically adjusted based on your historical service spend patterns.

Cost Anomaly Subscription sends an alert to SNS when cost monitor detects an anomaly and a threshold is exceeded. The threshold is configurable and it can be a fixed amount or a percentage.

Deployment options

• AWS recommendation is to use a Service Monitor which analizes the cost paterns of a single account and alerts when unexpected cost in any service is found. In such case, this module needs to be instantiated and deployed separately on each of the accounts that need to be monitored leaving the accounts variable empty This is the deployment recommended by AWS.

• It is possible to monitor all the member accounts of and AWS Organization, however, it's less granular, therefore less likely to find unexpected cost patterns. In this case, deploy this module on the root account and use the variable accounts in order to define which accounts should be monitored.

• Recommended deployment: In an environment with Control Tower enabled, instantiate this module individually on each of the main accounts, such as sandbox, staging, and production. In each deployment, do not use the accounts variable so that the monitors only focus on the account and do not deploy the lambda using the deploy lambda variable. On the root/main account, instantiate the module using the accounts variable, include the account number of every AWS account in your organization and deploy the Lambda. This way, you'll have granular monitoring at the service level on the accounts you consider more important, monitoring at the account level using the root account, and the lambda reporting the forecasted cost of the main account. Refer to the examples folder for more information.

Cost

The Cost Anomaly Detection service does not have a cost by itself. However, it sends its findings to SNS which has a cost of few cents per million messages. If the Lambda function is deployed, on each execution it will make 3 calls to the Cost Explorer API, which has a cost of 1 cent per call. Conclusion: The cost of running this solution is quite low, however not zero.

Slack integration

Before starting follow these steps to allow AWS to access your slack workspace

1. Access the AWS console on the account that the Cost alerts will monitor. In a CT environment, all billing is commonly centralized in the root account
2. Access AWS ChatBot service, choose Slack on the Chat client dropdown box, and click on Configure Client

3. Click on Allow on the next page.

4. Create a channel to receive the cost alerts in slack as usual.

5. In the Slack interface right click on the channel name and select copy link

6. From the URL, pick up the channel ID and use it on the repo as the value for the slack_channel_id variable. Example: https://caylent.slack.com/archives/C018WLGMXYZ (This is an example URL, C018WLGMXYZ is what needs to go into your tfvars file)

7. Access Slack on your web browser. Log in with your credentials, and pick up the Workspace ID from the URL and use its value in the repo as a value for the slack_workspace_id variable. Example: https://app.slack.com/client/T01JK23AB/slack-connect (This is an example URL, T01JK23AB is the workspace ID that you need in your tfvars file)

8. Invite the AWS ChatBot app to the channel.

   

Microsoft Teams integration

Before starting follow these steps to allow AWS to access your MS Teams workspace

1. Create a team in Microsoft Teams. (An existent Team can be also used)
2. Create a channel in the team created before. This channel will be used to send the alerts.
3. Add AWS Chatbot app to the channel. Select the team created before and go to "Manage Team" -> "Apps". Click on "+ Get more apps" and search for aws. Add the "aws" app and select the channel where we want to add the bot.
4. Right-click on the channel that will be used to publish the alerts and select "Get link to channel". Copy the channel URL.
5. Access the AWS console on the account that the Cost alerts will monitor. In a CT environment, all billing is commonly centralized in the root account.
6. Access AWS ChatBot service, choose Microsoft Teams on the Chat client dropdown box, and click on Configure Client.
7. Paste the URL from step 4 and click on "Configure". (You'll need MS Teams admin privileges to authorize access from AWS to MS Teams)
8. Continue with the module deployment.

Once this is done, Terraform can be applied to create the alerts, subscriptions, SNS topic, and the configuration that maps the slack channel with the alerts.

Requirements

Name	Version
terraform	~> 1.1
archive	2.4.0
aws	~> 4.63
awscc	~> 0.68
null	3.2.1

Providers

Name	Version
archive	2.4.0
aws	4.67.0
awscc	0.65.0

Modules

No modules.

Resources

Name	Type
aws_ce_anomaly_monitor.linked_account_anomaly_monitor	resource
aws_ce_anomaly_monitor.service_anomaly_monitor	resource
aws_ce_anomaly_subscription.anomaly_subscription	resource
aws_cloudwatch_event_rule.lambda_trigger	resource
aws_cloudwatch_event_target.event_target	resource
aws_iam_policy.chatbot_channel_policy	resource
aws_iam_role.chatbot_role	resource
aws_iam_role.iam_for_lambda	resource
aws_iam_role_policy_attachment.chatbot_role_attachement	resource
aws_lambda_function.cost_alert	resource
aws_lambda_permission.allow_events_bridge_to_run_lambda	resource
aws_sns_topic.cost_anomaly_topic	resource
awscc_chatbot_microsoft_teams_channel_configuration.chatbot_ms_teams_channel	resource
awscc_chatbot_slack_channel_configuration.chatbot_slack_channel	resource
archive_file.lambda_deployment_package	data source
aws_caller_identity.current	data source
aws_iam_policy_document.assume_role	data source
aws_iam_policy_document.chatbot_assume_role_policy	data source
aws_iam_policy_document.chatbot_channel_policy_document	data source
aws_iam_policy_document.lambda_policy	data source
aws_iam_policy_document.sns_topic_policy_document	data source
aws_region.current	data source

Inputs

Name	Description	Type	Default	Required
accounts	List of AWS accounts to monitor. Use it when deploying the module on the root account of an organization	list(string)	[]	no
alert_threshold	Defines the value to trigger an alert. Depending on the value chosen for the threshold_type variable, it will represent a percentage or an absolute ammount of money	number	n/a	yes
deploy_lambda	flag to choose if the lambda will be deployed or not	bool	true	no
enable_ms_teams_integration	Set to false if Microsoft Teams integration is not needed and another subscriber to the SNS topic is preferred	bool	true	no
enable_slack_integration	Set to false if slack integration is not needed and another subscriber to the SNS topic is preferred	bool	true	no
lambda_frequency	Frequency to run the lambda (cron formating is also accepted)	string	"cron(0 13 ? * MON *)"	no
lambda_timeout	maximum amount of time in seconds that the Lambda function can run	number	3	no
name	name for the monitors, topic, etc	string	"cost-anomaly-monitor"	no
slack_channel_id	right click on the channel name, copy channel URL, and use the letters and number after the last /	string	""	no
slack_workspace_id	ID of your slack slack_workspace_id	string	""	no
sns_topic_arn	ARN of an already existing SNS topic to send alerts. If a value is provided, the module will not create a SNS topic	string	""	no
tags	Map of tags to apply to resources	map(string)	{}	no
team_id	The id of the Microsoft Teams team	string	""	no
teams_channel_id	The id of the Microsoft Teams channel	string	""	no
teams_tenant_id	The id of the Microsoft Teams tenant	string	""	no
threshold_type	Indicate if the alert will trigger based on a absolute amount or a percentage	string	n/a	yes

Outputs

Name	Description
anomaly_monitor_arn	n/a
anomaly_subscription_arn	n/a
sns_topic_arn	n/a