Comments (9)
Hello @idesofoctober,
I apologize for the error you are experiencing.
I attempted to run the command steampipe check benchmark.cis_v300
multiple times, but I was unable to replicate the error.
In the plugin, the API DescribeMetricFilters
is only used in the aws_cloudwatch_log_metric_filter
table. Generally, we do not handle the InvalidParameterException
error. We currently have a few tables that handle this error for specific cases. However, we are planning to remove them if they are deemed unnecessary. Additionally, in the compliance pack, I did not find any queries that use both MetricName
and MetricNamespace
together in the WHERE clause.
Could you please provide more context about which specific control or query is failing for you? This will help us determine if the issue needs to be addressed on the compliance query side.
Thanks!
from steampipe-plugin-aws.
Hi @ParthaI , I appreciate you digging into this, I'm actually having quite a lot of problems right now, and this is only one of them :(
benchmark cisv_300 runs the below from the compliance pack, I believe and that particular control is one of the ones that errors across all my accounts. (However I cannot reliably reproduce running locally with an arm64 build, only when running in fargate ecs with an amd64 container/infrastructure....I don't mean to add any red herrings here, but wondering if you have any insight or if that brings anything interesting front of mind re: other issues you may have seen?
Can you remind me where to find the specific query referenced below in the code? Thanks!
control "log_metric_filter_unauthorized_api" {
title = "Ensure a log metric filter and alarm exist for unauthorized API calls"
description = "Real-time monitoring of API calls can be achieved by directing CloudTrail Logs to CloudWatch Logs and establishing corresponding metric filters and alarms. It is recommended that a metric filter and alarm be established for unauthorized API calls."
query = query.log_metric_filter_unauthorized_api
from steampipe-plugin-aws.
Hi @idesofoctober,
You can find the query at the line query = query.log_metric_filter_unauthorized_api
. The log_metric_filter_unauthorized_api
query is being used in this context. You can view the actual query by visiting the Powerpipe Hub and filtering by the name log_metric_filter_unauthorized_api
. The specific query used for that control can be seen like: https://hub.powerpipe.io/mods/turbot/aws_compliance/queries/log_metric_filter_unauthorized_api#sql.
Please try running the query on your local setup to see if you encounter any issues with it.
Thanks!
from steampipe-plugin-aws.
Additionally, in the compliance pack, I did not find any queries that use both
MetricName
andMetricNamespace
together in the WHERE clause.
The error is that one or the other was used but not both - so a query that had both together should work whereas one that only has metric_name would fail with that error.
from steampipe-plugin-aws.
@ParthaI I think the error is coming from upstream re: the compliance mod->aws plugin when it tries to run some hydrate functions. Is that possible?
I am now running:
steampipe v0.23.2
powerpipe v0.4.0
aws plugin v 0.139.0
and the latest compliance mod
When I run
powerpipe benchmark run aws_compliance.benchmark.cis_v300 --query-timeout 600 --export=<account_id>_cis_v300_06242024.csv
with the additional environment variables in my docker container (some of which may be redundant now that i've transitioned to powerpipe):
#disable auto-update
ENV STEAMPIPE_UPDATE_CHECK=false
#disable telemetry
ENV STEAMPIPE_TELEMETRY=none
#limit parallel connections to DB
ENV STEAMPIPE_MAX_PARALLEL=1
ENV POWERPIPE_MAX_PARALLEL=1
#SQL Query Timeout Update
ENV STEAMPIPE_QUERY_TIMEOUT=600
I get this error:
Operation error CloudWatch Logs: DescribeMetricFilters, https response error StatusCode: 400, RequestID: , InvalidParameterException: Describe Metric Filters request must contain both MetricName and MetricNamespace (SQLSTATE HV000)
for the below controls:
4.1, 4.2, 4.3, 4.4, 4.5, 4.6, 4.7. 4.8, 4.9, 4.10, 4.11, 4.12, 4.13, 4.14, 4.15
from steampipe-plugin-aws.
@ParthaI this DOES NOT happen with all of our accounts FYI, is there potentially something to look at re: what is different in accounts where this error occurs vs. accounts where it does not?
from steampipe-plugin-aws.
Hi @idesofoctober,
I apologize for misunderstanding the context initially. We were able to reproduce the error you are experiencing.
From the plugin side, the API is behaving as expected. However, I have raised a PR based on the API behavior(Preventing API calls based on query parameters). Another PR will be raised from the compliance side to resolve the issue.
Note: The plugin behavior requires that if we pass the MetricName
, it must be accompanied by the MetricNamespace
.
Thank you so much for providing detailed information!
from steampipe-plugin-aws.
Thanks @ParthaI I was looking at the aws compliance mod github repo and didn't see any issues/prs for this, but not understanding the code well, is the v.95 release meant to address this (not sure if you are even the right person to ask?)?
from steampipe-plugin-aws.
Hi @idesofoctober,
The fixes have already been merged from the compliance side in the main branch. The v0.95 release contains the fix addressing this issue.
Previously, the table aws_cloudwatch_log_metric_filter
was queried by providing only the MetricName
as a query parameter. However, the AWS DescribeMetricFilters API requires that if MetricName
is provided, it must be accompanied by MetricNamespace
. We can't pass only one of these values in a single API call.
The plugin fix ensures that API calls are made correctly by manipulating the query parameters to adhere to the API's behavior.
Thanks!
from steampipe-plugin-aws.
Related Issues (20)
- Add table aws_IPAM_Public_IP_Insights
- Dynamic values are not evaluated HOT 3
- The `aws_kms_key_rotation` table throws `AccessDeniedException` despite the `ignore_error_codes` configuration in the connection config
- Add table aws_securityhub_enabled_products
- Add table aws_lightsail_bucket
- Excessive duplicate AWS API calls HOT 4
- Add table aws_memorydb_cluster
- Add table aws_timestreamwrite_table
- Add table aws_timestreamwrite_database
- Add table aws_keyspaces_table
- Add table aws_qldb_ledger HOT 1
- Add table aws_qldb_stream HOT 1
- Parsing AWS config is broken in v.0.145.0 for chained assume role sessions (when used via steampipe-postgres-fdw >= v.1.11.5) HOT 9
- aws_ec2_ami unable to include disabled images
- Add table aws_mwaa_environment
- Can Custom Controls be queried ? HOT 3
- Add tables for Storage Gateway
- Strange behavior with aws_health_event and regions HOT 2
- aws_ec2_instance_type takes over 4 minutes HOT 6
- aws_securityhub_finding - Add key column on created_at and updated_at
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from steampipe-plugin-aws.