Comments (4)
NotEnoughValidWindowsException
means that CC hasn't been able to collect enough data yet about the MSK cluster.
In my experience metrics from MSK must be fetched from the OpenTelemetry ports using Prometheus. The default instructions do no work as with MSK you can't just "drop a jar in the Kafka server classpath".
I started my journey here: https://docs.aws.amazon.com/msk/latest/developerguide/cruise-control.html
from cruise-control.
WARN Skip generating metric sample for broker 2 because the following required metrics are missing [ALL_TOPIC_REPLICATION_BYTES_OUT, ALL_TOPIC_BYTES_OUT, BROKER_PRODUCE_TOTAL_TIME_MS_MEAN, BROKER_FOLLOWER_FETCH_LOCAL_TIME_MS_MAX, ALL_TOPIC_BYTES_IN, BROKER_PRODUCE_REQUEST_QUEUE_TIME_MS_MEAN, BROKER_CONSUMER_FETCH_TOTAL_TIME_MS_MEAN, BROKER_REQUEST_QUEUE_SIZE, ALL_TOPIC_FETCH_REQUEST_RATE, BROKER_CONSUMER_FETCH_REQUEST_QUEUE_TIME_MS_MAX, ALL_TOPIC_MESSAGES_IN_PER_SEC, BROKER_FOLLOWER_FETCH_TOTAL_TIME_MS_MAX, BROKER_CONSUMER_FETCH_LOCAL_TIME_MS_MEAN, BROKER_FOLLOWER_FETCH_REQUEST_QUEUE_TIME_MS_MEAN, ALL_TOPIC_PRODUCE_REQUEST_RATE, BROKER_FOLLOWER_FETCH_REQUEST_RATE, BROKER_PRODUCE_TOTAL_TIME_MS_MAX, BROKER_FOLLOWER_FETCH_LOCAL_TIME_MS_MEAN, BROKER_PRODUCE_LOCAL_TIME_MS_MEAN, BROKER_FOLLOWER_FETCH_TOTAL_TIME_MS_MEAN, BROKER_REQUEST_HANDLER_AVG_IDLE_PERCENT, BROKER_PRODUCE_REQUEST_QUEUE_TIME_MS_MAX, BROKER_CONSUMER_FETCH_LOCAL_TIME_MS_MAX, ALL_TOPIC_REPLICATION_BYTES_IN, BROKER_CONSUMER_FETCH_REQUEST_QUEUE_TIME_MS_MEAN, BROKER_PRODUCE_LOCAL_TIME_MS_MAX, BROKER_FOLLOWER_FETCH_REQUEST_QUEUE_TIME_MS_MAX, BROKER_RESPONSE_QUEUE_SIZE, BROKER_CONSUMER_FETCH_TOTAL_TIME_MS_MAX]. (com.linkedin.kafka.cruisecontrol.monitor.sampling.SamplingUtils)
I followed the instructions in the developer guide but a lot of broker metrics are missing. Please help me figure out whats wrong.
from cruise-control.
Hello @UdayaPriyaKannan, have you solved this issue?
I'm getting the same and i have replicated conf from AWS labs. It seems that as we're already scraping metrics from MSK there is some conflicts happening. There is left big window to make sure that cc has enough time for getting the metrics.
from cruise-control.
@micr01996 No, the issue is not solved yet
My training stopped at 20%
I'm able to do a PLE dry run but other operations in Kafka cluster administration tab throws Not Enough Valid Windows exception.
Kafka partition load, resource distribution tab also throws the same exception.
from cruise-control.
Related Issues (20)
- Slow task execution due to long retry backoff in ReplicationThrottleHelper
- Does CC support MSK/Kafka 2.8.1? - NoSuchMethodError: 'void kafka.zk.AdminZkClient.<init> (CC 2.5.137) HOT 4
- log4j not working when synced to latest main branch HOT 3
- Is there a final jar available with all dependency jars in jfrog HOT 5
- How do I enable DEBUG logging for Cruise Control? HOT 1
- Failed to describe Kafka cluster configs when using KafkaAdminTopicConfigProvider HOT 8
- Get cluster load API (sometimes) ignores `start` and `end` arguments
- CC reporting less disk for partition load
- Save Cruise Control proposal plans for later uses
- Fix Denial of Service vulnerability in SnappyInputStream HOT 1
- Upgrade vertx to 4.5.8 due to CVE-2024-1300 and CVE-2024-1023
- Suppress warning logs(Skip generating) in CC HOT 2
- Logfile rotation isn't removing old log files
- OptimizationFailureException for DiskCapacityGoal wrong computation of disk usage
- Cruise control reporter not recording Kafka metrics HOT 4
- java.lang.NoSuchMethodError: 'java.lang.String kafka.server.KafkaConfig.ListenersProp()' when running MetricsReporter for Kafka 3.8 HOT 4
- Maximum replication factor is not considered when assessing whether the cluster is over-provisioned
- ERROR [nioEventLoopGroup-4-1:NettyServerCnxnFactory$CertificateVerifier@468] - Unsuccessful handshake with session 0x0
- Executor doesn't move partitions on `add|remove|demote_broker` HOT 2
- Multiple SLF4j bindings in class path
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cruise-control.