jkorab / ameliant-tools Goto Github PK

May also be useful to add distribution strategy at this point

At the moment the key allocation strategy is just to use ProducerDefinition.uniqueKeyCount randomly generated strings for the key, and to modulo assign those keys to outgoing messages. The default partitioning strategy used in Kafka (key.hashcode()) is pretty crude, and does not ensure that these will be fairly assigned amongst the available topic partitions. In addition, the random generation means that the distribution will be different between test runs.

Not all topics will have the same number of partitions. Using an old computed hash that points to a non-existent partition will cause an explosion.

To test, set up a 2 topics with [2,1] partitions, [1,1] producers, [2, 2] consumers and use the sticky partitioner. Should start throwing exceptions from the single-partition producer.

Split ZK + Kafka into different packages. Resources isn't a great name.

Both the sticky and roundRobin strategies depend on instance state. Running multiple producers will mess with that and give inconsistent partition results.

• Auto-generated topic should be used if no topic defined, should not need to be a flag
• A child element should override, at present it is overridden

May want to split this out into a Gitbook, rather than blow out the README.

Programatically configure the logging system based on command-line flags.

Currently the performance report just dumps out amongst all of the console logging. The test should accumulate the performance stats of its drivers, and either dump them to console at the end or output to a file defined through a flag.

would be great to have this in mvn central so we can import them into projects using mvn dependency management

Test ProducerDriver.getPartitionerClassName() IllegalArgumentException