trendyol / kafka-cronsumer Goto Github PK

We can implement parallel testing

Like we did in kafka-konsumer, we can provide batch consume fn for our exception messages

We realized that at every iteration we created new goroutines without releasing previous ones, we can re-use the same goroutines or close and create new ones.

https://github.com/dependabot

Users want to use kafkaProducer in order to publish their messages to an exception topic for the first time.

Instead of they create Kafka producer implementation, we can expose our kafka producer and present them for use.

On the consumer side of Kafka topics, we want to ignore some messages or process only some messages depending on the message type. Mostly, message producers send their events with some details on the message header. So, could the cronsumer project gives us a feature about ignoring/accepting messages for a given header key/values.

We could add client id configuration to the producer.

producerConf := kafka.WriterConfig{
ClientID: "your-client-id",
...
}

Thanks to this change, we will get rid of the specific example dependencies like YAML

Even messages are processed in the iteration, cronsumer still try to processed again and again

We could add an example which SASL enabled.

Right now if we have more than one consumer and apiEnabled is true, for every consumer we are creating new api. Instead we can create the api once and use this API for all consumers.

Sometimes we have faced big lag of exception/retry topics, In that kind of situation we need close the cron and finish the lag asap so we can provide this feature

While consuming and producing, we can ensure that we propagate related tracing headers. If not, we need to do this integration.