nmaquet / kasper Goto Github PK

I found your lib is pretty interesting and we share many similar ideas. Maybe we can collaborate somehow.

How about this distributed map reduce?

https://github.com/chrislusf/gleam/blob/master/examples/kafka_reader/kafka_reader.go

Greetings. I read your intro blog post and talk.

Very intrigued by this project (as well as goka), but noticed that it's still in beta and doesn't seem to be under active development.

So, curious ... are you still using it in production? Did the micro-batching / centralized state-management design prove to be problematic?

For testing, it would be nice to be able to use the noopMetric as the MetricsProvider but since it's not exported, you have to copy the implementation. It would be nice if this was exported so that you can use it.

I use this for batch processing, but when multiple consumers are configured in the same consumer group, each consumer consumes the full amount of data;

is this design like this, or is there a bug?

i want use the consumer group , what should i do ?

You're defining two interfaces Map and MultiMap but in the concrete implementation of Redis in the functions NewMultiRedis and NewRedis you're returning internal structure instead of the interface (MultiStore and Store respectfully). Same for elasticsearch.

This makes it difficult to leverage the interface when swapping out Store implementations.

For example, instead of:

func NewRedis(config *Config, conn redis.Conn, keyPrefix string) *Redis

It should be:

func NewRedis(config *Config, conn redis.Conn, keyPrefix string) Store

Since the Redis struct implements the Store interface.

If a user would like to use the concrete implementation (for some reason), they can always do something like the following to check the implementation and cast:

if r, ok := thestore.(*Redis); ok {
}

If we can make this change, it will make it much easier to actually use different implementations of these interfaces.

I'm happy to submit a PR if you'd like.