kapperchino / jet-stream Goto Github PK

Add integration tests to test with the client + hosted shards

Need to setup pipelines with cr + deployment when there's a change.

Currently envoy is setup, but there's an issue where the grpc call is said to be unimplemented, need to fix it so that we can proceed with client testing + adding features.

Currently if we publish and connection to one node fails, the publish will already be done on the previous nodes. The solution is to add ack support for publishers, when first published the message will stay in a temp cache that has a ttl of a short period of time. They'll only be written to the main disk storage when the ack is called. This will make sure that publish will be written to all the correct partitions.

Currently the client is pretty bad, needs testing + improvements for resiliency and such, especially needs testing on GKE atm.