seanparsons / glue Goto Github PK

It should probably take a function (MultiService m a b -> IO MultiGetResponse a b) or some such to trigger the actual running of the preloading.

Cool project, do you know of Netflix's Hystrix and Facebook's Haxl?

Some patterns that are useful (some that I've used before):

• Request Coalescing (similar to batching, but instead of just batching all requests as a multi-request, it eliminates redundant requests, this was used in the case each request is asking for the same thing, then only send 1 request for all the multi-requests, and copy the response to each requester, instead of a time window, I did this with locks, because I needed low latency responses)
• Circuit Pattern (as seen in: https://github.com/jlouis/fuse)
• Workflow Monad (https://hackage.haskell.org/package/Workflow) - Useful for dropped connections and resumable connections/downloads (see http://oboejs.com/why)
• Idempotence Monad (as seen in Stripe and https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/popl38-ramalingam.pdf) - this would enhance the retry pattern
• All the stuff TCP did: ARQ, Forward Error Correction (useful for high bandwidth - high latency/low reliability connections or connections with only a forward link and no return link), Checksumming (I saw you mentioned that HTTP isn't the only thing you're focused on)
• Dealing with Streams and not just Request Response, and handling exceptions (one may want to use on top of something like conduit (distributed conduits))

Also I'm thinking about how this would integrate with RPC frameworks like Cap'n Proto and promise pipelining feature.

Citing from http://hydra.cryp.to/build/945325/log/raw:

example/Glue/Example/BatcherExample.hs:33:7:
    Non type-variable argument
      in the constraint: Control.Monad.Base.MonadBase IO m
    (Use FlexibleContexts to permit this)
    When checking that ‘service’ has the inferred type
      service :: forall (m :: * -> *) v.
                 (Eq v, Num v, hashable-1.2.3.2:Data.Hashable.Class.Hashable v,
                  Control.Monad.Base.MonadBase IO m) =>
                 S.HashSet v -> m (M.HashMap v v)
    In the expression:
      do { counter <- newIORef 0;
           let listOfNums = ...;
           let requests = ...;
           let service request = ...;
           .... }
    In an equation for ‘runTest’:
        runTest store name shouldBatch
          = do { counter <- newIORef 0;
                 let listOfNums = ...;
                 let requests = ...;
                 .... }

Citing from http://hydra.cryp.to/build/947718/log/raw:

Preprocessing test suite 'tests' for glue-0.1.1.0...

test/Main.hs:4:18:
    Could not find module ‘Spec’
    Use -v to see a list of the files searched for.