Hi,
Thanks for putting this library together it looks really great. I haven't had a chance to fully test it out yet, but currently implementing it. This isn't an issue, more a question or request for advice. In pretty much every apollo subscription tutorial or example, they are always so rudimentary it's hard to think about how they might apply to a real application and also how they might perform at scale.
Going through the source to see how things hang together and thinking about how subscriptions should be implemented in particular the amount of data having to be read from Dynamo on every event. If event names are so simplistic as per all the examples around the web, if we just had a 'newMessage' event, is my understanding correct that if we had 20,000 users subscribed to this event, that this library would essentially need to fetch 20,000 records from Dynamo every time, even if the there was a withFilter of eg payload.topic === variables.topic.
Am I correct in thinking it would have to fetch all of the subscribers before processing the filter as the variables requested by the client are stored in the subscription record?
So what happens if that is thousands of records. I noticed the recent added limit to number of records retrieved from dynamo per request. Do you think this could start to cause issues at scale? Also processing large number of subscriptions to an event in the same lambda instance, could that have an effect on a realtime application? Maybe batching and invoking lambdas in parallel to process the responses.
So my question is, in this sort of scenario (ie serverless, external pubsub) in your experience and given the current functionality should we be taking a different approach to the withFilter pattern and looking to bake some of the filtering elements into the event name, 'newMessage::team:some_team_topic:cars'. How have you gone about it in any more production like systems?
I noticed the event name is stored as the hashkey, with Dynamos ability to filter on the range key with operators like beginsWith and lte, gte, between maybe some future support for specifying a range key in the event name passed from graphql could give an almost redis like feature set to this library ('newMessage::everything_passed_the_colons_is_range_key').
Being able to the subscribe just to eg all messages for all topics in a team by using
beginsWith('newMessage::team:some_team')
Any advice much appreciated.
Cheers
Paul