Comments (4)
Here's the brief description of how subscription mechanism works or supposed to work (at least in my understanding)
- The request is received by cacherpc instance to perform some remote procedure call
- If the request is getAccountInfo or getProgramAccounts, cacherpc handles it, otherwise the request is routed through to validator
- For the public key in request system should try to find an active web socket subscription (currently logic for that is broken),
presence of active subscription would indicate that the entry in cache is being maintained in up to date state, absence of it means that we have never cached data for that key before or that web socket subscription has been terminated and data in cache might be stale. - If subscription is present, then we can try to get data from cache and return it immediately
- If subscription is not present or the data itself is not present in cache then we forward the request to validator
- Then there's some magic with concurrent async code...
- After receiving response from validator, put the data into cache, and initiate subscription for that request, to keep it in sync.
- Finally system returns the response
So currently, instead of checking whether we have an active subscription in step 3, we just compare the number of confirmed subscriptions with the number of initiated subscriptions (but not necesserally confirmed), which leads to the problem described in issue, that even if one the subscriptions, managed by some worker, isn't confirmed, then none of the requests that should be handled by this worker will get answered from cache, until all its subscriptions are confirmed.
So proposed solution is to map public keys with subscriptions (that's a logical thing to do). So here's one of the ways to implement it (admittedly not the elegant one):
- There's already a mapping between public keys and subscriptions, but not a direct one:
pub struct AccountUpdateManager { ... sub_to_id: HashMap<(Subscription, Commitment), u64>, // should contain only active confirmed subscriptions connection: Connection, ... }
So the presence of the key (tuple of Subscription and Commitment) in sub_to_id
field is an indication of existence of active web socket subscription, but here's the catch:
Subscription is an enum which has two variants (for now), as well as Commitment (3 variants), so in order to verify that the map contains the Pubkey, one has to go through all the combinations of the variants of those two enums (worst case scenario). For now the number of combinations is 2 x 3 = 6, but in case if new variants are added this number will increase in arithmetic fashion, e.g. if the number of supported methods increases from 2 to 10, then the number of combinations will grow to 30 (again that's the worst case).
So here's the summary of this hacky solution: lazily generate a sequence of combinations of Subscription and Commitment variants, and check the presence of the entry for each combination. If the entry is found then active subscription exists. Upsides: easy to implement (almost no changes are necessary), downsides: not really performant.
UPDATE:
Going through the code once more, I've noticed that we do have all the information to compose the Subscription, Commitment tuple, so no combination generation is actually necessary!
from cacherpc.
Your analysis is correct. What do you propose to solve the problem, I mean, how to pass information about subscription status to rpc?
from cacherpc.
There's a required method on Cacheable
trait named is_cacheable
, which has the following signature
fn is_cacheable(&self, state: &State) -> Result<(), UncacheableReason>;
As you can see it has access to self, which is type responsible to hold the information about current request, those types (currently there's 2) have an associated method called commitment
(will need to move it to a trait) which can be used to retrieve Commitment for the Pubkey (with which the request is made). Also the the implementors of the is_cacheable
can hardcode the variant of Subscription into method itself. Thus when making a call to subscription_active
function from the is_cacheable
(that seems to be the only call site for now), instead of passing just Pubkey, one must pass Subscription and Commitment as parameters, and the body will look something like this:
pub fn subscription_active(&self, sub: Subscription, commitment: Commitment) -> bool {
let idx = self.get_idx_by_key(key);
self.0[idx].0.sub_to_id.contains(&(Subscription, commitment))
}
The problem however is that self.0[idx].0 is not AccountUpdateManager
but rather actix::Addr<AccountUpdateManager>
, which is just a transmitting end of channel. So the problem boils down to how to access the map inside AccountUpdateManager
without having a direct access to AccountUpdateManager
.
With the solution proposed above the second member of tuple in vector which is part of
pub struct PubSubManager(Vec<(actix::Addr<AccountUpdateManager>, Arc<AtomicBool>)>);
type, actually becomes useless, and we can substitute it with RwLock<HashMap<K,V>>
or Arc<DashMap<K,V>>
for better performance, while changing the type of sub_to_id
respectively. So we can check subscription status as before, but for a particular Pubkey.
All in all this approach will require some rewrite.
from cacherpc.
Now before serving request from cache we check if we have active ws connection, after which we send a query to subscription manager, which is responsible for handling updates for the cache entry of the request. The query's purpose is figure out if we have maintained active subscription for the given account|program, and if we do, application will serve the request from cache.
from cacherpc.
Related Issues (20)
- Content type with charset returns 415 HOT 1
- Any path component should be accepted HOT 1
- Cannot request Stake11111 (Gateway timeout) HOT 7
- Make SLOT_DISTANCE configurable HOT 1
- Log updated config HOT 1
- `filter::tests::tree_matches_overall` can randomly fail
- Pubsub reconnect counter (and possibly rpc)
- Pubsub reconnect backoff settings
- Detect situations when when ws subscriptions are active but no updates received HOT 1
- Unix socket based, command listener implementation
- Batch requests support implementatin in WAF
- Detect infinite reconnections on websocket
- Implement getMultipleAccounts
- Add ability to reload waf script without restart HOT 1
- Implement metrics endpoint with Prometheus format HOT 1
- MailboxError panics
- getAccountInfo occasionally returns error HOT 5
- Caching getProgramAccounts with filters HOT 1
- Slots & commitments
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cacherpc.