Comments (18)
For file based persister, you can check out com.dropbox.mobile.store:filesystem:4.0.0-alpha02
It's definitally not as well supported as store
but might be good for your needs.
from store.
Correct coordinate is com.dropbox.mobile.store:filesystem4:4.0.0-alpha02
.
from store.
Hello, Can you please help by providing a minimal repro. This includes:
- The store creation code you are using
- The actual key types, which for simplicity sake should be a basic type like
String
(to rule out bugs resulting from issues withhashCode
impl)
Thanks for the report!
from store.
-
For the key I just use java BigInteger but I believe u can reproduce with String as well.
private val blockInfoStore = StoreBuilder
.fromNonFlow<BigInteger, GetBlockResponse> {
rpc.getBlock(GetBlockRequest(it.toString()))
}.cachePolicy(
MemoryPolicy.builder()
.setMemorySize(5)
.setExpireAfterAccess(10)
.setExpireAfterTimeUnit(TimeUnit.MINUTES)
.build()
)
.build()
if u use get
or fresh
extension functions no issue at all. The problem only with stream
itself.
You can clone code I left link above and test yourself.
from store.
On first glance - you're not providing a local source of Truth (disk cache) and you are setting your memory cache to expire. I this case the fetcher will be called when the cache is evicted. Can you confirm by one of the following:
- removing the cache expiration
- add a source of Truth
from store.
local source of Truth is database or what? Removing the cache expiration still trigger the several times with same state and origin and key.
I even tried very simple configuration:
StoreBuilder
.fromNonFlow<BigInteger, GetBlockResponse> {
rpc.getBlock(GetBlockRequest(it.toString()))
}
.build()
Still similar result. Please try my example I submitted here early.
How to add disk cache as simple file without setup database?
Updated:
But at least if I remove cache expiration stream never hang with Loading state.
from store.
are you using alpha01? we had a bug that always triggered fetcher when there is no source of truth, we've fixed it in alpha02.
#75
from store.
Failed to resolve: com.dropbox.mobile.store:filesystem:4.0.0-alpha02
@yigit
Chek source code in example is using alpha02.
You know even if use StoreRequest.fresh the behaviour still the same.
Then if persister is so essential and mandatory what is point to use store if I only need temporary memory/file cache for example? At least library should not allow to build store without persister and throw error during compilation (or at runtime).
from store.
Hi @atonamy, in order for allow us to investigate. Can you please provide a minimal repro. The current repro you provided includes rpc.getBlock
etc. that makes it impossible for us to repro. Can you provide something very basic e.g a fetcher that translates an Int
to it's String
. (Extra cupcake points if you can just put it in a self contained junit test)
from store.
I've forked your app and added more logging,
it is always getting new ids from block numbers.
If you look at these logs:
You can see that it always gets a new block id so it is a new key for a new request.
Btw, it did help find a bug in cache though 👍
#89
I'm not sure if i'm looking at the right part of the code, as @eyalgu mentioned, it would be much more clear to understand if this was a sample app but from what i can see in this app, it keeps sending different keys all the time hence getting new values.
here is a sample output from a run:
2020-02-03 21:22:18.777 13201-13252/? I/System.out: request for 103388547. seen before? false, []
2020-02-03 21:22:19.036 13201-13241/? I/System.out: request for 103388546. seen before? false, [103388547]
2020-02-03 21:22:19.292 13201-13242/? I/System.out: request for 103388545. seen before? false, [103388547, 103388546]
2020-02-03 21:22:19.540 13201-13255/? I/System.out: request for 103388544. seen before? false, [103388547, 103388546, 103388545]
2020-02-03 21:22:19.792 13201-13252/? I/System.out: request for 103388543. seen before? false, [103388547, 103388546, 103388545, 103388544]
2020-02-03 21:22:38.410 13201-13255/? I/System.out: request for 103388584. seen before? false, [103388547, 103388546, 103388545, 103388544, 103388543]
2020-02-03 21:22:38.663 13201-13252/? I/System.out: request for 103388583. seen before? false, [103388547, 103388546, 103388545, 103388544, 103388543, 103388584]
2020-02-03 21:22:38.913 13201-13241/? I/System.out: request for 103388582. seen before? false, [103388547, 103388546, 103388545, 103388544, 103388543, 103388584, 103388583]
2020-02-03 21:22:39.166 13201-13255/? I/System.out: request for 103388581. seen before? false, [103388547, 103388546, 103388545, 103388544, 103388543, 103388584, 103388583, 103388582]
2020-02-03 21:22:39.422 13201-13240/? I/System.out: request for 103388580. seen before? false, [103388547, 103388546, 103388545, 103388544, 103388543, 103388584, 103388583, 103388582, 103388581]
Lmk if i'm looking at the wrong thing.
from store.
Yes this #89 crashing issue also exists but if you have a look log carefully you may notice with this different keys Flow.collect {}
triggers with the same origin and status more than one time (but must be one only right?) and keep increasing this duplicate calls when more different keys you fetch which is obviously somewhere issue there.
Sorry I'm a bit busy, but I can create more testable solution maybe on weekends if it still valuable.
from store.
Not sure if relevant, but in your original example I noticed that you launch one coroutine and in it iterate over all your keys and collect the streams on them. A store stream does not finish collecting, so in that example the code will only start to collect the first key and will never get to the second.
Again that may be just specific to the example you have so a full repro would be the best next step here.
Thanks again sure trying out store and taking the time to report bugs!
from store.
@eyalgu but collection streams always happening with new unique keys in this example. So even if previous streams is not finished how come it affect new streams which is not related to previous? And u can even wait when previous stream collection will finish you still will get this duplicate calls on collection with new stream and with new different keys anyway. You can even try to cancel coroutine after complition which collect stream and new stream still produce duplicate collection with same origin and state.
Update:
I suspect the main issue when we collect streams in parallel, if we do it sequently strictly one by one the store with streams works fine.
Need more tests to confirm.
from store.
Flow.collect {}
triggers with the same origin and status more than one time
In my log, it always gets called w/ a new key so store makes 1 request for each new key. Not sure if I understand the problem correctly, that code seems to work as expected. If there are different keys, there will be different requests. I feel like i'm still not understanding what the problem is :/.
Every time you call the runStoreStreamTest
function, you are asking for a fresh
value so it skips cache, calls rpc.info.headBlockNum
which returns new keys (from its previous call) and the query to the second store always receives new keys, hence makes new requests.
from store.
When we collect stream with cache and new key it first goes to fetcher since we don't have a cache yet because of new key so the first we receive in collect
loading
status and fetcher
origin right? Then once data received successfully from remoute source collect
will call one more time and show in the status that we have data now correct?
Now if start to collect multiple streams in parallel the above described procedure the same for each stream with new unique key ?
Then if we split multiple streams collection in several batches with equal quantity of parallel streams which is run sequentially when one batch of multiple streams collection is complete and start another one we can see that described above behavior is different now the successful status with fetcher in collect
called twice now in our second batch which is shouldn't be like this? And if we continue to run third, fourth batches sequentially we can see that collect
calls keep increasing with same status and origin like in this described case.
Do I clearly described the problem?
In all streams and batches the keys is always new and unique never reuse same key anywhere in any stream in this scenario.
from store.
You had me until the last paragraph. To be honest the simplest way to get a fix is to submit a failing test case. This will both allow us to reproduce your issue and verify a fix. Seems like you might be hitting on some use case we did not consider.
from store.
@digitalbuddha no problem let's wait untill end of the week I'll try my best to write unit tests on weekend with isolated and mocking infrastructure if it still will be the case.
thank you
from store.
Closing, please reopen when you have a verifiable test case
from store.
Related Issues (20)
- [Feature Request] Add converter to StoreBuilder HOT 7
- [Feature Request] get() and fresh() extensions on MutableStore
- [Feature Request] HOT 1
- Inconsistent SourceOfTruth argument types HOT 2
- [BUG] Fix breaking changes with source of truth HOT 7
- [Documentation] Update Store documentation HOT 1
- Integrate website with CI
- [Feature Request] The stream of StoreResponse<T> should emit Loading(origin = SourceOfTruth) HOT 1
- [Question] Data with nested keys HOT 1
- [BUG] Stop with the breaking API changes in beta HOT 4
- [BUG] 5.0.0-beta02 breaks StoreReadRequest.cached + Validator HOT 2
- [BUG] Cached(refresh = true) will collect SoT reader twice HOT 3
- [Feature Request] fromLocalToOutput converter method went missing
- [BUG] Deadlock during synchronization in MutableStore
- [BUG] Examples on https://mobilenativefoundation.github.io/Store/mutable-store/building/builder/ are wrong
- [Feature Request] Common Testing Utils
- [BUG] Documentation "Edit this page" button 404's HOT 1
- Store Not Fetching or Read/Writing HOT 6
- CLA HOT 1
- [Question] Why should anyone consider using Store if they are already using NetworkBoundResource concept/framework? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from store.