Comments (11)
I think this was originally done because expressions in an aggregate expressions are a lot simpler, e.g. you can not nest EXISTS
in there, i.e. only RegularFunctions, not SpecialFunctions. But now I'm not so sure. I can not immediately find it in the spec. So could definitely make sense.
Be aware, aggregation is very statefull. There is some shenanigans in the AggregateEvaluator to provide a nice API (replacing functions on the first call), both outwards and inwards. Take care when implementing an async alternative.
The BaseAggregators should be able to be reused.
from sparqlee.
Thank you for the clarification.
from sparqlee.
It is clear that an AsyncAggregateEvaluator should have some locking or transaction mechanism in place to handle certain scenarios.
- Result should be async. When a user calls put and doesn't await, the output of result will be arbitrary. the result function should wait until all puts are done.
- Locking init: since init changes the behaviour of some functions, it should not be called multiple times. We will need some kind of transaction system to make sure it is only called once. Even when put is called multiple times at once.
Within the BaseAggregators I see two aggregators that might have unexpected async behavior. This being GroupConcat and Sample, GroupConcat
because the aggregator is defined by an order. Sample
forms a problem to because it will return the first term.
I don't know how we can handle the unexpected behavior of GroupConcat
. Either we throw an error if we notice multiple puts are called at the same time or, we expect the user to await every put.
To handle all these problems we could also expect the user to use await between every call to the AyncAggregateEvaluator
but this raises the question on why to make one in the first place?
I'm open to suggestions on how to tackle these problems.
from sparqlee.
Result should be async
👍
Since everything else in Comunica is async, this shouldn't be a problem.
Either we throw an error if we notice multiple puts are called at the same time
We definitely do not want this
we expect the user to await every put.
Sounds reasonable to me
this raises the question on why to make one in the first place
Not sure what you mean by this
from sparqlee.
So, we expect a user using the aggregates GroupConcat
and Sample
are responsible for using await between every put so the behavior of these aggregators is as expected?
Than all that's left is finding a way to use transactions for the mentioned functions.
from sparqlee.
for using await between every put
No, I'd make put
async
, and await
it when it's invoked.
from sparqlee.
I don't get that. Assuming we use a transaction system on put so only one put is can be executed at a time, a user writing
await Promise.all([aggreg.put(a), aggreg.put(b), aggreg.put(c)]);
const res = await aggreg.result();
where aggreg
is a GroupConcat
migt expect a, b, c
. But, since it is not clear what will exectute first it might be b, a, c
. So than if a user wants a, b, c
. We can expect him to write
aggreg.put(a);
aggreg.put(b);
aggreg.put(c);
right?
from sparqlee.
I think this will be required:
await aggreg.put(a);
await aggreg.put(b);
await aggreg.put(c);
from sparqlee.
Yes, of course, I forgot the awaits.
from sparqlee.
After further research about how JS works and finding out that it doesn't have the concurrency problems other languages have. I need to revise my previous message.
- I think result shouldn't be async. If we expect a user to do this we might as well expect him to await all puts before calling result. It would be pretty hard to implement a result function that awaits all puts. There would be some questions to be answered like 'what about puts that where called after calling results?' If these questions arise when implementing I don't think users will find it natural to work with either. Making result not async would give a clear signal to the user that he/she needs to await all puts.
- The locking mechanism isn't needed since we could just check a simple state to see whether init has already been called.
from sparqlee.
This issue has been handled in the pull requests. I will close it.
from sparqlee.
Related Issues (20)
- Auto casting arguments using `OverloadTree` HOT 26
- Document spec link explaining `arithmeticWidening` HOT 1
- Clean NumericLiterals specificFormatterCreator HOT 1
- OverloadTree caching and open world types HOT 2
- Async super type discovery and getting full context in regular functions. HOT 1
- Wrong implementation of substr HOT 3
- Handle invalid lexical form better HOT 6
- Make new type system default HOT 1
- Breaking changes HOT 9
- Dependency Dashboard
- Include Sparqlee into the main Comunica monorepo HOT 15
- DISTINCT is not supported yet on aggregates
- `month` function not erroring on invalid dates HOT 1
- Equality check Error should not be thrown
- Refactor out redundancies in RegularFunctions and Ordering HOT 3
- Suport overriding functions HOT 12
- AggregateEvaluators is a mess HOT 4
- Migrate to eslint HOT 1
- Enable `no-redeclare` rule HOT 52
- Enable `no-implicit-globals` rule HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sparqlee.