Comments (4)
So, my idea to use algorithms in the likes of the ones we've developed for the partial
namespace is parsers combinators. When I reach a token for which I want to partial::skip()
over, I'll just instantiate a new json::chunk_reader
to consume the current element. Once I'm over (parser for inner sequence emits token::end
), I'll call chunk_reader.next<token::null>()
on the outer parser. This idea might help to give you new scenarios to think on while you decide on what to do about #53.
from trial.protocol.
Getting json::partial::skip()
to support chunk_reader
can be done by moving the implementation of skip()
into a detail
namespace that takes a generic typename Reader
argument instead of basic_reader<CharT>
, and then add two public skip()
overloads for chunk_reader
.
That only partially solves the problem though, because we need to distinguish between a error and end-of-input in order to indicate that the chunk_reader
needs more data (e.g. by return an empty view from skip()
.) However, chunk_reader
restores its internal state, so we cannot query the reader afterwards.
A possible workaround could be to return a tri-bool instead of a bool, which indicates
- true = token available
- false = error
- inderterminate = end of input
from trial.protocol.
we need to distinguish between a error and end-of-input in order to indicate that the
chunk_reader
needs more data (e.g. by return an empty view fromskip()
.)
That kinda was my point about coroutines. Once skip()
returns, a new call to skip()
won't know how much more tail needs to be consumed. I don't like the coroutine approaches tho (C++ coroutines are the worse implementation of coroutines I've ever seen).
My idea was to keep the state in a new chunk_reader
object. You instantiate a new chunk_reader
to consume the subtree defined by the current token. That way you know you must consume (the inner) chunk_reader
until the end
token. So this inner chunk_reader
accidentally already keeps all the state you need about the substream. Once the subtree is parsed, you just need to advance the state for the outer chunk_reader
(we could use next("null")
but I'd like to see a next<token::null>()
instead).
Honestly it's pretty easy to reimplement skip()
and that's what I've done in my working copy of tabjson to implement chunked parsing. However there are more elaborate algorithms such as partial::parse()
that I prefer to reuse instead of reimplement (the "extra" typing to reuse a ready abstraction pays the price here). I don't plan to touch partial::parse()
anytime soon tho as tabjson serves all my needs so far w/o touching any DOM handle.
A possible workaround could be to return a tri-bool instead of a bool, which indicates [...]
That's the first time tribool ever solved a problem for me. It works for me.
The current lack of error vs end of input is also the only blocker for tabjson's chunked parsing as I can workaround everything else already.
from trial.protocol.
I have added a feature branch with the tribool changes.
from trial.protocol.
Related Issues (20)
- Change Allocator template to Allocator type HOT 1
- AllocatorAwareContainer HOT 4
- Use to_address() HOT 2
- Do not inherit from allocator HOT 1
- Use error specializations thoroughly HOT 1
- Compare current string-value with a string_view w/o allocating HOT 12
- reader.value<std::string>() should allow the allocator to be provided HOT 5
- String collector API should only be noexcept if `Collector()` is noexcept HOT 11
- Parser gets confused on non-NULL terminated string HOT 1
- Isn't boost::serialization::nvp missing in serialization support? HOT 1
- Literal whitespace vs literal node HOT 7
- Split symbol::string and symbol::object_key events HOT 3
- Tests fail to build on boost 1.76 HOT 6
- Parsers combinators HOT 7
- Merged matching & decode on Boost.Hana serialization
- Use asciidoc to document new pages HOT 4
- fails to build with latest visual studio
- Issues with constness in std::make_pair HOT 4
- Generator for real numbers shouldn't print the fractional part if there is none HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from trial.protocol.