Comments (4)
@mraleph Do you see any potential performance improvements from merging the underlying regular UTF8 automaton with an automaton that lexes JSON and using that as the lexer for the JsonDecoder (to have a Utf8JsonDecoder) so that no fusing is needed?
from sdk.
The fusing creates a UTF-8 specialized JSON decoder, _JsonUtf8Decoder
.
from sdk.
Thank you for the hint @lrhn. It looks to me like the specialized UTF8-JSON decoder is using a table-driven UTF8 decoder and a non-table-driven JSON parser.
What I meant to say in this comment was, since the language of UTF8 appears to be regular, and the lexical structure of JSON seems to be regular too, it feels like there might be performance gains to be had by merging both automata into one and having the tokenization phase be based on a single automaton that does UTF8 decoding and JSON lexing at the same time.
from sdk.
I think the UTF-8 is a red herring here.
A table based tokenizer would work equally well for both UTF-8 and string based JSON parsing, since everything outside of string values is ASCII anyway, and what's inside strings need to be interpreted of it contains escapes -- or non-ASCII characters in a UTF-8 source.
The reason the current implementation doesn't use a table based tokenizer is that it doesn't need to tokenize at all. The moment it sees the first character of a value, it can start interpreting it. It tries to collect the value of a number and the contents of a string while scanning, in one pass, so it doesn't have to first tokenize, and then interpret the token afterwards.
Maybe that process can be made more efficient, but doing an extra tokenization step first doesn't sound like an improvement.
(Unless it can be done very efficiently using SIMD operations or using parallelism, or something.)
from sdk.
Related Issues (20)
- Function argument rejected based on faulty inferred parameter type
- AOT compilation runs into speculative input use by `CalculateElementAddressInstr` HOT 1
- Analyzer Feedback from IntelliJ
- Type inference for abstract classes with recurring generics can't be inferred HOT 10
- The front end is not happy about wrong constructor declaration
- The front end is not happy about wrong extension type constructor declaration
- pkg/analyzer/test/src/dart/resolution/augmented_invocation_test fails with RuntimeError on analyzer-asserts-win bot HOT 2
- Code completion suggestions are irrelevant / context unaware HOT 2
- [Wildcard Variables] Language Tests in `tests/language` HOT 5
- [Wildcard Variables] All tests in `tests/language` and co19 passing.
- [Wildcard Variables] Experiment Flag HOT 3
- [Wildcard Variables] CFE Implementation
- [CP] Cherry-Pick "[cfe] Ensure default values in synthesized function nodes" HOT 6
- [vm] use_code_comments_flag_test timing out on vm-aot-linux-release-simarm_x64 HOT 1
- [analysis_server] Non-const constructors aren't suggested in a const context HOT 2
- Move analyzer test utilities to analyzer_utilities package HOT 3
- [Wildcard Variables] Feature Specification HOT 6
- Problems with moving comments when automatically moving child arg to last parameter position of widget constructor
- Unreachable code during SelectRepresentations HOT 1
- "Stop on uncaught exceptions" sometimes stops on caught exceptions HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sdk.