metthal / pog Goto Github PK
View Code? Open in Web Editor NEWC++ library for generating LALR(1) parsers
License: MIT License
pog's People
Contributors
Stargazers
Watchers
pog's Issues
when using "parser.enter_tokenizer_state" at midrule will cause a Syntax error.
The reason :
the parser first tokenize a token by the _tokens at @default_state , then find a action for reduce. but when using "parser.enter_tokenizer_state" at midrule , we hope to tokenize a token by the _tokens at @state_after_enter. So we need first execute midrule then to tokenize a token.
(actually, not only using midrule wil cause this error, but also using "parser.enter_tokenizer_state" at production is so. However, we can limit it to only midrule for simpleness)
Fix thinking
Adding a special empty symbol use for lookahead. when current rule is a midrule , the key of _action_table use the empty symbol instead of real next_symbol for searching action. then we can try search a maybe_action first by using empty symbol, if not finding an action, then get next token for searching action.
hoping fix it , thanks.
🙏🙏🙏
Assertion failed: false && "Reduction happened but corresponding GOTO table record is empty"
Help , please .
#include <iostream>
#include <any>
#define POG_DEBUG_TOKENIZER
#define POG_DEBUG_PARSER
#include "pog/pog.h"
using namespace pog;
using Value = std::any;
int main(int argc, char const *argv[])
{
Parser<Value> parser;
parser.token("\\s+");
parser.token("test").symbol("test").action(
[](auto&& sv) {
return sv;
});
parser.rule("TEST_LIST")
.production("TEST_LIST", "TEST",
[](auto&& p){ return std::move(p); })
.production(
[](auto&& p){ return std::move(p); })
;
parser.rule("TEST")
.production("test",
[](auto&& p){ return std::move(p); })
;
parser.set_start_symbol("TEST_LIST");
std::cout << parser.prepare().to_string() << std::endl;
try{
auto v = parser.parse(std::cin);
}
catch (const SyntaxError& err) {
//HtmlReport html(parser);
//html.save("parser.html");
std::cerr << err.what() << std::endl;
}
return 0;
}
DEBUG LOG:
test
^Z
[tokenizer] Matched 'test' with token 'test' (index 2)
[parser] Tokenizer returned new token with symbol 'test'
[parser] Top of the stack is state 0
[parser] Reducing by rule 'TEST_LIST -> <eps>'
[parser] Pushing state 1
[parser] Reusing old token with symbol 'test'
[parser] Top of the stack is state 1
[parser] Shifting state 2
[tokenizer] Matched '
' with token '\s+' (index 1)
[tokenizer] Reached end of input
[tokenizer] Matched '' with token '$' (index 0)
[tokenizer] At the end of input
[parser] Tokenizer returned new token with symbol '@end'
[parser] Top of the stack is state 2
[parser] Reducing by rule 'TEST -> test'
[parser] Pushing state 3
[parser] Reusing old token with symbol '@end'
[parser] Top of the stack is state 3
[parser] Reducing by rule 'TEST_LIST -> TEST_LIST TEST'
Assertion failed: false && "Reduction happened but corresponding GOTO table record is empty", file external\pog\include\pog/parser.h, line 206
Provide expected symbols in case of syntax error
As of right now, pog raises an exception in case of syntax error. It takes all symbols that are expected on the input in the particular parsing state it is in at the moment and serializes it into the string of parser error. So you'll get something like unexpected symbol <XYZ>, expected one of <A>, <B>, <C>, ....
It would be proficient for users of pog to have these expected symbols available for the purpose of building language servers. Instead of raising an exception, please return something like ParsingResult object which would contain all of these and would essentially bear the same information as the exception in case of syntax error.
Add command-line tool wich will generate C++ file with parser
Pog is able to generate parser during the runtime but it takes some time to actually create the parser itself. In the same way as bison and other parser generators, we should add CLI tool which generates C++ files with the parser itself. It would give an option to people which do not want to generate parser during the runtime.
Even though pog was created as a mean to replace bison because of problems of integrating it into CMake build system, we can still use the generated C++ file without actually running the pog each build.
Could you provide an example that how use states ?
I always arise a syntax error . thanks.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.