Comments (7)
How do I attach a file? I've pasted it in below:
import yacc,lex
def t_ID(t):
r'[a-zA-Z_][a-zA-Z_0-9]*'
return t
tokens = [
'ID',
'PLUS',
'LSQUARE',
'RSQUARE',
]
t_PLUS = r'+'
t_LSQUARE = r'['
t_RSQUARE = r']'
def t_error(t):
pass
def p_error(t):
pass
def lexer():
return lex.lex()
def p_expr1(p):
'''expr : expr PLUS expr'''
p[0] = (p[1],p[2],p[3])
def p_expr2(p):
'''expr : expr LSQUARE expr RSQUARE'''
p[0] = (p[1],"*",p[3])
def p_expr_atom(p):
'''expr : ID'''
p[0] = p[1]
def parse(data):
lexer()
parser = yacc.yacc()
return parser.parse(data)
precedenceTest = [
(( # this is the precedence
('left', 'PLUS'),
('left','RSQUARE'),
),# this is the expected output - PLUS is lower priority
(('a', '', 'b'), '+', ('d', '', 'b'))), # Fails
((
('left', 'PLUS'),
('left','LSQUARE'),
),
(('a', '*', 'b'), '+', ('d', '*', 'b'))), # Works
((
('left', 'PLUS'),
('left','LSQUARE','RSQUARE'),
),
(('a', '*', 'b'), '+', ('d', '*', 'b'))), # Works
((
('left', 'PLUS','RSQUARE'),
),# Now we expect left associativity
((('a', '*', 'b'), '+', 'd'), '*', 'b')), # Works
((
('left', 'PLUS','LSQUARE'),
),
((('a', '*', 'b'), '+', 'd'), '*', 'b')), # Works
((
('left', 'PLUS','LSQUARE','RSQUARE'),
),
((('a', '*', 'b'), '+', 'd'), '*', 'b')), # Works
((
('right', 'PLUS','RSQUARE'),
),# Now we expect right associativity
('a', '*',( 'b', '+',( 'd', '*', 'b')))), # Fails
((
('right', 'PLUS','LSQUARE'),
),
('a', '*',( 'b', '+',( 'd', '*', 'b')))), # Fails
((
('right', 'PLUS','LSQUARE','RSQUARE'),
),
('a', '*',( 'b', '+',( 'd', '*', 'b')))), # Fails
((
('left','RSQUARE'),
('left', 'PLUS'),
),
(('a', '*',( 'b', '+', 'd')), '*', 'b')), # Fails
((
('left','LSQUARE'),
('left', 'PLUS'),
),
(('a', '*',( 'b', '+', 'd')), '*', 'b')), # Fails
((
('left','LSQUARE','RSQUARE'),
('left', 'PLUS'),
),
(('a', '*',( 'b', '+', 'd')), '*', 'b')), # Fails
((
('right','RSQUARE'),
('left', 'PLUS'),
),
('a', '*',(( 'b', '+', 'd'), '*', 'b'))), # Fails
((
('right','LSQUARE'),
('left', 'PLUS'),
),
('a', '*',(( 'b', '+', 'd'), '*', 'b'))), # Fails
((
('right','LSQUARE','RSQUARE'),
('left', 'PLUS'),
),
('a', '*',(( 'b', '+', 'd'), '*', 'b'))), # Fails
]
for p,t in precedenceTest:
precedence = p
ans = parse("a[b]+d[b]")
print ans == t,t,ans
from ply.
Sorry that came out more rubbish than I expected. Anyway, if you cut and paste the code into your editor it should look OK.
from ply.
The precedence table and associativity rules only come into play during the resolution of shift/reduce conflicts and even then, it's only the right-most terminal that matters. You would really need to sit down with the parser.out debugging output and study it to know for certain what's happening here.
from ply.
OK good point - missed that.
However, firstly the parser is changing its output as the precedence changes, even though it doesn't have conflicts. So I don't really get that.
Secondly, if I make the operator infix, so we do get conflicts, I still have it coming out wrong. Program enclosed below.
Here, I've made the operator infix but with two terminals. I get shift/reduce conflicts. I use the right-most terminal and the conflicts are not being resolved in the way I would expect. I've cut it down to one example - just paste the examples in from the previous code to see the whole set;
The output looks like the operators are same precedence, left associative whereas I thought I was specifying that [] was higher priority.
It seems like maybe this is me being stupid but the example looks to me exactly like the usual expression example, but where the conflict is resolved by precedence, yet the precedence is not working here.
import yacc,lex
def t_ID(t):
r'[a-zA-Z_][a-zA-Z_0-9]*'
return t
tokens = [
'ID',
'PLUS',
'LSQUARE',
'RSQUARE',
]
t_PLUS = r'+'
t_LSQUARE = r'['
t_RSQUARE = r']'
def t_error(t):
pass
def p_error(t):
pass
def lexer():
return lex.lex()
def p_expr1(p):
'''expr : expr PLUS expr'''
p[0] = (p[1],p[2],p[3])
def p_expr2(p):
'''expr : expr LSQUARE RSQUARE expr'''
p[0] = (p[1],"*",p[4])
def p_expr_atom(p):
'''expr : ID'''
p[0] = p[1]
def parse(data):
lexer()
parser = yacc.yacc()
return parser.parse(data)
precedenceTest = [
(( # this is the precedence
('left', 'PLUS'),
('left','RSQUARE'),
),# this is the expected output - PLUS is lower priority
(('a', '', 'b'), '+', ('d', '', 'b'))), # Fails
]
for p,t in precedenceTest:
precedence = p
ans = parse("a[]b+d[]b")
print ans == t,t,ans
from ply.
And, I've tried this example in bison now, and get the expected output with * having higher precedence. So it really seems like ply is doing something different to bison/yacc.
I have a feeling that it does the right thing when the multi-token operator is on the left, but not when it is on the right. But I'm not sure.
I have the bison input if you want it but I don't really want to paste it in here. Is there some way to attach files?
Ben
from ply.
OK - I think I see where I am going wrong.
I need to put both terminals in the precedence list because the rightmost defines the precedence of the RULE, while the left terminal is the token which will appear in the lookahead. So I need to define the precedence of both the rule and the tokens separately (though presumably they should be the same in my case).
When I do that I get the same results with bison and ply, I think.
The fact that they differ in cases where I only specify one is probably no big deal, since the input is ambiguous in any case. In fact, if I turn on Bison's glr functionality then it rejects these ones as ambiguous.
So it looks like this is all just down to my misunderstanding. Just to be sure, I'm going to run through the comparisons again but I don't expect a problem.
Sorry for wasting your time and thanks for putting me on the right track.
Ben
from ply.
This issue bit me as well. I had assumed that if used a fictitious token and the %prec rule in the grammar then it would set the precedence correctly for the entire grammar rule. I had to explicitly set the precedence of the left-most terminal and the in addition to using the fictious terminal. I guess this set the precedence on the lookahead token and the rule is required when using multiple terminals.
Really appreciate PLY but I think the section on precedence could be expanded to cover this issue of multiple terminals.
from ply.
Related Issues (20)
- sequence item 0: expected str instance, float found HOT 1
- What types of codes are the PLY suitable for? HOT 2
- Are there any recommendations for grammar rules references to parse C language?
- Exporting grammar for documentation
- Invalid Syntax when importing module in yacc
- Configuring ID
- Space as a part of lexeme HOT 2
- Allow cloning a parser HOT 1
- How can I assign p arguments when there are two non-terminals side by side (PLY) HOT 7
- Whether tok.type can be acquired at parser
- Parse HQL with STRUCT Comment returns empty
- Simple typo in doc/ply.md about error handler example HOT 1
- Python 3.11: inability to use global inline flags HOT 3
- yacc argument "write_tables" is missing in 3.11 HOT 5
- Warn user about tuple flattening in precedence table HOT 3
- PyPI package HOT 2
- Trying to get in touch regarding a security issue
- "global flags not at the start of the expression" error when using in-line global regex flag with Python 11+ HOT 1
- t_AUTOINCREMENT error ?
- Please make a new release HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ply.