leastfixedpoint / java-parser-tools Goto Github PK

I have been ponering how to define quoted string in my Turtle grammar 
definition.   Currently I have this (which is not sufficient)...

quotedString ::= ("'"" {anychar} "'"")
anychar ::= (%alpha% | %digit% | " " | ":" | ".")  

What I really need is a quoted string matcher like the one defined in the 
info.reflectionsofmind.parser.Grammar class.

However, in order to be able to use a custom Matcher I would have to 
abandon my grammar defintion file and instead define the Turtle grammar in 
Java, using the info.reflectionsofmind.parser API.

However, if an extension syntax is added to the grammar definition then it 
would be possible to create Matchers in Java and refer to them in the 
grammer.
There is an example of EBNF extension syntax in this article...
http://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form

Basically, in java-parser-ttols, the extension syntax would allow a 
Matcher defined in Java to be assigned to a named matcher in the grammar 
definitionwork.  Here is an example...

quotedString= ?info.reflectionsofmind.parser.QuotedStringMatcher?

...where the contents between the question marks names a class that 
implements a Matcher.

It would be convenient if the Grammar class supported %whitespace% in 
grammar files.

Current if you pass an expression name into the Grammar.generate method 
that does not exist in the given grammer then Grammar.generate will simply 
return a null Matcher.

It would be nicer if it threw an exception intead.

XPath support (possibly using Jaxen) would make it much easier to navigate 
an AST. 
For instance, given this grammar...
   turtleDocument ::= {statement {%whitespace%}} 
   statement ::= ((directive {%whitespace%} ".") | (triples {%
whitespace%} ".")) 
   directive ::= (prefixID | base) 
   prefixID ::= ("@prefix" %whitespace% [prefixName] ":" {%whitespace%} 
uriref) 
   base ::= ("@base" %whitespace% uriref) 

I have to navigate down 6 levels of to get to a 'base' node.
With XPath I could just do...

  NamedNode base= XPath.selectNode("//base");

Adding getParent() to Abstract node will make it easier to navigate node 
trees.

Consider the following grammar...
   predicateObjectList ::= (verb objectList {";" verb  objectList}) 
   objectList ::= (object {"," object}) 
   verb ::= (resource | "a") 

Then consider the task of processing the verbs and associated objects in a 
predicateObjectList.


We can first get the all the verb nodes..

List<AbstractNode> verbNodes= Navigation.findAllDecendentsById
(predicateObjectListNode, "verb");


Now, given a verb node how can we get the associated object nodes?
There currently is no easy way except to manually navigate the 
predicateObjectList node and process the verbs and objects are we 
encounter them, yuck.

If we add getParent() to the AbstractNode class then we can easily find 
all the objects nodes for verb like so...

List<AbstractNode> objectNodes= Navigation.findAllDecendentsById
(verbNode.getParent(), "object");

My grammer is shown below.
Notice that the definition of the 'statement' expression includes a 
reference to the 'comment' expression.  However, the 'comment' expression 
is not defined.

Later, when the Matchers.fullMatch method is called a NullPointer 
exception is thrown.
Matchers.fullMatch(matcher, "@prefix 
bindingType:<meteor:net.sf.meteor.BindingType>.");


-----------------------------------------
My grammer...

turtleDocument ::= {statement {whitespace}} 
statement ::= ((directive {whitespace} ".") | (triples {whitespace} ".") | 
comment) 
directive ::= (prefixID | base) 
prefixID ::= ("@prefix" whitespace [prefixName] ":" {whitespace} uriref) 
base ::= ("@base" whitespace uriref) 
triples ::= (subject predicateObjectList) 
predicateObjectList ::= (verb objectList ";" {verb objectList ";"}) 
objectList ::= (object {"," object}) 
verb ::= (resource | "a") 
subject ::= (resource | blank) 
object ::= (resource | blank | literal) 
literal ::= (quotedString (["@" language] | datatypeString | integer | 
double | decimal | boolean)) 
datatypeString ::= (quotedString "^^" resource)
digits ::= (%digit% {%digit%})
sign ::= ("-" | "+")
integer ::= ([sign] digits) 
doubl ::= ([sign] ((digits "." [digits] exponent) | ("." digits exponent) 
| (digits exponent)))
decimal ::= ([sign] ((digits "." {digits}) | ("." digits) | digits)) 
exponent ::= (["e" | "E"] [sign] digits) 
boolean ::= ("true" | "false") 
blank ::= (nodeID | "[]" | ("[" predicateObjectList "]") | collection) 
collection ::= ("(" (object {"," object}) ")") 
resource ::= (uriref | qname) 
nodeID ::= ("_:" name)
qname ::= ([prefixName] ":" [name]) 
uriref ::= ("<" relativeURI ">")  
language ::= ((%lower% {%lower%}) {"-" ((%lower% | %digit%) {(%lower% | %
digit%)})})  
nameStartChar ::= (%alpha% | "_") 
nameChar ::= (%alpha% | %digit% | "-") 
name ::= (nameStartChar {nameChar})  
prefixName ::= (nameStartChar {nameChar}) 
relativeURI ::= (anychar {anychar}) 
quotedString ::= ("$" {anychar} "$")
anychar ::= (%alpha% | %digit% | " " | ":" | ".")  
whitespace ::= (" " {" "})   

Currently when a call is made to the Matchers.fullMatch(matcher, document) 
method and a document is passed with a syntax error in it then fullMatch 
simply returns an empty Results list.

There needs to be a way to indicate where the parsing failed so that the 
syntax error may be discovered. 
Perhaps the returned List should always have at least one Result object 
and the Result class should contain fields to indicate if a matching error 
happened and where in the document matching failed.

I need to define 'quoted strings' in my grammer.
Something like this...

quotedString ::= ("\"" {anychar} "\"")

I looked around at how other ebnf syntaxes handle this.
I can think of two ways to handle this.

1) Allow either single quotes or double quotes when defining text.
This is what the 'official' EBNF specification does.
http://www.cl.cam.ac.uk/~mgk25/iso-14977.pdf

2) Use backslash as an escape character like I did above.
This approach would be familiar to Java programmers.

I chose appoach #1 since it is easily implemented by changing java-parsing-
tools grammer definition.

To allow for better i18n, recognizing of different error types and overall 
static code verification we should store an Exception, not a String to 
indicate matching failure.