w3c / qt3tests Goto Github PK

In fn/matches.re.xml, there are tests re00461, re00981 which refer to IsHighSurrogates, and tests re00369 which refers to IsLowSurrogates. I'm trying to understand whether these should be accepted (as they are).

In the XML Schema 1.0 specification, in F.1.1:

"The blocks mentioned above exclude the HighSurrogates, LowSurrogates and HighPrivateUseSurrogates blocks. These blocks identify "surrogate" characters, which do not occur at the level of the "character abstraction" that XML instance documents operate on."

This seems clear: these should not be accepted.

In the XML Schema 1.1 specification things are more confusing.

In G.4.2.2 Category escapes, in the category definition at the bottom:

Note: The properties mentioned above exclude the Cs property.
The Cs property identifies "surrogate" characters, which do not
occur at the level of the "character abstraction" that XML instance
documents operate on.

but this is for IsCategory. It appears in XML Schema 1.1 IsHighSurrogates, IsLOwSurrogates and High Private Use Surrogates are accepted, unlike in XML Schema 1.1.

Perhaps we should add a dependency type on xsd-version 1.1 for the tests that refer to these?

These two tests use a sequence of two items for the Order spec key, and allow either a result or an error of XPTY004.
However the XQuery 1.0 and 3.1 specs both state:

The process of evaluating and comparing the orderspecs is based on the following rules:

• Atomization is applied to the result of the expression in each orderspec. If the result of atomization is neither a single atomic value nor an empty sequence, a type error is raised [err:XPTY0004].

As far as I can see these tests should only allow an error of XPTY004... or did I miss something?

In the XSLT tests, various test cases have been removed that use unicode blocks that have since disappeared w3c/xslt30-test#76

This includes IsCombiningMarksforSymbols. But various tests in fn/matches.re.xml still reference this obsolete unicode block. So I would propose we change those tests cases to use newer unicode blocks too (or remove them).

In particular:

• re00256 (IsCombiningMarksforSymbols)

• re00288 (IsPrivateUse)

• re00338 (IsCombiningMarksforSymbols)

• re00370 (IsPrivateUse)

• re00394 (IsGreek)

• re00429 (IsCombiningMarksforSymbols)

• re00480 (IsPrivateUse)

• re00738 (IsGreek)

Given the XPath expression:

let $m := map{'a': (1, 2), 'b': 2, 'c': 1, 'd': 3}
            return map:merge(
                for $key in map:keys($m) return for $val in map:get($m, $key) return map{$val : $key},
                map{'duplicates': 'combine'})

The test expects:

<assert>deep-equal($result(1), ("a", "c"))</assert>
<assert>deep-equal($result(2), ("a", "b"))</assert>
<assert>deep-equal($result(3), ("d"))</assert>

But isn't this dependent on the implementation-dependent order of keys?

If map:keys happens to produce ('d', 'c', 'b', 'a'):

then the inversion logic happens to produces a sequence with this structure:

({3: 'd'},  {'1': ''c'},  {'2': 'b'},  ({1: 'a'}, {2: 'a'}) 

then combining the values would result in: this map:

{1 : ('c', 'a'), 2 :  ('b', 'a'), 3: 'd'}

but that's not allowed according to the test.

A way to fix the test would be to sort the inverted sequence first.

The test metadata does not provide an adequate way of testing import module declarations with location hints. The test suite documentation says, for example: it can be argued that the language specification does not require a processor to allow multiple modules with the same module URI This may have been true in XQuery 1.0, but it is not true in 3.1, where §4.12.2 explicitly addresses this. This scenario seems to be untested in the current test suite, and recent email correspondence suggests that different processors are handling things in different ways, even where the spec is clear.

See also Saxon issue https://saxonica.plan.io/issues/4803

Hi,

a spec="XP30+" is needed for test cases "predicates-33" and "predicates-35" (concat operator || is used)

file: prod/Predicate.xml

    <test-case name="predicates-33">
      <description> Filter expression whose predicate compares boolean to untypedAtomic. Saxon bug 5187 </description>
      <created by="Michael Kay" on="2021-12-20"/>
      <environment ref="works"/>
      <test>count(/works/employee[(xs:integer(hours) gt 40) = xs:untypedAtomic("true"||substring(string(current-date()), 200))])</test>
      <result>
         <assert-eq>3</assert-eq>
      </result>
   </test-case>

   <test-case name="predicates-35">
      <description> Filter expression whose predicate compares multiple types to untypedAtomic. 
         Requires elimination of duplicates. Saxon bug 5187 </description>
      <created by="Michael Kay" on="2021-12-20"/>
      <environment ref="works"/>
      <test>(/works/employee[(xs:integer(hours) eq 40, empnum, number(hours) - 39) = xs:untypedAtomic("1"||substring(string(current-date()), 200))]) ! (empnum || pnum)</test>
      <result>
         <assert-deep-eq>"E1P1", "E2P1", "E4P4"</assert-deep-eq>
      </result>
   </test-case>

Thank you again

I think that test case fn-transform-82e is misinterpreting the spec.

If fn:transform() requests an XSLT 2.0 processor, and if no XSLT 2.0 processor is available, but an XSLT 3.0 processor is available, then the function executes with an XSLT 3.0 processor. This is the situation with Saxon 10, which no longer offers XSLT 2.0 processing as an option.

The question then is whether the "global-context-item" option to fn:transform() should be ignored. The wording in the spec is:

For invocation of an XSLT 2.0 processor (see [XSL Transformations (XSLT) Version 2.0]), the supplied options must include all of the following (if anything else is present, it is ignored):

followed by another section:

For invocation of an XSLT 3.0 processor...

Which of these two sections applies? I think it is the one relevant to the processor version that is actually invoked, not the one that is requested. With this interpretation, the global-context-item option in fn-transform-82e should not be ignored in the case where a 3.0 processor is invoked.

In the test re00984 the characters ⌈ and &#8969, that is, ⌈ and ⌉ are considered to be word characters, but they're in the set P, punctuation. According to the XML Schema 1.1 spec they should therefore not be considered to be in \w:

[#x0000-#x10FFFF]-[\p{P}\p{Z}\p{C}] (all characters except the set of "punctuation", "separator" and "other" characters)

See for instance here:

https://en.wiktionary.org/wiki/Appendix:Unicode/Miscellaneous_Technical

Or directly from the UCD's UnicodeData.txt:

2308;LEFT CEILING;Ps;0;ON;;;;;Y;;;;;
2309;RIGHT CEILING;Pe;0;ON;;;;;Y;;;;;

My proposal is to remove these two characters from the test.

This is going to be pretty long. I was fairly dismayed when I learned about this so please bear with me. I am going to talk about a test case, but I'm going to argue the XPath spec is ambiguous and unclear for this case, so perhaps this is more about improving the specification rather than interpreting it.

MapConstructor-025

Let's consider the test MapConstructor-025.

let $m := map{'a':1} return map:size(map{$m?a:true()})

The interesting bit here is:

map{$m?a:true()}

The test parses this as:

map { $m ? a : true() }

Now we turn to a piece of text in section A.2, which the description of the test references

When tokenizing, the longest possible match that is consistent with the EBNF is used.

The test interprets this as follows: a:true is the longest token, but it's only grammatical when the token is a, so the longest possible token is a.

But that's not really independent tokenizing like a lexer does. Tokenizing is not aware of the grammar; tokenizing creates tokens that can be used by the grammar.

So another interpretation of this same text could be:

a:true is the longest possible token allowed by the grammar, so a:true is the token. It's true that such a token leads to the above code not to be parsable, but that's okay: all kinds of sequences of correct tokens are not parsable.

What is a token?

But what is a token in the first place? We go back to A.2, which is titled "Lexical structure" and as we have seen, talks about tokenizing. It describes terminal symbols, at first sight a decent candidate for tokens.

What kind of terminal symbol is a:true? It would be a prefixed qname, as described by the QName production. QName is described as a terminal symbol.

QName refers to the grammar in the XML spec itself, where a QName is PrefixedName | UnprefixedName
Both are composed from NCName. An UnprefixedName is basically a NCName.

But wait. The XPath spec also describes an NCName by itself as a terminal symbol. So when we encounter an NCName in the grammar, is it to be interpreted as a QName or not? That cannot actually be determined without using the grammar.

So it appears that the terminal symbols are not really describing tokens at all.

It's unclear what is meant by "tokenizing".

What is a terminal symbol?

As a digression: A.2.1 says "Terminal symbols" but that evidently isn't the complete list of terminal symbols, as A.2.2 introduces a lot more delimiting and non-delimiting terminal symbols. Which I've interpreted as tokens, but you can't quite do that. Almost but not quite.

For instance, there is also an ambiguity with NCName for reserved function names such as if.

Is if an NCName or not? If it's used as a function if(), it's not, but in other contexts it is a valid ncname, and in another context it's a keyword in the language.

My adventure

In my XPath engine I have a separate lexer (tokenizing) and parsing phase. This is pretty common in programming language construction. The lexer produces tokens and the grammar then defines how those tokens combine into an AST. Whitespace is handled in the lexing phase and then disregarded during parsing.

This works pretty well in most cases, but I noticed it wasn't working well for some cases where ws: explicit is in use. It also didn't work well for QNames: foo : bar, with whitespace, could incorrectly be interpreted as a valid QName.

So I thought, let's handle these cases in the lexer. So now the lexer produces various new tokens, like URIQualifiedName and even wildcards (like foo:*) are interpreted as tokens as the "ws: explicit" rule is in place for them too.

But what to do about QName? As we discussed before, without the grammar it is ambiguous whether a given NCName is a QName or not. But, I thought, it's not ambiguous whether something is a PrefixedName.

So I introduced a token for that. And previous cases where the test were failing are now passing. But to my dismay, MapConstructor-025 started failing.

I am unclear how to resolve this: I really don't want to make the whole grammar whitespace aware for the few exceptional cases; I want to keep handling this in the lexer. But handling PrefixedName as a token leads to new trouble.

MapConstructor-025 takes one interpretation of the specification, but I really think the specification is ambiguous as it doesn't define what a token is. I think PrefixedName is a perfectly reasonable interpretation of "longest possible match according to the EBNF" .

What is a terminal?

[Definition: A terminal is a symbol or string or pattern that can appear in the right-hand side of a rule, but never appears on the left-hand side in the main grammar, although it may appear on the left-hand side of a rule in the grammar for terminals.]

Does this really say much at all beyond "if we put the rule in the grammar for terminals, it's a terminal, but if we put it in the main grammar it's not"?

Concrete steps

I'd love for the code in MapConstructor-025 to be considered invalid grammatically, but I don't know whether it can be, and the specification is unclear anyway.

I think a concrete step is to make the specification more clear. What's a token? (or avoid that word altogether). Is a terminal symbol a token?

If the interpretation in MapConstructor-025 is correct, make explicit that QName or PrefixedName cannot be a token (and safely tucked away in the XML specification), and that NCName is ambiguous on the token level and can only be resolved grammatically.

I'm going to have to think hard for a while to find a doable way to make MapConstructor-025 work in my implementation.

The test is testing for the vendor field being correctly recognized as - 'vendor' : 'Saxonica' - we are using SaxonHE in the eXist implementation of fn:transform, and the vendor string returned from Saxon matches; therefore we attempt to compile and run the stylesheet. Doing so,

the template used in this test:

<xsl:template name='main'>
              <out><xsl:value-of select=""saxon:in-summer-time(xs:dateTime('2016-07-01T00:00:00'), 'Europe/London')""/></out>
            </xsl:template> 

causes an error when run using Saxon HE: QueryError(FOXT0003,err:FOXT0003 Errors were reported during stylesheet compilation.

Further digging suggests that saxon:in-summer-time is restricted to Saxon EE. Presumably therefore the test passes when run with a version of Saxon EE.

We suggest that the test be modified to use a function valid in all versions of Saxon.

I just noticed the tests are here now. They are hard to find.

When you google for "XQuery/XPath/XSLT 3.* Test Suite" it still finds https://dev.w3.org/2011/QT3-test-suite where it does not mention github

And the instructions here (e.g. Overview.html or guide/running.html ) still tell you to report bugs to the disabled w3c bugzilla

I am looking at test case numberformat11 https://github.com/w3c/qt3tests/blob/master/fn/format-number.xml#L118

<test-case name="numberformat11">
      <description>
        Creator: David Marston 
        Purpose: Test default decimal-format on pattern-only characters, positive number. </description>
      <created by="Michael Kay, Saxonica" on="2010-12-13"/>
      <!--SOURCE: numberformat/numberformat11.xml-->
      <environment>
         <decimal-format digit="!" pattern-separator="\"/>
      </environment>
      <test>format-number(26931.4,'+!!!,!!!.!!!\-!!,!!!.!!!')</test>
      <result>
         <assert-string-value>+26,931.4</assert-string-value>
      </result>
   </test-case>

It seems to me that this should actually raise the error FODF1310 as the picture string is invalid. It is invalid because it fails this part of the specification:

https://www.w3.org/TR/xpath-functions-31/#syntax-of-picture-string

A sub-picture must not contain a passive character that is preceded by an active character and that is followed by another active character.

It seems to me that, after the decimal format in the environment is redefined, the 3 characters \-! form a sequence of active, passive, active characters.

If this is the case then I believe that numberformat12 is also incorrect and should also raise FODF1310 for a similar reason.

Can someone confirm my understanding?

Hi,

the test case "fn-replace-58" requires a spec="XP3.1+" because it uses the arrow operator =>:

file: fn/replace.xml

   <test-case name="fn-replace-58">
      <description>Evaluation of replace with emoji. Saxon bug 5174.</description>
      <created by="Michael Kay after Martin Honnen" on="2021-11-24"/>
      <test>let $in := codepoints-to-string((72, 101, 108, 108, 111, 32, 128512, 128515, 128516, 32, 
                                             128156, 129446, 32, 252, 228, 246, 36)) 
         return replace($in, "\p{IsEmoticons}+", "") => string-to-codepoints()</test>
      <result>
         <assert-deep-eq>72, 101, 108, 108, 111, 32, 32, 
            128156, 129446, 32, 252, 228, 246, 36</assert-deep-eq>
      </result>
   </test-case>

Thank you

I (CC @alanpaxton) just want to check my understand of K-RoundFunc-2 from the fn-round test-set: https://github.com/evolvedbinary/qt3tests/blob/master/fn/round.xml#L397

I see that it is different from K-RoundFunc-2a in terms of its dependencies:

1. K-RoundFunc-2 depends on XQuery 1.0
2. K-RoundFunc-2a depends on XQuery 3.0+

I have tested the following query against recent Saxon, BaseX, and eXist-db, and all of them return 1 instead of the error XPST0017:

xquery version "1.0";
round(1, 2)

They each return the same result (i.e. 1) as running:

xquery version "3.0";
round(1, 2)

Does this mean that each of those 3 processors is not correctly implementing the version declaration, and are failing this test?

Comparing the version declaration behaviour between XQuery 1.0 and XQuery 3.0 is not straight-forward:

1. https://www.w3.org/TR/2010/REC-xquery-20101214/#id-version-declaration
2. https://www.w3.org/TR/xquery-30/#id-version-declaration

With Saxon, fn:matches("AB", "^(.*)+B") returns false, and I don't understand why. I've ported Saxon's regex engine to Rust, and I'm also getting a false here (once I fixed another bug), which surprised me, so I verified that Saxon does the same.

Looking at it, the .* should consume "A", and then "B" should be matched. But it doesn't do that.

On regex101 all the flavors have it match https://regex101.com/r/JfmqWf/1

I'm trying to understand why it wouldn't match for the XPath regex flavor. And whether it should match or not, I think it's worthwhile adding a test case to the test suite to nail this behavior down.

This behavior is implemented inside of the Repeat op, for greedy matching. I suspect it's in advance() (in the Java code). I see a bug was fixed previously (3787).

MapTest-058 has

 let $s as function(xs:anyAtomicType) as xs:string := map { 1:'A','x':'B' }

But the type of this map is function(xs:anyAtomicType) as xs:string? (2.5.6.2.35) which is not function(xs:anyAtomicType) as xs:string

re00984 tests that ⌈,⌉ (U+2308 and U+2309) should match \w

But they have category code Ps,Pe, so they match \p{P} and should not match \w = [#x0000-#x10FFFF]-[\p{P}\p{Z}\p{C}]

 <dependency type="spec" value="XQ10+"/>
 <test><![CDATA[count(string(text { }))]]></test>

text { } is a syntax error before xq3.1

@michaelhkay: In the section on fn:generate-id in the XQFO specification, it is written that the returned identifier of the function

must consist of ASCII alphanumeric characters and must start with an alphabetic character. Thus, the string is syntactically an XML name.

Various tests in https://github.com/w3c/qt3tests/blob/master/fn/generate-id.xml require the resulting id to match the regular expression ^[A-Za-z][A-Za-z0-9]*$. Maybe we need to relax this pattern?

This test uses the document { node constructor from XQuery but it doesn't specify an XQuery dependency.

<![CDATA[
         document{<M id="1"/>}/M/count(subsequence(. ! (., @id), 1, 2))
]]>

@michaelhkay We now have two test suites that are actively maintained (this one and https://github.com/qt4cg/qt4tests/). Do we already have a standard procedure to synchronize the two repositories?

If not, I’d be glad to cherry-pick recent qt3tests commits to the QT4 repository.

about the decimal type the spec says:

For the xs:decimal type, the maximum number of decimal digits (totalDigits facet) MUST be at least 18. This limit SHOULD be at least 20 digits in order to accommodate the full range of values of built-in subtypes of xs:integer, such as xs:long and xs:unsignedLong.

Thus it is correct to perform all calculations with 18 digit total precision

But an expected outcome of op-numeric-dividedec2args-4 is -1.619760582531006901, which has 18 digit fraction precision, but 19 digit total precision.

I am looking at the fn-transform-20 test case here: https://github.com/w3c/qt3tests/blob/master/fn/transform.xml#L840

I am a little bit confused by its source definition:

<environment>
    <source role="$staticbaseuri" file="transform/staticbaseuri.xsl"  uri="http://www.w3.org/fots/fn/transform/staticbaseuri.xsl"/>
</environment>

I don't understand why it has both role and uri attributes. It would seem to me that the fn-transform-20 test itself only needs the uri attribute to make the document available to the stylesheet-location. Is this a small mistake?

Likewise, as I read in the catalog-schema.xsd, it states in English (but does not enforce in its grammar):

The role and uri attributes indicate how the source document is made available to queries (as the context item, as the value of an external variable, or as a URI that can be supplied to the doc() function.)

Perhaps it is my reading, but I interpret that to mean that there should be a role OR uri attribute, but not both.

A second issue with fn-transform-20 is that it appears to make a possible assertion that can never be fulfilled. I see:

<any-of>
    <assert>$result/x = 'http://www.example.com'</assert>
    <assert>$result/x = 'http://www.w3.org/fots/fn/transform/staticbaseuri.xsl'</assert>
</any-of>

However my reading of the test and the staticbaseuri.xsl is that it can only ever match:

<assert>$result/x = 'http://www.example.com'</assert>

Comments please

collation-key-010 and collation-key-014 do not use the html5 environment (unlike the other tests in that file depending on html-ascii-insensitive)

I am a little confused by test cases uri-collection-901 and uri-collection-903, see: https://github.com/w3c/qt3tests/blob/master/fn/uri-collection.xml#L66

Basically they look like this:

  <test>uri-collection(())</test>
  <result>
     <error code="FODC0002"/>
  </result>

If I am interpreting the XP 3.1 spec correctly, then I think the result they are asserting might be too strict.

The spec for fn:uri-collection states:

The zero-argument form of the function returns the URIs in the default URI collection described in Section C.2 Dynamic Context Components.

So I looked up the default URI collection definition in the Dynamic context here: https://www.w3.org/TR/xpath-31/#id-xp-evaluation-context-components, and it states:

Default URI collection | global; overwriteable by implementation

So if I understand it, an implementation is free to set any default URI collection they wish. If that is the case, would I be permitted to send a PR to change the result of the tests to be something like:

  <test>uri-collection(())</test>
  <result>
     <any-of>
       <assert>fn:not(fn:empty($result)) and every $uri in $result satisfies $uri instance of xs:anyURI</assert> 
       <error code="FODC0002"/>
     <any-of>
  </result>

Kind regards.

fo-test-fn-id-001 fo-test-fn-id-002 fo-test-fn-element-with-id-001 fo-test-fn-element-with-id-002 fo-test-fn-idref-001 fo-test-fn-idref-002 have a validate lax { expression, but no dependency on schema validation and do not allow XQST0075 as output

Hi, I'm a bit confused about the combination of the test cases fn-function-lookup-759 and fn-function-lookup-761.

The fn-function-lookup-759 says that in any case you should be able to lookup for the function load-xquery-module:

  <test-case name="fn-function-lookup-759" 
             covers="fn-load-xquery-module">
    <description>Attempts to look up function fn:load-xquery-module#1.</description>
    <created by="Tim Mills" on="2016-08-19"/>
    <dependency type="spec" value="XP31+ XQ31+"/>
    <test>exists(function-lookup(fn:QName('http://www.w3.org/2005/xpath-functions', 'load-xquery-module'), 1))</test>
    <result>
      <assert-true/>
    </result>
  </test-case>

Reminder:
The function signature of load-xquery-module#1 is fn:load-xquery-module($module-uri as xs:string) as map(*).

The test case fn-function-lookup-761 says now if you do not support the feature fn-load-xquery-module and make the same lookup as in fn-function-lookup-759 but now call the function with a wrong typed argument, this function must throw an error code FOQM0006:

  <test-case name="fn-function-lookup-761" 
             covers="fn-load-xquery-module"
             covers-30="dynamic-function-call">
    <description>Attempts to invoke function fn:load-xquery-module#1.</description>
    <created by="Tim Mills" on="2016-08-19"/>
    <dependency type="spec" value="XP31+ XQ31+"/>
    <dependency type="feature" value="fn-load-xquery-module" satisfied="false"/>  
    <test>function-lookup(fn:QName('http://www.w3.org/2005/xpath-functions', 'load-xquery-module'), 1)( 0 )</test>
    <result>
      <error code="FOQM0006" />
    </result>
  </test-case>

On my point of view, the error code XPTY0004 for the failed type conversion should be also acceptable, shouldn't it? Otherwise the function returned by function-lookup(fn:QName('http://www.w3.org/2005/xpath-functions', 'load-xquery-module'), 1) must have a signiture which is more lax as the specification requires.

Test re00984 tests a large number of code-points for the \w character sequence.
Characters ⌈ and ⌉ are in this list. These codepoints were moved from \p{S} to \p{P} in unicode version 6.3, and therefore out of the \w character sequence.

Perhaps the test should include the "unicode-version" dependency flag for version "6.2"?