Coder Social home page Coder Social logo

exist-db / exist Goto Github PK

View Code? Open in Web Editor NEW
416.0 62.0 179.0 666.3 MB

eXist Native XML Database and Application Platform

Home Page: https://exist-db.org

License: GNU Lesser General Public License v2.1

XQuery 5.25% Shell 0.15% Python 0.01% Perl 0.03% XSLT 0.27% HTML 0.24% Java 92.91% GAP 0.97% CSS 0.08% Batchfile 0.01% AMPL 0.02% Dockerfile 0.02% JavaScript 0.04%
xquery xml java xslt xproc tei-xml xml-schema application-server xml-database nosql

exist's People

Contributors

adamretter avatar alanpaxton avatar alexmilowski avatar brihaye avatar chaeron avatar chris-evolvedbinary avatar dependabot-preview[bot] avatar dependabot[bot] avatar dizzzz avatar duncdrum avatar fgeorges avatar gev avatar jamfuller avatar jensopetersen avatar jmfernandez avatar jmvanel avatar joewiz avatar lcahlander avatar line-o avatar ljo avatar marmoure avatar olvidalo avatar pkaminski avatar reinhapa avatar rhubner avatar rwalpole avatar shabanovd avatar windauer avatar wolfgangmm avatar xristy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

exist's Issues

unable to change password of users other than admin in Admin Client

One is unable to change the password of a user one has created in the Admin Client

On Mac OS 10.9.1, using Oracle build 1.7.0_51-b13, I have done the following:

1 download develop (8dc2e16)
2 build
2 start up eXist
3 open up admin client against new build
4 accept default settings (connect type: remote) and log in to admin account with the default empty password
5 choose Edit Users from Tools
6 set admin password to e.g. "secret"
7 create a user "foo" with password "bar" - assign "foo" to the group "guest" and click "Create"
9 open the user account for foo
10 change foo's password to e.g. "magic" and click "Save"
# One gets the error "org.xmldb.api.base.XMLDBException: Wrong password for user [admin]"
11 restart the (now unresponding) admin client and log in with admin/secret
12 change the password of the foo user to the password of the admin user, "secret", and click "Save"
# this works: the problem is that one can only change the password of existing users to the password of the admin user, since (I guess) the code for changing the admin user and changing other users is mixed up - when you change the admin user, a login with the new admin password is presumably performed, and this appears to be happening as well when you change the password of other users

Regression in ant adduser task; creating users break the admin client

Invoking the following ant task against a clean eXist 2.1 server, as well as the 2.2RC1:

<exist:adduser name="myuser" secret="mypassword" primaryGroup="dba"
            failonerror="false" uri="${exist.url}/xmlrpc/db" user="admin" password=""/>

Seems to create the user successfully, but when running the admin client I receive the following error:

$ bin/client.sh -ouri=xmldb:exist://localhost:8080/xmlrpc -u admin -P password -s
...
XMLDBException while retrieving collection contents: Failed to invoke method getSubCollectionPermissions in class org.exist.xmlrpc.
org.apache.xmlrpc.XmlRpcException: Failed to invoke method getSubCollectionPermissions in class org.exist.xmlrpc.RpcConnection: nul
    at org.apache.xmlrpc.client.XmlRpcStreamTransport.readResponse(XmlRpcStreamTransport.java:197)
    at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:156)
    at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143)
    at org.apache.xmlrpc.client.XmlRpcSunHttpTransport.sendRequest(XmlRpcSunHttpTransport.java:69)
    at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56)
    at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167)
    at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:158)
    at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:147)
    at org.exist.xmldb.RemoteCollection.getSubCollectionPermissions(RemoteCollection.java:342)
    at org.exist.xmldb.RemoteUserManagementService.getSubCollectionPermissions(RemoteUserManagementService.java:402)
    at org.exist.client.InteractiveClient.getResources(InteractiveClient.java:406)

Incidentally, since upgrading from eXist 2.0 I also see a change in behaviour when changing the password from the ant tasks, the target reports an error in making the change but the password change goes through.

jetty hangs in SelectChannelEndPoint.blockWritable

When doing repeated requests against jetty and connections are dropped before a response is given jetty will hang itself in a wait loop when trying to write ( and therefor will blocks all other threads ).

This has been tested by creating a simple test-case that uses XQJ/XQS to query exist-db and using a query which results in a decent-sized resultset.

Queries executed at a normal rate ( and with a small resultset ) are having no issues, but as soon as you will make continuous requests which will get dropped ( e.g. by constantly refreshing a browser ) this issue occurs and as shown in the logs below all threads will enter a wait-state.

snapshot-1382621601502-cpu-backtraces
screenshot from 2013-10-24 15 58 21

I have searched for "SelectChannelEndPoint.blockWritable" on Google and found numerous reports to related issues:

And one possible solution:

NPE when running util:serialize with doctype parameters

I'm trying to serialize DITA topic to string with DOCTYPE's added. So the following code block is used:

let $serialization-params := 
(
    "doctype-public=-//D//DTD P-Topic//EN", 
    "doctype-system=http://domain.com/xmetal/schemas/p-topic.dtd"
)
let $topic-string := util:serialize($topic-document, $serialization-params)

It fails with the following NPE:

Caused by: java.lang.NullPointerException
    at org.exist.xquery.functions.util.Serialize.parseSerializationOptions(Serialize.java:194)
    at org.exist.xquery.functions.util.Serialize.eval(Serialize.java:156)
    at org.exist.xquery.BasicFunction.eval(BasicFunction.java:70)
    at org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:55)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)

According to the code, it splits the string using space character as a delimiter. It looks like the same code is used to parse the serialization parameters that can be provided in declare option exist:serialize statement. However, in case of util:serialize the parameters format is different and I believe the space should not be considered as a delimiter.

I've tried to replace the space with the non-breaking space code &#160; This technique works well in declare option exist:serialize statement, however, with util:serialize it leads to another error:

exerr:ERROR XMLDB reported an exception while storing documentorg.xmldb.api.base.XMLDBException: The XML parser reported a problem: fatal error at (2,39) : An invalid XML character (Unicode: 0xa0) was found in the public identifier. [at line 57, column 22]

Not sure if it makes more sense to allow using space or &#160; or both but at least one of those ways should be working. Any considerations? It would be nice to fix it before 2.2 final release. I'm willing to help fixing but I'd like to hear developers' opinion on this.

I18N: format-number and decimal separator

The xquery 3.0 specification, section 4.10, introduces a declaration to change the decimal separator. This example from there:

declare decimal-format local:de decimal-separator = "," grouping-separator = ".";
declare decimal-format local:en decimal-separator = "." grouping-separator = ",";

let $numbers := (1234.567, 789, 1234567.765)
for $i in $numbers
return (
  format-number($i, "#.###,##", "local:de"),
  format-number($i, "#,###.##", "local:en")
)

It would be a very welcome feature in eg. Europe.

enable-xsl setting in conf.xml is ignored

In $EXIST_HOME/conf.xml there is a setting attribute on the "serializer" named "enable-xsl", by default this is set to "no".

Even when set to "no", eXist attempts to process XSL PIs on documents retrieved via REST, this is incorrect and is a regression.

Permissions incorrect in Java admin tool Properties

With multiple files selected in the Java admin client, opening the Properties window always shows the permissions for the selected files as rw-rw-rw-, regardless of the actual permissions of the selected files.

screen shot 2014-03-27 at 8 58 12 am

Notice here that the original files are rw-r--r--, but the Java admin client shows the permissions as rw-rw-rw-.

A better approach would be to pick up the actual properties of the selected items. This should be straightforward if the properties are identical. If not identical, then it would make sense to display the overlapping set of properties. Even better, some UIs allow for a "gray dash" mark over the property checkbox that are not common across all the items; then the user can click once to turn all on, again to turn all off, or not at all to leave the original properties unchanged. I'm not sure if this UI element is available to the Java admin client though.

when backing up with the Admin Client, an existing backup file is always overwritten

On Mac OS 10.9.1, using Oracle build 1.7.0_51-b13, I have done the following:

1 download latest develop (8dc2e16)
2 build
2 start up eXist
3 open up admin client against new build
4 accept default settings (connect type: remote) and log in to admin account with the default empty password
5 export the contents of the database to some destination
6 export the contents of the database to the same destination, using the same filename. Answer "No" to "OK to delete".
# It does not matter what you answer - the original backup is deleted in any case.

Stability of exist 2.1 against exist 1.4

This is a question
My company has an application developed on exist 1.4 which is hosted under tomcat7. There have been a lot of complaints from productions deployments about the stability of the recovery. Many times they had to resolve into a restore from a last backup.
We are working on a POC to see what are the differences between these two versions with regards to this subject
To achieve this I created collections independent form the existing's application.

While doing so I corrupted relatively easy the exist 1.4 while I only saw the recovery only once
I tried corrupting the exist 2.1 but I never got to a place that no query was executing. But even without the error, the result of the functions compared to what the database reports is not matching.

Here is the script

xquery version "3.0";

import module namespace xmldb="http://exist-db.org/xquery/xmldb";

let $log-in := xmldb:login("/db", "admin", "admin")

let $max := 10000
for $i in (1 to $max)
        let $filename := concat("raw", $i, ".xml")
        let $data:=<raw i="{xs:string($i)}"/>
return
xmldb:store('/db/TestStability', $filename, $data)

While the xmldb:store returns the name of the file the database reports only the raw1.xml being created.

Can you please provide some feedback about the above problem? Can you also provide what is actually better on the stability front between the major versions?

eXist seems to treat 'xquery' as reserved namespace prefix in xquery modules

When an imported xquery module declares 'xquery' as its namespace prefix, eXist-2.1 throws an error. A test case can be found at https://gist.github.com/rvdb/7152363. If ns_main.xq is executed with eXist (eXide / Java client / controller), following error is thrown:

Cannot compile xquery: exerr:ERROR error found while loading module test: 
Error while loading module ns_module.xq: error found while loading module 
from ns_module.xq: org.exist.xquery.XPathException: err:XPST0003 expecting
name, found 'xquery' [at line 2, column 18]

This seems incorrect: 'xquery' is not listed as reserved namespace prefix in the spec (http://www.w3.org/TR/xquery/#id-basics), which is confirmed by the fact that Saxon does execute ns_main.xq without problems.

Additionally, this problem seems limited to module namespace declarations. Another test case at https://gist.github.com/rvdb/7152340 is executed without errors, which demonstrates that 'xquery' can be used in a main module without complaints.

Remove warnings Java8

with java8 there are some warnings; over time we need to clean these

    [javac] ........./exist/src/org/exist/storage/md/MetaData.java:41: warning: '_' used as an identifier
    [javac]     protected static MetaData _ = null;
    [javac]                               ^
    [javac]   (use of '_' as an identifier might not be supported in releases after Java SE 8)
    [javac] ........./exist/src/org/exist/storage/md/MetaData.java:44: warning: '_' used as an identifier
    [javac]         return _;
    [javac]                ^
    [javac]   (use of '_' as an identifier might not be supported in releases after Java SE 8)

NPE in Lucene

Similar to the error reported in #166 (was #57), I am seeing an NPE at http://history.state.gov/search?q=peace&within=frus-history. Other search terms like http://history.state.gov/search?q=piece&within=frus-history do not return any error. This page is doing a Lucene full text search on a collection of TEI documents. The server is running v2.1 develop branch rev. a36fbcf from Mar 2. Restarting eXist-db didn't have an effect on the error. The error I get in browser window is:

HTTP ERROR 500

Problem accessing /search. Reason:

    An error occurred: null
Caused by:

javax.servlet.ServletException: An error occurred: null
    at org.exist.http.servlets.EXistServlet.doGet(EXistServlet.java:256)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:669)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:457)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:229)
    at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:103)
    at org.exist.http.urlrewrite.Forward.doRewrite(Forward.java:46)
    at org.exist.http.urlrewrite.XQueryURLRewrite.doRewrite(XQueryURLRewrite.java:577)
    at org.exist.http.urlrewrite.XQueryURLRewrite.service(XQueryURLRewrite.java:353)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:669)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:457)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:368)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:488)
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:932)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:994)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NullPointerException
    at org.exist.indexing.lucene.LuceneMatchListener.scanMatches(LuceneMatchListener.java:167)
    at org.exist.indexing.lucene.LuceneMatchListener.startElement(LuceneMatchListener.java:126)
    at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:203)
    at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:80)
    at org.exist.storage.serializers.Serializer.serializeToReceiver(Serializer.java:1070)
    at org.exist.storage.serializers.Serializer.toSAX(Serializer.java:878)
    at org.exist.dom.NodeProxy.toSAX(NodeProxy.java:791)
    at org.exist.xquery.functions.util.Expand.eval(Expand.java:111)
    at org.exist.xquery.BasicFunction.eval(BasicFunction.java:70)
    at org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:55)
    at org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:74)
    at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:214)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:102)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153)
    at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61)
    at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:327)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:232)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:65)
    at org.exist.xquery.ForExpr.eval(ForExpr.java:307)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:102)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:102)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:65)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:189)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153)
    at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:327)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:232)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.SequenceConstructor.eval(SequenceConstructor.java:84)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:100)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:102)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:65)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:189)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153)
    at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:327)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:232)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.XQuery.execute(XQuery.java:297)
    at org.exist.xquery.XQuery.execute(XQuery.java:226)
    at org.exist.xquery.functions.util.Eval.execute(Eval.java:443)
    at org.exist.xquery.functions.util.Eval.doEval(Eval.java:396)
    at org.exist.xquery.functions.util.Eval.eval(Eval.java:224)
    at org.exist.xquery.BasicFunction.eval(BasicFunction.java:70)
    at org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:55)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:100)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153)
    at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:327)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:232)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.SequenceConstructor.eval(SequenceConstructor.java:84)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.XQuery.execute(XQuery.java:297)
    at org.exist.xquery.XQuery.execute(XQuery.java:217)
    at org.exist.http.RESTServer.executeXQuery(RESTServer.java:1533)
    at org.exist.http.RESTServer.doGet(RESTServer.java:508)
    at org.exist.http.servlets.EXistServlet.doGet(EXistServlet.java:222)
    ... 42 more
Caused by:

java.lang.NullPointerException
    at org.exist.indexing.lucene.LuceneMatchListener.scanMatches(LuceneMatchListener.java:167)
    at org.exist.indexing.lucene.LuceneMatchListener.startElement(LuceneMatchListener.java:126)
    at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:203)
    at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:80)
    at org.exist.storage.serializers.Serializer.serializeToReceiver(Serializer.java:1070)
    at org.exist.storage.serializers.Serializer.toSAX(Serializer.java:878)
    at org.exist.dom.NodeProxy.toSAX(NodeProxy.java:791)
    at org.exist.xquery.functions.util.Expand.eval(Expand.java:111)
    at org.exist.xquery.BasicFunction.eval(BasicFunction.java:70)
    at org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:55)
    at org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:74)
    at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:214)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:102)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153)
    at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61)
    at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:327)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:232)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:65)
    at org.exist.xquery.ForExpr.eval(ForExpr.java:307)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:102)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:102)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:65)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:189)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153)
    at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:327)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:232)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.SequenceConstructor.eval(SequenceConstructor.java:84)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:100)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:102)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:65)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:189)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153)
    at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:327)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:232)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.XQuery.execute(XQuery.java:297)
    at org.exist.xquery.XQuery.execute(XQuery.java:226)
    at org.exist.xquery.functions.util.Eval.execute(Eval.java:443)
    at org.exist.xquery.functions.util.Eval.doEval(Eval.java:396)
    at org.exist.xquery.functions.util.Eval.eval(Eval.java:224)
    at org.exist.xquery.BasicFunction.eval(BasicFunction.java:70)
    at org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:55)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:100)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153)
    at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:327)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:232)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.SequenceConstructor.eval(SequenceConstructor.java:84)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.XQuery.execute(XQuery.java:297)
    at org.exist.xquery.XQuery.execute(XQuery.java:217)
    at org.exist.http.RESTServer.executeXQuery(RESTServer.java:1533)
    at org.exist.http.RESTServer.doGet(RESTServer.java:508)
    at org.exist.http.servlets.EXistServlet.doGet(EXistServlet.java:222)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:669)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:457)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:229)
    at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:103)
    at org.exist.http.urlrewrite.Forward.doRewrite(Forward.java:46)
    at org.exist.http.urlrewrite.XQueryURLRewrite.doRewrite(XQueryURLRewrite.java:577)
    at org.exist.http.urlrewrite.XQueryURLRewrite.service(XQueryURLRewrite.java:353)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:669)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:457)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:368)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:488)
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:932)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:994)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:744)
Powered by Jetty://

The error in exist.log appears to be identical (?):

2014-03-25 18:23:49,977 [eXistThread-51] ERROR (EXistServlet.java [doGet]:255) -  
java.lang.NullPointerException
    at org.exist.indexing.lucene.LuceneMatchListener.scanMatches(LuceneMatchListener.java:167)
    at org.exist.indexing.lucene.LuceneMatchListener.startElement(LuceneMatchListener.java:126)
    at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:203)
    at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:80)
    at org.exist.storage.serializers.Serializer.serializeToReceiver(Serializer.java:1070)
    at org.exist.storage.serializers.Serializer.toSAX(Serializer.java:878)
    at org.exist.dom.NodeProxy.toSAX(NodeProxy.java:791)
    at org.exist.xquery.functions.util.Expand.eval(Expand.java:111)
    at org.exist.xquery.BasicFunction.eval(BasicFunction.java:70)
    at org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:55)
    at org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:74)
    at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:214)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:102)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153)
    at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61)
    at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:327)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:232)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:65)
    at org.exist.xquery.ForExpr.eval(ForExpr.java:307)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:102)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:102)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:65)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:189)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153)
    at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:327)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:232)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.SequenceConstructor.eval(SequenceConstructor.java:84)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:100)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:102)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
    at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:65)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:189)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153)
    at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:327)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:232)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.XQuery.execute(XQuery.java:297)
    at org.exist.xquery.XQuery.execute(XQuery.java:226)
    at org.exist.xquery.functions.util.Eval.execute(Eval.java:443)
    at org.exist.xquery.functions.util.Eval.doEval(Eval.java:396)
    at org.exist.xquery.functions.util.Eval.eval(Eval.java:224)
    at org.exist.xquery.BasicFunction.eval(BasicFunction.java:70)
    at org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:55)
    at org.exist.xquery.ConditionalExpression.eval(ConditionalExpression.java:100)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153)
    at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:327)
    at org.exist.xquery.FunctionCall.eval(FunctionCall.java:232)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.SequenceConstructor.eval(SequenceConstructor.java:84)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.XQuery.execute(XQuery.java:297)
    at org.exist.xquery.XQuery.execute(XQuery.java:217)
    at org.exist.http.RESTServer.executeXQuery(RESTServer.java:1533)
    at org.exist.http.RESTServer.doGet(RESTServer.java:508)
    at org.exist.http.servlets.EXistServlet.doGet(EXistServlet.java:222)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:669)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:457)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:229)
    at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:103)
    at org.exist.http.urlrewrite.Forward.doRewrite(Forward.java:46)
    at org.exist.http.urlrewrite.XQueryURLRewrite.doRewrite(XQueryURLRewrite.java:577)
    at org.exist.http.urlrewrite.XQueryURLRewrite.service(XQueryURLRewrite.java:353)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:669)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:457)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:368)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:488)
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:932)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:994)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:744)

About collection design

I'm really interested in this project, and I'm ready to build our EHR project using this database. Native XML database is new to me. I don't know how to design the structure of the database(or collection). For example, in my EHR project, each user has his/her basic information along with his/her various clinical information. I can't just organize these information in one XML file. So I should create lots of XML files. The question is, how to express the relationship between these XML files? And XQuery is used to query information in single XML file, how is it used to searching information from a set of XML files. How to organize the XML files and their relationship. That really confused me, and I can not find any solution. Please help me, thank you.

Installing AtomicWiki from the dashboard doesn't work

Install AtomicWiki via Package Manager on a fresh exist-db instance.
The installation process runs without error and AtomicWiki appears in the dashboard.
When you click this icon you get:

<exception>
  <path>/db/apps/wiki/modules/view.xql</path>
  <message>
    err:XPDY0002 variable '$templates:TYPE_ERROR' is not set. [at line 206, column 20, source: /db/apps/wiki/modules/templates.xql]
  </message>
</exception>

System Status:

    Uptime: PT23M17.468S
    eXist Version:  2.1
    eXist Build:    20130717
    eXist Home: E:\devtools\exist-db
    SVN Revision:   18721
    Operating System:   Windows 7 6.1 amd64
    File encoding:  UTF-8
  Java
    Vendor: Oracle Corporation
    Version:    1.7.0_21
    Implementation: Java HotSpot(TM) 64-Bit Server VM
    Installation:   E:\devtools\jdk7\jre
    Temp file path: E:\devtools\exist-db\tools\jetty/tmp
    Memory Usage
    Max. Memory:    1398144K
    Current Total:  558016K
    Free:   253948K

Need for a more flexible catalog-loader

When trying to load the DocBook XSLT files, there are failures due to entities that are not recognized by eXist.

I tried to make another pass at creating an EXPath library that encapsulates the DocBook 5 XSLT files. The entities in entities.ent are not recognized since they are not in eXist's catalog.xml. It would be nice if the EXPath packaging allowed for a catalog.xml for the app?

When trying to load the XSLT files, the following error occurs:

storing document autoidx-kimber.xsl (6 of 99) ...XMLDBException during parse: Failed to invoke method parse in class org.exist.xmlrpc.RpcConnection: java.io.FileNotFoundException: /Applications/eXist-db/entities.ent (No such file or directory)

shutdown.bat not working

When you use bin/shutdown.bat you get error message

ERROR: HTTP server returned unexpected status: Not Found
CAUSE: HTTP server returned unexpected status: Not Found

when jetty is running on a non-standard http port, e.g. 8083 instead of 8080.

Looks like the standard http port is hardcoded in class org.exist.jetty.ShutdownServer line 73:

String uri = properties.getProperty("uri", "xmldb:exist://localhost:8080/exist/xmlrpc");

fuzzy search in XML syntax for Lucene queries

In the XML syntax for Lucene queries, one is supposed (https://en.wikibooks.org/wiki/XQuery/Lucene_Search#XML_Query_Syntax_vs._Default_Lucene_Syntax) to be able to fire off a query against the old Shakespeare files like

let $query := 
<query>
<fuzzy min-similarity="0.3">snake</fuzzy>
</query>
return //SPEECH[ft:query(., $query)]

However, it does not matter which value one gives @min-similarity - 379 records are returned no matter what. This is also what //SPEECH[ft:query(., 'snake~')], with the default value '0.5', returns, so @min-similarity does not seem to be taken into account. Is the name of this variable wrong or is this feature not implemented?

"boundary-space preserve" declaration not observed

With a "boundary-space preserve" declaration, white-space between elements is supposed to be maintained. However, when I feed the function below with the exact same contents from an in-memory variable and from a document stored in the database, the result of the in-memory fragment is returned with white-space preserved, whereas the result of the stored document is returned as if the declaration had told the function to strip white-space, if the text node is white-space-only.

xquery version "3.0";

declare namespace tei="http://www.tei-c.org/ns/1.0";

declare boundary-space preserve;

declare function local:get-text($input as node()*) as item()* {
   for $node in $input/node()
       return
           typeswitch($node)
               case text() return 
                   $node
                       default return local:get-text($node)
};

let $in-memory := 
<TEI xmlns="http://www.tei-c.org/ns/1.0">
<p>Government of new Territory of Nevada—Governor <name>Nye</name> <lb/>and the practical jokers—<name>Mr. Clemens</name> begins journalistic life <lb/>on <name>Virginia City</name> <name>Enterprise</name>.</p>
</TEI>

let $stored := doc('/db/test/stored.xml')

return
   <result>
       <inline>{string-join(local:get-text($in-memory))}</inline>
       <stored>{string-join(local:get-text($stored))}</stored>
   </result>

The result I get is:

<result>
    <inline>Government of new Territory of Nevada—Governor Nye and the practical jokers—Mr. Clemens begins journalistic life on Virginia City Enterprise. </inline>
    <stored>Government of new Territory of Nevada—Governor Nyeand the practical jokers—Mr. Clemens begins journalistic life on Virginia CityEnterprise.</stored>
</result>

where "Nye" and "and" and "City" and "Enterprise" are run together in the "stored" output, because the space between and and between and is stripped.

Setting preserve-whitespace-mixed-content="yes" in conf.xml works as a workaround, but is not ideal.

Betterform (at exist 2.2RC1) does multiply tbody if used with xf:repeat-nodeset

If a table is given with one tbody element, than the resulting HTML document does have multiple tbody elements:

<table>
  <thead>
    <tr>
      <th>my headline</th>
    </tr>
  </thead>
  <tbody xf:repeat-nodeset="instance('myinstance')/values/value">
    <tr>
      </td><xf:output ref="text()"/></td>
    </tr>
  </tbody>
</table>

will result in

<table>
  <thead>
    <tr>
      <th>my headline</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      </td>10</td>
    </tr>
  </tbody>
  <tbody>
    <tr>
      </td>20</td>
    </tr>
  </tbody>
  <tbody>
    <tr>
      </td>30</td>
    </tr>
  </tbody>
</table>

But there should allways be one <tbody> element as part of the HTML result.

copyCollection creates orphaned documents when target collection name is the same as source

To reproduce:

1.) Create a collection with documents in it.
2.) Copy that collection to a seperate location
3.) Copy it back over the original
4.) Take a backup and see all the original files are now in the lost_and_found

IHMO the correct solution to this is to throw an exception when the target collection exists to prevent data loss. The same should be applied to moveCollection, which will simply delete the target if it exists. Without an exception being thrown this creates a situation for data loss to occur very easily without warning.

Betterform (at exist 2.2RC1) is loosing tfoot elements

If a table is used with tbody and xf:repeat-nodeset="xyz" and tfoot, the later will be removed:

If the XForms is defining a table including the <tfoot> like this

<table>
<thead>
<tr>
<th>my headline</th>
</tr>
</thead>
<tbody xf:repeat-nodeset="instance('myinstance')/values/value">
<tr>
</td><xf:output ref="text()"/></td>
</tr>
</tbody>
<tfoot>
<tr>
<td>my footer buttons</td>
</tr>
</tfoot>
</table>

But on the resulting HTML the tfoot will allways be gone.

admin web application: indexes: contents of arbitrary files listed

I just added some files with names like a.a, coll.xc, ... in /db/system/config/db/test/. I was expecting that only the collection.xconf file has any influence on the configuration.

But it seems as if every file (indepedent of its name) is used for index configuration or at least displayed in the admin panel under "indexes". This results in the attached screenshot
screenshot from 2014-04-13 06 10 01

There are 6 different, contradictory entries for the collection /db/test.

eXist version 2.2RC1

Local mode: err:XPST0081 error found while loading module repair

The following xquery yields in a trace when running in localmode:

xquery version "3.0";
import module namespace repair="http://exist-db.org/xquery/repo/repair" 
at "resource:org/exist/xquery/modules/expathrepo/repair.xql";
repair:clean-all(),
repair:repair()
org.xmldb.api.base.XMLDBException: err:XPST0081 error found while loading module repair: Error while loading module resource:org/exist/xquery/modules/expathrepo/repair.xql: No namespace defined for prefix compression [at line 100, column 26]
    at org.exist.xmldb.LocalXPathQueryService.compile(LocalXPathQueryService.java:259)
    at org.exist.client.QueryDialog$QueryThread.run(QueryDialog.java:593)
Caused by: org.exist.xquery.XPathException: err:XPST0081 error found while loading module repair: Error while loading module resource:org/exist/xquery/modules/expathrepo/repair.xql: No namespace defined for prefix compression [at line 100, column 26]
    at org.exist.dom.QName.parse(QName.java:289)
    at org.exist.xquery.FunctionFactory.createFunction(FunctionFactory.java:48)
    at org.exist.xquery.parser.XQueryTreeParser.functionCall(XQueryTreeParser.java:9843)
    at org.exist.xquery.parser.XQueryTreeParser.primaryExpr(XQueryTreeParser.java:7071)
    at org.exist.xquery.parser.XQueryTreeParser.expr(XQueryTreeParser.java:3651)
    at org.exist.xquery.parser.XQueryTreeParser.expr(XQueryTreeParser.java:367)
    at org.exist.xquery.parser.XQueryTreeParser.expr(XQueryTreeParser.java:1668)
    at org.exist.xquery.parser.XQueryTreeParser.functionDecl(XQueryTreeParser.java:5688)
    at org.exist.xquery.parser.XQueryTreeParser.prolog(XQueryTreeParser.java:4851)
    at org.exist.xquery.parser.XQueryTreeParser.libraryModule(XQueryTreeParser.java:4108)
    at org.exist.xquery.parser.XQueryTreeParser.module(XQueryTreeParser.java:3943)
    at org.exist.xquery.parser.XQueryTreeParser.xpath(XQueryTreeParser.java:3731)
    at org.exist.xquery.XQueryContext.compileModule(XQueryContext.java:2905)
    at org.exist.xquery.XQueryContext.compileOrBorrowModule(XQueryContext.java:2834)
    at org.exist.xquery.XQueryContext.importModule(XQueryContext.java:2776)
    at org.exist.xquery.parser.XQueryTreeParser.importDecl(XQueryTreeParser.java:5804)
    at org.exist.xquery.parser.XQueryTreeParser.prolog(XQueryTreeParser.java:4844)
    at org.exist.xquery.parser.XQueryTreeParser.mainModule(XQueryTreeParser.java:4120)
    at org.exist.xquery.parser.XQueryTreeParser.module(XQueryTreeParser.java:4065)
    at org.exist.xquery.parser.XQueryTreeParser.xpath(XQueryTreeParser.java:3731)
    at org.exist.xquery.XQuery.compile(XQuery.java:160)
    at org.exist.xquery.XQuery.compile(XQuery.java:109)
    at org.exist.xquery.XQuery.compile(XQuery.java:93)
    at org.exist.xquery.XQuery.compile(XQuery.java:85)
    at org.exist.xmldb.LocalXPathQueryService.compileAndCheck(LocalXPathQueryService.java:272)
    at org.exist.xmldb.LocalXPathQueryService.compile(LocalXPathQueryService.java:257)
    ... 1 more
Caused by: org.exist.xquery.XPathException: err:XPST0081 error found while loading module repair: Error while loading module resource:org/exist/xquery/modules/expathrepo/repair.xql: No namespace defined for prefix compression [at line 100, column 26]
    at org.exist.dom.QName.parse(QName.java:289)
    at org.exist.xquery.FunctionFactory.createFunction(FunctionFactory.java:48)
    at org.exist.xquery.parser.XQueryTreeParser.functionCall(XQueryTreeParser.java:9843)
    at org.exist.xquery.parser.XQueryTreeParser.primaryExpr(XQueryTreeParser.java:7071)
    at org.exist.xquery.parser.XQueryTreeParser.expr(XQueryTreeParser.java:3651)
    at org.exist.xquery.parser.XQueryTreeParser.expr(XQueryTreeParser.java:367)
    at org.exist.xquery.parser.XQueryTreeParser.expr(XQueryTreeParser.java:1668)
    at org.exist.xquery.parser.XQueryTreeParser.functionDecl(XQueryTreeParser.java:5688)
    at org.exist.xquery.parser.XQueryTreeParser.prolog(XQueryTreeParser.java:4851)
    at org.exist.xquery.parser.XQueryTreeParser.libraryModule(XQueryTreeParser.java:4108)
    at org.exist.xquery.parser.XQueryTreeParser.module(XQueryTreeParser.java:3943)
    at org.exist.xquery.parser.XQueryTreeParser.xpath(XQueryTreeParser.java:3731)
    at org.exist.xquery.XQueryContext.compileModule(XQueryContext.java:2905)
    at org.exist.xquery.XQueryContext.compileOrBorrowModule(XQueryContext.java:2834)
    at org.exist.xquery.XQueryContext.importModule(XQueryContext.java:2776)
    at org.exist.xquery.parser.XQueryTreeParser.importDecl(XQueryTreeParser.java:5804)
    at org.exist.xquery.parser.XQueryTreeParser.prolog(XQueryTreeParser.java:4844)
    at org.exist.xquery.parser.XQueryTreeParser.mainModule(XQueryTreeParser.java:4120)
    at org.exist.xquery.parser.XQueryTreeParser.module(XQueryTreeParser.java:4065)
    at org.exist.xquery.parser.XQueryTreeParser.xpath(XQueryTreeParser.java:3731)
    at org.exist.xquery.XQuery.compile(XQuery.java:160)
    at org.exist.xquery.XQuery.compile(XQuery.java:109)
    at org.exist.xquery.XQuery.compile(XQuery.java:93)
    at org.exist.xquery.XQuery.compile(XQuery.java:85)
    at org.exist.xmldb.LocalXPathQueryService.compileAndCheck(LocalXPathQueryService.java:272)
    at org.exist.xmldb.LocalXPathQueryService.compile(LocalXPathQueryService.java:257)
    at org.exist.client.QueryDialog$QueryThread.run(QueryDialog.java:593)

``

Untranslated numerical entity &#160; in UTF-8 serialization Question + Patch

In the default context or encoding set to UTF-8, Every numérical entity is converted as a character represented by his binary value. By example: A became A (binary 65)

All except the non breaking space, follow this rule.
The non breaking space is always converted as  

This is happening at a so low level, it seems impossible to force it. Especially if you want to zip on the fly the result.

What is the reason of this exception ? Is it possible to define charset mapping ?

I identified the source of this behaviour in org.exist.util.serializer.XMLWriter

in the method:

    private void writeChars(final CharSequence s, final boolean inAttribute) throws IOException {
        final boolean[] specialChars = inAttribute ? attrSpecialChars : textSpecialChars;
        char ch = 0;
        final int len = s.length();
        int pos = 0, i;
        while(pos < len) {
            i = pos;
            while(i < len) {
                ch = s.charAt(i);
                if(ch < 128) {
                    if(specialChars[ch]) {
                        break;
                    } else {
                        i++;
                    }
                } else if(!charSet.inCharacterSet(ch) || ch == 160) {
                                break;
                } else {
                    i++;
                }
            }
            writeCharSeq(s, pos, i);
            // writer.write(s.subSequence(pos, i).toString());

            if (i >= len) {
                    return;
            }

            if(needsEscape(ch)) {
                switch(ch) {
                    case '<':
                        writer.write("&lt;");
                        break;
                    case '>':
                        writer.write("&gt;");
                        break;
                    case '&':
                        writer.write("&amp;");
                        break;
                    case '\r':
                        writer.write("&#xD;");
                        break;
                    case '\n':
                        writer.write("&#xA;");
                        break;
                    case '\t':
                        writer.write("&#x9;");
                        break;
                    case '"':
                        writer.write("&#34;");
                        break;
                    // non-breaking space:
                    case 160:
                        writer.write("&#160;");
                        break;
                    default:
                        writeCharacterReference(ch);
                }
            } else {
                writer.write(ch);
            }

            pos = ++i;
        }
    }

I replace it by

  private void writeChars(final CharSequence s, final boolean inAttribute) throws IOException {
        final boolean[] specialChars = inAttribute ? attrSpecialChars : textSpecialChars;
        char ch = 0;
        final int len = s.length();
        int pos = 0, i;
        while(pos < len) {
            i = pos;
            while(i < len) {
                ch = s.charAt(i);
                if(ch < 128) {
                    if(specialChars[ch]) {
                        break;
                    } else {
                        i++;
                    }
                } else if(!charSet.inCharacterSet(ch)) {
                                break;
                } else {
                    i++;
                }
            }
            writeCharSeq(s, pos, i);
            // writer.write(s.subSequence(pos, i).toString());

            if (i >= len) {
                    return;
            }

            if(needsEscape(ch)) {
                switch(ch) {
                    case '<':
                        writer.write("&lt;");
                        break;
                    case '>':
                        writer.write("&gt;");
                        break;
                    case '&':
                        writer.write("&amp;");
                        break;
                    case '\r':
                        writer.write("&#xD;");
                        break;
                    case '\n':
                        writer.write("&#xA;");
                        break;
                    case '\t':
                        writer.write("&#x9;");
                        break;
                    case '"':
                        writer.write("&#34;");
                        break;
                    default:
                        writeCharacterReference(ch);
                }
            } else {
                writer.write(ch);
            }

            pos = ++i;
        }
    }

I reexecuted the test suite and a full usage of exist-db demo site with success.

The only place where this was visible was inside the new exide in mode XML.
But I never used it before for XML so I can not compare.

I know the usage of the numerical entity is XML correct but this is blocking
some integration with other tools.

Best regards

Pierre

xqDoc annotations

In the master branch, commit 7ba4f55, some xqDoc annotation functionality seems to be broken:
In the code below, @see is not working as I expect. The following code produces the attached documentation, where @see is omitted.

(:~
 : This module …
 : 
 : @see http://www.example.org/
 : @author Christian Forster
 : @version 0.1
 :)

xquery function documentation 2013-10-17 13-25-47

Other issues occur in the functx documentation. The comment in its source code contains an email address and everything from the "@" part on does not appear in the generated documentation.
Also, within that module @author, @version, and @see annotations in function comments are ignored.

NPE in LuceneMatchListener.scanMatches

configuration was not OK -my fault-, but a NPE should not happen....

Caused by: java.lang.NullPointerException
    at org.exist.indexing.lucene.LuceneMatchListener.scanMatches(LuceneMatchListener.java:167)
    at org.exist.indexing.lucene.LuceneMatchListener.startElement(LuceneMatchListener.java:126)
    at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:203)
    at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:80)
    at org.exist.storage.serializers.Serializer.serializeToReceiver(Serializer.java:1070)
    at org.exist.storage.serializers.Serializer.toSAX(Serializer.java:878)
    at org.exist.dom.NodeProxy.toSAX(NodeProxy.java:791)

sm:passwd() returns "You must be a DBA to create a User Account."

[eXist 2.1]

Hi,

I tried to set it up so Joe Average user could change his own password instead of through me. Code:

if (not(empty($userName) or empty($userNewPass))) 
then (
    let $d := sm:passwd($userName,$userNewPass)
    return true()
 )
 else (false())

I can run this code as DBA, thereby being able to change any password, but when I run this code as Joe Average (using his username obviously) I get:

"You must be a DBA to create a User Account."

At the very least there's bug in this text as I'm not creating anything, but I'd much rather be able to achieve my original goal which is to lift the burden of password changes off my back. Am I doing something wrong or can someone confirm this is indeed a bug?

Thanks in advance,

Alexander Henket
Nictiz

Empty tag "source" should be self-closing in HTML5

When serializing output as HTML5, the <source> tag is output with a separate closing tag, instead of being a self-closing tag -- it's an empty tag according to the specification:

http://www.w3.org/TR/html51/embedded-content-0.html#the-source-element

I think this is because the HTML5 serializer is a subclass of the XHTML serializer, and this uses a list of element names (emptyTags) to determine which tags should be output as self-closing tags, and "source" isn't on the list:

https://github.com/eXist-db/exist/blob/develop/src/org/exist/util/serializer/XHTMLWriter.java

Outputting <source> with a separate closing tag results in validation errors from the HTML5 validator.

NPE in new lucene

While performing a 'search' in the documentation app an NPE appeared:

HTTP ERROR 500

Problem accessing /exist/apps/doc/search.html. Reason:

An unknown error occurred: null

Caused by:

javax.servlet.ServletException: An unknown error occurred: null
at org.exist.http.servlets.EXistServlet.doPost(EXistServlet.java:473)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)

(.......)

Caused by: java.lang.NullPointerException
at org.exist.indexing.lucene.LuceneIndexWorker$LuceneHitCollector.collect(LuceneIndexWorker.java:874)
at org.apache.lucene.search.Scorer.score(Scorer.java:65)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:624)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
at org.exist.indexing.lucene.LuceneIndexWorker.searchAndProcess(LuceneIndexWorker.java:499)
at org.exist.indexing.lucene.LuceneIndexWorker.query(LuceneIndexWorker.java:393)
at org.exist.xquery.modules.lucene.Query.preSelect(Query.java:198)

(....)

util:string-to-binary() leaks files

During intensive usage of util:string-to-binary() we found out that many files were written to jetty/tmp but these were never cleaned up.

As a result a system (linux, oracle java7) ran out of file handles over time.

example:

java 28644 29132 username 219u REG 202,1 1700 411310 /opt/exist-trunk/tools/jetty/tmp/_mmtfm_16f53166-157f-4f5d-82a1-5c8c2c42e6bf/mmtf_1382636042437789285362156175046.tmp (deleted)
java 28644 29132 username 222u REG 202,1 1700 411056 /opt/exist-trunk/tools/jetty/tmp/_mmtfm_16f53166-157f-4f5d-82a1-5c8c2c42e6bf/mmtf_1382636042437789285362156175046.tmp (deleted)
java 28644 29132 username 227u REG 202,1 1700 411056 /opt/exist-trunk/tools/jetty/tmp/_mmtfm_16f53166-157f-4f5d-82a1-5c8c2c42e6bf/mmtf_1382636042437789285362156175046.tmp (deleted)
java 28644 29132 username 229u REG 202,1 1700 411148 /opt/exist-trunk/tools/jetty/tmp/_mmtfm_16f53166-157f-4f5d-82a1-5c8c2c42e6bf/mmtf_1382636042437789285362156175046.tmp (deleted)
java 28644 29132 username 233u REG 202,1 2690 411152 /opt/exist-trunk/tools/jetty/tmp/_mmtfm_16f53166-157f-4f5d-82a1-5c8c2c42e6bf/mmtf_1382636042437789285362156175046.tmp (deleted)
java 28644 29132 username 234u REG 202,1 1700 411148 /opt/exist-trunk/tools/jetty/tmp/_mmtfm_16f53166-157f-4f5d-82a1-5c8c2c42e6bf/mmtf_1382636042437789285362156175046.tmp (deleted)
username@myserver:/opt/exist-trunk/tools/jetty/tmp$ lsof| grep mmtf | wc -l
2912

JVM Hang from RestXQ resource leak

As we've been ramping up for our first release using eXist we've discovered that the eXist server is completely unreliable failing after a couple days of continuous testing. The JVM would become unresponsive, we discovered that the process reached its file limit. No amount of forced garbage collections, or even shutting down the web app (we had eXist deployed in tomcat) would free the open file resources.

Fortunately, we've tracked the root cause down to a resource leak in RestXQ, specifically any PUT request processed.

Steps to reproduce:

  1. Launch a vanilla eXist server, install the demo package, specifically the "XForms and RestXQ" demo.
  2. Issue a PUT request (ie. an HTTP PUT to http://localhost:8080/exist/restxq/address, or run the demo app changing the address and saving the form)
  3. Observe from a shell that the open file count grows for each PUT request, i.e. on linux run:
sudo lsof -p `pgrep java` | grep 'COMMAND\|tmp'

If you have multiple java processes running replace "pgrep java" with the PID of your exist server.

The result of the listing of open files command above will show the eXist tmp directory filling with tmp files. Each PUT processed by RestXQ (even a function that immediately returns) will leave a dangling file in the tmp directory. The content's of the leaked file is the body of the PUT request. Shutting down the JVM removes the file from the filesystem, however while the JVM is running the file resource is never closed from the running process leading to the JVM eventually failing due to reaching its open file limit (i.e., after 4096 PUT requests on my system).

I have reproduced this on Linux and OSX running Oracle's JVM "1.7.0_51", I suspect that you will see different behaviour on MS Windows due to its different handling of files.

Spatial module doesn't build

Environment: ubuntu 13.04 64-bit, OpenJDK 7, latest exist from git

Steps to reproduce:

  1. Activate spatial module in $build-dir/extensions/indexes/local.build.properties
  2. ./build.sh download-additional-jars
  3. ./build.sh all

The end of the compile looks like this

compile-src:
     [echo] Compiling sources 'index-spatial'
    [mkdir] Created dir: /home/gtn/exist/extensions/indexes/spatial/build/classes
    [javac] Compiling 10 source files to /home/gtn/exist/extensions/indexes/spatial/build/classes
    [javac] warning: [options] bootstrap class path not set in conjunction with -source 1.6
    [javac] /home/gtn/exist/extensions/indexes/spatial/src/org/exist/indexing/spatial/GMLHSQLIndexWorker.java:61: error: GMLHSQLIndexWorker is not abstract and does not override abstract method getQueryRewriter(XQueryContext) in IndexWorker
    [javac] public class GMLHSQLIndexWorker extends AbstractGMLJDBCIndexWorker {
    [javac]        ^
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] 1 error
    [javac] 1 warning

BUILD FAILED
/home/gtn/exist/build/scripts/build-impl.xml:415: The following error occurred while executing this line:
/home/gtn/exist/build/scripts/extensions-build.xml:31: The following error occurred while executing this line:
/home/gtn/exist/build/scripts/extensions-common.xml:137: The following error occurred while executing this line:
/home/gtn/exist/build/scripts/extensions-common.xml:199: The following error occurred while executing this line:
/home/gtn/exist/build/scripts/extensions-common.xml:152: The following error occurred while executing this line:
/home/gtn/exist/build/scripts/extensions-common.xml:163: Compile failed; see the compiler error output for details.

Also, note that the fetch url in $build-dir/extensions/indexes/spatial/extension.xml pointing to geoapi-nogenerics-2.1.1.jar (line 19) gets a 404. The file is available at the osgeo.org location used elsewhere in the file.

wrapper does not start on MacOSX 10.9/Mavericks

The sysctl cannot be found on the path, as a result it falls back to 32bit:

STATUS | wrapper | 2013/10/24 20:27:14 | --> Wrapper Started as Daemon
STATUS | wrapper | 2013/10/24 20:27:14 | Java Service Wrapper Community Edition 32-bit 3.5.20
STATUS | wrapper | 2013/10/24 20:27:14 | Copyright (C) 1999-2013 Tanuki Software, Ltd. All Rights Reserved.
STATUS | wrapper | 2013/10/24 20:27:14 | http://wrapper.tanukisoftware.com
STATUS | wrapper | 2013/10/24 20:27:14 |
STATUS | wrapper | 2013/10/24 20:27:15 | Launching a JVM...
ERROR | wrapper | 2013/10/24 20:27:15 | JVM exited while loading the application.
INFO | jvm 1 | 2013/10/24 20:27:15 | Error: This Java instance does not support a 32-bit JVM.
INFO | jvm 1 | 2013/10/24 20:27:15 | Please install the desired version.
STATUS | wrapper | 2013/10/24 20:27:15 | Automatic JVM Restarts disabled. Shutting down.
STATUS | wrapper | 2013/10/24 20:27:15 | <-- Wrapper Stopped

solution: add /usr/sbin/ to the PATH, OSX only.....

Change to ExternalModuleImpl.isReady causes regression in lookup for external resources

Before this commit (2fa9b22) it was possible to reference a xsl stylesheet in an external .xqm like so:

transform:transform($doc,xs:anyURI('stylesheet.xsl'), $params)

Where xs:anyURI('stylesheet.xsl') was intended to be a relative reference to the stylesheet from the .xqm.

After this commit it seems that the calculated absolute path is now incorrect and an error like the following is thrown:

Malformed URL for stylesheet: /Users/someuser/dev/api/get/util/stylesheet/stylesheet.xsl

buggy bin/startup.bat

There is a small but serious bug in bin/startup.bat, which prevents starting exist-db with this script on WIndows systems:

Line 51:
set JAVA_ENDORSED_DIRS="%EXIST_HOME%"\lib\endorsed

Should read as:
set JAVA_ENDORSED_DIRS="%EXIST_HOME%\lib\endorsed"

If you use this script as it is, although jetty seems to start up and the logs look OK, jetty will not respond to any requests (like http://localhost:8080/exit, etc)

compression:zip is not supporting Xquery 3.0 serialize option

In exist 2.1 build 20130717 svn revision 18721

compression:zip is supporting
declare option exist:serialize "method=xml indent=no";

but not

declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization";
declare option output:method "xml";
declare option output:indent "yes";

The supported options should be documented in the function documentation.
Both modes of serialization pragma should be supported.

Best regards

Pierre Colot

Index usage breaks text() selector

I stumbled across the following bug while I was trying to setup indexing:

I have an easy Xquery:

xquery version "3.0";
declare default element namespace 'http://www.loc.gov/MARC21/slim';
<collection>{subsequence((
   doc("/db/od/books_export.xml")/collection/record[datafield/subfield/text() = 'algebraic']
),0,2)}
</collection>

This query returns one matching result.

If I add the following file as "/db/system/config/db/od/collection.xconf" (without reindexing the database!), the above Xquery returns an empty set of results.

<collection xmlns="http://exist-db.org/collection-config/1.0">
    <index xmlns:marc="http://www.loc.gov/MARC21/slim">
        <!-- Range indexes -->
        <range>
            <create qname="marc:subfield" type="xs:string"/>
        </range>
    </index>
</collection>

It seems that eXistDb tries to use the range index although it was not created so far. To me this is very confusing and this behaviour should be changed or at least documented.

Trace at startup: (NoClassDefFoundError): TarArchiveOutputStream

The tracelog explains it all

2014-03-06 14:57:33,968 [main] INFO  (Configuration.java [<init>]:183) - Reading configuration from file C:\Localdata\Development\GitHub\exist\conf.xml 
2014-03-06 14:57:34,060 [main] INFO  (Configuration.java [configureStartup]:1258) - Registered StartupTrigger: org.exist.extensions.exquery.restxq.impl.RestXqStartupTrigger 
2014-03-06 14:57:34,061 [main] INFO  (Configuration.java [configureStartup]:1258) - Registered StartupTrigger: org.exist.repo.AutoDeploymentTrigger 
2014-03-06 14:57:34,335 [main] ERROR (Configuration.java [lookupModuleClass]:462) - Module http://exist-db.org/xquery/compression could not be initialized due to a missing dependancy (NoClassDefFoundError): org/apache/commons/compress/archivers/tar/TarArchiveOutputStream 
java.lang.NoClassDefFoundError: org/apache/commons/compress/archivers/tar/TarArchiveOutputStream
    at org.exist.xquery.modules.compression.CompressionModule.<clinit>(CompressionModule.java:47)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:190)
    at org.exist.util.Configuration.lookupModuleClass(Configuration.java:446)
    at org.exist.util.Configuration.loadModuleClasses(Configuration.java:412)
    at org.exist.util.Configuration.configureXQuery(Configuration.java:348)
    at org.exist.util.Configuration.<init>(Configuration.java:253)
    at org.exist.xmldb.DatabaseImpl.configure(DatabaseImpl.java:107)
    at org.exist.xmldb.DatabaseImpl.getLocalCollection(DatabaseImpl.java:183)
    at org.exist.xmldb.DatabaseImpl.getCollection(DatabaseImpl.java:162)
    at org.exist.xmldb.DatabaseImpl.getCollection(DatabaseImpl.java:153)
    at org.xmldb.api.DatabaseManager.getCollection(Unknown Source)
    at org.exist.client.InteractiveClient.connect(InteractiveClient.java:329)
    at org.exist.client.InteractiveClient.connectToDatabase(InteractiveClient.java:2339)
    at org.exist.client.InteractiveClient.run(InteractiveClient.java:2427)
    at org.exist.client.InteractiveClient.main(InteractiveClient.java:291)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.exist.start.Main.invokeMain(Main.java:126)
    at org.exist.start.Main.run(Main.java:448)
    at org.exist.start.Main.main(Main.java:50)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.compress.archivers.tar.TarArchiveOutputStream
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    ... 23 more
2014-03-06 14:57:34,355 [main] INFO  (eXistURLStreamHandlerFactory.java [init]:53) - Succesfully registered eXistURLStreamHandlerFactory. 

Proxy firewall: trace at startup

Behind a proxy/firewall it seems during startup a call is made to a server. Need to verify why, it makes exist hang for some time.

XQ-3.1 import-module() with support for function-lookup()

I am trying to get a import-module() function added into XQuery 3.1 at the moment. Sometime ago I managed to convince them to add function-lookup, but they said that import-module would have to wait for the future... well now it is the future ;-)

Personally I suspect that there is a bug/deficiency with the implementation of util:import-module/fn:function-lookup.

I can’t exactly remember why, but I think function-lookup cannot see a
module imported dynamically via util:import-module. In the templating
module, I use the following approach, which has the advantage of
avoiding the non-standard util:import-module function: I have a main
module, which knows all modules to be used by the application and
imports them. In this main module, I create an inline function which
calls function-lookup. This inline lookup function is passed to the
templating module as a function parameter. Because of the closure, the
inline function will be in the context of the module in which it was
created and has access to any module imported there.

History trigger fails

Current (pull #128) develop branch:

General
Uptime: PT49M2.568S
eXist Version: 2.1
eXist Build: 20140131
eXist Home: C:\exist
SVN Revision:
Operating System: Windows 7 6.1 amd64
File encoding: UTF-8
Java
Vendor: Oracle Corporation
Version: 1.7.0_51
Implementation: Java HotSpot(TM) 64-Bit Server VM
Installation: C:\Java\jre7
Temp file path: C:\exist\tools\jetty/tmp
Memory Usage
Max. Memory: 932352K
Current Total: 357376K
Free: 153699K

Added a HistoryTrigger and received a Stack Overflow error:

2014-01-31 09:41:13,555 [eXistThread-61] DEBUG (RpcConnection.java [handleException]:124) -  
java.lang.StackOverflowError
    at sun.misc.Unsafe.putInt(Native Method)
    at java.nio.DirectByteBuffer.putInt(Unknown Source)
    at java.nio.DirectByteBuffer.putInt(Unknown Source)
    at org.exist.storage.btree.UpdateValueLoggable.write(UpdateValueLoggable.java:73)
    at org.exist.storage.journal.Journal.writeToLog(Journal.java:244)
    at org.exist.storage.btree.BTree.writeToLog(BTree.java:759)
    at org.exist.storage.btree.BTree.access$2(BTree.java:757)
    at org.exist.storage.btree.BTree$BTreeNode.addValue(BTree.java:1424)
    at org.exist.storage.btree.BTree$BTreeNode.addValue(BTree.java:1414)
    at org.exist.storage.btree.BTree$BTreeNode.access$0(BTree.java:1406)
    at org.exist.storage.btree.BTree.addValue(BTree.java:272)
    at org.exist.storage.index.BFile.update(BFile.java:986)
    at org.exist.storage.index.BFile.update(BFile.java:934)
    at org.exist.storage.index.BFile.put(BFile.java:692)
    at org.exist.storage.NativeBroker.saveCollection(NativeBroker.java:1646)
    at org.exist.collections.triggers.HistoryTrigger.makeCopy(HistoryTrigger.java:101)
    at org.exist.collections.triggers.HistoryTrigger.beforeCopyDocument(HistoryTrigger.java:143)
    at org.exist.collections.triggers.DocumentTriggersVisitor.beforeCopyDocument(DocumentTriggersVisitor.java:86)
    at org.exist.storage.NativeBroker.copyResource(NativeBroker.java:2509)
    at org.exist.collections.triggers.HistoryTrigger.makeCopy(HistoryTrigger.java:102)
    at org.exist.collections.triggers.HistoryTrigger.beforeCopyDocument(HistoryTrigger.java:143)
    at org.exist.collections.triggers.DocumentTriggersVisitor.beforeCopyDocument(DocumentTriggersVisitor.java:86)
    at org.exist.storage.NativeBroker.copyResource(NativeBroker.java:2509)
    at org.exist.collections.triggers.HistoryTrigger.makeCopy(HistoryTrigger.java:102)
    at org.exist.collections.triggers.HistoryTrigger.beforeCopyDocument(HistoryTrigger.java:143)
    at org.exist.collections.triggers.DocumentTriggersVisitor.beforeCopyDocument(DocumentTriggersVisitor.java:86)
    at org.exist.storage.NativeBroker.copyResource(NativeBroker.java:2509)
    at org.exist.collections.triggers.HistoryTrigger.makeCopy(HistoryTrigger.java:102)
    at org.exist.collections.triggers.HistoryTrigger.beforeCopyDocument(HistoryTrigger.java:143)
    at org.exist.collections.triggers.DocumentTriggersVisitor.beforeCopyDocument(DocumentTriggersVisitor.java:86)
    at org.exist.storage.NativeBroker.copyResource(NativeBroker.java:2509)
    at org.exist.collections.triggers.HistoryTrigger.makeCopy(HistoryTrigger.java:102)
    at org.exist.collections.triggers.HistoryTrigger.beforeCopyDocument(HistoryTrigger.java:143)
    at org.exist.collections.triggers.DocumentTriggersVisitor.beforeCopyDocument(DocumentTriggersVisitor.java:86)
    at org.exist.storage.NativeBroker.copyResource(NativeBroker.java:2509)
    at org.exist.collections.triggers.HistoryTrigger.makeCopy(HistoryTrigger.java:102)
    at org.exist.collections.triggers.HistoryTrigger.beforeCopyDocument(HistoryTrigger.java:143)
    at org.exist.collections.triggers.DocumentTriggersVisitor.beforeCopyDocument(DocumentTriggersVisitor.java:86)
    at org.exist.storage.NativeBroker.copyResource(NativeBroker.java:2509)
    at org.exist.collections.triggers.HistoryTrigger.makeCopy(HistoryTrigger.java:102)
    at org.exist.collections.triggers.HistoryTrigger.beforeCopyDocument(HistoryTrigger.java:143)
    at 
.
.
.
org.exist.collections.triggers.DocumentTriggersVisitor.beforeCopyDocument(DocumentTriggersVisitor.java:86)
    at org.exist.storage.NativeBroker.copyResource(NativeBroker.java:2509)
    at org.exist.collections.triggers.HistoryTrigger.makeCopy(HistoryTrigger.java:102)
    at org.exist.collections.triggers.HistoryTrigger.beforeCopyDocument(HistoryTrigger.java:143)
    at org.exist.collections.triggers.DocumentTriggersVisitor.beforeCopyDocument(DocumentTriggersVisitor.java:86)
    at org.exist.storage.NativeBroker.copyResource(NativeBroker.java:2509)
    at org.exist.collections.triggers.HistoryTrigger.makeCopy(HistoryTrigger.java:102)

Coding of path components in URIs

This is a port of SF ticket http://sourceforge.net/p/exist/bugs/766/ - for short:

In a few places the eXist codebase uses "java.net.URLDecoder" (which is intended to be used on http query strings) instead of "java.net.URI", resulting in wrong coding of path components in URIs and thereby creation of ressources, that cannot be fetched by the name, they were uploaded with.

Simplest way to show the (mis)behaviour:

  • From the dashboard start collections browser,
  • create collection "/db/testing",
  • upload a ressource, eg. "A+A.svg"
  • note that it gets represented as "A%2BA.svg" in the list widget,
  • which is the same as displayed in "/exist/rest/db/testing/",
  • and also in "/exist/webdav/db/testing/", while
  • in eXide open dialog it will show up as "A+A.svg".

BUT

  • fetching /exist/rest/db/testing/A%2BA.svg will result in a 404,
  • same as /exist/rest/db/testing/A+A.svg,
  • while fetching /exist/rest/db/testing/A%252BA.svg will work,
  • same as /exist/webdav/db/testing/A%252BA.svg,
  • all the while the file is inaccessible from eXide's "open", but
  • double clicking in coll.browser will fire up eXide to "A%2BA.svg".

That is: there is double encoding happening. The plus sign is encoded to %2B on store, and the newly introduced percent sign has to be coded to %25 on retrieve. This applies to all the characters, that java.net.URLDecoder codes different from java.net.URI in path components (most prominently the plus sign, but also others).

Missing dirs for dist-war target (sourceforge bug #826)

Running ./build.sh dist-war I get two errors:

  1. BUILD FAILED $home/exist/build/scripts/dist.xml:35: $MYBUILDDIR/extensions/indexes/spatial/lib does not exist.
  2. BUILD FAILED $home/exist/build/scripts/dist.xml:35: $MYBUILDDIR/extensions/betterform/main/xrx does not exist.
    The problem is that the $home/exist/extensions/indexes/spatial/lib/ and $home/exist/extensions/betterform/main/xrx/ directories do not exist. Manually creating them before running dist-war stops the errors from recurring.
    Platform: Ubuntu 13.04, OpenJDK 1.7.0_25
    GIT info: develop branch

Move to Java7

Let's move to Java7.......

time is ready, tests run well

matches(): non-compliant to the spec and buggy

eXist's implementation of matches(xs:string*, xs:string) differs w.r.t. the official matches(xs:string?, xs:string) function. As the cardinality for the first string parameter suggests, eXist accepts multiple strings as first argument for matches(), while this throws an error against the standard XPath function. This can be demonstrated by running following query with Saxon and eXist (tested with eXist-2.1):

  let $data :=
    <data>
      <entry n="1">
        <val>abc</val>
      </entry>
      <entry n="2">
        <val>xyz</val>
        <val>bcd</val>
      </entry>
      <entry n="3">
        <val>123</val>
        <val>haha</val>
      </entry>
      <entry n="4">
        <val>haha</val>
        <val>123</val>
      </entry>
  </data>
  return $data//entry[matches(val,'a')]

While Saxon throws an error ("XPTY0004: A sequence of more than one item is not allowed as the first argument of matches()"), eXist lets this pass. Yet, there seems to be a bug when the first argument of matches is a sequence of strings. Above query returns following results:

  <entry n="1">
    <val>abc</val>
  </entry>
  <entry n="4">
    <val>haha</val>
    <val>123</val>
  </entry>

While both entries 3 and 4 should be returned, 3 is omitted from the results. This illustrates that if the first argument to eXist's matches() function is a sequence, only the first item is evaluated.

This bug aside, the non-compliant behaviour is documented in the eXist function documentation at http://demo.exist-db.org/exist/apps/fundocs/view.html?uri=http://www.w3.org/2005/xpath-functions&location=java:org.exist.xquery.functions.fn.FnModule#matches.2. Still, I'm curious about what motivated this discrepancy from the official spec, and the decision to keep the eXist matches() function unchanged in the http://www.w3.org/2005/xpath-functions namespace? IMO, this makes it too easy to write non-compliant XQuery code.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.