wexsol / mongrel

eXist-db extension for MongoDB and GridFS

Mongrel: the eXist-db MongoDB extension

The Mongrel eXist-db extension provides xquery extension functions to access MongoDB database functions.

In addition to "regular" MongoDB operations the extension also provides GridFS functions to manage arbitrary sized documents that are stored in MongoDB.

Extensive documentation can be found on the Wiki. Downloads and release notes are on the GitHub Releases page.

Versions

Mongrel 0.6.1 requires eXist-db 3.0 (or newer).

The function namespaces are http://expath.org/ns/mongo and http://expath.org/ns/mongo/gridfs

This version supports XQuery 3.1 JSON structures. The extension is based on the 2.x version of the official mongodb driver

Requirements

• eXist-db 3.0+
• Java8

License

The work is released AS-IS under the LGPL license.

Support

Support on this extension is provided by weXsol, eXist Solutions GmbH, and Evolved Binary.

mongrel's Issues

Use XpathExceptions errVal to report json formatted errors

Making an EXPath module?

Hi Dannes, I've just discovered your MongoDB module for eXist-db. We've done a similar module recently, based on XQuery maps and arrays; maybe we can join forces and make an EXPath module out of it? Would you be interested?

Wrapup and make 1.0 release for eXist-db v3.x

Focus on last release only

Aggregation issue with Mongo 3.6, & 4.0+ (enterprise builds) : how do we process cursor results?

Getting the following successful response when performing an aggregation summary query.

{  
   "ok" : 0.0 , 
   "errmsg" : "The 'cursor' option is required, except for aggregate with the explain argument" , 
   "code" : 9 , 
   "codeName" : "FailedToParse"
}

<?xml version="1.0" encoding="UTF-8"?>
<response>
    <request type="json" contractor="11050">
        <queryPath>/db/apps/invoice/query/getContractorStatistics2.js</queryPath>
        <querySource>
            .... blah blah blah....
        </querySource>
    </request>
    <payload>
        <record item="1">
            <record>
                { "ok" : 0.0 , "errmsg" : "The 'cursor' option is required, except for aggregate with the explain argument" , "code" : 9 , "codeName" : "FailedToParse"}
            </record>
        </record>
    </payload>
</response>   

Is there anyway I can process cursor record results using an XQuery like the following:
My example is using existDB. Here is a short simple XQuery function:

declare function invoiceLib:getContractorStatistics2( $mongodbClientId as xs:string, $contractor as xs:string ) as node()
{ 	
    let $options := map { "liberal": true(), "duplicates": "use-last" }

    let $employeeId := if ( $contractor castable as xs:integer )
                      then xs:integer( $contractor )
                      else 0
    
    let $queryFilePath := '/db/apps/invoice/query/getContractorStatistics2.js' 
    let $querySourceText := if ( util:binary-doc-available( $queryFilePath ) )  
                            then util:binary-to-string( util:binary-doc ( $queryFilePath ) ) 
                            else 'not found'

    let $cmdresult := mongodb:command($mongodbClientId, $invoiceLib:DATABASE, $querySourceText)

    return element response {
        element request { 
            attribute type { "json" }, 
            attribute contractor { $employeeId }, 
            element queryPath { $queryFilePath }, 
            element querySource { $querySourceText } 
        }, 
        element payload { 
            for $record at $position in $cmdresult
            let $jsonRecord := parse-json($record, $options)
            return element record { 
                attribute item { $position }, 
                element record { $record }
                (: doing some guessing below:  
                element minInvoice { $jsonRecord?1?("minInvoice") }, 
                element maxInvoice { $jsonRecord?1?("maxInvoice") }
                element minDate { $jsonRecord?minDate }, 
                element maxDate { $jsonRecord?maxDate }, 
                :)
            }
        }
    }
};

I'd be happy to discuss further.
Thank you for your help in advance...
I believe there was a big change between Mongo 3.4 and 3.6.

Cheers,
Allan Cuthbertson.

Completely revamp to java8

Code is mostly on Java7 now, can be improved a lot.

Implement more driver functions

not deprecated methods from

• https://api.mongodb.org/java/2.13/com/mongodb/DB.html
• https://api.mongodb.org/java/2.13/com/mongodb/gridfs/GridFS.html

Implement map-reduce functionality

The README is referring to the MongoDB driver version 2.0 and you are at 3.x

Make sure the broker is returned correctly

During this Mongrel logging, we are also seeing a /WARN (BrokerPool.java
[release]:1278) - Broker org.exist.storage.NativeBroker_exist_5 was returned
with extraneous Subjects, cleaning...
java.lang.IllegalStateException: DBBroker pushSubject/popSubject mismatch
.../ Do we have a problem?

Mongrel/java/src/org/exist/mongodb/shared/ContentSerializer.java

Line 97 in bb9e621

final Serializer serializer = context.getBroker().newSerializer();

Migrate API to support XQuery 3.1

keywords: Maps, Arrays

Re-implement document compression selection

At this moment documents are stored compressed in GridFS when the file extension is not in a list. Better to have a white list of extensions , and let's add mime/type check too.

Status of other database commands?

Hi Dannes,

I got my first connection to MongoDB from eXist up and running! Thanks so much for this tool!

The tutorial I followed for MongoDB was The Little MongoDB Book (initially found via the iBooks store, but a newer edition covering MongoDB 2.6 is available as PDF and epub), which explained how to use the "update" command, e.g.:

db.unicorns.update({name: 'Roooooodles'}, {$set: {weight: 590}})

db.hits.update({page: 'unicorns'}, {$inc: {hits: 1}}, true);

I don't see update in the list of available XQuery functions. This raises the question - what is the status of the implementation of other of MongoDB's database commands, as listed at http://docs.mongodb.org/manual/reference/command/?

Also, the wiki page on MongoDB examples mentions mongodb:query(), but I believe it should be mongodb:find(). Didn't want to make this change without checking with you first.

Bravo!
Joe

Add unit tests gridfs

find() : ArrayOutofBound

2015-01-03 17:46:53,308 [eXistThread-30] ERROR (Find.java [eval]:132) - 4 
java.lang.ArrayIndexOutOfBoundsException: 4
        at org.exist.mongodb.xquery.mongodb.collection.Find.eval(Find.java:88)
        at org.exist.xquery.BasicFunction.eval(BasicFunction.java:70)