lbdremy / solr-node-client

A solr client for node.js.

solr-client - a node.js solr client

Installation

Install the library with:

npm install solr-client

or

yarn add solr-client

Documentation

Node.js version 12+ is supported. Solr versions 3-8 are supported.

• See the official documentation for more details.
• If you are upgrading from an earlier version, please see the migration guide.
• You can also check out the changelog.

Usage (callback API in 0.9.0 and older)

// Load dependency
const solr = require('solr-client');

// Create a client
const client = solr.createClient();

// Add a new document
const obj = await client.add({ id : 12, title_t : 'Hello' });
console.log('Solr response:', obj);

Usage (promise API in 0.10.0+`)

// Load dependency
const solr = require('solr-client');

// Create a client
const client = solr.createClient();

// Add a new document
const obj = await client.add({ id : 12, title_t : 'Hello' });
console.log('Solr response:', obj);

Roadmap

v0.3.x - v0.x.x

• Implement all features available in Solr 4 (SolrCloud API in particular)
• Provide all low-level commands
• Complete documentation

v1.0.x

• First stable version
• the API is frozen until v2.0.x, only new features and bug fixes can be introduced

Test

Tests are executed against a Solr instance in a Docker container.

To execute them on latest supported Solr version, run:

npm run solr:current:start
npm run test:current

If you want to execute them on oldest Solr version supported, run:

npm run solr:legacy:start
npm run test:legacy

Windows 10 and Docker

Windows 10 Pro or Enterprise

If you have a version of Windows 10 Pro or Enterprise, simply download Docker Desktop. Ensure that Hyper-V is enabled, and virtualization is enabled in your motherboard's BIOS. It may prompt you to install updates to Windows during Docker Desktop installation and then restart the system.

Windows 10 Home

If you have a version of Windows 10 Home, a little more effort is needed. By default, Hyper-V cannot be turned on for Home edition. The steps that seem to work:

• Check to ensure you have WSL1, WSL2, and the Kernel Update. This can be done using the shell commands found [here](https://blog.devgenius.io installing-docker-onwindows-10-home-edition-2e7c1b79d76d).
• Follow the instructions in this link to install Hyper-V. Please note, that you must have WSL1/WSL2 and the update for Hyper-V to install properly or the changes may be undone during restart.
• After restarting and checking that Hyper-V is enabled, you may install Docker Desktop using the setup wizard.

Test coverage

Before to be able to run the command below, you will need to install jscoverage available here.

npm run-script test-cov

This command will generate a file named coverage.html, use your browser to visualize it.

Static analysis and complexity report

npm run-script report

Licence

(The MIT License)

Copyright 2011-2012 HipSnip Limited Copyright 2013-2014 Rémy Loubradou

solr-node-client's Issues

Support the replication HTTP API

Docs: http://wiki.apache.org/solr/SolrReplication

Should not escape Lucene special characters

I do not think we should escape those special characters, instead, we let the user decide whether to escape them.

e.g.:
query.q({"id": "_"})
What I want is to get all documents which have the 'id' field; but if escaped, it becomes {"id": "_"}, which means to get documents which have an 'id' field and its value is '*'.

Lucene document says these characters are special, but not unsafe. solr-client should pass these characters to Lucene/Solr.

Move the definition of the paths of all request handlers (update,select,admin...) in the `Client`

Create solr client with multiple endpoints

We're going to need to be able to create a client communicating with multiple endpoints, the requests should be load balanced between the endpoints.
The module poolee is a potential candidate to solve this.

addRemoteResource working?

There is no test for the addRemoteResource method of Client and I haven't been able to get it to work. I get
{ name: 'SolrError', message: 'HTTP status 400.Reason: Unknown command: id [8]' }.

Does this really work as-is?

Establish persistance connection with keep-alive

As it is pointed out in the RFC of HTTP (http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html), a persistance connection has several advantages when a client makes multiple requests on the same server in a short amount of time, which would be good to benefit since our case.

Rewrite all tests with mocha and chai.

Add config option for custom search handlers.

It should be possible to add an custom search handler in client config.

function createClient(host, port, core, path, handler){
  var options = (typeof host === 'object')? host : {
      host : host,
      port : port,
      core : core,
      path : path,
      handler : handler
   };
  return new Client(options);
}

[api] Will it pay to provide a more flexible query api.

Most of Query api accepts an options object. I was just wondering won't it be sugary to provide apis which accepts key-value pairs, boolean string attributes as arguments in any order?

For example:

// Below all are equivalent
Query#facet('on', 'missing', {field: 'field'});

Query#facet('on', {'missing': true}, {field: 'field'});

Query#facet({field: 'field'}, 'on', {'missing': true});

Query#facet({'field': 'field',
             'missing': true,
             'on': true
           });
//Later on merge all the arguments into a single options object

When using coffeescript this syntax would look really neat and descriptive inside apps.

Con: Will running code to merge all the arguments after every call add to overhead and impact performance?

SORL_CLIENT: Nothing to purge. Add update clear.

This event cannot be consider as an error in the worst case a info/warning.

Add a method to ask Solr for Remote Streaming

Remote streaming allows you to send the contents of a URL as a stream to a given SolrRequestHandler
http://wiki.apache.org/solr/ContentStream

Add method `createDisMaxQuery()` to `Client`

Implement ping command

Implement the ping command.
The request is http://host:port/solr/admin/ping?&wt=json

Response:

{
    "responseHeader": {
        "status": 0,
        "QTime": 5,
       "params": {
           "echoParams": "all",
           "rows": "10",
           "q": "solrpingquery",
           "qt": "search",
           "wt": "json"
        }
   },
   "status": "OK"
}

Implement More Like this

mlt() returning nothing

Hello again. For some reason, I can't get the MoreLikeThis feature to return any results using your client. I am storing term vectors in Solr, and am using a small data set (10 documents) just to debug this for now. My query goes as follows:

var recommend = function ( entry, numHits, callback ) {

  var query = client.createQuery().q('_id:' + entry._id).fl('_id,content').start(0).rows(1).sort({score:'desc'});

  client.search( query, function (err, res) {

    console.log(res.response.docs[0].content);
    var moreLikeThisQuery = client.createQuery().q(res.response.docs[0].content).mlt({ 
      fl: 'content',
      count: numHits,
    }).fl('_id');

    client.search( moreLikeThisQuery, function (err, res) {
      if (err) throw err;

      if (res) {
        callback(null, res.response.docs);
      }
    });
  });
};

You'll notice that I first query the indexed documents in order to get content to use for the MoreLikeThis query. Does something seem wrong here? I've been debugging for some time, to no avail. In my example data set I have a few documents that should have a high similarity score, eg.

{
    "_id": "1000",
    "publisher_id": "1000",
    "source_id": "1000",
    "item_id": "1000",
    "guid_hash": "abcd1000",
    "content": "Hello all. Let's talk business now. I like to make things happen, but this is just a bunch of crap. Derp a lerp a derp a herp a lerp.",
    "title": "First Post"
}

and

{
    "_id": "1002",
    "publisher_id": "1002",
    "source_id": "1002",
    "item_id": "1002",
    "guid_hash": "abcd1002",
    "content": "Hello all. Let's talk about business mmmkay? I like to make things occur, but this is just a bunch of crap shterf. Derp a lerp a derp a herp.",
    "title": "Third Post"
}

Any ideas?

Thanks for your time man!

Client#search should accept an Object as first argument and stringify it as a qs

Update NPM to use newest version

I would like to take advantage of your query.js script you recently added. Can you update npm to use the newest version of your project so i can install it easily from npm? Last time you updated npm was over 100 days ago. Thanks.

Sort option not being parsed correctly

We are using this query:

 var query = solr.createQuery()
 .q({keyword: keyword})
 .rangeFilter({field: 'timestamp', start: timestampArr[keyword], end: '*'})
 .sort(['created_at desc'])
 .start(0)
 .rows(2);

It works if we don't sort. But with sort, we get this output:

Can\'t determine Sort Order: \'0 created_at desc\', pos=1

So seems to me that it's prepending extra characters to the sort option ('0 ')

Support Core

Actually the core is in parameter when creating a new Client but it's not considered when building the full url to access the Solr API. Fix this!

Error handling

The callback that return the response and the error should respect the node convention, so

In node.js, it is considered standard practice to handle errors in asynchronous functions by returning them as the first argument to the current function's callback. If there is an error, the first parameter is passed an Error object with all the details. Otherwise, the first parameter is null.

Implement shorthand command `softCommit`

Implement shorthand command `deleteAll`

How to delete all documents?

I tried

client.deleteByQuery( "*:*", function( err, response ) {
    //....
});

but I always get

SolrError: HTTP status 404.Reason: Not Found

Also

 client.delete( "*", "*", function( err, response ) {

and

 client.delete( "id", "*", function( err, response ) {

lead to the same result!

What am I doing wrong?

Write better documentation

For :

• Querying
• Deleting
• Optimizing
• Adding

Add authentication support

It would be nice to support authentication. Need to take a look to the Solr documentation.
http://wiki.apache.org/solr/SolrSecurity
Basic Access Auth and Digest Access Auth?

Support for Solr 4

E.g. the path for update is now update/ and not update/json.

Question: How did you made the travis tests pass?

Can we put solr infrastructure on travis? I need this for elasticSearchClient.

solr-node-client vs. node-solr

I am new to solr with node.js and wondering which is the better lib to connect both: yours or node-solr (https://github.com/gsf/node-solr). I do not expect an objective answert but maybe you did a comparison to the other project by yourself that could help me find out which one I should choose depending on my scope.

Thanks for your help!

P.S. I have asked this question the other author too.

searching for multiple words in a field

How do you specify a query for documents with a field named "foo" that contains both the words "bar" and "baz"?
I tried this, but it must not be correct.

var query = client.createQuery().
  q({foo: 'bar AND baz'});

Support multiple facet fields

Solr allows multiple facet.field, remove limiting behavior in code which disallows this in the client

Implement shorthand command `searchAll`

Buffering documents in the `add` method and remove them after is wrong

I implemented this a while ago, and it appears it's not a good idea.
If buffering is necessary it should be done outside of this method.

Case where it causes trouble:

var solr = require('solr-client');
var client1 = solr.createClient({host : 'localhost',port : 8983});
var client2 = solr.createClient({host: '12.67.78.98',port : 8983});
var docs = [ { id : 3938, title: 'blalbkjj'},{ id :78783, title : 'broubrou'}];
client1.add(docs,function(err,obj){});
client2.add(docs,function(err,obj){}); // Well, I'm sorry for you client2 but your array of docs is empty

The quick fix, without breaking the buffering feature in the add method would be to clean the array of docs only if this.updateEach !=1 and so add a condition where the splice is done
if(this.updateEach != 1){ this.adds.splice(0,this.adds.length); // Remove every elements}

I will remove the buffering feature and so this.adds, and the method flushAdd in the next minor release but for now I will just add this condition above.

Undo overidding of `querystring.escape`

Support multiple group fields

Solr allows multiple group.field, remove limiting behavior in code which disallows this in the client

Added a whole bunch of features...

Hi
Would you like me to put together a pull request?
I have added the following:
sharding and multicore searching.
collection and cloud support.
add some DSE support for documents (sendToSolr api)
I am not sure if those features fit into your requirements.
Andy

Implement The TermsComponent

Look at :
http://wiki.apache.org/solr/TermsComponent

Publish schema.xml for Solr4

Hi @lbdremy, please publish your schema.xml file for Solr4 in the test directory. It would be easier for other people working on the 0.3.x branch and contribute.

Remove mock, use `travis-solr` instead when possible

There is a project called travis-solr available there https://github.com/moliware/travis-solr, making the installation of solr a breeze in the travis env.
Instead of mocking all Solr responses, have a proper Solr server is preferable.

vows 0.6.x from npm does not support wildcard

Hi All,
We use vows to test and as I research the vows project support the wildcard from this commit vowsjs/vows@70865cc
and the npm don't include this commit. And as I run the npm test the error will show below:

$ npm test

> solr-client@0.2.6 test d:\openSource\Aptana Studio 3 Workspace\solr-node-client
> vows --spec test/test-*


module.js:340
    throw err;
          ^
Error: Cannot find module 'd:\openSource\Aptana Studio 3 Workspace\solr-node-client\test\test-*'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at d:\openSource\Aptana Studio 3 Workspace\solr-node-client\node_modules\vows\bin\vows:520:19
    at Array.reduce (native)
    at importSuites (d:\openSource\Aptana Studio 3 Workspace\solr-node-client\node_modules\vows\bin\vows:515:18)
    at Object.<anonymous> (d:\openSource\Aptana Studio 3 Workspace\solr-node-client\node_modules\vows\bin\vows:261:15)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
npm ERR! Test failed.  See above for more details.
npm ERR! not ok code 0

Version number in trunk should be other then latest release

I suggest to change the version number of the project into something like:

(currentVersion + 1) + "-dev|snapshot"

would make for current trunk

"0.2.7-dev|snapshot"

I coming from java so I am used to the "snapshot" but dev I like more ;)

Add Support for Range Facets

Adding support for Range Facet handling (Solr 3.1+).

See "http://wiki.apache.org/solr/SimpleFacetParameters?highlight=%28%28SimpleFacetParameters%29%29#Facet_by_Range" for detail parameters.

How to specify collection

How does this client determines to which collection it needs to store or retrieve data from? As far as I can see from the source code, it needs to be specified into options object. Is there any other way? For example specifying a global collection name in the client object itself?

Implement shorthand command `prepareCommit`

Return an object and not the JSON response?

Remove sax module and parse errors differently

The sax module is absolutely not necessary, so parse errors differently like this the module will be without dependencies, expect vows.

Escape special characters in values

More informations about what need to be escaped there -> http://lucene.apache.org/core/3_6_1/queryparsersyntax.html#Escaping%20Special%20Characters

Allow to create custom command e.g : suggest ...

Rename purgeAdd method

It will be nice to renamed purgeAdd method to flashAdd that's more correct.

Grouping / Field Collapsing

Add a warning in README, because Grouping / Field Collapsing would work if the Apache Solr installed has a version >=3.3, according to the announcement of Apache Solr 3.3 available at http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_3/solr/CHANGES.txt

Error while using rangeFilter

Hi,
When trying to apply a range filter, I'm getting some Java errors.
Here is the query:
solr.createQuery()
.q(solr_query_params)
.rangeFilter({ field : 'created_at', start : 1343979397, stop : 1343979398})
.start(0)
.rows(20);

Error I get:
{ name: 'SolrError',
message: 'HTTP status 500.Reason: For input string: "undefined"

java.lang.NumberFormatException: For input string: "undefined"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:492)
at java.lang.Integer.parseInt(Integer.java:527)
at org.apache.solr.schema.TrieField.getRangeQuery(TrieField.java:234)
at org.apache.solr.search.SolrQueryParser.getRangeQuery(SolrQueryParser.java:213)
at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1489)
at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:1317)
at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1245)
at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1234)
at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:206)
at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:79)
at org.apache.solr.search.QParser.getQuery(QParser.java:143)
at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:123)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:165)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)' }