manticoresoftware / manticoresearch-typescript

Official TypeScript client for Manticore Search

Manticore TypeScript client

Сlient for Manticore Search.

❗ WARNING: this is a development version of the client. The latest release's readme is https://github.com/manticoresoftware/manticoresearch-typescript/tree/4.1.0

Requirements

Minimum Manticore Search version is 4.2.1 with HTTP protocol enabled.

Manticore Search	manticoresearch-typescript	Node
dev	manticoresearch-ts-dev	>= 18.0.0
>= 6.2.0	4.1.x	>= 18.0.0
>= 6.2.0	4.0.x	>= 18.0.0
>= 6.2.0	3.3.1	>= 18.0.0
>= 4.2.1	1.0.x	>= 18.0.0

Installation

npm install manticoresearch-ts-dev 

Getting Started

Please follow the installation instruction and execute the following typescript code:

import {
  Configuration,
  IndexApi,
  SearchApi,
  ResponseError,
} from "manticoresearch-ts";
(async () => {
  try {
    /*
    const config = new Configuration({
      basePath: 'http://localhost:9308',
      // fetchApi: <your own fetch API> // use node-fetch with node version < 18
    })
    const indexApi = new IndexApi(config);
    */
    const indexApi = new IndexApi();
    const docs = [
      { insert: { index: "test", id: 1, doc: { title: "Title 1" } } },
      { insert: { index: "test", id: 2, doc: { title: "Title 2" } } },
    ];
    const insertResponse = await indexApi.bulk(
      docs.map((doc) => JSON.stringify(doc)).join("\n")
    );
    console.info("Insert response:", JSON.stringify(insertResponse, null, 2));

    const searchApi = new SearchApi();
    const searchResponse = await searchApi.search({
      index: "test",
      query: { query_string: "Title 1" },
    });
    console.info("Search response:", JSON.stringify(searchResponse, null, 2));
  } catch (error) {
    const errorResponse =
      error instanceof ResponseError ? await error.response.json() : error;
    console.error("Error response:", JSON.stringify(errorResponse, null, 2));
  }
})();

Documentation

Full documentation on the API Endpoints and Models used is available in docs folder as listed below.

Manticore Search server documentation: https://manual.manticoresearch.com.

Documentation for API Endpoints

All URIs are relative to http://127.0.0.1:9308

Class	Method	HTTP request	Description

| Manticoresearch.IndexApi | bulk | POST /bulk | Bulk index operations |

| Manticoresearch.IndexApi | delete | POST /delete | Delete a document in an index |

| Manticoresearch.IndexApi | insert | POST /insert | Create a new document in an index |

| Manticoresearch.IndexApi | replace | POST /replace | Replace new document in an index |

| Manticoresearch.IndexApi | update | POST /update | Update a document in an index |

| Manticoresearch.IndexApi | update_0 | POST /{index}/_update/{id} | Partially replaces a document in an index |

| Manticoresearch.SearchApi | percolate | POST /pq/{index}/search | Perform reverse search on a percolate index |

| Manticoresearch.SearchApi | search | POST /search | Performs a search on an index |

| Manticoresearch.UtilsApi | sql | POST /sql | Perform SQL requests |

Documentation for Authorization

All endpoints do not require authorization.

manticoresearch-typescript's Issues

BigInt Support is not working

Im using manticoresearch-ts in order to implement a Helper that communicates with the manticore database. In my specific use case, i need to replace some documents based on the manticore id column. But when calling the IndexApi.replace function with the BigInt, a JSON Type error gets thrown, indicating that "BigInts cannot be serialized".

I know that js JSON.stringify does not have native support for bigint types (as BigInt.toJSON is not implemented) but this is something you'd imagine the official Manticore API library for javascript should handle internally.

By now Im trying to implement a solution that involves using the json-bigint javascript dependency, but im worried that i should quit using the official manticoresearch-ts library as it uses JSON.parse and JSON.stringify internally when calling the API methods (such as IndexApi.replace). My thoughts are that just by using json-bigint parse and stringify methods inside the library, the issue will be solved.

In other words, there's a bug while sending or recieving bigint values with the API.

• ¿Am I missing something?
• ¿Is this a known bug within the ts library?
• ¿Is there any workaround to work with Manticore IDs JSON and javascript?

IndexApi.replace() seems to have wrong signature

1.) Wrong type definition

The autocomplete from my IDE (PHPStorm) suggests the following:

However, the data returned looks like this:

{
  _index: 'db',
  _id: 8865561979561969000,
  created: false,
  result: 'updated',
  status: 200
}

The attribute found from the type definition is not present in the data and the attribute status from the data is not present in the type definition.

2.) Missing type definition
When the query does not succeed the returned data is something like this:

{
  error: {
    type: "table 'main' absent, or does not support INSERT",
    index: 'main'
  },
  status: 409
}

The return type of the type definition does not mention this. AFAICT the method need to return Promise<SuccessResponse | ErrorResponse> and not only Promise<SuccessResponse>.

This may occur on more than just this method. At least IndexApi.insert() seems to have the same problem.

SearchApi source missing underscore

Versions:
manticoresearch-ts: v4.1.0
manticoresearch: v6.2.12

I think the SearchApi is sending the wrong payload to the /search endpoint for the source field. When I prepare a searchRequest like shown below:

const results: SearchResponse =  await searchApi.search({
            index: "my_index",
            query: { query_string: "a query" },
            source: ["id", "title", "abstract"],
            highlight: { 
           
            },
 
        });

The payload that is sent (verified with wireshark) is :

{"index":"my_index","query":{"query_string":"a query"},"source":["id", "title", "abstract"],"highlight":{"}}

But the backend doesn't restrict the response objects to only the source fields because the key is missing the underscore _source which causes larger than expected payloads to be returned in my case. I verified this with curl also.

Changing the request to include the underscore works just fine though.

const results: SearchResponse =  await searchApi.search({
            index: "my_index",
            query: { query_string: "a query" },
            _source: ["id", "title", "abstract"],
            highlight: { 
           
            },
 
        });

The only impact now is that typescript linters get upset and throw errors like this:

Object literal may only specify known properties, but '_source' does not exist in type 'SearchRequest'. Did you mean to write 'source'?ts(2561)

I just started using this library so maybe I'm just using the API incorrectly, but I didn't see this mentioned in the docs.

SearchApi does not handle manticore error responses

Maybe this behavior is intended but I wanted to ask to be sure. Looking at the simple search example below from the README I made the assumption (perhaps incorrectly) that errors returned from Manticore would be handled by the catch block, but that only appears to be the case when there is something like a network error that gets thrown. However, there is no error thrown when Manticore returns an error object like in the case of a syntax error.

try {
   
<snip>

    const searchApi = new SearchApi();
    const searchResponse = await searchApi.search({
      index: "test",
      query: { query_string: "Title 1" },
    });
    console.info("Search response:", JSON.stringify(searchResponse, null, 2));

  } catch (error) {
    const errorResponse =
      error instanceof ResponseError ? await error.response.json() : error;
    console.error("Error response:", JSON.stringify(errorResponse, null, 2));
  }

Example error response during a syntax error:

[
    {
      total: 0,
      warning: '',
      error: "index test_table: query error: P08: syntax error, unexpected '~' near '~'"
    }
]

If this is intended behavior then it looks like the SearchResponse interface does not account for Manticore errors. As a result I ended up extending the SearchResponse interface, like shown below, in my app.

export interface ErrorMessage {

    total: number
    warning: string
    error: string
}

export interface SearchResponseWithError extends SearchResponse {

    message?: ErrorMessage | null

} 

Then I just handled the error messages specifically in my results handling code. If this in intended then maybe just a small update to the example code would clarify usage better. Another option is to have SearchApi throw errors.

Fix `aggs` and `expressions` in `SearchRequest`

Edit: After further testing, it turns out, that some types (i tried to use) are incorrect - here are failing test cases:

describe('Search Api Tests', () => {
  it('should aggregate by field', async function () {
    try {
      const query: Manticoresearch.SearchRequest = {
        index: 'test',
        query: {
          match_all: {},
        },
        aggs: [
          {
            name: 'cat',
            field: 'cat',
          },
        ],
      };

      const result = await searchApi.search(query);
      expect(result).to.deep.nested.property('hits.total', 5);
    } catch (err) {
      const errorResponse = err instanceof Manticoresearch.ResponseError ? await err.response.json() : err;
      console.error('Error response:', JSON.stringify(errorResponse, null, 2));
      expect(err).to.be.null;
    }
  });
});

which results in this error on manticoresearch 6.2.12:

{
      "total": 0,
      "warning": "",
      "error": "\"aggs\" property should be an object\""
}

According to the documentation, the schema should look like this instead:

"aggs": {
    "group name" : {
        "terms" : {
            "field":"attribute name",
            "size": 1000
      },
      "sort": [ {"attribute name": { "order":"asc" }} ]
  }
}

cc @sanikolaev - is there a documentation for the complete json-api schema?

While creating these test cases i've also stumbled upon some issues (or at least "you have to know"s) when migrating from the javascript library:

utilsApi.sql('drop table if exists test');

works in the javascript library, but throws this error in typescript:

{
  "error": "only SELECT queries are supported"
}

because the rawResponse option defaults to true in javascript, but false in typescript.

I'd be happy to contribute to this library (e.g. creating a PR with the updated types for aggs and expressions and updating the test cases) but there seems to be a lot of things not being right yet. The PR #2 of @mannol tidies up some things though.

Let me know what are your thoughts on this, if you accept PRs and how to contribute.

I want to order a facet result by the document counts to get my "most used buckets" like its described here: https://manual.manticoresearch.com/Searching/Faceted_search#Ordering-in-facet-result

Currently the Aggregation-Type only has name, field and size.

/cli is missing in SDK

There are no implementations for https://manual.manticoresearch.com/Connecting_to_the_server/HTTP#/cli and json version

Would be a great enhancement for UtilsApi

Unable to import manticoresearch

Unable to import manticoresearch into a node project due to Cannot find module 'manticoresearch-ts' or its corresponding type declarations.ts(2307)

node version: v20.9.0
typescript version: v5.3.3
package manager: npm
npm version: v10.1.0

Cannot find module 'manticoresearch-ts' or it's type declarations

The following error appears when copying the imports from the README:

Fix:

import * from "manticoresearch-ts/dist/src"

// OR

import * from "manticoresearch-ts/dist/src/index"

This probably happens because the index.d.ts file is under dist/src in the npm package.