weaviate / weaviate-javascript-client Goto Github PK

View Code? Open in Web Editor NEW

12.0 25.0 8.0 854 KB

No longer maintained, please see the TypeScript client

Home Page: https://github.com/weaviate/typescript-client

License: BSD 3-Clause "New" or "Revised" License

JavaScript 0.28% Shell 0.46% TypeScript 99.26%

generative-search nlp semantic-search vector-database vector-search javascript

weaviate-javascript-client's Introduction

Weaviate

Overview

Weaviate is a cloud-native, open source vector database that is robust, fast, and scalable.

To get started quickly, have a look at one of these pages:

Quickstart tutorial To see Weaviate in action
Contributor guide To contribute to this project

For more details, read through the summary on this page or see the system documentation.

Why Weaviate?

Weaviate uses state-of-the-art machine learning (ML) models to turn your data - text, images, and more - into a searchable vector database.

Here are some highlights.

Speed

Weaviate is fast. The core engine can run a 10-NN nearest neighbor search on millions of objects in milliseconds. See benchmarks.

Flexibility

Weaviate can vectorize your data at import time. Or, if you have already vectorized your data, you can upload your own vectors instead.

Modules give you the flexibility to tune Weaviate for your needs. More than two dozen modules connect you to popular services and model hubs such as OpenAI, Cohere, VoyageAI and HuggingFace. Use custom modules to work with your own models or third party services.

Production-readiness

Weaviate is built with scaling, replication, and security in mind so you can go smoothly from rapid prototyping to production at scale.

Beyond search

Weaviate doesn't just power lightning-fast vector searches. Other superpowers include recommendation, summarization, and integration with neural search frameworks.

Who uses Weaviate?

Software Engineers
- Weaviate is an ML-first database engine
- Out-of-the-box modules for AI-powered searches, automatic classification, and LLM integration
- Full CRUD support
- Cloud-native, distributed system that runs well on Kubernetes
- Scales with your workloads
Data Engineers
- Weaviate is a fast, flexible vector database
- Use your own ML model or third party models
- Run locally or with an inference service
Data Scientists
- Seamless handover of Machine Learning models to engineers and MLOps
- Deploy and maintain your ML models in production reliably and efficiently
- Easily package custom trained models

What can you build with Weaviate?

A Weaviate vector database can search text, images, or a combination of both. Fast vector search provides a foundation for chatbots, recommendation systems, summarizers, and classification systems.

Here are some examples that show how Weaviate integrates with other AI and ML tools:

Use Weaviate with third part embeddings

Use Weaviate as a document store

DocArray
Haystack (blogpost)

Use Weaviate as a memory backend

Demos

These demos are working applications that highlight some of Weaviate's capabilities. Their source code is available on GitHub.

How can you connect to Weaviate?

Weaviate exposes a GraphQL API and a REST API. Starting in v1.23, a new gRPC API provides even faster access to your data.

Weaviate provides client libraries for several popular languages:

There are also community supported libraries for additional languages.

Where can You learn more?

Free, self-paced courses in Weaviate Academy teach you how to use Weaviate. The Tutorials repo has code for example projects. The Recipes repo has even more project code to get you started.

The Weaviate blog and podcast regularly post stories on Weaviate and AI.

Here are some popular posts:

Blogs

Podcasts

Join our community!

At Weaviate, we love to connect with our community. We love helping amazing people build cool things. And, we love to talk with you about you passion for vector databases and AI.

Please reach out, and join our community:

To keep up to date with new releases, meetup news, and more, subscribe to our newsletter

weaviate-javascript-client's People

Contributors

Stargazers

Watchers

Forkers

halvespereira philippeitis erikvullings evenfrost marcussorealheis rosshartmann mloenow

weaviate-javascript-client's Issues

Client breaks in certain React setups

We should probably transpile (babel, etc.) the client before exporting it to remove any features considered experimental.

Add JS/TS formatter

We curerntly do not have a formatter for the JS code. I don't have any preferences so please use the most common one :)

This should be added:

In the CI
In precommit

Complete TS migration type conversions

The PR which migrated this client from untyped javascript to typescript included some types, but not all of them. Now there are a remainder of any types which should be assigned their proper types, where possible.

A quick way to determine where these changes need to be made can be done by grepping the repo for : any.

Hybrid, Selected properties only

The documentation states that I can use 'withHybrid' with the 'properties' property (https://weaviate.io/developers/weaviate/search/hybrid), but it seems that it is not included yet.

Error: invalid usage: Error: hybrid filter: unrecognized key 'properties'

!npm show weaviate-client

[email protected] | SEE LICENSE IN LICENSE | deps: 3 | versions: 26
Javascript client for Weaviate
https://github.com/weaviate/weaviate-javascript-client#readme

keywords: weaviate

dist
.tarball: https://registry.npmjs.org/weaviate-client/-/weaviate-client-2.14.5.tgz
.shasum: 30585c9632b3ee636a018595ba30dc9b6e44b06f
.integrity: sha512-pGC1AAYAyBF8Gl8D45AE9JOmvyP4cMHC+TeF6New4ISEq/RefxtZH7F5A26FLHx+3aLGTBGXJhziB6r9x6dyVQ==
.unpackedSize: 544.9 kB

dependencies:
@babel/runtime: ^7.20.7 graphql-request: ^5.1.0 isomorphic-fetch: ^3.0.0

maintainers:

semi-technologies [email protected]

dist-tags:
latest: 2.14.5

!node --version
v19.9.0

referencesBatcher is cumbersome to use: new function suggested

The current API to update items in batch is rather cumbersome to use, e.g. according to the docs, I should use:

const weaviate = require("weaviate-client");

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
});

client.batch
      .referencesBatcher()
      .withReference({
        from: `weaviate://localhost/Author/36ddd591-2dee-4e7e-a3cc-eb86d30a4303/wroteArticles`,
        to: `weaviate://localhost/Article/6bb06a43-e7f0-393e-9ecf-3c0f4e129064`,
        // prior to v1.14 omit the class name as part of the "to" beacon and specify it as weaviate://localhost/<id>
      })
      .withReference({
        from: `weaviate://localhost/Author/36ddd591-2dee-4e7e-a3cc-eb86d30a4303/wroteArticles`,
        to: `weaviate://localhost/Article/b72912b9-e5d7-304e-a654-66dc63c55b32`,
        // prior to v1.14 omit the class name as part of the "to" beacon and specify it as weaviate://localhost/<id>
      })
      .withReference({
        from: `weaviate://localhost/Author/36ddd591-2dee-4e7e-a3cc-eb86d30a4304/wroteArticles`,
        to: `weaviate://localhost/Article/b72912b9-e5d7-304e-a654-66dc63c55b32`,
        // prior to v1.14 omit the class name as part of the "to" beacon and specify it as weaviate://localhost/<id>
      })
      .do()
      .then(res => {
        console.log(res)
      })
      .catch(err => {
          console.error(err)
      });

In my case, however, I do not know how many cross references I need to add (variable number), so it would be much easier and less verbose to use something as

const weaviate = require("weaviate-client");

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
});

client.batch
      .referencesBatcher()
      .withReferences([{
        from: `weaviate://localhost/Author/36ddd591-2dee-4e7e-a3cc-eb86d30a4303/wroteArticles`,
        to: `weaviate://localhost/Article/6bb06a43-e7f0-393e-9ecf-3c0f4e129064`,
        // prior to v1.14 omit the class name as part of the "to" beacon and specify it as weaviate://localhost/<id>
      }, {
        from: `weaviate://localhost/Author/36ddd591-2dee-4e7e-a3cc-eb86d30a4303/wroteArticles`,
        to: `weaviate://localhost/Article/b72912b9-e5d7-304e-a654-66dc63c55b32`,
        // prior to v1.14 omit the class name as part of the "to" beacon and specify it as weaviate://localhost/<id>
      }, {
        from: `weaviate://localhost/Author/36ddd591-2dee-4e7e-a3cc-eb86d30a4304/wroteArticles`,
        to: `weaviate://localhost/Article/b72912b9-e5d7-304e-a654-66dc63c55b32`,
        // prior to v1.14 omit the class name as part of the "to" beacon and specify it as weaviate://localhost/<id>
      }])
      .do()
      .then(res => {
        console.log(res)
      })
      .catch(err => {
          console.error(err)
      });

I've briefly looked at the code, and it seems very easy to implement. Just add the following method to the referencesBatcher.js

withReferences = (obj) => {
  this.references = obj;
  return this;
};

Add support for types.

Currently, javascript client doesn't have type definitions. It would be really useful to add type definitions for development with typescript.

Possibility to update class

It would be great to have the possibility to update a class as currently it seems there is no way to do it.

Add support for rerank parameter in ask argument

Add support for nearImage argument

GraphQL Get `group` filter missing

The GraphQL query with group filter for entity merging is missing. Queries like this are impossible with the client now:

{
  Get {
    Things {
      Publication(
        group:{
          type: merge,
          force:0.05
        }
      ) {
        name
      }
    }
  }
}

Add support for spellcheck module

ERROR in ./node_modules/weaviate-client/lib.js

After installing weaviate-client for React
I get the following error

Compiled with problems:
ERROR in ./node_modules/weaviate-client/lib.js 672:11-24
Module not found: Error: Can't resolve 'fs' in 'C:\Users\User\react-projects\toing\smart-select-schema\node_modules\weaviate-client'

package JSON:
package.zip

Adjust to Weaviate v1

Add GraphQL Aggregate

function for RESTful GET parameter `limit` is missing

withLimit() does work with .graphql.get functions, but not for the .getter function. This is expected to work:

const weaviate = require("weaviate-client");

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
});

client
    .data
    .getter()
    .withLimit(2)  // THIS IS CURRENTLY NOT IMPLEMENTED 
    .withUnderscoreClassification()
    //.withUnderscoreInterpretation()
    //.withUnderscoreNearestNeighbors()
    //.withUnderscoreFeatureProjection()
    //.withUnderscoreVector()
    .do()
    .then(res => {
        console.log(res)
    })
    .catch(err => {
        console.error(err)
    });

security warning from isomorphic-fetch

might not matter, but leaving it here just in case so I can fix it later

npm install weaviate-client                                                                         
"4 vulnerabilities (3 low, 1 high)"                                            
                                 
Run `npm audit` for details.

node-fetch  <=2.6.6
Severity: high
The `size` option isn't honored after following a redirect in node-fetch - https://github.com/advisories/GHSA-w7rc-rwvf-8q5r
node-fetch is vulnerable to Exposure of Sensitive Information to an Unauthorized Actor - https://github.com/advisories/GHSA-r683-j2x4-v87g
fix available via `npm audit fix --force`
Will install [email protected], which is a breaking change
node_modules/node-fetch
  isomorphic-fetch  2.0.0 - 2.2.1
  Depends on vulnerable versions of node-fetch
  node_modules/isomorphic-fetch

Add REST capabilities

schema
objects
classifications
other

Support ask argument

ERROR - GENERATIVE AI JS COMPONENT

Thanks for all the heard work on the Weaviate platform. We are testing both your javascript and python sdks, and seem to keep hitting issues with the javascript component.

Our schema and db instances are here

https://product.weaviate.network/v1/schema
https://product.weaviate.network/v1/objects

We migrated ~11k item product catalog to Weaviate in order to test the generative ai capability that you recently announced

Here is the code, closely following your example to run a simple test:

 import weaviate from 'weaviate-client'

let url = process.env.DB
let WEAVIATE_HOST = process.env.WEAVIATE_PRODUCT_HOST
let OPEN_API_KEY = process.env.OPENAI_API_KEY

const client = weaviate.client({
  scheme: 'https',
  host: WEAVIATE_HOST,
  headers: {'X-OpenAI-Api-Key': OPEN_API_KEY},
});

// instruction for the generative module
const generateTask = '"Summarize the selected products in a table and provide prices on each"';

client.graphql
  .get()
  .withClassName('Product')
  .withFields(
    'name ' +
    `_additional {generate(groupedResult:{ task: ${generateTask} }) { groupedResult }}`
  )
  .withNearText({
    concepts: ['drill bits and drills']
  })
  .withLimit(5)
  .do()
  .then(res => {
    console.log(res)
  })
  .catch(err => {
    console.error(err)
  });

Here is the console error

ClientError: Cannot query field "generate" on type "ProductAdditional".: {"response":{"errors":[{"locations":[{"column":87,"line":1}],"message":"Cannot query field "generate" on type "ProductAdditional".","path":null}],"status":200,"headers":{}},"request":{"query":"{Get{Product(nearText:{concepts:["drill bits and drills"]},limit:5){name _additional {generate(groupedResult:{ task: "Summarize the selected products in a table and provide prices on each" }) { groupedResult }}}}}"}}
at C:\Users\patri\Desktop\gpt\gptserver\node_modules\graphql-request\dist\index.js:416:31
at step (C:\Users\patri\Desktop\gpt\gptserver\node_modules\graphql-request\dist\index.js:67:23)
at Object.next (C:\Users\patri\Desktop\gpt\gptserver\node_modules\graphql-request\dist\index.js:48:53)
at fulfilled (C:\Users\patri\Desktop\gpt\gptserver\node_modules\graphql-request\dist\index.js:39:58)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
response: {
errors: [ [Object] ],
status: 200,
headers: Headers { [Symbol(map)]: [Object: null prototype] }
},
request: {
query: '{Get{Product(nearText:{concepts:["drill bits and drills"]},limit:5){name _additional {generate(groupedResult:{ task: "Summarize the selected products in a table and provide prices on each" }) { groupedResult }}}}}',
variables: undefined
}
}

Any tips on what I might be doing wrong would be greatly appreciated

Thank you!

Creating a data object with custom vectors

Add support for nearObject argument

Error: Cannot find module './httpClient.js'

When initiating a weaviate client like so:

const client = weaviate.client({
  scheme: "http",
  host: "localhost:3000",
});

I gett the exception Error: Cannot find module './httpClient.js' originating from lib.js. There is in fact no httpClient.js and yet it is required from within lib.js

Add support for where argument in Aggregate

API Feature Completion

Still missing

Prototype GraphQL Client

Inline with the updated Python client, we want to have a slightly more typed interface.

Current Draft:

Minimal

client.graphql.get.things("Person", "{name street city}").do()

With filters/explore/limit

client.graphql.get.things("Person", "{name street city}")
  .withWhere({operator: "Equal", valueString:"John Doe", path: ["name"]})
  .explore({concepts: ["musician"]})
  .do()

Add JS/TS linter

We curerntly do not have a linter for the JS code. I don't have any preferences so please use the most common one :)

This should be added:

In the CI
In precommit

Add support for offset param

Support of Weaviate 1.14.1 still missing (in referencesBatcher)?

I'm using Weaviate 1.14.1, and when I want to use the batch API to add cross references, I get the following error:

Usage of beacons paths with className is not supported in Weaviate . className parameter is ignored

Is the referencesBatcher not yet updated to support 1.14?

Add `.requiresOIDC` method

Add a method to misc which returns true if the instance uses OpenID Connect auth and returns false if it doesn't.

Convert to Typescript?

Would you be open to converting this package to TS so people can get autocomplete when using the builder pattern?

Add support for object check request

Add support for /v1/schema/{className}

Update Weaviate test docker compose version

GraphQL Aggregate `groupBy` filter gives inconsistent result

Important to note is that the property isAccessible is not filled, so it only has null values.

The following query in plain GraphQL gives the following result:

GraphQL Query:

{
  Aggregate {
    Things {
      Article(groupBy: ["isAccessible"]) {
        meta {
          count
        }
        wordCount {
          mean
        }
        groupedBy {
          value
          path
        }
      }
    }
  }
}

GraphQL Result:

{
  "data": {
    "Aggregate": {
      "Things": {
        "Article": []
      }
    }
  },
  "errors": null
}

Expected is the same output with the JS code, but the result is inconsistent:

JS Query:

const weaviate = require("weaviate-client");

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
});

client.graphql
      .aggregate()
      .withClassName('Article')
      .withGroupBy(["isAccessible"])
      .withFields('meta { count } wordCount {mean} groupedBy {value path}')
      .do()
      .then(res => {
        console.log(JSON.stringify(res))
      })
      .catch(err => {
        console.error(err)
      });

JS response:

{
  "data": {
    "Aggregate": {
      "Things": {
        "Article": [
          {
            "groupedBy": null,
            "meta": {
              "count": 2548
            },
            "wordCount": {
              "mean": 1013.82025
            }
          }
        ]
      }
    }
  },
  "errors": null
}

Add support for generative openai module

Using the generative module is currently possible, but he UX is not that great. Its usage could be improved by adding a explicit support for this kind of query:

Note:

The module generative-openai needs to be enabled using ENABLE_MODULES=generative-openai
You need an openai API key:OPENAI_APIKEY=some_key
The tests should be skipped if no openai key is available
For inspiration, check the python or go imlpementation

Automatically transmit API keys found in the environment variables along with the request

When importing data for a class with the vectorizer set to 'text2vec-openai', I got the error

OpenAI API Key: no api key found neither in request header: X-OpenAI-Api-Key nor in environment variable under OPENAI_APIKEY

However, OPENAI_APIKEY was in fact set. The error was thus misleading.

Would it be within the scope of the client to automatically pick up the OpenAI API key from the OPENAI_APIKEY environment variable and add it into the request header, i.e. by setting headers: { 'X-OpenAI-Api-Key': process.env.OPENAI_APIKEY }?

how to include the library directly in the browser?

Hi, Thanks for this interesting semweb framework!

Any tips on how to include this library directly in the browser (without the "require" build step)?
I could not find a bundled "dist" build, just the "lib.js" - which does not work when loaded directly in the browser.

CORS issue with JS Client and Multi Parameters issue with TS Client.

Issues:

JS Client: I would like to start of by saying that i am aware that it is no longer maintained. However since I'm working with react 18, naturally i went for the JS Client first. Problem is that after setting up my code like so:
const weave = require('weaviate-client'); const client = weave.client({ scheme: 'https', host: 'cluster-something.weaviate.network', headers: {'X-OpenAI-Api-Key': 'secret-key'}, authClientSecret: new weave.AuthUserPasswordCredentials({ username: 'email', password: 'password!', }), })
I am unable to get past openconfiguration as the one and only header that is required for queries in my case, ie, the 'x-open-ai-api-key' happens to be a forbidden header. This is also applicable for all 'Acess-Control-X' headers. In short, it fails preflight.
The issue is however resolved when i turn on CORS everywhere for firefox. Not reallu sure if this will negatively impact the production build.
The exact error is attached below:
Access fetch at api-endpoint' from origin 'http://localhost:3000' has been blocked by CORS policy: Request header field x-openai-api-key is not allowed by Access-Control-Allow-Headers in preflight response.
To me it looks like you need an https server to run queries. But surely i have to be wrong. I mean there should be some wiggle room for development environments.
TS Client: Since i work with react 18, switching to the ts library was not easy. I had to downgrade react-scripts v5 to v4 which was already not ideal. But once i got it running, on using the code above in issue 1, I got an error saying that "You can either provide an API Key or Auth Credentials" which had me confused because the documentation says that you need to provide both. Yes, i did infact double check that i am using the 'x-open-ai-api-key' in headers. So i was not in any way providing an auth token alongside auth credentials.

If Anyone has any suggestions or can help in any way i would really appreciate it. Thanks!

`geoCoordinates` where filter error

A GraphQL query like this with a where filter on geoCoordinates works:

{
        operator: "WithinGeoRange",
        path: ["headquartersGeoLocation"],
        valueGeoRange: {
          geoCoordinates: {
            latitude: 51.51,    
            longitude: -0.09    
          },
          distance: {
            max: 2000 
          }
        }
      }

However, if this is used with the client like the following, the geoCoordinates object is parsed wrong, resulting in an error message:

client.graphql
      .get()
      .withClassName('Article')
      .withFields('title InPublication{... on headquartersGeoLocation{latitude longitude}}')
      .withWhere({
        operator: "WithinGeoRange",
        path: ["headquartersGeoLocation"],
        valueGeoRange: {
          geoCoordinates: {
            latitude: 51.51,    
            longitude: -0.09    
          },
          distance: {
            max: 2000 
          }
        }
      })
      .do()
      .then(res => {
        console.log(JSON.stringify(res))
      })
      .catch(err => {
        console.error(err)
      });

error message:

{
  "errors": [
    {
      "locations": [
        {
          "column": 67,
          "line": 1
        }
      ],
      "message": "Syntax Error GraphQL request (1:67) Expected Name, found String \"geoCoordinates\"\n\n1: {Get{Things{Article(where:{operator:WithinGeoRange,valueGeoRange:{\"geoCoordinates\":{\"latitude\":51.51,\"longitude\":-0.09},\"distance\":{\"max\":2000}},path:[\"headquartersGeoLocation\"]}){title InPublication{... on headquartersGeoLocation{latitude longitude}}}}}}\n                                                                     ^\n",
      "path": null
    }
  ],
  "highlightQuery": "{Get{Things{Article(where:{operator:WithinGeoRange,valueGeoRange:{\"geoCoordinates\":{\"latitude\":51.51,\"longitude\":-0.09},\"distance\":{\"max\":2000}},path:[\"headquartersGeoLocation\"]}){title InPublication{... on headquartersGeoLocation{latitude longitude}}}}}}\n                                                                   ~~~~~~~~\n"
}