Deploy with docker compose and similar setup vespa to other vector dbs about pyvespa

[kkraune]

Hi, and thanks for valuable feedback!

It seems to me from your comments above that it is easier for you to work on the schema file (.sd) directly and not generate it using the Python API. It also seems that you prefer to spin up the container manually. All of this is great, and resembles what one usually does in https://docs.vespa.ai/en/vespa-quick-start.html.

A more barebones example using pyvespa is https://pyvespa.readthedocs.io/en/latest/examples/pyvespa-examples.html#Neighbors

We have learned that it can be a bit overwhelming to get started, as using vectors in Vespa is almost just like using any other field type. Most applications do not use vectors in isolation but in a combo with other filters and ranking signals. So bear with us ;-)

The two key config files are the schema and services definition, e.g.:

• https://github.com/vespa-engine/sample-apps/blob/master/vector-streaming-search/schemas/mail.sd
• https://github.com/vespa-engine/sample-apps/blob/master/vector-streaming-search/services.xml

I think this is what you are looking for wrt simple configuration - it is essentially the schema file that is important. services.xml is about how to run it, multiple options at https://docs.vespa.ai/en/getting-started.html - from laptop deployment, through multinode system and also the Vespa Cloud service.

To keep this short, maybe run trough https://docs.vespa.ai/en/vespa-quick-start.html to familiarize with the two files / deployment, then explore how to add a field for the embeddings (curious to learn about your thoughts there), and finally look at options for how to query Vespa - let's do that in later comments

[ricoms]

For the initialization I found a schema setup example here, I'll be following that. But then I want to define a simple client in which I can insert random embeddings and query for random embeddings, without too many configurations initially.

[ricoms]

My suggestion would be to simplify a lot the Quick Start example. It's showing too many features, too many initial configurations. When the main functionality is vector archive, indexing and search.

Thanks a lot for the great work you are doing, and please consider my comments as of a fan. It looks great all of Vespa and pyvespa. I hope there is something useful for you to be extracted from my comments.

best regards.

[ricoms]

these are a couple of open source benchmark tools that I'm kind of following for insights as well - https://github.com/zilliztech/VectorDBBench/tree/main/vectordb_bench/backend/clients and https://github.com/qdrant/vector-db-benchmark/tree/master/engine/clients

maybe they are of interest for you. :)

[ricoms]

Thanks for your swift and complete answer @kkraune . As I said I'm working on a client so we here can loadtest vespa db. Hope I can get some of it open source, at least on those benchmarks I shared.

I'll study the documentations you shared and try to make the best of it.

Best regards!