qdrant / demo-distributed-deployment-docker

An example of setting up the distributed deployment of Qdrant with docker-compose

demo-distributed-deployment-docker

This repository contains the source code of the tutorial describing how to run Qdrant in a distributed mode using docker-compose.

Tutorial: https://youtu.be/53qQNUsCx2M

demo-distributed-deployment-docker's Issues

Upload Points API is returning status 400

I have followed the tutorial(video) and run the same commands on my machine but I am facing error: 400 while uploading the points given in the GitHub repository, It may be possible that we are using the newer version of qdrant/qdrant:latest docker image but just assuming.

Please help how can it work. I have run the below steps.

docker-compose Config

version: "3.9"
services:
  qdrant_primary:
    image: "qdrant/qdrant:latest"
    ports:
      - "6333:6333"
    environment:
      QDRANT__CLUSTER__ENABLED: "true"
    command: ["./qdrant", "--uri", "http://qdrant_primary:6335"]
  qdrant_secondary:
    image: "qdrant/qdrant:latest"
    environment:
      QDRANT__CLUSTER__ENABLED: "true"
    command: ["./qdrant", "--bootstrap", "http://qdrant_primary:6335"]

Starting the primary and secondary with scale

docker-compose -f qdrant-compose.yaml up qdrant_primary
docker-compose -f qdrant-compose.yaml up qdrant_secondary
docker-compose -f qdrant-compose.yaml up --scale qdrant_secondary=5

docker stats after running nodes

CONTAINER ID   NAME                                    CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O    PIDS
efb1475ee35c   qdrant-experiments-qdrant_secondary-5   1.37%     335.6MiB / 7.657GiB   4.28%     37.1kB / 31.5kB   0B / 274kB   70
f9caafbef1c8   qdrant-experiments-qdrant_secondary-3   1.38%     343.6MiB / 7.657GiB   4.38%     35.7kB / 30kB     0B / 279kB   70
1da6eb3d8ee3   qdrant-experiments-qdrant_secondary-2   1.43%     329.5MiB / 7.657GiB   4.20%     34.6kB / 29kB     0B / 283kB   70
4126333c73e6   qdrant-experiments-qdrant_secondary-4   1.28%     340.5MiB / 7.657GiB   4.34%     33.7kB / 28.2kB   0B / 287kB   70
f2a86694cc98   qdrant-experiments-qdrant_secondary-1   1.25%     343.7MiB / 7.657GiB   4.38%     46.8kB / 41.2kB   0B / 274kB   70
7a3c4f2b1603   qdrant-experiments-qdrant_primary-1     2.24%     338.2MiB / 7.657GiB   4.31%     155kB / 152kB     0B / 266kB   70

Cluster Info

{
  "result": {
    "status": "enabled",
    "peer_id": 6096081247485373,
    "peers": {
      "6205091360379412": {
        "uri": "http://172.18.0.4:6335/"
      },
      "1489837326465628": {
        "uri": "http://172.18.0.6:6335/"
      },
      "2556765209810808": {
        "uri": "http://172.18.0.5:6335/"
      },
      "1844334719089466": {
        "uri": "http://172.18.0.3:6335/"
      },
      "6096081247485373": {
        "uri": "http://qdrant_primary:6335/"
      },
      "6690136135998057": {
        "uri": "http://172.18.0.7:6335/"
      }
    },
    "raft_info": {
      "term": 1,
      "commit": 11,
      "pending_operations": 0,
      "leader": 6096081247485373,
      "role": "Leader",
      "is_voter": true
    },
    "consensus_thread_status": {
      "consensus_thread_status": "working",
      "last_update": "2024-03-26T12:49:22.878880807Z"
    },
    "message_send_failures": {}
  },
  "status": "ok",
  "time": 0.000009542
}

Create Collection

curl --fail -X PUT \
  -H "Content-type: application/json" \
  "http://localhost:6333/collections/example_collection" \
  --data-binary "@docs/collection.json"

Upload Points

curl --fail  --verbose -X PUT \
  -H "Content-type: application/json" \
  "http://localhost:6333/collections/example_collection/points?wait=true" \
  --data-binary "@docs/points.json"

Getting Error

*   Trying [::1]:6333...
* Connected to localhost (::1) port 6333
> PUT /collections/example_collection/points?wait=true HTTP/1.1
> Host: localhost:6333
> User-Agent: curl/8.4.0
> Accept: */*
> Content-type: application/json
> Content-Length: 7457254
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 400 Bad Request
< content-length: 87
< content-type: application/json
< vary: Origin, Access-Control-Request-Method, Access-Control-Request-Headers
< date: Tue, 26 Mar 2024 13:05:39 GMT
* The requested URL returned error: 400
* Closing connection
curl: (22) The requested URL returned error: 400

Logs from One of the node

INFO qdrant::tonic::logging: gRPC /qdrant.PointsInternal/Upsert failed with Client specified an invalid argument "Wrong input: Not existing vector name error: " 0.012165