Server crash when inserting with ivfflat index with high number of clusters about pgvector

[ankane]

Just pushed 0.3.1 with a fix. This uncovered a corruption issue introduced in 0.2.7, so recreate all ivfflat indexes after upgrading.

Thanks again for the great report!

[ArthurMelin]

The crash that happens when building a new index doesn't happen in the same location, I'm assuming this is a different problem and creating a 2nd issue.

[ankane]

Hey @ArthurMelin, thanks for another report! Unfortunately, I'm not able to reproduce this one:

CREATE TABLE embed (id integer NOT NULL, vec vector(384) NOT NULL);
CREATE INDEX ON embed USING ivfflat (vec vector_cosine_ops) WITH (lists = 10000);
INSERT INTO embed SELECT i, (SELECT array_agg(random()) FROM generate_series(1, 384)) FROM generate_series(1, 1000) i;

From the GDB output, it looks like the page is in an unexpected state.

Edit: Tried to reproduce with COPY as well, but no luck.

COPY embed FROM 'rand1k.csv' WITH (FORMAT csv);

[ArthurMelin]

Hi thanks for the reply, I tried with your method of insertion and it crashed as well, do you have assertions enabled in your Postgres build? Maybe this bug doesn't break enough stuff to trigger a crash if they are not?

[ankane]

Thanks @ArthurMelin, was able to reproduce. Working on a fix.