Does tantivy::IndexWriter support multi-process? about tantivy

[fulmicoton]

Would this approach violate any internal implementation contracts of Tantivy, and is it feasible?

It will most likely not work yes.

The problem has to do with the delete and commit work.
Deletes are performed right before serialization.

The reason something like

 - add_doc(1)
 - delete_doc(1)
 - delete_doc(2)
 - add_doc(2)

work the way you expect, is because we attach an opstamp to each document and each delete operation, to know in which order those operations happened.

With your scheme, two concurrent writes could end up with very different outcomes.

 - add_doc(1)
 - add_doc(2)

 - delete_doc(1)
 - delete_doc(2)

You could end up with
no docs, doc1, doc2, doc1 and doc2 in the resulting tantivy index.

It will NOT look like the transaction were executed in the order of them taking the write lock.

[cyccbxhl]

The problem has to do with the delete and commit work. Deletes are performed right before serialization.

Can you explain more about it?

Tantivy's commit would be called in pg's commit command, I actually don't need tantivy operations executed in the order of them taking the write lock, I need them meet the RC(Read Committed) transaction isolation level: Only committed operation/data is visible for other concurrent transaction.
The behavior I expect:
(1)

Because the doc1 and doc2 is invisible for delete ops in txn2, when txn1 and txn2 all committed, there are doc1 and doc2 in the resulting tantivy index;
(2)

when txn1 and txn2 all committed, there are no doc in the resulting tantivy index;

Can tantivy be able to do that?

[fulmicoton]

Actually I think you are right it might work.

[cyccbxhl]

Thank you very much. I'll try implementing the code based on this plan first to see if there are any other issues. There might be some questions I will need to ask you later. I will close this issue for now.