0.36 Nested Collections not stored in a separate Sub Index about docarray

[dwlmt]

Another issue is I have structures like the following. These contain multiple embeddings at the top level and also TermDoc has embeddings for each item in the list. It seems as if now it only works with multiple embeddings at the parent doc, or a single embedding with the parent and the nested embeddings but not both. It will give an error from QDrant saying "title_embedding" does not exist which I assume is because this isn't in the TermDoc in the same collection.

class ProductDoc(ProductCoreDoc):
    ...
    terms: Optional[DocList[TermDoc]]
    title_embedding: Optional[AnyTensor] = Field(
        space=similarity_space, dim=dim_size)
    description_embedding: Optional[AnyTensor] = Field(
        space=similarity_space, dim=dim_size)
    category_embedding: Optional[AnyTensor] = Field(
        space=similarity_space, dim=dim_size)
    embedding: Optional[AnyTensor] = Field(
        space=similarity_space, dim=dim_size)

[JoanFM]

First of all,

Can you confirm that it did not work the same in 0.35.0?

[dwlmt]

Yes, if I downgrade to 0.35.0 I get this from Qdrant where the paths is created as a separate collection:

collections=[CollectionDescription(name='channel_category_nested'), ... CollectionDescription(name='channel_category_nested__paths')]

In 0.36.0 the paths vectors are stored in the parent collection.

[JoanFM]

@kacperlukawski may be able to help here and give better insights.

[JoanFM]

I believe this issue is caused by #1705

[dwlmt]

@JoanFM thanks for this. I see a fix has been merged to main. I assume this will be in the next 0.37.0 release.

[JoanFM]

yes, u can also have it if u use the prerelease by pip install docarray --pre