Cannot alloc memory error about ngt

[masajiro]

You might be trying to handle a huge amount of data beyond our assumption.
Could you tell me the number of the objects, dimensionality and data type for our future improvement?

Although in fact there are parameters to define the maximum memory size that can be allocated at once, they cannot be changed from the outside.
Could you increase these three parameter's default values below?
https://github.com/yahoojapan/NGT/blob/master/lib/NGT/Index.h#L73-L75

[anxietymonger]

Thanks a lot for your reply. This helps a lot.

I am processing an index with over 100M objects and 40 dimensions.

In addition, I found that NGT_SHARED_MEMORY_MAX_SIZE (default value 1024MB) in defines.h.in should also be changed.

I am trying again with these parameter changes. Since this will cost a few time, I may report the result later.

Thanks again.

[masajiro]

Thank you for your additional information.

NGT_SHARED_MEMORY_MAX_SIZE might also work for this problem.

[anxietymonger]

I was able to build index successfully with the settings above. Appreciate your help.