`Chunk not found` exception when trying to drop index about nitrite-java

[AngryGami]

Thanks :)
First of all, I don't consider this a bug in Nitrite, just a lack of tools to gracefully handle situations like this (i.e., without clearing the entire database).

Unfortunately, I don't have the corrupted database file anymore. I had to continue with other tasks and deleted it. I was just thinking about a mechanism to recover from such issues when the app is in production.

I'll try that branch with the fix if I encounter the problem again (I've seen this a couple of times already, so it's likely to happen again). As far as I understand, this happens because the app crashes for some other reason, and the database fails to fully write either data or metadata for some chunk. Therefore, the approach of opening a previous version should work. I was considering this myself, for example, by accessing the MVStore instance and calling rollbackTo on it.

[AngryGami]

Nitrite can be less strict when removing map from MVStore here:

    public void removeMap(String name) {
       // why do `openMap` here? MVStore is totally capable of removing map by just name
        MVMap<?, ?> mvMap = this.mvStore.openMap(name);
        this.mvStore.removeMap(mvMap);
        this.getCatalog().remove(name);
        this.nitriteMapRegistry.remove(name);
    }

[anidotnet]

Hi @AngryGami did you try rebuilding index instead of dropping it, if you think the index is corrupted?

why do openMap here? MVStore is totally capable of removing map by just name

Inside the MVStore, it is opening the map by name anyway if it's not already opened. And openMap has no overhead in Nitrite if the MV map is already opened.

[AngryGami]

did you try rebuilding index instead of dropping it, if you think the index is corrupted?

I didn't, but I'm actually operating not on index, I'm trying to drop Repository/Collection (so internally it looks for indexes associated with it and drop them, and one of them apparently got corrupted and refuse get dropped). I don't really care about fixing the index - it doesn't make a lot of sense, I would prefer to drop collection and be done with it.

Inside the MVStore, it is opening the map by name anyway if it's not already opened. And openMap has no overhead in Nitrite if the MV map is already opened.

Well, I'm not sure it does open it internally. Here is the method for removing map by name:

    public void removeMap(String name) {
        int id = getMapId(name);
        if(id > 0) {
            MVMap<?, ?> map = getMap(id);
            if (map == null) {
                map = openMap(name, MVStoreTool.getGenericMapBuilder());
            }
            removeMap(map);
        }
    }

It only goes inside openMap method if getMap returns null and I hoped that map was already opened at this point somehow.

[anidotnet]

It only goes inside openMap method if getMap returns null and I hoped that map was already opened at this point somehow.

That's what I said (Inside the MVStore, it is opening the map by name anyway if it's not already opened). If the map is already opened, removeMap(String) is calling removeMap(MVMap) and if the map is not opened, it is first opened in removeMap(String) and then call removeMap(MVMap). Essentially, you can't avoid the situation you are currently in even if removeMap(String) is used.

Now back to the problem itself. As this kind of issue is almost impossible to reproduce, I need your help here in debugging it. Here is the branch which contains a slightly modified NitriteMVStore which tries to open the map with previous version if the db is corrupted. I am not sure if it will solve your issue, but as I said, I have no means to reproduce this issue and test the fix myself.

After cloning the repo branch to your local, just do a mvn clean install (may be change the pom version before) and use it in your project and see if the fix resolves the issue.