Just for some queries: MDB_CORRUPTED: Located page was wrong type. about meilisearch

[sznowicki]

I'm sure I've been moving data while meili process was not running though. I'll observe it with fresh index and report if it comes back. This time I'll make a proper dump ;)

[sznowicki]

I thought about making a dump and recreate indexes but this also failed with also quite cryptic message:

Task {
  indexUid: null,
  status: 'failed',
  type: 'dumpCreation',
  uid: 2162947,
  details: { dumpUid: null },
  canceledBy: null,
  error: {
    message: 'internal: MDB_PAGE_NOTFOUND: Requested page not found.',
    code: 'internal',
    type: 'internal',
    link: 'https://docs.meilisearch.com/errors#internal'
  },
  duration: 'PT35.734438274S',
  startedAt: 2024-02-23T19:59:11.542Z,
  enqueuedAt: 2024-02-23T19:59:11.533Z,
  finishedAt: 2024-02-23T19:59:47.277Z
}

[Kerollmops]

Hey @sznowicki 👋

Thank you for the issue and report. The MDB_CORRUPTED kind of errors is clearly showing that the database is corrupt on disk.

It brings the question of what kind of volume did you mount on Docker? Is it a network disk (NTFS)?

Meilisearch, and more specifically LMDB (our internal key-value store), doesn't support network file systems. Always use a raw SSD or NVMe.

The best solution to avoid having to deal with this kind of issues is to use the cloud offering. We provide more features like analytics and auto-updates. We never saw any database corruption in the past.

[sznowicki]

It's RAID 1 setup with LVM and LUKS encryption. For the process it should be visible as a regular ext4 filesystem. I'm reindexing all from scratch at the moment and so far don't observe any problems.

Possibly something went wrong when I moved the data from one disk to another.

For future problems, is there any way to recover from such state using some LMDB processes?

[Kerollmops]

Possibly something went wrong when I moved the data from one disk to another.

Ho! Indeed, copying a file while it is being written to (indexing documents) is unsafe as the copy process is iterative (pages by pages) while data is being modified.

For future problems, is there any way to recover from such state using some LMDB processes?

There is maybe a feature in LMDB to read the pages from the previous transaction (before the one indexing new documents). But not sure it is available in LMDB 0.9...