[Question] Does the Mmap object need to be kept alive in order for the byte buffer to be valid? about memmap2-rs

[adamreichold]

this would mainly occur when using unsafe code to turn the bytes buffer into a slice or something else

Ideally, such transformations would keep the lifetimes intact, e.g starting from a &'a Mmap going through a &'a [u8] should yield a &'a [T] which could also not outlive the Mmap itself.

[adamreichold]

Yes, Mmap calls munmap(2) on Drop meaning that the memory mapping becomes invalid.

It should also not be possible to access it using safe code because it is access via impl Deref for Mmap which means that the borrow cannot continue after the Mmap is dropped?

[coreylowman]

Okay thanks! And yeah, this would mainly occur when using unsafe code to turn the bytes buffer into a slice or something else

[coreylowman]

I'm investigating whether I can avoid causing undefined behavior by creating a Vec<T> from an mmap'd file. Currently looking at something like:

struct MemoryMappedVec<T> {
    data: Option<Vec<T>>,
    #[allow(unused)]
    mmap: memmap2::Mmap,
}

impl<T> MemoryMappedVec<T> {
    pub unsafe fn new(path: &str, length: usize) -> Self {
        let file = std::fs::File::open(path).unwrap();
        let mmap: memmap2::Mmap = unsafe { memmap2::Mmap::map(&file).unwrap() };
        let bytes: &[u8] = &mmap;
        let ptr = bytes.as_ptr() as *mut T;
        assert!(bytes.len() < (isize::MAX as usize));
        assert_eq!(bytes.len(), length * std::mem::size_of::<T>());
        assert_eq!(ptr.align_offset(std::mem::align_of::<T>()), 0);
        let data = unsafe { Vec::from_raw_parts(ptr, length, length) };
        Self {
            data: Some(data),
            mmap,
        }
    }
}

impl<T> Drop for MemoryMappedVec<T> {
    fn drop(&mut self) {
        if let Some(data) = std::mem::take(&mut self.data) {
            // Avoid dropping the Vec, since it wasn't actually allocated in the first place.
            std::mem::forget(data);
        }
    }
}

I know this is insanely unsafe, especially the ownership of Vec stuff. I'm wondering in what situations I can get away with this.

[adamreichold]

I am not sure why you would want to do use a Vec<T> if you can never hand it out anyway (because it cannot be dropped, which should also be ingrained into the type using ManuallyDrop).

Why not just implement Deref<Target=[T]> for your struct which could contain just the Mmap as turning the &[u8] into a &[T] should not cost anything as long as the size, alignment, etc. is checked upfront?