Comments (7)
Can you please illustrate the C or at least plain mmap
call sequence you would use?
From your description, I wonder why you just don't pass the byte slice itself to the other threads? And what kind of synchronization exists between them as multiple threads writing into the same mapping without synchronization is likely to imply a data race.
from memmap2-rs.
(It is also not clear to me what "thread-local memory map" means here as a memory mapping is by design process-wide as all threads share a single address space in which all memory maps are contained.)
from memmap2-rs.
Here's some code from the slice_deque
crate that calls to the POSIX mmap:
let ptr = mmap(
ptr::null_mut(),
size,
PROT_READ | PROT_WRITE,
MAP_SHARED,
fd,
0,
);
if ptr == MAP_FAILED {
print_error("@first: mmap failed");
if close(fd) == -1 {
print_error("@first: close failed");
}
return Err(AllocError::Oom);
}
let ptr2 = mmap(
(ptr as *mut u8).offset(half_size as isize) as *mut c_void,
half_size,
PROT_READ | PROT_WRITE,
MAP_SHARED | MAP_FIXED,
fd,
0,
);
In ptr2
, a second map is created by offsetting into the first map at ptr
.
The reason for this is because I'm implementing a concurrent Prolog runtime in Rust with a consolidated heap. The first section of the heap (the memory map shared among all threads) is the atom table. The second section that comes immediately after it is the Prolog heap. Cells in the Prolog heap compose Prolog terms. Prolog terms can point down into the atom table.
From your description, I wonder why you just don't pass the byte slice itself to the other threads?
Because of the Prolog VM design, it's most convenient to map the atom table contiguously in memory next to the thread-local Prolog heap. In particular, converting atoms to strings (lists of characters) is very efficient this way.
(It is also not clear to me what "thread-local memory map" means here as a memory mapping is by design process-wide as all threads share a single address space in which all memory maps are contained.)
By that I mean that every VM thread will have its own local MMapMut
object that will never traverse threads but that contains the atom table MMap*
as a sub-map.
And what kind of synchronization exists between them as multiple threads writing into the same mapping without synchronization is likely to imply a data race.
Absolutely, I plan to use RCU to manage that. But that's outside the scope of the memmap-rs
library.
Also, I'm aware that memory mapping is process-wide and not per-thread. I mention threads just to give context to my use case. I don't expect memmap-rs
to be aware of threading and synchronization.
from memmap2-rs.
So this appears to be a duplicate of #35, i.e. support for MAP_FIXED
.
To be honest, using MAP_FIXED
safely appears even harder to just plain non-overlapping mappings and it does not appear to be supported on Windows, so I am unsure whether this crate will be particularly helpful compared to lower-level wrappers like libc
, nix
or rustix
.
Finally, but that is besides questions on this crate's API, why could the terms not just point into a global atom table wherever that is allocated, i.e. why must the two sections of the heap be contiguous?
from memmap2-rs.
Heap cells are tagged 64-bit pointers, so their addresses are relative rather than absolute. There would then need to be further tags to distinguish the locations, which would complicate the VM design.
from memmap2-rs.
If PRs are welcome for this, I'll try to implement MAP_FIXED
. I need it to work on Windows too, so I'll be looking at that as well.
from memmap2-rs.
You are right, fixing memory maps is not possible in Windows. Closing this issue.
from memmap2-rs.
Related Issues (20)
- Migrate to `safer_owning_ref` HOT 1
- why does memmap2::Mmap::map need to be unsafe? HOT 4
- Implement AsRawFd, IntoRawFd, AsRawHandle, IntoRawHandle and Into<Stdio> for MMap HOT 2
- mmap'd anon pages fail in io_uring IOSQE_BUFFER_SELECT usage HOT 2
- advise_writes_unsafely_to_part_of_map test fails on powerpc64le-unknown-linux-gnu HOT 2
- error[E0433]: failed to resolve: could not find `Advice` in `memmap2` HOT 1
- PermissionDenied, message: "Access is denied." at remove file after read on Github Action WIndows HOT 2
- Doesn't compile for freebsd HOT 3
- v0.9.2 build failed for android target. HOT 3
- Support for `MS_INVALIDATE`? HOT 8
- libc dependency is too loose HOT 11
- Implementing DerefMut for Mmap? HOT 3
- support non-slice based APIs HOT 2
- Support multiple advice HOT 2
- Contributing a migration from rustix to libc HOT 9
- MSRV and edition HOT 5
- Merging changes with memmapix HOT 24
- memmap2 `advise` is unsound HOT 8
- Interest is supporting mbind HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from memmap2-rs.