craftspider / department Goto Github PK
View Code? Open in Web Editor NEWAn implementation of the Storage API in Rust
License: Apache License 2.0
An implementation of the Storage API in Rust
License: Apache License 2.0
Currently, release 0.3 is waiting on a couple things, hopefully:
JustMetadata<T>
: Allows implementing CoerceUnsized on most handle types - rust-lang/rust#97052Handle<T, for<T> This<T> = Self::Handle<T>>
to be added as a bound to handlesCurrently, GATs and Pointer Metadata are the two biggest nightly features we need.
GATs are planned to stabilize in the not-too-distant future. This is a major step towards the crate compiling on stable, and will allow removing the GAT feature. Would like to have them stable before the next big release.
Relevant PR: rust-lang/rust#96709
Pointer Metadata has proposals to stabilize some types, but looks to still be a while off. This won't block the next release, but I intend to keep an eye on any movement in this area.
Oh hi! I hadn't seen this until I saw it pop up in rust-lang/rust#97052.
I'm part of alloc-wg and about two months ago I picked up the Storage API again in this repo: https://github.com/CAD97/storages-api (docs: https://cad97.github.io/storages-api/storage_api/index.html)
As part of this refinement and simplification push, I ended up converting the Storage
trait to be untyped: that is, Storage::Handle
is no longer a GAT, and notably this means that handle resolution can return regular references to [MaybeUninit<u8>]
rather than pointers with difficult to explain invalidation rules.
There's some miscellaneous renaming involved, but also the simplification brought us down to a set of just four traits: Storage
, SharedMutabilityStorage
, PinningStorage
, and MultipleStorage
. I've been advised (and agree) that controlling complexity is important here for a path to std inclusion; even with this simplified API surface without GAT, it's still a contender for the most complex API exposed by std. And that complexity is safety critical.
I'm fairly convinced that the current shape in my repo is minimal for what we want to support, but I wasn't thinking the previous POC was terribly over-engineered1 either, so it's possible that a further simplification is hiding.
matthieu-m agrees that the untyping of storage is probably a good direction; the previous storage API was not typed due to some conscious decision due to necessity, but because it evolved from C++'s std::allocator
design, which is typed, and matthiue-m just hadn't considered using an untyped interface. This does mean that usage of the storage traits becomes more unsafe
/difficult, but this is easily addressed by adding a typed RawBox
around Storage
, RawVec
around Storage
of slices, and maybe RawArena
around MultipleStorage
? as a bridge between raw storage and high-level collections.
(It's also quite possible that matthieu-m overlooked the possibility of making handles untyped because of the relative newness of the pointee metadata APIs at the time, in order to draw the conclusion that S::Handle<T>
can be replaced by (S::Handle, <T as Pointee>::Metadata)
.)
Anyway, getting to the point:
I received a Rust Foundation Community Grant. My project is sort of two projects; the first is unrelated work in the tracing ecosystem, but the second half (scheduled Oct..=Dec) is to develop and push the Storage API to an MCP. rust-lang/rust#97052 is actually an early start on prerequisite work for upstreaming storages, as otherwise a storage-generic Box
can't be coerced.
pub struct RawBox<T: ?Sized, S: Storage> {
handle: S::Handle,
metadata: <T as Pointee>::Metadata, // -> JustMetadata<T>
storage: S,
}
My plan for MCPing the Storage API is basically broken down into the following three parallel steps:
#[no_std]
compatible, so while refactoring in-place has the benefits of full API coverage and ease of "build with my fork of std" testing, wanting to move things into libcore means the diff would be little better than an external crate. Plus having to deal with std iteration times.tower::Service
" article.Allocator
trait.Ultimately, I think that this MCP is where the Storage API either sinks or swims. Either it gets accepted for in-tree implementation/experimentation, or it gets rejected, likely on the matter of being insufficiently motivated for std inclusion given its complexity. Especially given the async-wg group w.r.t. async in traits (e.g. dyn*
), now really does feel like the time to say if we want to make Box
the "owning handle to some storage" or if it's just an owning pointer to allocated memory. My claim is that Box<T, ?>
can be both &move T
and dyn* T
, as well as InlineMaxSizeDst<T>
and so much more, so its a matter of proving it worth it.
I'd like to work together, or at the very least avoid duplicating work you've already done unnecessarily. Adding your experience working with (your modified copy of) the storage API will help to strengthen the motivation provided in the MCP.
...I don't really know how to end this info dump. I guess...
TL;DR: Hi ๐ I'm working on the Storage API and would like to incorporate your experience working with it.
Even back then I disagreed on some details with matthieu-m, which ended up being integrated into the most recent iteration. Notably, matthieu-m had separate API trees for "Element
" and "Range
" storage, but I've always advocated for "Range
" storage to just be "Element
" storage of an array/slice. Along with this difference is that matthieu-m's element storage takes T
to place it, whereas mine previously took <T as Pointee>::Metadata
(when it was still typed) and now just deals in Layout
. โฉ
Currently, only a small number of handles can be unsized. Preferably, all handles should be coerce-able. This requires some Rust features though:
JustMetadata<T>
: Allows MetaHandle and OffsetMetaHandle to be unsized - rust-lang/rust#97052This is currently blocked on a lack of for<T>
or similar style of bound. The metadata
and addr
methods work fine generically, but anything using casts immediately runs into unproveable bounds, so it won't be added as a bound until those can be resolved somehow.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.