gilnaa / memoffset Goto Github PK

I know it's hard/impossible right now, but are you tracking the ongoing const_fn changes to ensure that offset_of! can be used in constants at some point? It's usually very useful in tandem with size_of(), which is already const fn.

Playground containing an extract of the code from the current master (d3f5f23)

#[repr(C)]
struct Foo {
    a: core::sync::atomic::AtomicUsize,
}

assert_eq!([0; offset_of!(Foo, a)].len(), 0);

Building this fails with the following error:

error[E0492]: cannot borrow a constant which may contain interior mutability, create a static instead
  --> src/offset_of:112:66
    |
112 |             let base_ref = ::core::mem::transmute::<_, &$parent>(&uninit);
    |                                                                  ^^^^^^^
...
NNN |     assert_eq!([0; offset_of!(Foo, a)].len(), 0);
    |                    ------------------ in this macro invocation

Note that there is no error if Foo::a is usize instead.

e.g. #26, #28

I will report https://www.cvedetails.com/cve/CVE-2019-15553/ to the Rust security advisory database. So it becomes possible to automatically detect this CVE in Rust CIs. I create this issue to track it.

memoffset 0.5.1 and 0.5.3 cause clippy to warn in the consumer's crate. This should either be fixed or suppressed within memoffset.

warning: All the struct fields are matched to a wildcard pattern, consider using `..`.
  --> bfffs-fio/src/lib.rs:95:17
   |
95 |                 offset_of!(BfffsOptions, pool_name),
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(clippy::unneeded_field_pattern)]` on by default
   = help: Try with `BfffsOptions { .. }` instead
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unneeded_field_pattern
   = note: this warning originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

Hi! Thank you for this crate. I was going to update it in our dependency tree and wanted to check what the changes were. But I'm not able to find a changelog. I can of course not be lazy and do a git diff. But having a human readable changelog is really nice. A changelog can outline caveats to not miss and other things that might not be obvious from reading just the code diff. So this is just a friendly request to have one added :)

https://keepachangelog.com/en/1.0.0/

We should have compile-fail tests (as doctests, I heard that's possible) to make sure we properly check for deref coercions, and whatever else might be relevant.

The feature const_fn is no longer supported on Rust nightly, hence memoffset 0.6.3 no longer compiles when using the unstable_const feature.

memoffset does not work with unions. Would it be possible to add this feature?
For example, this code:

use memoffset::offset_of;

pub union Foo {
    foo32: i32,
    foo64: i64
}

#[test]
fn alignment() {
    assert_eq!(offset_of!(Foo, foo32), offset_of!(Foo, foo64));
}

Produces this result:

error: `..` cannot be used in union patterns
  --> test/memoffset.rs:10:16
   |
10 |     assert_eq!(offset_of!(Foo, foo32), offset_of!(Foo, foo64));
   |                ^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: this error originates in the macro `_memoffset__field_check` (in Nightly builds, run with -Z macro-backtrace for more info)

error: `..` cannot be used in union patterns
  --> test/memoffset.rs:10:40
   |
10 |     assert_eq!(offset_of!(Foo, foo32), offset_of!(Foo, foo64));
   |                                        ^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: this error originates in the macro `_memoffset__field_check` (in Nightly builds, run with -Z macro-backtrace for more info)

offset_of! (and probably the other macros in this crate) consumes too much stack space in dev build, especially when the type is big. It's probably because it's declaring a (uninitialized) variable of the type. It's optimized out in the release build.

It affects my workflow in developing an OS in Rust. Its dev build won't boot because offset_of! of a big struct (2KiB) used up the limited amount of kernel stack (4KiB), incurring stack overflow.

Some crates.io releases do not have corresponding tags in the repository: 0.5.1, 0.2.1, 0.2.0.

Previously on memoffset:

• #4

Constant evaluation mostly stable with the exception of the inability to take address of a Cell in const which still requires nightly and #![feature(const_refs_to_cell)].

The primary macro implementations in memoffset 0.5 use other secondary macros, but don't scope them with $crate (or something like that, I don't actually know what the proper syntax is).

This causes a problem at the use site: if only the main macro is imported, the macro invocation expands to code that won't compile.

use memoffset::{offset_of, span_of};

struct Hmm {
    mmm: u32,
}

fn main() {
    println!("{}", offset_of!(Hmm, mmm));
    println!("{}", span_of!(Hmm, mmm));
}

error: cannot find macro `field_check!` in this scope
 --> src/main.rs:8:20
  |
8 |     println!("{}", offset_of!(Hmm, mmm));
  |                    ^^^^^^^^^^^^^^^^^^^^
  |
  = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: cannot find macro `let_base_ptr!` in this scope
 --> src/main.rs:8:20
  |
8 |     println!("{}", offset_of!(Hmm, mmm));
  |                    ^^^^^^^^^^^^^^^^^^^^
  |
  = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: cannot find macro `let_base_ptr!` in this scope
 --> src/main.rs:9:20
  |
9 |     println!("{}", span_of!(Hmm, mmm));
  |                    ^^^^^^^^^^^^^^^^^^
  |
  = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: cannot find macro `field_check!` in this scope
 --> src/main.rs:9:20
  |
9 |     println!("{}", span_of!(Hmm, mmm));
  |                    ^^^^^^^^^^^^^^^^^^
  |
  = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error[E0425]: cannot find value `base_ptr` in this scope
 --> src/main.rs:8:20
  |
8 |     println!("{}", offset_of!(Hmm, mmm));
  |                    ^^^^^^^^^^^^^^^^^^^^ not found in this scope
  |
  = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error[E0425]: cannot find value `base_ptr` in this scope
 --> src/main.rs:8:20
  |
8 |     println!("{}", offset_of!(Hmm, mmm));
  |                    ^^^^^^^^^^^^^^^^^^^^ not found in this scope
  |
  = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error[E0425]: cannot find value `root` in this scope
 --> src/main.rs:9:20
  |
9 |     println!("{}", span_of!(Hmm, mmm));
  |                    ^^^^^^^^^^^^^^^^^^ not found in this scope
  |
  = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error[E0425]: cannot find value `root` in this scope
 --> src/main.rs:9:20
  |
9 |     println!("{}", span_of!(Hmm, mmm));
  |                    ^^^^^^^^^^^^^^^^^^ not found in this scope
  |
  = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: aborting due to 8 previous errors

Travis stopped working for a few projects of mine, it can probably stop working any day here as well. (Looks like projects are granted a specific number of "credits" overall and once those are out, it's over.)

We should probably switch to GHA. I will prepare a PR.

It's very likely that https://docs.rs/autocfg/1.0.0/autocfg/ becomes the crate for rustc feature detection:

• it's 1.0
• it's maintained by a high-profile Rust contributor
• it is based on a generally better patrern of feature detection (as opposed to version sniffing)

It might make sense to replace rustc_version with autocfg

memoffset does not compile with actual nightly:

error[E0635]: unknown feature const_transmute

Version 0.4 contains the union-transmute-based version of span_of corresponding to the offset_of in the yanked 0.3. Would it be a good idea to yank 0.4 as well?

Current implementation creates a dummy object. It's possible to get alignment with just pointer manipulation.

        let base = std::mem::align_of::<Foo>(); // First valid non-NULL pointer address
        let ptr = base as *const Foo;
        assert_eq!(0, (&(*ptr).field) as *const _ as usize - base);

I think &(*ptr) is valid and does not actually deference the object, because such construct is also allowed for non-movable values (where *obj causes an error, but &*obj is fine).

There's a cfg(memoffset_constant_expression) in there that never applies.

With the implementation in:

macro_rules! offset_of {
    ($parent:ty, $($field:tt)+) => (unsafe {
        let x: &'static $parent = $crate::Transmuter::<$parent> { int: 0 }.ptr;
        $crate::Transmuter { ptr: &x.$($field)+ }.int
    });
}

you are constructing a &'static T at address 0. This is undefined behavior because Rust may assume that you have a valid &'static T but you do not.

Moreover, in the old pre-1.33.0 implementation you have:

macro_rules! offset_of {
    ($father:ty, $($field:tt)+) => ({
        #[allow(unused_unsafe)]
        let root: $father = unsafe { $crate::mem::uninitialized() };

        let base = &root as *const _ as usize;

        // Future error: borrow of packed field requires unsafe function or block (error E0133)
        #[allow(unused_unsafe)]
        let member =  unsafe { &root.$($field)* as *const _ as usize };

        $crate::mem::forget(root);

        member - base
    });
}

Note that when you say &root you have already triggered undefined behavior because again, you assert with &root that you have a valid reference but you do not in fact.

See rust-lang/rfcs#2582 and rust-lang/unsafe-code-guidelines#77 for a discussion.

As for your comment about // Future error, do note that unsafe { ... } here does not make what you are doing any less undefined behavior. Instead, while you might not get an error from the compiler, you might get miscompilation should LLVM or rustc's behavior change.

cc @RalfJung

offset_of is UB because we are creating a reference to a field that has not been initialized yet. (Miri does not currently check references transitively, so tests still pass in Miri.) There is currently no way to avoid this UB when implementing offset_of!. progress is blocked on rust-lang/rfcs#2582.

We are also "more UB" for older versions of rustc, because we are dereferencing a dangling pointer. We only do it to compute a field address, so no memory access happens, but the dereference is enough to cause UB.

See #9 for other ways in which there used to be "more UB".

[This is perhaps more of a documentation request - it's information I might have expected to find in the README]

I understand that Rust generally does not make any guarantees regarding the layout of structs. The book states:

Type layout can be changed with each compilation. Instead of trying to document exactly what is done, we only document what is guaranteed today.

If you apply offset_of! to a struct with the default representation, do you get garbage, or do you get whatever the offset is under the current compiler and current compilation pass?

span_of creates a reference to an uninitialized field for size_of_val.

We should switch to a different implementation that works with raw pointers. It doesn't even have to support unsized types (we just need the type inference), so we could use

fn size_of_pointee<T>(_ptr: *const T) -> T {
  mem::size_of::<T>()
}

This doesn't work with memoffset:

struct Foo {
    bar: f32,
    baz: [f32],
}
offset_of!(Foo, bar);

Even though this (unsound) code works:

struct Foo {
    bar: f32,
    baz: [f32],
}
let foo_ptr: &Foo = unsafe { mem::zeroed() };
let foo_offs = (&foo_ptr.bar) as *const _ as usize;

I ran into this problem when trying to port glium to memoffset in glium/glium#1782. The gpgpu example has relied on offset of calculations supporting unsized structs. I could work around the issue by changing the gpgpu example, but that isn't really perfect.

It would be cool to get a release out which includes #50 , now that 1.51 is released.