rust bindings fail to build about cachelib

[leozzx]

Hi @brayniac, I'm not familiar with rust, but it is complaining cachelib/rust/src/cachelib.h not found. The file does exist, so can you try to build in the root directory (CacheLib/) instead of rust directory?

[brayniac]

@leozzx - unfortunately that won't work because the repo root is not a crate or workspace root.

CacheLib$ cargo build --release
error: could not find `Cargo.toml` in `/home/brian/CacheLib` or any parent directory

[farnz]

Can you try and teach the cc::Build returned by the build.rs that /home/brian/CacheLib is a root for includes? I think this would involve constructing a Path using the result of std::env::current_dir and passing it to the include method on the Build object.

[therealgymmy]

cc @agordon for advice.

[brayniac]

@therealgymmy @agordon @farnz - any updates on this?

[journaux]

"Can you try and teach the [cc::Build](https://docs.rs/cc/1.0.72/cc/struct.Build.html) returned by the build.rs that /home/brian/CacheLib is a root for includes? I think this would involve constructing a Path using the result of [std::env::current_dir](https://doc.rust-lang.org/std/env/fn.current_dir.html) and passing it to the include method on the Build object."

@farnz resolves the first issue,
then in debian/ubuntu missing dependency headers forms new errors

  cargo:warning=/CacheLib/cachelib/rust/../../cachelib/rust/src/cachelib.h:24:10: fatal error: cachelib/facebook/admin/CacheAdmin.h: No such file or directory
  cargo:warning=   24 | #include "cachelib/facebook/admin/CacheAdmin.h"
  cargo:warning=      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  cargo:warning=compilation terminated.

when removing the admin (a private facebook header file), the build fails on lack of rust bindings for folly string type.

i removed the references to std::chrono::timestamp & folly::String types, replacing them with primitives i64 & std::string respectively in order to achieve a successful compilation

by example

all in all => need to have the folly rust bindings in order to compile cachelib - otherwise, the community can not implement Cxx Extern Type for ffi.

more than happy to help if a contributor wants to hop on a video call so that we can consume cachelib in rust/c.

thank u for a great library. the only high quality, multi threaded caching library for ssd we've found. twitter pelikan single threaded & does not support 10TB ssd drives.

[therealgymmy]

@vitruvvius: thanks for figuring out a fix! Could you send out a PR?

[journaux]

i'd rather see folly & std::chrono::timestamp resolved w/ FFI bindings than the strategy provided

[journaux]

what are we doing about this, if at all?
happy to contribute if the facebook team ok w/ transitioning off folly/chrono dependency on the ffi,
albeit i doubt that they want to fork + maintain an oss mirrior.

[journaux]

one thing that might be interesting is to move away from hand-writing the FFI & integrating autocxx from google https://github.com/google/autocxx

[nickamorim]

Even with the change proposed by @journaux, the Rust bindings still fail to build.

  cargo:warning=In file included from /Users/nickamorim/src/github.com/Shopify/CacheLib/cachelib/rust/src/cachelib.h:22:0,

  cargo:warning=                 from /Users/nickamorim/src/github.com/Shopify/CacheLib/cachelib/rust/target/debug/build/cachelib-7bbe50bf6a183fab/out/cxxbridge/sources/cachelib/src/lib.rs.cc:1:

  cargo:warning=/Users/nickamorim/src/github.com/Shopify/CacheLib/cachelib/allocator/CacheAllocator.h:19:10: fatal error: folly/CPortability.h: No such file or directory

  cargo:warning= #include <folly/CPortability.h>

  cargo:warning=          ^~~~~~~~~~~~~~~~~~~~~~

  cargo:warning=compilation terminated.

  exit status: 1

  --- stderr

  CXX include path:
    /Users/nickamorim/src/github.com/Shopify/CacheLib/cachelib/rust/target/debug/build/cachelib-7bbe50bf6a183fab/out/cxxbridge/include
    /Users/nickamorim/src/github.com/Shopify/CacheLib/cachelib/rust/target/debug/build/cachelib-7bbe50bf6a183fab/out/cxxbridge/crate


  error occurred: Command "c++" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "-m64" "-I" "/Users/nickamorim/src/github.com/Shopify/CacheLib/cachelib/rust/target/debug/build/cachelib-7bbe50bf6a183fab/out/cxxbridge/include" "-I" "/Users/nickamorim/src/github.com/Shopify/CacheLib/cachelib/rust/target/debug/build/cachelib-7bbe50bf6a183fab/out/cxxbridge/crate" "-I" "/Users/nickamorim/src/github.com/Shopify/CacheLib" "-Wall" "-Wextra" "-std=c++11" "-o" "/Users/nickamorim/src/github.com/Shopify/CacheLib/cachelib/rust/target/debug/build/cachelib-7bbe50bf6a183fab/out/2198180abe54d98b-lib.rs.o" "-c" "/Users/nickamorim/src/github.com/Shopify/CacheLib/cachelib/rust/target/debug/build/cachelib-7bbe50bf6a183fab/out/cxxbridge/sources/cachelib/src/lib.rs.cc" with args "c++" did not execute successfully (status code exit status: 1).

[journaux]

@nickamorim that seems a folly version / include path bug, nonetheless a lot of adjustments needed circa 8 months ago to compile properly (watch out for stack smashing in rust :: c++ ffi). anecdotally, somebody informed me facebook never released their internal cache eviction policy for cachelib, implying the public implementation does not match the private deployment.