Coder Social home page Coder Social logo

redislabsmodules / redismodule-rs Goto Github PK

View Code? Open in Web Editor NEW
267.0 12.0 62.0 932 KB

Rust API for Redis Modules API

License: BSD 3-Clause "New" or "Revised" License

Rust 75.30% C 23.33% Shell 0.14% Dockerfile 0.27% Makefile 0.58% Python 0.39%
redis rust-lang redis-modules redismodule-rs rust

redismodule-rs's Introduction

license Releases crates.io docs CircleCI

redismodule-rs

This crate provides an idiomatic Rust API for the Redis Modules API. It allows writing Redis modules in Rust, without needing to use raw pointers or unsafe code. See here for the most recent API documentation.

Running the example module

  1. Install Rust
  2. Install Redis, most likely using your favorite package manager (Homebrew on Mac, APT or YUM on Linux)
  3. Run cargo build --example hello
  4. Start a redis server with the hello module
    • Linux: redis-server --loadmodule ./target/debug/examples/libhello.so
    • Mac: redis-server --loadmodule ./target/debug/examples/libhello.dylib
  5. Open a Redis CLI, and run HELLO.MUL 31 11.

Writing your own module

See the examples directory for some sample modules.

This crate tries to provide high-level wrappers around the standard Redis Modules API, while preserving the API's basic concepts. Therefore, following the Redis Modules API documentation will be mostly relevant here as well.

Redis Modules based on this crate

The following are some modules that are built on this crate:

Several member of the community have written their own modules based on this.

redismodule-rs's People

Contributors

alonre24 avatar anton-tokarev-softeq avatar chayim avatar danni-m avatar dependabot-preview[bot] avatar dependabot[bot] avatar dvirdukhan avatar edman avatar gavrie avatar gkorland avatar grippy avatar iddm avatar kerollmops avatar matobet avatar meirshpilraien avatar messense avatar oshadmi avatar phartleyp avatar prashanthpai avatar quchen88 avatar rafie avatar rcelha avatar realaravinth avatar siscia avatar slavak avatar socs avatar yihuang avatar yoav-steinberg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

redismodule-rs's Issues

Please document init

It looks like init is defined within the redis_module! macro but I don't see it documented anywhere. My module requires background timers, and so it would be ideal to start this using init, though this can also be done with a startup command

Compilation error on aarch64

I was trying to compile a module on a Raspberry Pi(aarch64) and I encountered the following error:

  --> /srv/mcaptchaguard/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.21.0/src/context/mod.rs:175:21
    |
175 |                     s.as_ptr() as *const i8,
    |                     ^^^^^^^^^^^^^^^^^^^^^^^ expected `u8`, found `i8`
    |
    = note: expected raw pointer `*const u8`
               found raw pointer `*const i8`

I applied the suggested fix and the compilation succeeded.

Rust doc says std::os::raw::c_char can either be i8 or u8 but I don't understand why i8 would fail to compile.

Here's the module source code


The same module compiles fine on x86_64.

`cargo test` panics on crates with dependency on `redis_module`

It appears that just the presence of #[macro_use] extern crate redis_module causes the resulting binary produced by cargo test to panic. I had the intention to write some integration tests for my Redis module in Rust and have them run by cargo test but this essentially prevents any testing whatsoever.

Steps to reproduce

Create a new library project

$ cargo new --lib mod_test

Tests pass

$ cd mod_test
$ cargo test
...

running 1 test
test tests::it_works ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

   Doc-tests mod_test

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

Add dependency on redis_module and crate import to src/lib.rs.

$ cat Cargo.toml
[package]
name = "mod_test"
version = "0.1.0"
authors = ["Martin Betak <[email protected]>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
redis-module = "0.9"
$ cat src/lib.rs
#[macro_use] extern crate redis_module;

#[cfg(test)]
mod tests {
    #[test]
    fn it_works() {
        assert_eq!(2 + 2, 4);
    }
}

Tests panic

    Finished test [unoptimized + debuginfo] target(s) in 0.13s
     Running target/debug/deps/mod_test-278b37354b464b59
thread panicked while processing panic. aborting.
error: test failed, to rerun pass '--lib'

Caused by:
  process didn't exit successfully: `/home/matobet/projects/rust/mod_test/target/debug/deps/mod_test-278b37354b464b59` (signal: 4, SIGILL: illegal instruction)

Curious, how production ready is this? Another question...

Great work by the way.

  1. how production ready is this?
  2. redis is single threaded, rust is multiproc, so when ran, does that mean the rust part can do multiproc but overall, the app is still single threaded? the "bottleneck" will be at redis side but i think it's very minimal or not worth the consideration?

Sample not building on RaspberriPi

I:

  1. Installed clang9
  2. Installed rust
  3. downloaded redis 6.0
  4. cloned the redismodule-rs repo
  5. tried to build following the read.me instructions.

No luck

pi@raspberrypiB:~/redis/redis-6.0.10/rust/redismodule-rs $ ./build.sh
Compiling redis-module v0.12.0 (/home/pi/redis/redis-6.0.10/rust/redismodule-rs)
error: failed to run custom build command for redis-module v0.12.0 (/home/pi/redis/redis-6.0.10/rust/redismodule-rs)

Caused by:
process didn't exit successfully: /home/pi/redis/redis-6.0.10/rust/redismodule-rs/target/debug/build/redis-module-1b4fff876549888d/build-script-build (exit code: 101)
--- stdout
TARGET = Some("armv7-unknown-linux-gnueabihf")
OPT_LEVEL = Some("0")
HOST = Some("armv7-unknown-linux-gnueabihf")
CC_armv7-unknown-linux-gnueabihf = None
CC_armv7_unknown_linux_gnueabihf = None
HOST_CC = None
CC = None
CFLAGS_armv7-unknown-linux-gnueabihf = None
CFLAGS_armv7_unknown_linux_gnueabihf = None
HOST_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("true")
CARGO_CFG_TARGET_FEATURE = None
running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-march=armv7-a" "-I" "src/include/" "-Wall" "-Wextra" "-DREDISMODULE_EXPERIMENTAL_API" "-o" "/home/pi/redis/redis-6.0.10/rust/redismodule-rs/target/debug/build/redis-module-2b3d6b368d48fecf/out/src/redismodule.o" "-c" "src/redismodule.c"
exit code: 0
AR_armv7-unknown-linux-gnueabihf = None
AR_armv7_unknown_linux_gnueabihf = None
HOST_AR = None
AR = None
running: "ar" "cq" "/home/pi/redis/redis-6.0.10/rust/redismodule-rs/target/debug/build/redis-module-2b3d6b368d48fecf/out/libredismodule.a" "/home/pi/redis/redis-6.0.10/rust/redismodule-rs/target/debug/build/redis-module-2b3d6b368d48fecf/out/src/redismodule.o"
exit code: 0
running: "ar" "s" "/home/pi/redis/redis-6.0.10/rust/redismodule-rs/target/debug/build/redis-module-2b3d6b368d48fecf/out/libredismodule.a"
exit code: 0
cargo:rustc-link-lib=static=redismodule
cargo:rustc-link-search=native=/home/pi/redis/redis-6.0.10/rust/redismodule-rs/target/debug/build/redis-module-2b3d6b368d48fecf/out

--- stderr
/usr/include/arm-linux-gnueabihf/sys/types.h:144:10: fatal error: 'stddef.h' file not found
/usr/include/arm-linux-gnueabihf/sys/types.h:144:10: fatal error: 'stddef.h' file not found, err: true
thread 'main' panicked at 'error generating bindings: ()', build.rs:76:10
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
pi@raspberrypiB:~/redis/redis-6.0.10/rust/redismodule-rs $

Functions suppressing not_unsafe_ptr_arg_deref clippy lint are unsound

All of these functions supressing the clippy lint clippy::not_unsafe_ptr_arg_deref seem to be trivially unsound:

redismodule-rs/src/raw.rs

Lines 185 to 591 in 56a8082

#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn call_reply_type(reply: *mut RedisModuleCallReply) -> ReplyType {
unsafe {
// TODO: Cache the unwrapped functions and use them instead of unwrapping every time?
RedisModule_CallReplyType.unwrap()(reply).into()
}
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn free_call_reply(reply: *mut RedisModuleCallReply) {
unsafe { RedisModule_FreeCallReply.unwrap()(reply) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn call_reply_integer(reply: *mut RedisModuleCallReply) -> c_longlong {
unsafe { RedisModule_CallReplyInteger.unwrap()(reply) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn call_reply_array_element(
reply: *mut RedisModuleCallReply,
idx: usize,
) -> *mut RedisModuleCallReply {
unsafe { RedisModule_CallReplyArrayElement.unwrap()(reply, idx) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn call_reply_length(reply: *mut RedisModuleCallReply) -> usize {
unsafe { RedisModule_CallReplyLength.unwrap()(reply) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn call_reply_string_ptr(reply: *mut RedisModuleCallReply, len: *mut size_t) -> *const c_char {
unsafe { RedisModule_CallReplyStringPtr.unwrap()(reply, len) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn call_reply_string(reply: *mut RedisModuleCallReply) -> String {
unsafe {
let mut len: size_t = 0;
let reply_string: *mut u8 =
RedisModule_CallReplyStringPtr.unwrap()(reply, &mut len) as *mut u8;
String::from_utf8(
slice::from_raw_parts(reply_string, len)
.iter()
.copied()
.collect(),
)
.unwrap()
}
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn close_key(kp: *mut RedisModuleKey) {
unsafe { RedisModule_CloseKey.unwrap()(kp) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn open_key(
ctx: *mut RedisModuleCtx,
keyname: *mut RedisModuleString,
mode: KeyMode,
) -> *mut RedisModuleKey {
unsafe { RedisModule_OpenKey.unwrap()(ctx, keyname, mode.bits).cast::<RedisModuleKey>() }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn reply_with_array(ctx: *mut RedisModuleCtx, len: c_long) -> Status {
unsafe { RedisModule_ReplyWithArray.unwrap()(ctx, len).into() }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn reply_with_error(ctx: *mut RedisModuleCtx, err: *const c_char) {
unsafe {
let msg = Context::str_as_legal_resp_string(CStr::from_ptr(err).to_str().unwrap());
RedisModule_ReplyWithError.unwrap()(ctx, msg.as_ptr());
}
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn reply_with_long_long(ctx: *mut RedisModuleCtx, ll: c_longlong) -> Status {
unsafe { RedisModule_ReplyWithLongLong.unwrap()(ctx, ll).into() }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn reply_with_double(ctx: *mut RedisModuleCtx, f: c_double) -> Status {
unsafe { RedisModule_ReplyWithDouble.unwrap()(ctx, f).into() }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn reply_with_string(ctx: *mut RedisModuleCtx, s: *mut RedisModuleString) -> Status {
unsafe { RedisModule_ReplyWithString.unwrap()(ctx, s).into() }
}
// Sets the expiry on a key.
//
// Expire is in milliseconds.
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn set_expire(key: *mut RedisModuleKey, expire: c_longlong) -> Status {
unsafe { RedisModule_SetExpire.unwrap()(key, expire).into() }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn string_dma(key: *mut RedisModuleKey, len: *mut size_t, mode: KeyMode) -> *const c_char {
unsafe { RedisModule_StringDMA.unwrap()(key, len, mode.bits) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn hash_get_multi<T>(
key: *mut RedisModuleKey,
fields: &[T],
values: &mut [*mut RedisModuleString],
) -> Result<(), RedisError>
where
T: Into<Vec<u8>> + Clone,
{
assert_eq!(fields.len(), values.len());
let mut fi = fields.iter();
let mut vi = values.iter_mut();
macro_rules! rm {
() => { unsafe {
RedisModule_HashGet.unwrap()(key, REDISMODULE_HASH_CFIELDS as i32,
ptr::null::<c_char>())
}};
($($args:expr)*) => { unsafe {
RedisModule_HashGet.unwrap()(
key, REDISMODULE_HASH_CFIELDS as i32,
$($args),*,
ptr::null::<c_char>()
)
}};
}
macro_rules! f {
() => {
CString::new((*fi.next().unwrap()).clone())
.unwrap()
.as_ptr()
};
}
macro_rules! v {
() => {
vi.next().unwrap()
};
}
// This convoluted code is necessary since Redis only exposes a varargs API for HashGet
// to modules. Unfortunately there's no straightforward or portable way of calling a
// a varargs function with a variable number of arguments that is determined at runtime.
// See also the following Redis ticket: https://github.com/redis/redis/issues/7860
let res = Status::from(match fields.len() {
0 => rm! {},
1 => rm! {f!() v!()},
2 => rm! {f!() v!() f!() v!()},
3 => rm! {f!() v!() f!() v!() f!() v!()},
4 => rm! {f!() v!() f!() v!() f!() v!() f!() v!()},
5 => rm! {f!() v!() f!() v!() f!() v!() f!() v!() f!() v!()},
6 => rm! {f!() v!() f!() v!() f!() v!() f!() v!() f!() v!() f!() v!()},
7 => rm! {
f!() v!() f!() v!() f!() v!() f!() v!() f!() v!() f!() v!()
f!() v!()
},
8 => rm! {
f!() v!() f!() v!() f!() v!() f!() v!() f!() v!() f!() v!()
f!() v!() f!() v!()
},
9 => rm! {
f!() v!() f!() v!() f!() v!() f!() v!() f!() v!() f!() v!()
f!() v!() f!() v!() f!() v!()
},
10 => rm! {
f!() v!() f!() v!() f!() v!() f!() v!() f!() v!() f!() v!()
f!() v!() f!() v!() f!() v!() f!() v!()
},
11 => rm! {
f!() v!() f!() v!() f!() v!() f!() v!() f!() v!() f!() v!()
f!() v!() f!() v!() f!() v!() f!() v!() f!() v!()
},
12 => rm! {
f!() v!() f!() v!() f!() v!() f!() v!() f!() v!() f!() v!()
f!() v!() f!() v!() f!() v!() f!() v!() f!() v!() f!() v!()
},
_ => panic!("Unsupported length"),
});
match res {
Status::Ok => Ok(()),
Status::Err => Err(RedisError::Str("ERR key is not a hash value")),
}
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn hash_set(key: *mut RedisModuleKey, field: &str, value: *mut RedisModuleString) -> Status {
let field = CString::new(field).unwrap();
unsafe {
RedisModule_HashSet.unwrap()(
key,
REDISMODULE_HASH_CFIELDS as i32,
field.as_ptr(),
value,
ptr::null::<c_char>(),
)
.into()
}
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn hash_del(key: *mut RedisModuleKey, field: &str) -> Status {
let field = CString::new(field).unwrap();
// TODO: Add hash_del_multi()
// Support to pass multiple fields is desired but is complicated.
// See hash_get_multi() and https://github.com/redis/redis/issues/7860
unsafe {
RedisModule_HashSet.unwrap()(
key,
REDISMODULE_HASH_CFIELDS as i32,
field.as_ptr(),
REDISMODULE_HASH_DELETE,
ptr::null::<c_char>(),
)
.into()
}
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn list_push(
key: *mut RedisModuleKey,
list_where: Where,
element: *mut RedisModuleString,
) -> Status {
unsafe { RedisModule_ListPush.unwrap()(key, list_where as i32, element).into() }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn list_pop(key: *mut RedisModuleKey, list_where: Where) -> *mut RedisModuleString {
unsafe { RedisModule_ListPop.unwrap()(key, list_where as i32) }
}
// Returns pointer to the C string, and sets len to its length
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn string_ptr_len(s: *const RedisModuleString, len: *mut size_t) -> *const c_char {
unsafe { RedisModule_StringPtrLen.unwrap()(s, len) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn string_retain_string(ctx: *mut RedisModuleCtx, s: *mut RedisModuleString) {
unsafe { RedisModule_RetainString.unwrap()(ctx, s) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn string_to_longlong(s: *const RedisModuleString, len: *mut i64) -> Status {
unsafe { RedisModule_StringToLongLong.unwrap()(s, len).into() }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn string_to_double(s: *const RedisModuleString, len: *mut f64) -> Status {
unsafe { RedisModule_StringToDouble.unwrap()(s, len).into() }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn string_set(key: *mut RedisModuleKey, s: *mut RedisModuleString) -> Status {
unsafe { RedisModule_StringSet.unwrap()(key, s).into() }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn replicate_verbatim(ctx: *mut RedisModuleCtx) -> Status {
unsafe { RedisModule_ReplicateVerbatim.unwrap()(ctx).into() }
}
fn load<F, T>(rdb: *mut RedisModuleIO, f: F) -> Result<T, Error>
where
F: FnOnce(*mut RedisModuleIO) -> T,
{
let res = f(rdb);
if is_io_error(rdb) {
Err(RedisError::short_read().into())
} else {
Ok(res)
}
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn load_unsigned(rdb: *mut RedisModuleIO) -> Result<u64, Error> {
unsafe { load(rdb, |rdb| RedisModule_LoadUnsigned.unwrap()(rdb)) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn load_signed(rdb: *mut RedisModuleIO) -> Result<i64, Error> {
unsafe { load(rdb, |rdb| RedisModule_LoadSigned.unwrap()(rdb)) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn load_string(rdb: *mut RedisModuleIO) -> Result<RedisString, Error> {
let p = unsafe { load(rdb, |rdb| RedisModule_LoadString.unwrap()(rdb))? };
let ctx = unsafe { RedisModule_GetContextFromIO.unwrap()(rdb) };
Ok(RedisString::from_redis_module_string(ctx, p))
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn load_string_buffer(rdb: *mut RedisModuleIO) -> Result<RedisBuffer, Error> {
unsafe {
let mut len = 0;
let buffer = load(rdb, |rdb| {
RedisModule_LoadStringBuffer.unwrap()(rdb, &mut len)
})?;
Ok(RedisBuffer::new(buffer, len))
}
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn replicate(ctx: *mut RedisModuleCtx, command: &str, args: &[&str]) -> Status {
let terminated_args: Vec<RedisString> =
args.iter().map(|s| RedisString::create(ctx, s)).collect();
let inner_args: Vec<*mut RedisModuleString> = terminated_args.iter().map(|s| s.inner).collect();
let cmd = CString::new(command).unwrap();
unsafe {
RedisModule_Replicate.unwrap()(
ctx,
cmd.as_ptr(),
FMT,
inner_args.as_ptr() as *mut c_char,
terminated_args.len(),
)
.into()
}
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn load_double(rdb: *mut RedisModuleIO) -> Result<f64, Error> {
unsafe { load(rdb, |rdb| RedisModule_LoadDouble.unwrap()(rdb)) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn load_float(rdb: *mut RedisModuleIO) -> Result<f32, Error> {
unsafe { load(rdb, |rdb| RedisModule_LoadFloat.unwrap()(rdb)) }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn save_string(rdb: *mut RedisModuleIO, buf: &str) {
unsafe { RedisModule_SaveStringBuffer.unwrap()(rdb, buf.as_ptr().cast::<c_char>(), buf.len()) };
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn save_double(rdb: *mut RedisModuleIO, val: f64) {
unsafe { RedisModule_SaveDouble.unwrap()(rdb, val) };
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn save_signed(rdb: *mut RedisModuleIO, val: i64) {
unsafe { RedisModule_SaveSigned.unwrap()(rdb, val) };
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn save_float(rdb: *mut RedisModuleIO, val: f32) {
unsafe { RedisModule_SaveFloat.unwrap()(rdb, val) };
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn save_unsigned(rdb: *mut RedisModuleIO, val: u64) {
unsafe { RedisModule_SaveUnsigned.unwrap()(rdb, val) };
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn string_append_buffer(
ctx: *mut RedisModuleCtx,
s: *mut RedisModuleString,
buff: &str,
) -> Status {
unsafe {
RedisModule_StringAppendBuffer.unwrap()(ctx, s, buff.as_ptr() as *mut c_char, buff.len())
.into()
}
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn subscribe_to_server_event(
ctx: *mut RedisModuleCtx,
event: RedisModuleEvent,
callback: RedisModuleEventCallback,
) -> Status {
unsafe { RedisModule_SubscribeToServerEvent.unwrap()(ctx, event, callback).into() }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn register_info_function(ctx: *mut RedisModuleCtx, callback: RedisModuleInfoFunc) -> Status {
unsafe { RedisModule_RegisterInfoFunc.unwrap()(ctx, callback).into() }
}
#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn add_info_section(
ctx: *mut RedisModuleInfoCtx,
name: Option<&str>, // assume NULL terminated
) -> Status {
let name = name.map(|n| CString::new(n).unwrap());
if let Some(n) = name {
unsafe { RedisModule_InfoAddSection.unwrap()(ctx, n.as_ptr() as *mut c_char).into() }
} else {
unsafe { RedisModule_InfoAddSection.unwrap()(ctx, ptr::null_mut()).into() }
}
}

There are more instances of supressions of this lint in this repository: https://github.com/RedisLabsModules/redismodule-rs/search?q=not_unsafe_ptr_arg_deref

Even if they were not exposed in the crate's public API, having unsound functions like that internally would be error prone.

At least one of these is available in the public API. Example safe rust code that triggers Undefined Behaviour:

redis_module::raw::call_reply_type(0usize as *mut _);

I found these after reading rust-lang/rust-clippy#7666 and searching all of github for code supressing this clippy lint.

Expose SubscribeToKeyspaceEvents

It might be useful to expose in a rust-native way registration of event handlers to keyspace events (using the RedisModule_SubscribeToKeyspaceEvents API). E.g. a macro or such.

Support for non-utf8/binary-safe data

The C API for command implementation is as follows:

int Some_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {

where RedisModuleString objects are binary safe C buffers.

However, the Rust's implementation is:

fn some_cmd(ctx: &Context, args: Vec<String>) -> RedisResult {

where args is a vector of Strings which can only contain valid UTF-8 bytes.

Further, most APIs implemented in redismodule-rs accept and return either String or &str.

How about changing the command implementation signature to accept Vec<RedisString> rust structure which abstracts access to the C structureRedisModuleString ?

Brittleness of `commands` and `event_handlers` for improperly named functions

redis_command! and redis_event_handler! macros are internally implemented by respectively defining do_command and handle_event functions which do the actual heavy lifting. Problem is that user might actually attempt to use function(s) with such name(s) in which the macro's version will overshadow the user's one and will lead to some really confusing compilation errors, see:

#[macro_use]
extern crate redis_module;

use redis_module::{Context, NotifyEvent, RedisResult, REDIS_OK};

pub fn do_command(ctx: &Context, args: Vec<String>) -> RedisResult {
    ctx.log_debug("Perfectly valid Command Handler!");

    REDIS_OK
}

pub fn handle_event(ctx: &Context, _event_type: NotifyEvent, _event: &str, _key: &str) {
    ctx.log_debug("Perfectly valid Event Handler!");
}

//////////////////////////////////////////////////////

redis_module! {
    name: "brittleness",
    version: 1,
    data_types: [],
    commands: [
      ["do_command", do_command, "write deny-oom no-cluster", 1, 1, 1]
    ],
    event_handlers: [
      [@ALL: handle_event]
    ]
}
$ cargo build
   Compiling reproduce v0.1.0 (/home/matobet/projects/rust/reproduce)
error[E0061]: this function takes 3 arguments but 2 arguments were supplied
   --> src/lib.rs:23:22
    |
23  |       ["do_command", do_command, "write deny-oom no-cluster", 1, 1, 1]
    |                      ^^^^^^^^^^ expected 3 arguments
    | 
   ::: /home/matobet/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.12.0/src/macros.rs:34:46
    |
34  |                 .map(|args| $command_handler(&context, args))
    |                                              --------  ---- supplied 2 arguments
    |
note: function defined here
   --> /home/matobet/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.12.0/src/macros.rs:14:23
    |
2   |  / macro_rules! redis_command {
3   |  |     ($ctx:expr,
4   |  |      $command_name:expr,
5   |  |      $command_handler:expr,
...    |
14  |  |         extern "C" fn do_command(
    |  |                       ^^^^^^^^^^
15  |  |             ctx: *mut $crate::raw::RedisModuleCtx,
    |  |             -------------------------------------
16  |  |             argv: *mut *mut $crate::raw::RedisModuleString,
    |  |             ----------------------------------------------
17  |  |             argc: c_int,
    |  |             -----------
...    |
55  |  |     }};
56  |  | }
    |  |_- in this expansion of `redis_command!` (#2)
...
100 | /  macro_rules! redis_module {
101 | |      (
102 | |          name: $module_name:expr,
103 | |          version: $module_version:expr,
...   |
174 | |                  redis_command!(ctx, $name, $command, $flags, $firstkey, $lastkey, $keystep);
    | |                  ---------------------------------------------------------------------------- in this macro invocation (#2)
...   |
199 | |      }
200 | |  }
    | |__- in this expansion of `redis_module!` (#1)
    | 
   ::: src/lib.rs:18:1
    |
18  | /  redis_module! {
19  | |      name: "brittleness",
20  | |      version: 1,
21  | |      data_types: [],
...   |
27  | |      ]
28  | |  }
    | |__- in this macro invocation (#1)

error[E0308]: mismatched types
   --> /home/matobet/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.12.0/src/macros.rs:35:37
    |
2   |  / macro_rules! redis_command {
3   |  |     ($ctx:expr,
4   |  |      $command_name:expr,
5   |  |      $command_handler:expr,
...    |
35  |  |                 .unwrap_or_else(|e| Err(e));
    |  |                                     ^^^^^^ expected `i32`, found enum `Result`
...    |
55  |  |     }};
56  |  | }
    |  |_- in this expansion of `redis_command!` (#2)
...
100 | /  macro_rules! redis_module {
101 | |      (
102 | |          name: $module_name:expr,
103 | |          version: $module_version:expr,
...   |
174 | |                  redis_command!(ctx, $name, $command, $flags, $firstkey, $lastkey, $keystep);
    | |                  ---------------------------------------------------------------------------- in this macro invocation (#2)
...   |
199 | |      }
200 | |  }
    | |__- in this expansion of `redis_module!` (#1)
    | 
   ::: src/lib.rs:18:1
    |
18  | /  redis_module! {
19  | |      name: "brittleness",
20  | |      version: 1,
21  | |      data_types: [],
...   |
27  | |      ]
28  | |  }
    | |__- in this macro invocation (#1)
    |
    = note: expected type `i32`
               found enum `Result<_, RedisError>`

error[E0308]: mismatched types
   --> /home/matobet/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.12.0/src/macros.rs:37:27
    |
2   |  / macro_rules! redis_command {
3   |  |     ($ctx:expr,
4   |  |      $command_name:expr,
5   |  |      $command_handler:expr,
...    |
37  |  |             context.reply(response) as c_int
    |  |                           ^^^^^^^^ expected enum `Result`, found `i32`
...    |
55  |  |     }};
56  |  | }
    |  |_- in this expansion of `redis_command!` (#2)
...
100 | /  macro_rules! redis_module {
101 | |      (
102 | |          name: $module_name:expr,
103 | |          version: $module_version:expr,
...   |
174 | |                  redis_command!(ctx, $name, $command, $flags, $firstkey, $lastkey, $keystep);
    | |                  ---------------------------------------------------------------------------- in this macro invocation (#2)
...   |
199 | |      }
200 | |  }
    | |__- in this expansion of `redis_module!` (#1)
    | 
   ::: src/lib.rs:18:1
    |
18  | /  redis_module! {
19  | |      name: "brittleness",
20  | |      version: 1,
21  | |      data_types: [],
...   |
27  | |      ]
28  | |  }
    | |__- in this macro invocation (#1)
    |
    = note: expected enum `Result<RedisValue, RedisError>`
               found type `i32`

error[E0308]: mismatched types
   --> /home/matobet/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.12.0/src/macros.rs:77:17
    |
60  | / macro_rules! redis_event_handler {
61  | |     (
62  | |         $ctx: expr,
63  | |         $event_type: expr,
...   |
77  | |                 &context,
    | |                 ^^^^^^^^ types differ in mutability
...   |
96  | |     }};
97  | | }
    | |_- in this expansion of `redis_event_handler!` (#2)
...
100 | / macro_rules! redis_module {
101 | |     (
102 | |         name: $module_name:expr,
103 | |         version: $module_version:expr,
...   |
179 | |                     redis_event_handler!(ctx, $(raw::NotifyEvent::$event_type |)+ raw::NotifyEvent::empty(), $event_handler);
    | |                     --------------------------------------------------------------------------------------------------------- in this macro invocation (#2)
...   |
199 | |     }
200 | | }
    | |_- in this expansion of `redis_module!` (#1)
    | 
   ::: src/lib.rs:18:1
    |
18  | / redis_module! {
19  | |     name: "brittleness",
20  | |     version: 1,
21  | |     data_types: [],
...   |
27  | |     ]
28  | | }
    | |_- in this macro invocation (#1)
    |
    = note: expected raw pointer `*mut RedisModuleCtx`
                 found reference `&redis_module::Context`

error[E0308]: mismatched types
   --> /home/matobet/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.12.0/src/macros.rs:78:17
    |
60  | / macro_rules! redis_event_handler {
61  | |     (
62  | |         $ctx: expr,
63  | |         $event_type: expr,
...   |
78  | |                 $crate::NotifyEvent::from_bits_truncate(event_type),
    | |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `i32`, found struct `NotifyEvent`
...   |
96  | |     }};
97  | | }
    | |_- in this expansion of `redis_event_handler!` (#2)
...
100 | / macro_rules! redis_module {
101 | |     (
102 | |         name: $module_name:expr,
103 | |         version: $module_version:expr,
...   |
179 | |                     redis_event_handler!(ctx, $(raw::NotifyEvent::$event_type |)+ raw::NotifyEvent::empty(), $event_handler);
    | |                     --------------------------------------------------------------------------------------------------------- in this macro invocation (#2)
...   |
199 | |     }
200 | | }
    | |_- in this expansion of `redis_module!` (#1)
    | 
   ::: src/lib.rs:18:1
    |
18  | / redis_module! {
19  | |     name: "brittleness",
20  | |     version: 1,
21  | |     data_types: [],
...   |
27  | |     ]
28  | | }
    | |_- in this macro invocation (#1)

error[E0308]: mismatched types
   --> /home/matobet/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.12.0/src/macros.rs:79:17
    |
60  | / macro_rules! redis_event_handler {
61  | |     (
62  | |         $ctx: expr,
63  | |         $event_type: expr,
...   |
79  | |                 event_str.to_str().unwrap(),
    | |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `i8`, found `str`
...   |
96  | |     }};
97  | | }
    | |_- in this expansion of `redis_event_handler!` (#2)
...
100 | / macro_rules! redis_module {
101 | |     (
102 | |         name: $module_name:expr,
103 | |         version: $module_version:expr,
...   |
179 | |                     redis_event_handler!(ctx, $(raw::NotifyEvent::$event_type |)+ raw::NotifyEvent::empty(), $event_handler);
    | |                     --------------------------------------------------------------------------------------------------------- in this macro invocation (#2)
...   |
199 | |     }
200 | | }
    | |_- in this expansion of `redis_module!` (#1)
    | 
   ::: src/lib.rs:18:1
    |
18  | / redis_module! {
19  | |     name: "brittleness",
20  | |     version: 1,
21  | |     data_types: [],
...   |
27  | |     ]
28  | | }
    | |_- in this macro invocation (#1)
    |
    = note: expected raw pointer `*const i8`
                 found reference `&str`

error[E0308]: mismatched types
   --> /home/matobet/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.12.0/src/macros.rs:80:17
    |
60  | / macro_rules! redis_event_handler {
61  | |     (
62  | |         $ctx: expr,
63  | |         $event_type: expr,
...   |
80  | |                 redis_key,
    | |                 ^^^^^^^^^ types differ in mutability
...   |
96  | |     }};
97  | | }
    | |_- in this expansion of `redis_event_handler!` (#2)
...
100 | / macro_rules! redis_module {
101 | |     (
102 | |         name: $module_name:expr,
103 | |         version: $module_version:expr,
...   |
179 | |                     redis_event_handler!(ctx, $(raw::NotifyEvent::$event_type |)+ raw::NotifyEvent::empty(), $event_handler);
    | |                     --------------------------------------------------------------------------------------------------------- in this macro invocation (#2)
...   |
199 | |     }
200 | | }
    | |_- in this expansion of `redis_module!` (#1)
    | 
   ::: src/lib.rs:18:1
    |
18  | / redis_module! {
19  | |     name: "brittleness",
20  | |     version: 1,
21  | |     data_types: [],
...   |
27  | |     ]
28  | | }
    | |_- in this macro invocation (#1)
    |
    = note: expected raw pointer `*mut RedisModuleString`
                 found reference `&str`

error: aborting due to 7 previous errors

Some errors have detailed explanations: E0061, E0308.
For more information about an error, try `rustc --explain E0061`.
error: could not compile `reproduce`

To learn more, run the command again with --verbose.

I myself was bitten by this when migrating my existing event handlers to the new declarative mechanism introduced in 0.12 and accidentally leaving the function name as handle_event and spending disproportionate amount of time debugging this.... Whereas the fix is actually quite simple - just rename the function(s).

It would be probably beneficial to try to make those names of functions used by the macros somehow "unique" as to eliminate (or at the very least reduce) the possibility of such naming clashes. I guess changing the do_command to something like __do_command__ would already bring immediate improvement over the current situation or alternatively we could really somehow generate a unique identifier for each command/event handler (perhaps on the textual representation of arguments passed to the command/event macro?) ...

I am interested in your thoughts on this.

Macos build fail

cargo build fail

Some days ago I can cargo build success, But today when i cargo clean, then cargo build again will fail, the environment and output show below.

environment

  • cargo 1.56.0 (4ed5d137b 2021-10-04)
  • macos 10.15.7

output

Compiling redis-module v99.99.99 (/Users/admin/proj/rust/redismodule-rs)
error: failed to run custom build command for `redis-module v99.99.99 (/Users/admin/proj/rust/redismodule-rs)`

Caused by:
  process didn't exit successfully: `/Users/admin/proj/rust/redismodule-rs/target/debug/build/redis-module-21ad79dd1c811e6b/build-script-build` (signal: 6, SIGABRT: process abort signal)
  --- stdout
  TARGET = Some("x86_64-apple-darwin")
  OPT_LEVEL = Some("0")
  HOST = Some("x86_64-apple-darwin")
  CC_x86_64-apple-darwin = None
  CC_x86_64_apple_darwin = None
  HOST_CC = None
  CC = None
  CFLAGS_x86_64-apple-darwin = None
  CFLAGS_x86_64_apple_darwin = None
  HOST_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2,sse3,ssse3")
  running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-arch" "x86_64" "-I" "src/include/" "-Wall" "-Wextra" "-o" "/Users/admin/proj/rust/redismodule-rs/target/debug/build/redis-module-8ecb2f490f35fd5f/out/src/redismodule.o" "-c" "src/redismodule.c"
  exit status: 0
  AR_x86_64-apple-darwin = None
  AR_x86_64_apple_darwin = None
  HOST_AR = None
  AR = None
  running: "ar" "cq" "/Users/admin/proj/rust/redismodule-rs/target/debug/build/redis-module-8ecb2f490f35fd5f/out/libredismodule.a" "/Users/admin/proj/rust/redismodule-rs/target/debug/build/redis-module-8ecb2f490f35fd5f/out/src/redismodule.o"
  exit status: 0
  running: "ar" "s" "/Users/admin/proj/rust/redismodule-rs/target/debug/build/redis-module-8ecb2f490f35fd5f/out/libredismodule.a"
  exit status: 0
  cargo:rustc-link-lib=static=redismodule
  cargo:rustc-link-search=native=/Users/admin/proj/rust/redismodule-rs/target/debug/build/redis-module-8ecb2f490f35fd5f/out

  --- stderr
  fatal runtime error: failed to initiate panic, error 5

I need your help. Thanks.

Improve the developer docs

We should have good and consistent doc comments that explain the usage of the module, its concepts, etc. The current autogenerated docs are very minimal.

Initialization failed. Module not loaded

Is there a good way to diagnose this error?

I have a module that works fine locally, and works in a docker image.

When I deployed to a machine (in a docker container) I get the error Module /opt/.../lib_redis.so initialization failed. Module not loaded when trying to load via the redis.conf file.

If I get redis running first then do MODULE LOAD /opt/.../lib_redis.so it also works fine.

Any clues?

Schema evolution in practice

Dear developers,
I'd like to write some modules for Redis in Rust.

  1. To avoid serde I'd like to use custom types. How can I do schema evolution for them in future?
  2. For production grade code should I care about handlers?
        rdb_load: None,
        rdb_save: None,
        aof_rewrite: None,
        free: Some(free),

        // Currently unused by Redis
        mem_usage: None,
        digest: None,

        // Aux data
        aux_load: None,
        aux_save: None,
        aux_save_triggers: 0,

        free_effort: None,
        unlink: None,
        copy: None,
        defrag: None,

Thanks in advance.

EXC_BAD_ACCESS: pointer being freed was not allocated

Currently the RedisAlloc is enforced at compilation and then at runtime a flag is used to enabled the allocator or not.

The first problem is that if the application do not want to use it it is currently not possible as it is always enabled in the macro.

The second problem is that the global allocator is enforced and even if the use_redis_alloc function is not called the allocator will fallback on the System allocator, therefore it is not possible to use, for example, the jemalloc one (which can be way more performant than the system's one).

The third and last problem is that I faced BAD_ACCESS crashes that were triggered inside the RedisAlloc::dealloc function of the kind: "pointer used after being freed". I think about it and it was probably because the USE_REDIS_ALLOC static was enabled after some alloc calls were already made to the allocator and therefore allocations were made by the system's one but freed by the redis one.

One solution could have been to enforce the allocator only when a specific feature is set and do not rely on a secondary runtime setting (i.e. the flag) to enable or disable the allocator, it is dangerous.

# Cargo.toml
[features]
default = ["redis-allocator"]
experimental-api = []
redis-allocator = []
// lib.rs
#[cfg_attr(feature="redis-allocator", global_allocator)]
static ALLOC: crate::alloc::RedisAlloc = crate::alloc::RedisAlloc;

By the way I did not take the time to look at what does declaring a global_allocator do in a library, wasn't it made to be used in a binary project? What happen if I declare another global_allocator inside my main.rs file?

Memory Leak in RedisKey::hash_get_multi()

It looks like redis_module::key::hash_get_multi() and/or redis_module::key::HMGetResult cause a memory leak.

Reproducing

Generate a Redis plugin module containing the following command. Redis memory will leak whenever the command is called.

Command syntax

command test_key

Preconditions

Redis must contain a HASH with name test_key, and the hash must contain a key whose name is "field". Leak does not occur if "field" is not a member of test_key.

pub fn cmd_doesnothing(ctx: &Context, args: Vec<RedisString>) -> RedisResult {
    let mut args = args.into_iter().skip(1);
    let hash_key = args.next_arg()?;
    let hash = ctx.open_key(&hash_key);
    let hash_result: redis_module::key::HMGetResult<&str, RedisString> = hash.hash_get_multi(&["field"])?.ok_or(RedisError::Str("error"))?;

    REDIS_OK
}

Memory is not reclaimed by FLUSHDB or MEMORY PURGE commands.

Fix typo in README

I tried to open a PR but I lack permissions.

Anyway, the command in the sample is HELLO.MUL and not HELLO.ADD.

Cheers, Martin

Accessing context within tests

I may be being a little bit dense but I'm having a go at porting some lua scripts over to rust and I've got a barebone setup and I'm having trouble reading keys etc from within the tests as the context is null.

I can see that the source of the null context is &Context::dummy(), but I'm wondering what the alternative is here? Do I simply need to run my tests integration style and not actually in rust?

Is this intended or am I missing a step somewhere?

Makefile is broken

Example output:

 make all 
Makefile:5: *** Submodules not present. Please run 'git submodule update --init --recursive'.  Stop.

Cnetos8 docker container and vm RedisJSON loadmodule failed

OS : Centos8

Process step:
1 - Clone RedisJSON
2 - Install rust tool
3 - Ind RedisJSON dir run " cargo build --release" success

# redis-server --loadmodule ./target/release/librejson.so
5477:C 26 Jul 2021 05:54:11.176 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
5477:C 26 Jul 2021 05:54:11.176 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=5477, just started
5477:C 26 Jul 2021 05:54:11.176 # Configuration loaded
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 5.0.3 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 5477
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

5477:M 26 Jul 2021 05:54:11.177 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
5477:M 26 Jul 2021 05:54:11.177 # Server initialized
5477:M 26 Jul 2021 05:54:11.178 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
5477:M 26 Jul 2021 05:54:11.178 * <ReJSON> Exported RedisJSON_V1 API
thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.21.0/src/raw.rs:558:42
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 5
Aborted

I don't know why this happen, please give me some advise , thanks !

FLUSHALL command failed after loading example module

FLUSHALL, BGSAVE, SYNC and similar commands failed after trying out the module provided as example.

Note that, I can do FLUSHALL even after the redis server started with the module. The problems started after I run alloc.set. alloc.set is doing a open_key_writable(key).set_value(...)

Am I missing something?

=== REDIS BUG REPORT START: Cut & paste starting from here ===
87858:M 25 Oct 2019 00:14:11.549 # Redis 5.0.4 crashed by signal: 11
87858:M 25 Oct 2019 00:14:11.549 # Accessing address: 0x0
87858:M 25 Oct 2019 00:14:11.549 # Failed assertion: (:0)

------ STACK TRACE ------
0 redis-server 0x000000010250d1a0 logStackTrace + 110
1 redis-server 0x000000010250d527 sigsegvHandler + 236
2 libsystem_platform.dylib 0x00007fff7e18db3d _sigtramp + 29
3 ??? 0x0000000000000001 0x0 + 1
4 redis-server 0x00000001024cac29 _dictClear + 128
5 redis-server 0x00000001024cb4e6 dictEmpty + 25
6 redis-server 0x00000001024e2ba1 emptyDb + 121
7 redis-server 0x00000001024e2dbd flushallCommand + 71
8 redis-server 0x00000001024cf857 call + 223
9 redis-server 0x00000001024d00dc processCommand + 1537
10 redis-server 0x00000001024dc7e1 processInputBuffer + 285
11 redis-server 0x00000001024c8dff aeProcessEvents + 550
12 redis-server 0x00000001024c90ee aeMain + 43
13 redis-server 0x00000001024d2f6d main + 1377
14 libdyld.dylib 0x00007fff7dfa2ed9 start + 1
15 ??? 0x0000000000000003 0x0 + 3

Redis module command not persisted in the AOF file

Hi,
I'm trying to set up persistence for redis module with custom type.
With

127.0.0.1:6379> config get appendonly
1) "appendonly"
2) "yes"

default redis commands are persisted in AOF file but my custom commands are not written there.
Is there anything I need to setup to make this work? Thank you

how can i use ThreadSafeContext in my project?

i try to add feature in my cargo.toml, but has a error.

[dependencies]
redis-module = {version = "0.11.0", feature = ["experimental-api"]}

error:
error[E0432]: unresolved import redis_module::ThreadSafeContext
--> src/server.rs:6:54
|
6 | use redis_module::{Context, RedisError, RedisResult, ThreadSafeContext};
| ^^^^^^^^^^^^^^^^^ no ThreadSafeContext in the root

how should i do? thanks for your help!

Convert from RedisValue?

Hi rust newbie here, when i do ctx.call(...); I get back a RedisValue. I'm expecting to be something like Vec<RedisString> how do I convert it to a vector to interate on the data?

Thanks in advance!

Expose the RedisModule_OnLoad argv vector

Currently there appears to be no way how the user-provided init function can access the arguments vector passed to the MODULE LOAD mymod arg1 arg2 ... that is present in the C-api int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc).

Exposing this would also in Rust code would be useful to pass some static configuration to the module at load time.

Issues building examples modules

I am super new to rust so please excuse me if this is trivial question but I really struggle to build any of the examples for linux. Building natively on my Mac works just fine but I wanted to build for linux so that I can use modules with redis running in a container.

First I tried cross compiling from mac os by following various articles such as e.g. https://timryan.org/2018/07/27/cross-compiling-linux-binaries-from-macos.html but to no avail. The issues always where that build.rs complained that some files e.g. types.h could not be found. After reinstalling xcode cli tools etc. I finally gave up on getting cross compilation to work for now. The next thing I tried was building using docker

docker run --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp rust:latest cargo build --example hello

which at least did't fail immidiately but at the end failed with

[...]
Compiling redis-module v0.17.0 (/usr/src/myapp)
error: failed to run custom build command for `redis-module v0.17.0 (/usr/src/myapp)`

Caused by:
  process didn't exit successfully: `/usr/src/myapp/target/debug/build/redis-module-ba440a6132e376c2/build-script-build` (exit code: 101)
  --- stdout
  TARGET = Some("x86_64-unknown-linux-gnu")
  OPT_LEVEL = Some("0")
  HOST = Some("x86_64-unknown-linux-gnu")
  CC_x86_64-unknown-linux-gnu = None
  CC_x86_64_unknown_linux_gnu = None
  HOST_CC = None
  CC = None
  CFLAGS_x86_64-unknown-linux-gnu = None
  CFLAGS_x86_64_unknown_linux_gnu = None
  HOST_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-I" "src/include/" "-Wall" "-Wextra" "-o" "/usr/src/myapp/target/debug/build/redis-module-fa94a2fb353d85d5/out/src/redismodule.o" "-c" "src/redismodule.c"
  exit code: 0
  AR_x86_64-unknown-linux-gnu = None
  AR_x86_64_unknown_linux_gnu = None
  HOST_AR = None
  AR = None
  running: "ar" "cq" "/usr/src/myapp/target/debug/build/redis-module-fa94a2fb353d85d5/out/libredismodule.a" "/usr/src/myapp/target/debug/build/redis-module-fa94a2fb353d85d5/out/src/redismodule.o"
  exit code: 0
  running: "ar" "s" "/usr/src/myapp/target/debug/build/redis-module-fa94a2fb353d85d5/out/libredismodule.a"
  exit code: 0
  cargo:rustc-link-lib=static=redismodule
  cargo:rustc-link-search=native=/usr/src/myapp/target/debug/build/redis-module-fa94a2fb353d85d5/out

  --- stderr
  thread 'main' panicked at 'Unable to find libclang: "couldn\'t find any valid shared libraries matching: [\'libclang.so\', \'libclang-*.so\', \'libclang.so.*\', \'libclang-*.so.*\'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/bindgen-0.58.1/src/lib.rs:2057:31
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Any suggestions how to quickly get up and running building the examples in this repo would be very appreciated :)

Parameters shouldn't be owned by command

Should this signature:

fn hello_mul(_: &Context, args: Vec<String>) -> RedisResult {

become:

fn hello_mul(_: &'ae Context, args: &'a [&'a str]) -> RedisResult {

Right now, to convert the params into Rust's String requires to_string() which allocates. Also, it's semantically more correct.

Add support for mem_usage2

RedisModuleTypeMemUsageFunc2 mem_usage2;


typedef size_t (*RedisModuleTypeMemUsageFunc2)(RedisModuleKeyOptCtx *ctx, const void *value,
                                               size_t sample_size);

Compilation issue in redis_module macro when updating from 0.18.0(to any version further)

error[E0308]: mismatched types
   --> redis-calc/src/lib.rs:138:1
    |
138 | / redis_module! {
139 | |     name: "crusty.calc",
140 | |     version: 1,
141 | |     data_types: [],
...   |
145 | |     ],
146 | | }
    | |_^ expected struct `std::string::String`, found struct `RedisString`
    |
    = note: expected struct `Vec<std::string::String>`
               found struct `Vec<RedisString>`
    = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)

module definition:

redis_module! {
    name: "crusty.calc",
    version: 1,
    data_types: [],
    commands: [
        ["crusty.calc.topk.add", topk_add, "", 0, 0, 0],
        ["crusty.calc.topk.consume", topk_consume, "", 0, 0, 0],
    ],
}

I don't see any code in examples to be updated to reflect needed changes

Existing key has wrong Redis type

Question

I had created a redis module to store erlang term struct here

When do some test it return {error, <<"Existing key has wrong Redis type">>}, I am a beginner for rust .
I don't know how to fix it.

I will appreciate your help!

Environment

  • redis 6.2.6
  • macos catalina 10.15.7 x86-64
(37473:C 23 Dec 2021 16:09:50.603 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
37473:C 23 Dec 2021 16:09:50.603 # Redis version=6.2.6, bits=64, commit=00000000, modified=0, pid=37473, just started
37473:C 23 Dec 2021 16:09:50.603 # Configuration loaded
37473:M 23 Dec 2021 16:09:50.604 * Increased maximum number of open files to 10032 (it was originally set to 256).
37473:M 23 Dec 2021 16:09:50.604 * monotonic clock: POSIX clock_gettime
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 6.2.6 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 7777
 |    `-._   `._    /     _.-'    |     PID: 37473
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           https://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'

Test

This a erlang redis client to test Just a client console shell.

db_controller@127.0.0.1)153> db_controller_redis:cmd(['big_data.set',<<"2">>,erlang:term_to_binary({row_data,<<"3">>,{a,9,0},1640240267786})]).
<<"ok">>
(db_controller@127.0.0.1)154> db_controller_redis:cmd(['big_data.set',<<"1">>,erlang:term_to_binary({row_data,<<"3">>,{a,9,0},1640240267786})]).
{error,<<"Existing key has wrong Redis type">>}
(db_controller@127.0.0.1)155> erlang:binary_to_term(db_controller_redis:cmd(['big_data.get',2, <<"3">>])).
{row_data,<<"3">>,{a,9,0},1640240267786}

`cargo build --example hello` fails on os/x

Env:

redismodule-rs on master is πŸ“¦ vredis-module:0.15.0 via π—₯ v1.49.0
19.6.0 Darwin Kernel Version 19.6.0
root:xnu-6153.141.28.1~1/RELEASE_X86_64 x86_64
mac os catalina - 10.15.7 (19H1030)

Output:

➜ cargo build --example hello
   Compiling redis-module v0.15.0 (/Users/briansam-bodden/Code/redis-modules/learning/redismodule-rs)
error[E0425]: cannot find value `RedisModule_ExportSharedAPI` in this scope
   --> src/raw.rs:502:14
    |
502 |     unsafe { RedisModule_ExportSharedAPI.unwrap()(ctx, name, func as *mut ::std::os::raw::c_void) };
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^ not found in this scope

error: aborting due to previous error

For more information about this error, try `rustc --explain E0425`.
error: could not compile `redis-module`

Build on CentOS7 docker image fails with "'GLIBC_2.18' not found"

OS is CentOS7 via docker image "image: centos:7"

the build is made using docker-compose via a .yaml file which has this image as its base:
services: rpm_build-centos7: image: centos:7

https://hub.docker.com/_/centos?tab=description
https://github.com/CentOS/sig-cloud-instance-images/blob/b2d195220e1c5b181427c3172829c23ab9cd27eb/docker/Dockerfile

the important section in the .yaml file that performs the build is this command sequence

            yum upgrade -y &&
            yum update -y &&
            
            yum install -y  glibc glibc-common glibc-devel wget curl gcc make clang clang-devel &&

            curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | /bin/sh -s -- -y &&
            
            export PATH=/usr/local/cargo/bin:/usr/sbin:/usr/bin:/sbin:/bin &&

            cargo build --release &&
            
            cargo install cargo-generate-rpm &&
            cargo generate-rpm
            "

the project .toml has this dependency:

[dependencies]
redis-module = { version="0.23", features = ["experimental-api"] }

The "cargo build" fails with this last error:

rpm_build-centos7_1  |   Downloaded clang-sys v1.2.2
rpm_build-centos7_1  |   Downloaded libc v0.2.103
rpm_build-centos7_1  |  Downloading crates ...
rpm_build-centos7_1  |   Downloaded anyhow v1.0.44
rpm_build-centos7_1  |    Compiling redis-module v0.23.0
rpm_build-centos7_1  | error: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /build/target/release/deps/libenum_primitive_derive-525852ee7a2d0d06.so)
rpm_build-centos7_1  |  --> /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.23.0/src/raw.rs:5:1
rpm_build-centos7_1  |   |
rpm_build-centos7_1  | 5 | extern crate enum_primitive_derive;
rpm_build-centos7_1  |   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
rpm_build-centos7_1  |
rpm_build-centos7_1  | error: could not compile `redis-module` due to previous error
local_rpm_build-centos7_1 exited with code 101

The error is
"error: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /build/target/release/deps/libenum_primitive_derive-525852ee7a2d0d06.so)"

I checked around and there are a lot of result with confusing messages (to me) involving
Rust / GLIBC_2.18 / CentOS7
I could not find a solution to this. (note build is working fine on CentOS8, Ubuntu 20.04)
Does anyone know of this issue and hopefully a workaround?
We would very much like to support CentOS7 with our custom redis-module development

Thank you!

failed loading module after modifying source code

I tried to extend original code of hello.rs module by adding support for avro-rs.
I made changes in cargo.toml

[dependencies]
bitflags = "1.2"
libc = "0.2"
enum-primitive-derive = "0.1.2"
num-traits = "^0.2"
regex = "1"
strum_macros = "0.24"
#failure = "0.1"
backtrace = "0.3"
avro-rs = "0.13.0"
serde = { version = "1.0", features = ["derive"] }
serde_json = "*"
serde_derive = "*"

and here changes in the source code:

use serde::{Deserialize, Serialize};
use avro_rs::{from_value, types::Record, Codec, Error, Reader, Schema, Writer};

fn hello_mul(_: &Context, args: Vec<RedisString>) -> RedisResult {
    if args.len() < 2 {
        return Err(RedisError::WrongArity);
    }

    let raw_schema = r#"
        {
            "type": "record",
            "name": "test",
            "fields": [
                {"name": "a", "type": "long", "default": 42},
                {"name": "b", "type": "string"}
            ]
        }
    "#;

    // if the schema is not valid, this function will return an error
    let schema = Schema::parse_str(raw_schema).unwrap();

the rest of the code the same as in original.

Build is ok.

From server log:

1:M 08 Apr 2022 19:46:40.854 # Module /data/hello.so initialization failed. Module not loaded

Thank you very much

fail to call RedisGraph module

when i call RedisGraph in redismodule-rs, redis fail.

fn hello_person(ctx: &Context, _: Vec<RedisString>) -> RedisResult {
    let r = ctx.call("GRAPH.QUERY", &["person", r#"MATCH (n) RETURN n"#]);
    match r {
        Ok(v) => {
            ctx.log_notice(&format!("{:?}", v));
        }
        Err(e) => {
            ctx.log_warning(&format!("{:?}", e));
        }
    }
    return Ok("OK".into());
}

Result:

$ redis-cli
127.0.0.1:6379> hello.person
Could not connect to Redis at 127.0.0.1:6379: Connection refused
(0.68s)
not connected>

Version:

redis-module: v0.21

Redis server: v6.2.4

RedisGraph: master

Publish to crates.io?

Looks like redismodule crate name is taken. How about renaming this crate to something like redis-module / redis_module then publish to crates.io?

How to use binary/bytes/stringbuffer type

I'm making a module that wants to read/write non-utf8 bytes to Redis. However the RedisKeyWritable read / write function only accept &str and also the higher level call function only accepts strings.

Is there a way to do this currently (without using the raw C bindings) ?

Currently i worked around this by creating my own type and using the set_value / get_value functions.

Using RedisString::create(....) and running cargo test throws panicked at 'called `Option::unwrap()` on a `None` value

Hi,
I've upgraded my module to redismodule-rs 0.26.0.
When upgrading I had to make changes and use RedisString::create(...) function.
When I ran cargo build the build passed.
When I ran cargo test --features test --all --all-targets --verbose the following error appeared on each test that used RedisString:create(...):

---- get_sets_single stdout ----
thread 'get_sets_single' panicked at 'called `Option::unwrap()` on a `None` value', /home/danitseitlin/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.26.0/src/redismodule.rs:107:60

Here is an example for the test:

#[test]
fn get_sets_single() {
    let ctx = Context::dummy().ctx;
    let args = vec![RedisString::create(ctx, "member1"), RedisString::create(ctx, "10"), RedisString::create(ctx, "20")];
    let sets = get_sets(args.into_iter());
    let sets = sets.expect("one member");
    assert_eq!(
        sets.0,
        vec![Set {
            member: "member1".to_string(),
            min_score: 10,
            max_score: 20,
        }]
    );
}

I hope you can help, thanks ✌🏽

Implement sorted set API

It would be nice to have safe abstractions for the following:

  • RedisModule_ZsetAdd
  • RedisModule_ZsetFirstInScoreRange
  • RedisModule_ZsetRangeCurrentElement
  • RedisModule_ZsetRangeEndReached
  • RedisModule_ZsetRangeNext
  • RedisModule_ZsetRangeStop

I'd be happy to help implement this, given some guidance.

Incorrect Return Type from `Context::call()`

The following Redis Module command simply wraps the Redis "GET" command and returns the value of the simple string key with the name "key". It contains a bug for some string values.

pub fn cmd_example(ctx: &Context, _args: Vec<RedisString>) -> RedisResult {
    ctx.call("GET", &["key"])
}
  • For single-line strings, it works fine.
  • For multi-line strings (i.e. those containing \r\n), it will cause the Redis client to fail. On the redis-py client, for example, the client hangs because RESP protocol is not respected.

Root cause is that Context::call() always returns strings as RedisValue::SimpleString. But strings containing \r\n should be RedisValue::BulkString.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.