seniorjoinu / ic-stable-memory Goto Github PK

I have data in a SHashMap and I've realized that I need to iterate over it. But there's no method that allows me to do so.

Even the std::collection::HashMap has an iter method that allows this. Any solution for this?

Here is the error message I'm seeing

Seems to be an incompatibility between the two types. Thoughts?
@seniorjoinu

Hi @seniorjoinu

https://github.com/gabrielnic/ic-rust/blob/main/backend/master/src/lib.rs

Based on this btree

#[derive(StableType, AsFixedSizeBytes)] pub struct Db { data: SBTreeMap<u64, SBox<Vec<u8>>>, }

if you run this commands:

`build master and install and in order to recreate it run:
dfx canister call master len_val -> should be 0
dfx canister call master insert
dfx canister call master len_val -> should be 3
dfx canister call master all

it should return 3 values but all I get back is

(
vec {
record { 1 : nat64; blob "\01\02\03\04\05\06\07\08\09\0a" };
record { 2 : nat64; blob "\01\02\03\04\05\06\07\08\09\0a" };
},
)`

Not really sure what the issue is and why the last element is omitted but see if you can replicate and find out what the issue is.

This is the error I'm seeing

Panicked at 'Unable to decode: Error { kind: UnexpectedEndOfInput }', /home/saikat/.cargo/registry/src/github.com-1ecc6299db9ec823/ic-stable-memory-0.2.2/./src/primitive/s_unsafe_cell.rs:39:58

Here's the minimal reproduction (same repo, updated code):
https://github.com/saikatdas0790/stable_memory_allocator_bug_illustration

Steps to Reproduce error

• dfx start --clean
• dfx deploy --no-wallet
• Open the Candid UI explorer
• Run one of the query methods. No issues
• Run the update method to insert SPrincipal into stable collections
• Run the query method again. Error
• Try the separate methods for String type which has no issues running

Let me know if you need any further details. Thank you

When a stable vector grows it can allocate more memory than a limit says, this will lead to an OOM error, while the canister actually have more memory for other collections to grow.

This is not critical, but annoying.

I might be confused about the inner workings of ICP but it is my understanding that regardless of subnet allowance a canister can be at most 4gb in size . This would seem to imply that you need to dynamically create new canisters no? Is this done here and I missed it?

I'm running into this error when I add ic-stable-memory to my Cargo.toml packages.

Do I have to switch to the nightly release channel to use this library or am I doing something wrong?

Remove the S prefix

Hello @seniorjoinu.

In candid version 0.9.0 or later, an error occurs in candid_decode_one_allow_trailing when trying to implement AsDynSizeBytes for a dynamic size enum that satisfies StableType, CandidType and Deserialize The following error occurs in candid_decode_one_allow_trailing.

#[derive(StableType, CandidType, Deserialize)]]
pub struct DetailsV001 {
    pub name: String,
    pub description: String,
    pub tag_ids: Vec<u64>,
}

#[derive(StableType, CandidType, Deserialize, CandidAsDynSizeBytes)]]
pub enum Details {
    V001(DetailsV001),
}

then,

error[E0277]: the trait bound `Details: candid::types::CandidType` is not satisfied
   --> src/backend/src/something.rs:14:47
    |
14  | #[derive(StableType, CandidType, Deserialize, CandidAsDynSizeBytes)]
    |                                               ^^^^^^^^^^^^^^^^^^^^ the trait `candid::types::CandidType` is not implemented for `Details`
    |
    = help: the following other types implement trait `candid::types::CandidType`:
              bool
              isize
              i8
              i16
              i32
              i64
              i128
              usize
            and 131 others
note: required by a bound in `candid_decode_one_allow_trailing`
   --> /Users/me/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ic-stable-memory-0.4.4/./src/encoding/dyn_size.rs:118:26
    |
116 | pub fn candid_decode_one_allow_trailing<'a, T>(bytes: &'a [u8]) -> Result<T>
    |        -------------------------------- required by a bound in this function
117 | where
118 |     T: Deserialize<'a> + CandidType,
    |                          ^^^^^^^^^^ required by this bound in `candid_decode_one_allow_trailing`
    = note: this error originates in the derive macro `CandidAsDynSizeBytes` (in Nightly builds, run with -Z macro-backtrace for more info)

The same error occurred when I manually implemented AsDynSizeBytes.

This does not happen with candid 0.8.4.

Here's a minimal reproduction
https://github.com/saikatdas0790/stable_memory_allocator_bug_illustration

Please note I'm on stable rust release. Not sure if that's related

Steps to Reproduce error

• dfx start
• dfx deploy --no-wallet
• Make a minimal change
• dfx deploy --no-wallet

Let me know if there are any other details I can provide

https://github.com/dfinity/stable-structures

Hi @seniorjoinu, would love to see a comparison benchmark between ic-stable-memory's implementation of stable BTreeMap and the one Dfinity recently made available. Thoughts?

Hi in my case i need to get &[u8] from SVec. I n typical Vec i would do &&Vec but with SVec i tryed blob.as_dyn_size_bytes() and to get typical vec blob.iter().map(|el| *el).collect::<Vec<_>>();.

In first case I got error that i have shape [256, 24] insted of [u8] in second Reject text: Canister bkyz2-fmaaa-aaaaa-qaaaq-cai trapped: stable memory out of bounds.

How can i get [u8] from SVec or Typical Vec from SVec

It seems like with the introduction of copy_as_bytes in is possible now to create a bunch of helper methods for SSlice like:

fn read_u64() -> u64;
fn write_u64(u64);

fn read_bool() -> bool
fn write_bool(bool);

fn read_u8_arr<const N: usize>() -> [u8; N];
fn write_u8_arr<const N: usize>([u8; N]);

It would be nice to implement them and to rewrite all the logic using them. This would both make it faster and easier to read.

I believe the answer is a BTreeMap but the stable variant implemented in this repo is missing a mechanism to get a range of values instead of just an individual value.

My needs are:

• Need to be able to maintain an order sorted by most recent first
• Need to support being able to add and remove items which will update the sort order
• Need to be able to fetch a range of items at a time instead of the entire list

BTreeMap looks ideal for it where I can specify the key to be the insertion time and have it sorted. The Rust std::collection implementation has a range API here

Any plans to expose the same for the stable btreemap?