foundationdb-rs / foundationdb-rs

FoundationDB Rust client api

FoundationDB Rust Client

The repo consists of multiple crates:

Library	Status	Description
foundationdb		High level FoundationDB client API
foundationdb-sys		C API bindings for FoundationDB
foundationdb-gen	n/a	Code generator for common options and types of FoundationDB

The current version requires rustc 1.70+ to work. The previous version (0.3) is still maintained and is available within the 0.3 branch.

You can access the main branch documentation here.

Supported platforms

Supported platforms are listed on the foundationdb's README.

Develop with Nix

A flake.nix is provided to develop the bindings. We recommend add a cluster-file on the configuration.nix file:

{
  environment.etc."foundationdb/fdb.cluster" = {
    mode = "0555";
    text = ''
      docker:[email protected]:4500
    '';
  };
}

A FoundationDB cluster can be run using these commands:

docker run -p 4500:4500 --name fdb -it --rm -d foundationdb/foundationdb:7.1.19
docker exec fdb fdbcli --exec "configure new single memory"

Correctness

Special care has been set up to be sure that the crate is correct, like official bindings. Every hour, we are running thousands of seeds on the BindingTester.

License

Licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

foundationdb-rs's Issues

Add script to generate an Joshua archive

Joshua is a tool designed to coordinate ephemeral tests of the FoundationDB. We should be able to create an archive for the bindingtester in order to massively run correctness tests on the crate.

Fix `RESULT_NOT_PRESENT` on directory tests

Correctness tests found three seeds with what appeared to be the same error:

Seed 4275610547

+ ./bindings/bindingtester/bindingtester.py --num-ops 1000 --api-version 630 --test-name directory --concurrency 1 rust --no-directory-snapshot-ops --compare python

Creating test at API version 630
Generating directory test at seed 4275610547 with 1000 op(s) and 1 concurrent tester(s)...

Inserting test into database...

Running tester '/home/runner/work/foundationdb-rs/foundationdb-rs/target/debug/bindingtester test_spec 630'...

Reading results from '('tester_output', 'stack')'...
Reading results from '('tester_output', 'directory_log', 'directory')'...
Reading results from '('tester_output', 'directory_log', 'subspace')'...

Inserting test into database...

Running tester 'python /home/runner/work/foundationdb-rs/foundationdb-rs/target/foundationdb_build/foundationdb/bindings/bindingtester/../python/tests/tester.py test_spec 630'...

Reading results from '('tester_output', 'stack')'...
Reading results from '('tester_output', 'directory_log', 'directory')'...
Reading results from '('tester_output', 'directory_log', 'subspace')'...

Comparing results from '('tester_output', 'stack')'...

Incorrect result: 
  rust   - ('tester_output', 'stack', 50, 386) = b'\x01RESULT_NOT_PRESENT\x00'
  python - ('tester_output', 'stack', 50, 386) = b'\x01\x01ERROR\x00\xff\x011020\x00\xff\x00'

Comparing results from '('tester_output', 'directory_log', 'directory')'...
Comparing results from '('tester_output', 'directory_log', 'subspace')'...

Test with seed 4275610547 and concurrency 1 had 1 incorrect result(s) and 0 error(s) at API version 630
Completed directory test with random seed 4275610547 and 1000 operations

Seed 1951034301

+ ./bindings/bindingtester/bindingtester.py --num-ops 1000 --api-version 630 --test-name directory --concurrency 1 rust --no-directory-snapshot-ops --compare python

Creating test at API version 630
Generating directory test at seed 1951034301 with 1000 op(s) and 1 concurrent tester(s)...

Inserting test into database...

Running tester '/home/runner/work/foundationdb-rs/foundationdb-rs/target/debug/bindingtester test_spec 630'...

Reading results from '('tester_output', 'stack')'...
Reading results from '('tester_output', 'directory_log', 'directory')'...
Reading results from '('tester_output', 'directory_log', 'subspace')'...

Inserting test into database...

Running tester 'python /home/runner/work/foundationdb-rs/foundationdb-rs/target/foundationdb_build/foundationdb/bindings/bindingtester/../python/tests/tester.py test_spec 630'...

Reading results from '('tester_output', 'stack')'...
Reading results from '('tester_output', 'directory_log', 'directory')'...
Reading results from '('tester_output', 'directory_log', 'subspace')'...

Comparing results from '('tester_output', 'stack')'...

Incorrect result: 
  rust   - ('tester_output', 'stack', 120, 1030) = b'\x01RESULT_NOT_PRESENT\x00'
  python - ('tester_output', 'stack', 120, 1030) = b'\x01\x01ERROR\x00\xff\x011020\x00\xff\x00'

Comparing results from '('tester_output', 'directory_log', 'directory')'...
Comparing results from '('tester_output', 'directory_log', 'subspace')'...

Test with seed 1951034301 and concurrency 1 had 1 incorrect result(s) and 0 error(s) at API version 630
Completed directory test with random seed 1951034301 and 1000 operations

Seed 1700644945

+ ./bindings/bindingtester/bindingtester.py --num-ops 1000 --api-version 630 --test-name directory --concurrency 1 rust --no-directory-snapshot-ops --compare python

Creating test at API version 630
Generating directory test at seed 1700644945 with 1000 op(s) and 1 concurrent tester(s)...

Inserting test into database...

Running tester '/home/runner/work/foundationdb-rs/foundationdb-rs/target/debug/bindingtester test_spec 630'...

Reading results from '('tester_output', 'stack')'...
Reading results from '('tester_output', 'directory_log', 'directory')'...
Reading results from '('tester_output', 'directory_log', 'subspace')'...

Inserting test into database...

Running tester 'python /home/runner/work/foundationdb-rs/foundationdb-rs/target/foundationdb_build/foundationdb/bindings/bindingtester/../python/tests/tester.py test_spec 630'...

Reading results from '('tester_output', 'stack')'...
Reading results from '('tester_output', 'directory_log', 'directory')'...
Reading results from '('tester_output', 'directory_log', 'subspace')'...

Comparing results from '('tester_output', 'stack')'...

Incorrect result: 
  rust   - ('tester_output', 'stack', 73, 560) = b'\x01RESULT_NOT_PRESENT\x00'
  python - ('tester_output', 'stack', 73, 560) = b'\x01\x01ERROR\x00\xff\x011020\x00\xff\x00'

Comparing results from '('tester_output', 'directory_log', 'directory')'...
Comparing results from '('tester_output', 'directory_log', 'subspace')'...

Test with seed 1700644945 and concurrency 1 had 1 incorrect result(s) and 0 error(s) at API version 630
Completed directory test with random seed 1700644945 and 1000 operations

Fix CI

On Clikengo's repo, CI was failing. It should still be failing.

Create a `tracing` feature

Even if fdb-7.0 is adding a tracing feature, we should add a feature that will use tracing.rs.

I really like what @KrzysFR did on the dotnet binding:

=== FoundationDB.Layers.Directories.DirectoryFacts.Test_Concurrent_Directory_Creation_With_Custom_Prefix() === 16:56:30.6413691
Transaction #113 (read/write, 20 operations, '#' = 0.5 ms, started 14:56:30.6132900Z, ended 14:56:30.6413691Z)
┌  oper. ┬─────────────────────────────────────────────────────────┬──── start ──── end ── duration ──┬─ sent  recv ┐
│ 0   rv°│ ######X                                                 │ T+  5.825 ~   8.473 (  2,648 µs) │             │ GetReadVersion => 62110623213066
│ 1   G  │ ___________________:.                                   │ T+ 13.654 ~  13.755 (    101 µs) │    24    12 │ Get T<02>DL<00><FE><01>T<02>DL<00><FF><FE><00><01>version<00> => <01><00><00><00><00><00><00><00><00><00><00><00>
│ 2   // │ ____________________`                                   │ T+ 13.760                        │     -     - │ // Looking for child Second under node T<02>DL<00><FE><01>T<02>DL<00><FF><FE><00>...
│:2   G  │ ____________________&$                                  │ T+ 13.763 ~  14.443 (    680 µs) │    24     0 │ Get T<02>DL<00><FE><01>T<02>DL<00><FF><FE><00><14><02>Second<00> => not_found
│ 3   G  │ _____________________.                                  │ T+ 14.448 ~  14.470 (     22 µs) │    24    12 │ Get T<02>DL<00><FE><01>T<02>DL<00><FF><FE><00><01>version<00> => <01><00><00><00><00><00><00><00><00><00><00><00>
│ 4   R  │ _____________________.;                                 │ T+ 14.474 ~  14.614 (    139 µs) │    44    30 │ Snapshot.GetRange fGE{T<02>DL<00><FE><01>T<02>DL<00><FF><FE><00><01>hca<00><14><00>} <= k < fGE{T<02>DL<00><FE><01>T<02>DL<00><FF><FE><00><01>hca<00><14><FF>}, limit(1), reverse, Exact => 1 result(s), has_more
│ 5   a  │ ______________________`                                 │ T+ 14.618 ~  14.622 (      4 µs) │    30       │ Atomic_Add T<02>DL<00><FE><01>T<02>DL<00><FF><FE><00><01>hca<00><14><14>, <01 00 00 00 00 00 00 00>
│ 6   G  │ ______________________:                                 │ T+ 14.623 ~  14.706 (     83 µs) │    24     0 │ Get T<02>DL<00><FE><01>T<02>DL<00><FF><FE><00><01>hca<00><15><01><15><13> => not_found
│ 7   s  │ ______________________`                                 │ T+ 14.707 ~  14.710 (      3 µs) │    24       │ Set T<02>DL<00><FE><01>T<02>DL<00><FF><FE><00><01>hca<00><15><01><15><13> = ''
│ 8   // │ ______________________`                                 │ T+ 14.714                        │     -     - │ // Allocated prefix 19 from window [0..63] (2 used)
│:8   // │ ______________________`                                 │ T+ 14.717                        │     -     - │ // Ensure that there is no data already present under prefix T<02>DL<00><15><13>
│:8   R  │ ______________________;                                 │ T+ 14.720 ~  14.825 (    105 µs) │    14     0 │ GetRange fGE{T<02>DL<00><15><13>} <= k < fGE{T<02>DL<00><15><14>}, limit(1), Exact => 0 result(s)
│ 9   // │ ______________________`                                 │ T+ 14.828                        │     -     - │ // Ensure that the prefix T<02>DL<00><15><13> has not already been allocated
│:9   R  │ ______________________:`                                │ T+ 14.831 ~  14.921 (     90 µs) │    24     0 │ Snapshot.GetRange fGE{T<02>DL<00><FE><00>} <= k < fGE{T<02>DL<00><FE><01>T<02>DL<00><FF><15><13><00><00>}, limit(1), reverse, Exact => 0 result(s)
│ 10  R  │ _______________________;                                │ T+ 14.927 ~  15.033 (    105 µs) │    32     0 │ Snapshot.GetRange fGE{T<02>DL<00><FE><01>T<02>DL<00><FF><15><13><00>} <= k < fGE{T<02>DL<00><FE><01>T<02>DL<00><FF><15><14><00>}, limit(1), Exact => 0 result(s)
│ 11  // │ _______________________`                                │ T+ 15.038                        │     -     - │ // Registering the new prefix T<02>DL<00><15><13> into the folder sub-tree
│:11  s  │ _______________________`                                │ T+ 15.040 ~  15.043 (      3 µs) │    31       │ Set T<02>DL<00><FE><01>T<02>DL<00><FF><FE><00><14><02>Second<00> = T<02>DL<00><15><13>
│ 12  s  │ _______________________`                                │ T+ 15.044 ~  15.045 (      1 µs) │    23       │ Set T<02>DL<00><FE><01>T<02>DL<00><FF><15><13><00><01>layer<00> = ''
│ 13  s  │ _______________________`                                │ T+ 15.051 ~  15.052 (      1 µs) │    43       │ Set T<02>DL<00><15><13> = 'This belongs to the second directory'
│ 14  Co°│ _________________________________________$############# │ T+ 22.086 ~  27.530 (  5,445 µs) │             │ Commit
└────────┴─────────────────────────────────────────────────────────┴──────────────────────────────────┴─────────────┘
> Read 54 bytes and Committed 263 bytes in 27.557 ms and 1 attempt(s)

Is it possible to bundle fdb client sources and build on compilation?

Required a system-wise installed fdb client seems a tricky prerequisite.

Add support for `7.1`

Merge Directory support

Use Github actions to check correctness

FoundationDB is an awesome database with a lot of strength built out of correctness. Bindings developers can leverage the bindingTester, which is available. We should configure Github actions to check for correctness.

I propose that we are taking advantage of Github actions free-tier to use two actions:

• A first action that can be runned every hour with some degree of parallelism. It will pull the master/main branch and run in a loop the bindingTester. The goal is to make it run for a bit less that an hour.
• A second action that will be triggered on a PR with a certain label. This will trigger a long-running correctness job (2 or 3h) before merging. As it is only triggered on PR with the right label, we will be able to choose which PR needs more testing.

Note that this is not replacing #3, but more like a short-term version of #3, when:

• we want something more integrated with Github
• the needed tests are compatible with our CI, i.e. not testing an unreleased fdb version

Merge support for release-7.0

Add a `run` method

The java binding has a nice method called run:

run​(java.util.function.Function<? super Transaction,​T> retryable) Runs a transactional function against this Database with retry logic.

It could be nice to be able to write this:

    db.run(|trx| {
        assert!(trx.get(b"hello", false).await?.is_none());
    }).await?;

You can set options about retry and time-limit within a transaction

Implement Eq and Hash traits for Subspace

Hello 😀

I'm currently working on a deserializer without parsing, I need to create a Map<Subspace, Element>.

But because Eq + Hash traits aren't bound to Subspace I need to wrap it into a Wrapper(Subspace) structure to satisfy the bound.

Is it possible to implement both of these traits ?

Thanks.

Fix seed 3864917676

Found an faulty seed during tests:

+ python2 ./bindings/bindingtester/bindingtester.py --num-ops 1000 --api-version 610 --test-name api --compare python rust

Creating test at API version 610
Generating api test at seed 3864917676 with 1000 op(s) and 1 concurrent tester(s)...

Inserting test into database...

Running tester '/home/runner/work/foundationdb-rs/foundationdb-rs/target/debug/bindingtester test_spec 610'...

Enable traces on BindingTester

It is fairly useful to enable traces on the bindingTester. We should be enable to trace transactions when we are detecting some env vars.

Add support for 7.2

Merge support for release-6.3

Use testcontainers to spawn an FoundationDB during tests

Currently, we are using an github action called Clikengo/foundationdb-actions-install that is installing FDB. In order to ease local testing, we could use testcontainers.

However, the rust version of testcontaienrs is not as developed as the java version, and things like ryuk are not yet implemented.

We should also be careful as we need to check if testcontainers is supporting all platform defined as Tier 1 and 2.

Update instructions to install FDB

In the crate's README, we are giving indications on how to install FDB:

foundationdb-rs/foundationdb/README.md

Lines 45 to 67 in d8a388b

	### Install FoundationDB
	Install FoundationDB on your system, see [FoundationDB Local Development](https://apple.github.io/foundationdb/local-dev.html), or these instructions:
	- Ubuntu Linux (this may work on the Linux subsystem for Windows as well)
	```console
	$> curl -O https://www.foundationdb.org/downloads/6.2.15/ubuntu/installers/foundationdb-clients_6.2.25-1_amd64.deb
	$> curl -O https://www.foundationdb.org/downloads/6.2.15/ubuntu/installers/foundationdb-server_6.2.25-1_amd64.deb
	$> sudo dpkg -i foundationdb-clients_6.2.25-1_amd64.deb
	$> sudo dpkg -i foundationdb-server_6.2.25-1_amd64.deb
	```
	- macOS
	```console
	$> curl -O https://www.foundationdb.org/downloads/6.2.25/macOS/installers/FoundationDB-6.2.25.pkg
	$> sudo installer -pkg FoundationDB-6.2.25.pkg -target /
	```
	- Windows
	https://www.foundationdb.org/downloads/6.2.25/windows/installers/foundationdb-6.2.25-x64.msi

We should update with FDB 6.3 and the new links from Github releases.

Fix 584458794

Automated tests found a bug in the directory: https://github.com/foundationdb-rs/foundationdb-rs/runs/5849103291?check_suite_focus=true

 + ./bindings/bindingtester/bindingtester.py --num-ops 1000 --api-version 630 --test-name directory --concurrency 1 rust --no-directory-snapshot-ops --compare python

Creating test at API version 630
Generating directory test at seed 584458794 with 1000 op(s) and 1 concurrent tester(s)...

Inserting test into database...

Running tester '/home/runner/work/foundationdb-rs/foundationdb-rs/target/debug/bindingtester test_spec 630'...

Reading results from '('tester_output', 'stack')'...
Reading results from '('tester_output', 'directory_log', 'directory')'...
Reading results from '('tester_output', 'directory_log', 'subspace')'...

Inserting test into database...

Running tester 'python /home/runner/work/foundationdb-rs/foundationdb-rs/target/foundationdb_build/foundationdb/bindings/bindingtester/../python/tests/tester.py test_spec 630'...

Reading results from '('tester_output', 'stack')'...
Reading results from '('tester_output', 'directory_log', 'directory')'...
Reading results from '('tester_output', 'directory_log', 'subspace')'...

Comparing results from '('tester_output', 'stack')'...

Incorrect result: 
  rust   - ('tester_output', 'stack', 184, 1684) = b'\x01RESULT_NOT_PRESENT\x00'
  python - ('tester_output', 'stack', 184, 1684) = b'\x01\x01ERROR\x00\xff\x011020\x00\xff\x00'

Incorrect result: 
  rust   - None
  python - ('tester_output', 'stack', 185, 1693) = b'\x01DIRECTORY_ERROR\x00'

Incorrect result: 
  rust   - ('tester_output', 'stack', 204, 1832) = b'\x01DIRECTORY_ERROR\x00'
  python - None

Incorrect result: 
  rust   - ('tester_output', 'stack', 206, 1842) = b'\x01DIRECTORY_ERROR\x00'
  python - None

Comparing results from '('tester_output', 'directory_log', 'directory')'...

Incorrect result: 
  rust   - ('tester_output', 'directory_log', 'directory', 5, 'children') = b'\x021\x00'
  python - ('tester_output', 'directory_log', 'directory', 5, 'children') = b''

Incorrect result: 
  rust   - ('tester_output', 'directory_log', 'directory', 7, 'children') = b'\x021\x00'
  python - ('tester_output', 'directory_log', 'directory', 7, 'children') = b''

Incorrect result: 
  rust   - ('tester_output', 'directory_log', 'directory', 130, 'children') = b'\x021\x00'
  python - ('tester_output', 'directory_log', 'directory', 130, 'children') = b''

Incorrect result: 
  rust   - ('tester_output', 'directory_log', 'directory', 131, 'children') = b''
  python - None

Incorrect result: 
  rust   - ('tester_output', 'directory_log', 'directory', 131, 'exists') = b'\x15\x01'
  python - None

Incorrect result: 
  rust   - ('tester_output', 'directory_log', 'directory', 131, 'layer') = b'\x01\x00'
  python - None

Incorrect result: 
  rust   - ('tester_output', 'directory_log', 'directory', 131, 'path') = b'\x02default1\x00'
  python - None

Incorrect result: 
  rust   - None
  python - ('tester_output', 'directory_log', 'directory', 143, 'children') = b''

Incorrect result: 
  rust   - None
  python - ('tester_output', 'directory_log', 'directory', 143, 'exists') = b'\x15\x01'

Incorrect result: 
  rust   - None
  python - ('tester_output', 'directory_log', 'directory', 143, 'layer') = b'\x01\x00'

Incorrect result: 
  rust   - None
  python - ('tester_output', 'directory_log', 'directory', 143, 'path') = b'\x021\x00'

Incorrect result: 
  rust   - None
  python - ('tester_output', 'directory_log', 'directory', 144, 'children') = b''

Incorrect result: 
  rust   - None
  python - ('tester_output', 'directory_log', 'directory', 144, 'exists') = b'\x15\x01'

Incorrect result: 
  rust   - None
  python - ('tester_output', 'directory_log', 'directory', 144, 'layer') = b'\x01\x00'

Incorrect result: 
  rust   - None
  python - ('tester_output', 'directory_log', 'directory', 144, 'path') = b'\x02default1\x00'

Comparing results from '('tester_output', 'directory_log', 'subspace')'...

Test with seed 584458794 and concurrency 1 had 19 incorrect result(s) and 0 error(s) at API version 630
Completed directory test with random seed 584458794 and 1000 operations

Error: Process completed with exit code 1.

0.7 could not compile `foundationdb`

Hello! Great work on this project; I'm excited to use Fdb and Rust for a brand new project.

My issue: When the version of foundationdb is set to 0.7, a compiler error occurs at foundationdb-0.7.0/src/transaction.rs:622:13

This can be reproduced with the following:

1. Create a new project with, e.g., cargo new fdbtest
2. Populate the [dependencies] section of Cargo.toml with foundationdb = "0.7"
3. Run cargo build
4. Expected: Project builds. Actual: A compiler error.

Full contents of Cargo.toml:

[package]
name = "fdbtest"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
foundationdb = "0.7"

Rust version:

$ rustc --version
rustc 1.66.0 (69f9c33d7 2022-12-12)

Compiler error:

$ cargo build
   Compiling proc-macro2 v1.0.51
   Compiling quote v1.0.23
   Compiling memchr v2.5.0
   Compiling unicode-ident v1.0.6
   Compiling autocfg v1.1.0
   Compiling cfg-if v1.0.0
   Compiling libc v0.2.139
   Compiling syn v1.0.109
   Compiling glob v0.3.1
   Compiling log v0.4.17
   Compiling hashbrown v0.12.3
   Compiling futures-core v0.3.26
   Compiling termcolor v1.2.0
   Compiling minimal-lexical v0.2.1
   Compiling os_str_bytes v6.4.1
   Compiling regex-syntax v0.6.28
   Compiling libloading v0.7.4
   Compiling clap_lex v0.2.4
   Compiling strsim v0.10.0
   Compiling futures-channel v0.3.26
   Compiling bitflags v1.3.2
   Compiling indexmap v1.9.2
   Compiling slab v0.4.8
   Compiling futures-task v0.3.26
   Compiling bindgen v0.60.1
   Compiling textwrap v0.16.0
   Compiling humantime v2.1.0
   Compiling clang-sys v1.6.0
   Compiling either v1.8.1
   Compiling futures-util v0.3.26
   Compiling futures-sink v0.3.26
   Compiling shlex v1.1.0
   Compiling aho-corasick v0.7.20
   Compiling nom v7.1.3
   Compiling lazycell v1.3.0
   Compiling rustc-hash v1.1.0
   Compiling lazy_static v1.4.0
   Compiling peeking_take_while v0.1.2
   Compiling xml-rs v0.8.4
   Compiling futures-io v0.3.26
   Compiling getrandom v0.2.8
   Compiling pin-utils v0.1.0
   Compiling atty v0.2.14
   Compiling which v4.4.0
   Compiling pin-project-lite v0.2.9
   Compiling ppv-lite86 v0.2.17
   Compiling async-trait v0.1.64
   Compiling clap v3.2.23
   Compiling static_assertions v1.1.0
   Compiling uuid v1.3.0
   Compiling rand_core v0.6.4
   Compiling rand_chacha v0.3.1
   Compiling foundationdb-gen v0.7.0
   Compiling rand v0.8.5
   Compiling regex v1.7.1
   Compiling foundationdb v0.7.0
   Compiling cexpr v0.6.0
   Compiling env_logger v0.9.3
   Compiling futures-macro v0.3.26
   Compiling async-recursion v1.0.2
   Compiling foundationdb-macros v0.1.1
   Compiling foundationdb-sys v0.7.0
   Compiling futures-executor v0.3.26
   Compiling futures v0.3.26
error[E0061]: this function takes 18 arguments but 17 arguments were supplied
   --> /Users/adam/.cargo/registry/src/github.com-1ecc6299db9ec823/foundationdb-0.7.0/src/transaction.rs:622:13
    |
622 |               fdb_sys::fdb_transaction_get_mapped_range(
    |  _____________^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^-
623 | |                 self.inner.as_ptr(),
624 | |                 key_begin.as_ptr(),
625 | |                 fdb_len(key_begin.len(), "key_begin"),
...   |
639 | |                 fdb_bool(opt.reverse),
640 | |             )
    | |_____________- an argument of type `i32` is missing
    |
note: function defined here
   --> /Users/adam/fdbtest/target/debug/build/foundationdb-sys-9bc961251a948a75/out/bindings.rs:899:12
    |
899 |     pub fn fdb_transaction_get_mapped_range(
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: provide the argument
    |
622 |             fdb_sys::fdb_transaction_get_mapped_range(self.inner.as_ptr(), key_begin.as_ptr(), fdb_len(key_begin.len(), "key_begin"), fdb_bool(begin.or_equal()), begin.offset(), key_end.as_ptr(), fdb_len(key_end.len(), "key_end"), fdb_bool(end.or_equal()), end.offset(), mapper.as_ptr(), fdb_len(mapper.len(), "mapper_length"), fdb_limit(opt.limit.unwrap_or(0)), fdb_limit(opt.target_bytes), opt.mode.code(), fdb_iteration(iteration), fdb_bool(snapshot), fdb_bool(opt.reverse), /* i32 */)
    |                                                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For more information about this error, try `rustc --explain E0061`.
error: could not compile `foundationdb` due to previous error

Use `run` in the class-sheduling example

Publish `0.6.0` on crates.io

As soon as #38 is fixed, we should release the current version to crates.io.

From what I can see, @yjh0502, @bluejekyll and @Speedy37 are listed as owners in the crate page.

Could you add me as owner of the crate? Or we could setup a Github Team to ease management. WDYT?

Found a faulty seed 4203526853

Scheduled correctness found an faulty seed:

+ ./bindings/bindingtester/bindingtester.py --num-ops 1000 --api-version 630 --test-name directory --concurrency 1 rust --no-directory-snapshot-ops --compare python

Creating test at API version 630
Generating directory test at seed 4203526853 with 1000 op(s) and 1 concurrent tester(s)...

Inserting test into database...

Running tester '/home/runner/work/foundationdb-rs/foundationdb-rs/target/debug/bindingtester test_spec 630'...

Reading results from '('tester_output', 'stack')'...
Reading results from '('tester_output', 'directory_log', 'directory')'...
Reading results from '('tester_output', 'directory_log', 'subspace')'...

Inserting test into database...

Running tester 'python /home/runner/work/foundationdb-rs/foundationdb-rs/target/foundationdb_build/foundationdb/bindings/bindingtester/../python/tests/tester.py test_spec 630'...

Reading results from '('tester_output', 'stack')'...
Reading results from '('tester_output', 'directory_log', 'directory')'...
Reading results from '('tester_output', 'directory_log', 'subspace')'...

Comparing results from '('tester_output', 'stack')'...

Incorrect result: 
  rust   - ('tester_output', 'stack', 12, 155) = b'\x01DIRECTORY_ERROR\x00'
  python - None

Comparing results from '('tester_output', 'directory_log', 'directory')'...

Incorrect result: 
  rust   - ('tester_output', 'directory_log', 'directory', 5, 'children') = b'\x021\x00'
  python - ('tester_output', 'directory_log', 'directory', 5, 'children') = b'\x022\x00'

Incorrect result: 
  rust   - ('tester_output', 'directory_log', 'directory', 8, 'exists') = b'\x15\x01'
  python - ('tester_output', 'directory_log', 'directory', 8, 'exists') = b'\x14'

Incorrect result: 
  rust   - None
  python - ('tester_output', 'directory_log', 'directory', 15, 'children') = b''

Incorrect result: 
  rust   - None
  python - ('tester_output', 'directory_log', 'directory', 15, 'exists') = b'\x15\x01'

Incorrect result: 
  rust   - None
  python - ('tester_output', 'directory_log', 'directory', 15, 'layer') = b'\x01\x00'

Incorrect result: 
  rust   - None
  python - ('tester_output', 'directory_log', 'directory', 15, 'path') = b'\x023\x00\x022\x00'

Comparing results from '('tester_output', 'directory_log', 'subspace')'...

Test with seed 4203526853 and concurrency 1 had 7 incorrect result(s) and 0 error(s) at API version 630
Completed directory test with random seed 4203526853 and 1000 operations

need to update bindgen for dependencies

I run cargo audit as part of my build pipeline. It spit out an error today:

 cargo audit
    Fetching advisory database from `https://github.com/RustSec/advisory-db.git`
      Loaded 592 security advisories (from /usr/local/cargo/advisory-db)
    Updating crates.io index
    Scanning Cargo.lock for vulnerabilities (217 crate dependencies)
Crate:     shlex
Version:   1.2.0
Title:     Multiple issues involving quote API
Date:      2024-01-21
ID:        RUSTSEC-2024-0006
URL:       https://rustsec.org/advisories/RUSTSEC-2024-0006
Solution:  Upgrade to >=1.3.0
Dependency tree:
shlex 1.2.0
├── bindgen 0.69.2
│   └── fdb-sys 0.4.0
│       └── fdb 0.4.0
│           └── fdblibrs 0.1.1
└── bindgen 0.65.1
    └── foundationdb-sys 0.8.0
        └── foundationdb 0.8.0

add a `update_metadata_version`

More context here: apple/foundationdb#1213

Move the repo from `Clikengo` to `FoundationDB-rs`

This meta-issue is set to keep track of the work that needs to be done in order to complete the move from Clikengo to foundationdb-rs

Context

Why moving the repo?

The original repo, Clikengo/foundationdb-rs is no longer active. There is no activity from Clikengo's employees on any repo owned by Clikengo. PRs are stacking and cannot be merged, as former maintainers like @Speedy37 only have simple contributor rights to the project. Also, developers like @Diggsey or @rajivr have forked the repo and are using their fork with various branches and/or fixes.

A change of main repository to merge the efforts seems the most logical to us for the sustainability of the project and the community. The overall goal is to improve and maintain the Rust binding as a community-maintained bindings. At that point, it could even be moved to the official FoundationDB's Github org.

Tasks

• add maintainers from crates.io back. I added back the owners of the crate(according to crates.io) @bluejekyll, @Speedy37 and @yjh0502 as maintainers of the repo. Feel free to ping me if you don't want to be maintainer.
• recreate codecov.io integration
• Add other maintainers. Did I forget someone?
• Change mention of Clikengo in the repo

Feel free to comment if you are seeing other tasks that needs to be done.

Improve CI by reusing steps

Right now, cron-correctness.yml and pr-correctness.yml almost have the same steps runned. The only difference is the number of steps which can be injected as an variable.

It would be nice to import or reuse the steps.

Fix misaligned pointer dereference on Rust beta and nightly

CI link

thread 'test_range' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7f0e5986d166', foundationdb/src/future.rs:308:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'test_range' panicked at 'panic in a function that cannot unwind', library/core/src/panicking.rs:126:5

Introduce platform support

As the FDB community is officially discontinuing the Windows build(see apple/foundationdb#5135 for more info), we should clarify our posture towards platform like Windows, Apple Silicon and so on.

One way to handle this can be to introduce platform support tiers, like in Rust:

• Tier 1 targets can be thought of as "guaranteed to work", meaning that the bindingtester is running on it to check correctness,
• Tier 2 targets can be thought of as "guaranteed to build", meaning that CI are running some tests on it.

What do you think about this?

Add a `db.run_with_options` methods

Almost the same as db.run, but allowing passing a vec of transactions options

Issues in the Class Scheduling example (class-scheduling.rs)

First of all, I want to say I think this example code is extremely valuable. Many details of using the library are non-obvious after reading the documentation, and a "hello world" example would definitely not be enough. It's also helpful to be able to compare the Rust code with the Python equivalent from the official documentation.

These two bugs don't cause obvious failures when running the code unmodified, because get_available_classes() is unlikely to be called, and transaction conflicts are rare, but they show up immediately if you reduce the class count and seat count both to 2:

• get_available_classes() unpacks the keys as single strings, but they are tuples of two strings (subspace prefix and class name) so it will always fail
• The Transaction::get() calls are passing snapshot=true, so the transactions don't get canceled and retried if there is a conflict, and the class counts are not atomically decremented; you end up with too many students per class.

This relates to #71 and #72:

• Example code uses get_range() instead of get_ranges_keyvalues(), which is not recommended, and it's confusing (what does the iteration=1_024 parameter do?)

Also:

• main() function does not drop(_guard) as advised elsewhere.
• Code uses Subspace in some places, and just tuples in other places, which is confusing when comparing with the Python version.
• Issue with the logic for the switch operation: if the student is already in the new class, signup_trx returns Ok(()), but the old class is still "ditched". Would it be good to use this as an example of how to cancel a transaction?
• There are a couple of code comments that I guess relate to internals of the library (or an early version of it?) and don't make much sense to someone new to the project.

The Transaction::get() snapshot=true issue is by far the most serious, since it could cause hard-to-spot bugs if someone copies and pastes into their own code.

Deprecate support for 5.X

5.0.0 version has been released in 3 August 2018, which is starting to be quite old. We should remove the relevant code.

Is anyone using newest version of the crate with old fdb versions ?

RUSTSEC-2021-0139: ansi_term is Unmaintained

ansi_term is Unmaintained

Details
Status	unmaintained
Package	ansi_term
Version	0.12.1
URL	ogham/rust-ansi-term#72
Date	2021-08-18

The maintainer has advised that this crate is deprecated and will not receive any maintenance.

The crate does not seem to have much dependencies and may or may not be ok to use as-is.

Last release seems to have been three years ago.

Possible Alternative(s)

The below list has not been vetted in any way and may or may not contain alternatives;

• ansiterm
• anstyle
• console
• nu-ansi-term
• owo-colors
• stylish
• yansi

Dependency Specific Migration(s)

• structopt, clap2

See advisory page for additional details.

Check seed 2911707775

Automatic correctness found an issue on this run:

+ ./bindings/bindingtester/bindingtester.py --num-ops 1000 --api-version 630 --test-name api --compare python rust

Creating test at API version 630
Generating api test at seed 2911707775 with 1000 op(s) and 1 concurrent tester(s)...

Inserting test into database...

Running tester '/home/runner/work/foundationdb-rs/foundationdb-rs/target/debug/bindingtester test_spec 630'...

Reading results from '('tester_output', 'workspace')'...
Reading results from '('tester_output', 'stack')'...

Inserting test into database...

Running tester 'python /home/runner/work/foundationdb-rs/foundationdb-rs/target/foundationdb_build/foundationdb/bindings/bindingtester/../python/tests/tester.py test_spec 630'...

Reading results from '('tester_output', 'workspace')'...
Reading results from '('tester_output', 'stack')'...

Comparing results from '('tester_output', 'workspace')'...
Comparing results from '('tester_output', 'stack')'...

Incorrect result: 
  rust   - ('tester_output', 'stack', 205, 17453) = b'\x01\x01ERROR\x00\xff\x011025\x00\xff\x00'
  python - ('tester_output', 'stack', 205, 17453) = b'\x01\x02tester_output\x00\xff\x02workspace\x00\xff\x00'


Test with seed 2911707775 and concurrency 1 had 1 incorrect result(s) and 0 error(s) at API version 630
Completed api test with random seed 2911707775 and 1000 operations

We need to check if the error can be reproduced

Bump dependencies

Before releasing the 0.8.0, we should bump the dependenciesMost of the work will be to:

• run cargo upgrade,
• rework the foundationdb-macros crate, as the public api of syn changed.

Move default branch to `main`

Clarify the correct use of unsafe

Hello!

I love that this crate exists. Thanks for all your hard work ❤️

After reading the documentation and examples, I have some confusion about the use of unsafe and drop in this code:

#[tokio::main]
async fn main() {
    let network = unsafe { foundationdb::boot() };
    // do some interesting things with the API...
    drop(network);
}

Which is in the documentation for foundationdb::boot and in the hello world example.

My concern is not about the use of unsafe in general. It's about how to use this pattern correctly when adopting this library. The documentation describes this constraint:

You MUST ensure drop is called on the returned object before the program exits. This is not required if the program is aborted.

Which leaves me with two questions:

1. Doesn't Rust call drop automatically when network goes out of scope - why must I call it manually?
2. What happens if my code panics before calling drop?

If you could help me understand this better, I'm happy to raise a PR updating the documentation with a deeper description of the motivations and usage recommendations.

Add clarification about get_range* methods

Following #71 and our experience, we should highly highlight the fact that get_range should be avoided by most users.

Add support for 7.3

Query issue

We have a process performing many range queries using Transaction.get_range. Each query has a limit of 50 rows. At some point mid-way through processing the query returned only 1 row instead of the expected 50 (there were 1000s of rows left in the range). The query uses the default Iterator streaming mode; we tried WantAll too but it did the same thing. This is using version 0.6.0 of the library, running the 6.3.23 version of FoundationDb. I have never seen this occur in years of usage. What is even more puzzling is that we took that specific query with exactly the same keys and ran it through a testing tool we built (also using this library and version) and it worked perfectly. It's only when we did it in a big loop that it didn't bring back all we expected.

I suspect that this is because we never implemented GetRangeResult.next to check if there are more results. 2 questions:

• Could not checking GetRangeResult.next cause this, as sometimes with heavier load the number of result rows that come back can be less than what was requested?
• Can the same thing happen with the WantAll mode? We did see exactly the same behavior with this. If so we'll go back to Iterator mode and implement GetRangeResult.next as documented.

correctness: fix seed 3153055325

Scheduled correctness found a issue:

+ ./bindings/bindingtester/bindingtester.py --num-ops 1000 --api-version 630 --test-name api --concurrency 5 rust

Creating test at API version 630
Generating api test at seed 3153055325 with 1000 op(s) and 5 concurrent tester(s)...

Inserting test into database...

Running tester '/home/runner/work/foundationdb-rs/foundationdb-rs/target/debug/bindingtester test_spec 630'...
thread 'b"thread\x5fspec3"' panicked at 'attempt to subtract with overflow', foundationdb-bindingtester/src/main.rs:718:72
stack backtrace:
   0: rust_begin_unwind
             at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/panicking.rs:498:5
   1: core::panicking::panic_fmt
             at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/core/src/panicking.rs:107:14
   2: core::panicking::panic
             at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/core/src/panicking.rs:48:5
   3: bindingtester::StackMachine::run_step::{{closure}}
             at /home/runner/work/foundationdb-rs/foundationdb-rs/foundationdb-bindingtester/src/main.rs:718:72
   4: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/core/src/future/mod.rs:80:19
   5: bindingtester::StackMachine::run::{{closure}}
             at /home/runner/work/foundationdb-rs/foundationdb-rs/foundationdb-bindingtester/src/main.rs:1578:21
   6: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/core/src/future/mod.rs:80:19
   7: futures_executor::local_pool::block_on::{{closure}}
             at /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.19/src/local_pool.rs:315:23
   8: futures_executor::local_pool::run_executor::{{closure}}
             at /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.19/src/local_pool.rs:90:37
   9: std::thread::local::LocalKey<T>::try_with
             at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/thread/local.rs:399:16
  10: std::thread::local::LocalKey<T>::with
             at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/thread/local.rs:375:9
  11: futures_executor::local_pool::run_executor
             at /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.19/src/local_pool.rs:86:5
  12: futures_executor::local_pool::block_on
             at /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.19/src/local_pool.rs:315:5
  13: bindingtester::StackMachine::run_step::{{closure}}::{{closure}}
             at /home/runner/work/foundationdb-rs/foundationdb-rs/foundationdb-bindingtester/src/main.rs:1454:29
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'main' panicked at 'joined thread to not panic: Any { .. }', foundationdb-bindingtester/src/main.rs:1586:27
stack backtrace:
   0: rust_begin_unwind
             at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/std/src/panicking.rs:498:5
   1: core::panicking::panic_fmt
             at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/core/src/panicking.rs:107:14
   2: core::result::unwrap_failed
             at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/core/src/result.rs:1613:5
   3: core::result::Result<T,E>::expect
             at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/core/src/result.rs:1255:23
   4: bindingtester::StackMachine::join
             at /home/runner/work/foundationdb-rs/foundationdb-rs/foundationdb-bindingtester/src/main.rs:1586:13
   5: bindingtester::main
             at /home/runner/work/foundationdb-rs/foundationdb-rs/foundationdb-bindingtester/src/main.rs:1638:5
   6: core::ops::function::FnOnce::call_once
             at /rustc/02072b482a8b5357f7fb5e5637444ae30e423c40/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

'/home/runner/work/foundationdb-rs/foundationdb-rs/target/debug/bindingtester' did not complete succesfully (exit code: 101)

Test of type api failed to complete successfully with random seed 3153055325 and 1000 operations

Error: Process completed with exit code 2.

Source: https://github.com/foundationdb-rs/foundationdb-rs/runs/4811486807?check_suite_focus=true#step:7:6949

Fix infinite recursion on Display impl

Backport other recipes

The official documentation has design recipes that we can backport to Rust:

• Blob: Store binary large objects (blobs) in the database.

• Hierarchical Documents: Create a representation for hierarchical documents.

• Multimaps: Create a multimap data structure with multiset values.

• Priority Queues: Create a data structure for priority queues supporting operations for push, pop_min, peek_min, pop_max, and peek_max.

• Queues: Create a queue data structure that supports FIFO operations.

• Segmented Range Reads: Perform range reads in calibrated batches.

• Simple Indexes: Add (one or more) indexes to allow efficient retrieval of data in multiple ways.

• Spatial Indexing: Create a spatial index for the database.

• Subspace Indirection: Employ subspace indirection to manage bulk inserts or similar long-running operations.

• Tables: Create a table data structure suitable for sparse data.

• Vector: Create a vector data structure.

RUSTSEC-2021-0145: Potential unaligned read

Potential unaligned read

Details
Status	unsound
Package	atty
Version	0.2.14
URL	softprops/atty#50
Date	2021-07-04

On windows, atty dereferences a potentially unaligned pointer.

In practice however, the pointer won't be unaligned unless a custom global allocator is used.

In particular, the System allocator on windows uses HeapAlloc, which guarantees a large enough alignment.

atty is Unmaintained

A Pull Request with a fix has been provided over a year ago but the maintainer seems to be unreachable.

Last release of atty was almost 3 years ago.

Possible Alternative(s)

The below list has not been vetted in any way and may or may not contain alternatives;

• std::io::IsTerminal - Stable since Rust 1.70.0
• is-terminal - Standalone crate supporting Rust older than 1.70.0

See advisory page for additional details.