n0-computer / iroh-ffi

FFI bindings for iroh

iroh-ffi

Foreign Function Interface (FFI) bindings for Iroh

This repo defines Python, Swift, and Kotlin bindings for iroh, which is written in Rust.

Published Packages:

Python: pypi Swift: Cocoapods Swift: SwifPM Rust: Crates

Repo Status: Tier 2

This is a "tier 2" repo, which means it's a repo we care about, but don't apply the same level of rigor as a "tier 1" repo. All work is done through pull requests and must pass continuous integration, but the peer review process is much lighter, and our reaction times to issues will not be as fast as tier 1 repositories.

If you're blocked on something or need to draw attention to an issue, please reach out to the iroh discord.

Library Compilation

Running cargo build --release will produce a dynamic library and a static library.

For builds targeting older versions of macOS, build with: MACOSX_DEPLOYMENT_TARGET=10.7 && cargo build --target x86_64-apple-darwin --release.

Language-Specific READMEs

• Swift readme
• Python readme
• Kotlin readme

Developers

Check our DEVELOPERS.md for guides on how to translate from the iroh rust API to the iroh FFI API, as well as how to set up testing for Golang and Python.

License

This project is licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

iroh-ffi's Issues

test(swift): write basic swift tests

document translation between the iroh api and the iroh ffi bindings

add to readme

• translating parameters
• event progress notes
• streams: add note to README that any lists are, for now, collected and returned as vecs, w/ a note detailing how to file and issue if that doesn't work for the user's use case
• paths instead of readers / streaming: add note to README that we are not currently streaming through the language barriers, and are only accepting paths to import from and paths to export to. Add note to file an issue if that doesn't work for the user's use case
• each api method gets its own IrohError with the method transformed into CamelCase
• each struct should be represented by an interface with getter methods
  (and setter methods if appropriate)
• enums example
• install pytest for python testing
• virtualenv instructions

[node/js] Doc Subscription

The API is a bit awkward right now. It blocks everything (sync?) so I can't find an easy way of having it run in the background or just polling it occasionally.

In the example below, the setBytes function runs correctly, but then when I wait for getMany it never returns, due to the subscription function. If I remove the subscribe(), it can query successfully.

console.log("Listening...");
listen();

const key = Buffer.from("foo");
console.log("Setting bytes");
doc.setBytes(author, key, val);

const q = Query.keyExact(key);

console.log("Querying...");
const entries = await doc.getMany(q);

// -----> program never gets here
console.log(`Entries ---- ${entries.length}`);

async function listen() {
  console.log("Subscribing");
  const sub = await doc.subscribe();
  for (const event of sub) {
    const keys = Object.keys(event);
    for (let type of keys) {
      console.log("Event:", type);
      const params = event[type];
      console.log("Data:", params);
    }
  }
}

[node/js] Doc not syncing

I'm trying a basic doc sync test, code here:

https://gist.github.com/mattdesl/efdedcf47494a24946273d3b01d2fd79

server.js opens a node from a local tmp dir and subscribes for doc changes
peer.js runs a new tmp node that changes the "foo" key to a random value

If I close the server, I can re-open it but it will not behave correctly anymore.

Steps to Bug:

1. Run node src/server.js in terminal
2. Once its running, close the terminal process with Cmd + C
3. Then run it again, node src/server.js and copy the ticket
4. Now in another terminal, run node src/peer.js [copied_ticket]

The server will not print any doc changes. However, if you remove the data dir rm -rf tmp/ and redo the steps without step 2 (exiting process), each time you run the peer script it will print doc events in the server. It is just that closing and re-opening the server seems to be the issue.

Another more general issue I have with this, how can a remote node wait for the 'content ready' event on other nodes? Right now I have a delay of 5 seconds which feels like a hack. I'd rather wait until I know the content has been synced, and then close.

process for cutting FFI-bound language releases

Generally, I think we should:

0. get iroh-ffi up to parity with commands listed in https://iroh.computer/docs/api
1. cut an iroh release
2. audit any braking changes that affect iroh-ffi, make said chainges
3. cut release in each downstream language (currently: python, swift, go)

Open questions

1. Should we try to match version numbers to the main iroh version number? That would make life much easier for end users, and us as we deal with bugs. however, this can be hard to achieve in practice if you ever mess up a release on any platform

go: publish go module

Setup the repo so things can be nicely used using go mod

expose tags API over FFI

as of v0.6.0 we have a tagging API that isn't available over FFI. Let's expose it.

Cannot build kotlin bindings on Windows

Branch; feat-kotlin
Platform: windows
NDK: 26.1.10909125

Hi all! Figured I'd try to build the kotlin bindings today.

• I have added the aarch64-linux-android target with rustup
• NDK is present and is in path
• Noticed that in .cargo/config.toml the archiver used is not present in the prebuilt NDK toolchain but there was a llvm-ar shipped within that toolchain, so I modified the line to ar = "llvm-ar"

I'm getting a weird failure when the build reaches the redb crate compilation:

> cargo build --target aarch64-linux-android --release
   Compiling redb v1.1.0
error: linker `aarch64-linux-android29-clang` not found
  |
  = note: program not found

error: could not compile `redb` (lib) due to previous error

The curious thing is that aarch64-linux-android29-clang is on the path and works fine ( I'm guessing it worked for all the other packages that where compiled before redb)

Any direction would be appreciated !

Potentially remove typed structs for easier user experience

I just did this work to NamespaceId, whose struct only existed to get the namespace id from a string and back.

Here are a list of structs that could also potentially be changed, but I'm still auditing the full effects of what that change might be:

• AuthorId
• DocTicket
• Hash
• Url
• Tag
• SocketAddr

AuthorId

This would only really effect Query and the QueryBuilder, adding a bunch of Result<Query> rather than Query returns. This is potentially fine, I take a look at how the API would feel with those additions.

DocTicket

No downsides to removing this

Hash

Returning and accepting a String rather than the Hash feels a bit gross, especially since so many methods return and accept Hashes, which means that if we convert a Hash to a String to return it in one method, we will immediately have to convert it back in the next. The user would also lose the to_hex method.

Url

These will convert to DerpUrls anyway. There is very minimal impact to the current API to get rid of this, i think we should.

Tag

no downside to removing this

SocketAddr

Would have the same impact as Url, which I definitely think can be removed safely.

write powershell version of `./make_go.sh` script, so windows devs can build the go api correctly

refactor: switch to async methods

Avoid using block_on and use uniffis ability to generate async calls

publish kotlin bindings to maven

publish updated python module with latest iroh

v0.6.0 is out! ffi bindings have gotten better! we should update & publish

Make clear on published packages which iroh version they are

Currently the pypi package 0.2.0 is for iroh 0.6.0, but there is no indication whatsoever to the users about this. We should make it clear in some way what version it refers to.

Some options:

1. Match the version numbers.

   This could be tricky when we need to do new releases in bindings without new iroh releases.

2. Document prominently in the README with a table or similar.

   This would need manual updating and is easy to become stale and make mistakes.

3. ??

add CI for node-js builds and tests

running python bindings, transfers fail if no derper is available

Running two nodes on to machines on a local network. If a derper is not available (ie cut off from outside traffic) the nodes fail to connect directly with the ticket they exchanged.
The same setup works with regular nodes with iroh start and iroh get --ticket

discussion: use `client::Iroh` as a model for the API, rather than the console commands?

we are currently using the commands listed in the docs as our model for the FFI bindings. But if we want folks to use iroh as a library the way the can use rust, shouldn't we model the API off the methods on the client::Iroh struct instead?

Python bindings do not persist secret key

It seems that python bindings generate a new secret key every time.

add instructions in `DEVELOPERS.md` for how to upload a wheel to pipy

assumption is you have a pipy account, and have been added to the iroh project.

to upload to pypi you'll have to:

• install twine (pip install twine)
• create an API token on pypi
• put that token into ~/.pypirc
• download the artefacts from https://github.com/n0-computer/iroh-ffi/actions/workflows/wheels.yml picking the main branch build of your commit
• extract all the zip files
• run twine upload [--repository testpypi] iroh-$version-*.whl

to use testpypi there are a few more steps, including having a test.pipy account:
pypirc:
pypi]
username = token
password = pypi-AgE... paste token here

Steps to run tests:

Assuming we are testing on linux/macos boxes

For go:

• install go
• run ./make_go.sh
• run tests using:

LD_LIBRARY_PATH="${LD_LIBRARY_PATH:-}:<binaries path>" \
CGO_LDFLAGS="-liroh -L <binaries path>" \
go test ./...

is iroh-ffi/target/debug when running this locally

For python:

• install python
• install virtualenv
• create & activate a virtual env, for example:

virtualenv venv
source venv/bin/activate

• install maturin
• run maturin develop
• install pytest
• run python -m pytest

bug: rust tests fail

running cargo test currently fails.

implement efficient streaming apis

This depends on the language and needs some manual work.

Golang

We need to use file descriptors for this, for an example of the go side this would look like this
https://github.com/filecoin-project/filecoin-ffi/blob/master/proofs.go#L208-L214

and the rust side looks sth like this
https://github.com/filecoin-project/filecoin-ffi/blob/master/rust/src/proofs/types.rs#L8-L40

For windows this can also be done, but needs specific care, as the apis will be slightly different.

Swift

TODO

Python

TODO

APIs that would benefit from this

TODO

doc subscription events do not have payloads

the rust equivalent has all sorts of useful goodies that aren't present in the FFI bindings. let's expose them to make subscriptions more useful, and you know, parity.

`Package.swift` is not updated when we cut a new release of `iroh-ffi`

our iOS quickstart leads users to install a swift package dependency, which relies on the root Package.swift: https://github.com/n0-computer/iroh-ffi/blob/main/Package.swift

Looks like this file hasn't been updated in 5 months!

Steps to fix:

• do a "manual upload" to patch the current v0.12.0 release
• ensure the release is updated on each release, either with documentation that a number 0 engineer can run on each release, or (ideally) a CI script that updates the file

add `blobs.read_at` and `blobs.get_collection`

n0-computer/iroh#1962

Python: Collection.store() and IrohNode.blobs_add_bytes_named() missing

As the title says: I saw those methods referenced in the Rust docs, but I don't see the methods in the Python bindings. They'd be very nice to have. :)

new bindings: kotlin

Insane API tracking issue

Tracking issue for parity with the entire Iroh API (with a few exceptions, specifically methods that take a stream or reader as an input).

iroh client

• node.stats

/// Get statistics of the running node.
pub fn node_stats(&self) -> Result<HashMap<String, CounterStats>, IrohError>

• node.connections

/// Get information about the different connections we have made
/// Note: this allocates for each `ConnectioInfo`, if you have many `ConnectionInfo`s this may be a prohibitively large list.
/// Please file an [issue](https://github.com/n0-computer/iroh-ffi/issues/new) if you run into this issue
pub fn node_connections(&self) -> Result<Vec<ConnectionInfo>, IrohError>

• node.connection_info

/// Get connection information about a node
pub fn node_connection_info(&self, node_id: PublicKey) -> Result<Option<ConnectionInfo>, IrohError>

• node.status

/// Get status information about a node
pub fn node_status(&self) -> Result<NodeStatusResponse, IrohError>

• node.shutdown

/// Shutdown the node.
/// If `force` is true, the node will be killed instantly without waiting for things to
/// shutdown gracefully.
pub fn node_shutdown(&self, force: bool) -> Result<(), IrohError> 

• docs.create

/// Create a new document.
pub fn docs_create(&self) -> Result<Doc, IrohError>

• docs.delete

/// Delete a document from the local node.
///
/// This is a destructive operation. Both the document secret key and all entries in the
/// document will be permanently deleted from the node's storage. Content blobs will be deleted
/// through garbage collection unless they are referenced from another document or tag.
pub fn docs_drop_doc(&self, doc_id: NamespaceId) -> Result<(), IrohError>

• docs.import

 /// Import a document from a ticket and join all peers in the ticket.
 pub fn docs_import(&self, ticket: DocTicket) -> Result<Doc, IrohError>

• docs.list

/// List all documents.
pub fn docs_list(&self) -> Result<Vec<NamespaceId>, IrohError>

• docs.open

/// Get a [`Doc`] client for a single document. Return None if the document cannot be found.
pub fn docs_open(&self, id: NamespaceId) -> Result<Option<Doc>, IrohError>

• authors.create

/// Create a new document author.
pub fn authors_create(&self) -> Result<AuthorId, IrohError>

• authors.list

/// List document authors for which we have a secret key.
///
/// Note: this allocates for each `AuthorId`, if you have many `AuthorId`s this may be a prohibitively large list.
/// Please file an [issue](https://github.com/n0-computer/iroh-ffi/issues/new) if you run into this issue
pub fn authors_list(&self) -> Result<Vec<AuthorId>, IrohError>

• tags.list

/// List all tags.
/// Note: this allocates for each `ListTagsResponse`, if you have many `Tags`s this may be a prohibitively large list.
/// Please file an [issue](https://github.com/n0-computer/iroh-ffi/issues/new) if you run into this issue
pub fn tags_list(&self) -> Result<ListTagsResponse, IrohError> 

• tags.delete

/// Delete a tag.
pub fn tags_delete(&self, name: Tag) -> Result<(), IrohError>

• blobs_size FFI METHOD ONLY

/// Get the size information on a single blob.
///
pub fn blobs_size(&self, hash: Hash) -> Result<u64, IrohError>

• blobs_write_to_path FFI METHOD ONLY

/// Export the blob contents to a file path
/// The `path` field is expected to be the absolute path.
pub fn blobs_write_to_path(&self, hash: Hash, path: String) -> Result<(), IrohError>

• blobs.read_to_bytes

/// Read all bytes of single blob.
///
/// This allocates a buffer for the full blob. Use only if you know that the blob you're
/// reading is small. If not sure, use [`Self::blobs_size`] and check the size with
/// before calling [`Self::blobs_read_to_bytes`].
pub fn blobs_read_to_bytes(&self, hash: Hash) -> Result<Vec<u8>, IrohError>

• blobs.add_from_path

/// Import a blob from a filesystem path.
///
/// `path` should be an absolute path valid for the file system on which
/// the node runs.
/// If `in_place` is true, Iroh will assume that the data will not change and will share it in
/// place without copying to the Iroh data directory.
pub fn blobs_add_from_path(
        &self,
        path: PathBuf,
        in_place: bool,
        tag: SetTagOption,
        wrap: WrapOption,
        cb: Box<dyn AddCallback>
) -> Result<(), IrohError>

• blobs.add_bytes

    /// Write a blob by passing bytes.
    pub fn blobs_add_bytes(
        &self,
        bytes: Vec<u8>,
        tag: SetTagOption,
    ) -> Result<BlobAddOutcome, IrohError>

• blobs.validate Not implemented in core

/// Validate hashes on the running node.
///
/// If `repair` is true, repair the store by removing invalid data.
pub fn blobs_validate(
        &self,
        repair: bool,
        cb: Box<dyn ValidateCallback>
 ) -> Result<(), IrohError>

• blobs.download

/// Download a blob from another node and add it to the local database.
pub fn blobs_download(
        &self,
        req: BlobDownloadRequest,
        cb: Box<dyn DownloadCallback>
) -> Result<(), IrohError>

• blobs.list

/// List all complete blobs.
///
/// Note: this allocates for each `BlobListResponse`, if you have many `BlobListReponse`s this may be a prohibitively large list.
/// Please file an [issue](https://github.com/n0-computer/iroh-ffi/issues/new) if you run into this issue
pub fn blobs_list(&self) -> Result<Vec<BlobListResponse>, IrohError>

• blobs.list_incomplete

/// List all incomplete (partial) blobs.
///
/// Note: this allocates for each `BlobListIncompleteResponse`, if you have many `BlobListIncompleteResponse`s this may be a prohibitively large list.
/// Please file an [issue](https://github.com/n0-computer/iroh-ffi/issues/new) if you run into this issue
pub fn blobs_list_incomplete(
        &self,
) -> Result<Vec<BlobListIncompleteResponse>, IrohError>

• blobs.list_collections

/// List all collections.
///
/// Note: this allocates for each `BlobListCollectionsResponse`, if you have many `BlobListCollectionsResponse`s this may be a prohibitively large list.
/// Please file an [issue](https://github.com/n0-computer/iroh-ffi/issues/new) if you run into this issue
pub fn blobs_list_collections(
        &self,
) -> Result<Vec<BlobListCollectionsResponse>, IrohError>

• blobs.delete_blob

/// Delete a blob.
pub fn blobs_delete_blob(&self, hash: Hash) -> Result<(), IrohError>

structs and enums

• struct Doc - interface

pub struct Doc(Doc);

impl Doc {
/// Get the document id of this doc.
pub fn id(&self) -> Arc<NamespaceId>

/// Close the document.
pub fn close(&self) -> Result<(), IrohError>

/// Set the content of a key to a byte array.
pub fn set_bytes(
        &self,
        author_id: AuthorId,
        key: Vec<u8>,
        value: Vec<u8>,
) -> Result<Arc<Hash>, IrohError>

/// Set an entries on the doc via its key, hash, and size.
pub fn set_hash(
        &self,
        author_id: AuthorId,
        key: Vec<u8>,
        hash: Hash,
        size: u64,
) -> Result<(), IrohError>

/// Get the content size of an [`Entry`]
pub fn size(&self, entry: &Entry) -> Result<u64, IrohError>

/// Read all content of an [`Entry`] into a buffer.
/// This allocates a buffer for the full entry. Use only if you know that the entry you're
/// reading is small. If not sure, use [`Self::size`] and check the size with
/// before calling [`Self::read_to_bytes`].

pub fn read_to_bytes(&self, entry: &Entry) -> Result<Vec<u8>, IrohError>

/// Delete entries that match the given `author` and key `prefix`.
///
/// This inserts an empty entry with the key set to `prefix`, effectively clearing all other
/// entries whose key starts with or is equal to the given `prefix`.
///
/// Returns the number of entries deleted.
pub fn del(&self, author_id: AuthorId, prefix: Vec<u8>) -> Result<usize, IrohError>

/// Get the latest entry for a key and author.
pub fn get_one(&self, author: AuthorId, key: Vec<u8>) -> Result<Option<Entry>, IrohError>

/// Get entries.
///
/// Note: this allocates for each `Entry`, if you have many `Entry`s this may be a prohibitively large list.
/// Please file an [issue](https://github.com/n0-computer/iroh-ffi/issues/new) if you run into this issue
pub fn get_many(&self, filter: GetFilter) -> Result<Vec<Arc<Entry>>, IrohError>

/// Share this document with peers over a ticket.
pub fn share(&self, mode: ShareMode) -> anyhow::Result<Arc<DocTicket>, IrohError>

/// Start to sync this document with a list of peers.
pub fn start_sync(&self, peers: Vec<PeerAddr>) -> Result<(), IrohError>

/// Stop the live sync for this document.
pub fn leave(&self) -> Result<(), IrohError>

/// Subscribe to events for this document.
pub fn subscribe(&self, cb: Box<dyn SubscribeCallback>) -> Result<(), IrohError>

/// Get status info for this document
pub fn status(&self) -> Result<Arc<OpenState>, IrohError>

• struct CounterStats - interface

/// Counter stats
pub struct CounterStats (CounterStats);
impl CounterStats {
    /// The counter value
    pub fn value(&self) -> u64
    /// The counter description
    pub fn description(&self) -> String
}

• struct ConnectionInfo - interface

/// Details about an Endpoint
pub struct ConnectionInfo(EndpointInfo); {

impl ConnectionInfo {
    /// The public key of the peer
    pub fn public_key(&self) -> Arc<PublicKey>
    /// The derp region of the peer
    pub fn derp_region(&self) -> Option<u16>
    /// The addrs of the peer
    pub fn addrs(&self) -> Arc<DirectAddrInfo>
    /// The type of connection we have to that peer
    pub fn conn_type(&self) -> Arc<ConnectionKind>
    /// The latency of the [`ConnectionKind`]
    pub fn latency(&self) -> Option<Duration>
    /// The duration since the last time this peer was used
    pub fn last_used(&self) -> Option<Duration>
}

• struct DirectAddrInfo - interface

/// Information about a direct address
pub struct DirectAddrInfo(DirectAddrInfo);

impl DirectAddrInfo {
    /// The `SocketAddr` of the peer
    pub fn addr(&self) -> Arc<SocketAddr>
    /// The latency to the address, if any
    pub fn latency(&self) -> Option<Duration>
    /// Last control message received by this peer
    pub fn last_control(&self) -> Option<(Duration, ControlMsg)>
    /// How long ago was the last payload message for this peer
    pub last_payload(&self) -> Option<Duration>
}

• enum ControlMsg - enum

/// Type of control message we have received from the peer
pub enum ControlMsg {
    /// We received a Ping from the peer
    Ping,
    /// We received a Pong from the peer
    Pong,
    /// We received a CallMeMaybe from the peer
    CallMeMaybe
}

• enum ConnectionType - interface

/// The kind of Connection we have to the peer
pub enum ConnectionType {
    /// Direct UDP connection
    Direct(SocketAddr),
    /// Relay connection over DERP
    Relay(u16),
    /// We have no verified connection to this PublicKey
    None
}

/// The type of ConnectionType
pub enum ConnType {
    Direct,
    Relay,
    None
}

impl ConnectionType {
    /// Whether the connection is direct, a relay, or unverified (aka none)
    pub fn r#type(&self) -> ConnType

    /// Return the [`SocketAddr`] if this is a direct connection
    pub fn as_direct(&self) -> Result<SocketAddr, IrohError>

    /// Return the derp region id if this is a relay connection 
    pub fn as_relay(&self) -> Result<u16, IrohError>
}

• struct NodeStatusResponse - interface

/// The response to a version request
pub struct NodeStatusResponse(NodeStatusResponse)
impl NodeStatusResponse {
    /// The peer id and socket addresses of this node.
    pub fn peer_addr(&self) -> Arc<PeerAddr>
    /// The bound listening addresses of the node
    pub fn listen_addrs(&self) -> Vec<Arc<SocketAddr>>
    /// The version of the node
    pub fn version(&self) -> String
}

• ListTagsResponse - interface

/// A response to a list collections request
pub struct ListTagsResponse (ListTagsResponse)
impl ListTagsResponse {
    /// Name of the tag
    pub fn name(&self) -> Arc<Tag>
    /// Format of the data
    pub fn format(&self) -> BlobFormat
    /// Hash of the data
    pub fn hash(&self) -> Arc<Hash>
}

• SetTagOption - interface

pub enum SetTagOption {
    /// A tag will be automatically generated
    Auto,
    /// The tag is explicitly named
    Named(Tag),
}

impl SetTagOption {
    /// Indicate you want an automatically generated tag
    pub fn auto() -> Self
    /// Indicate you want a named tag
    pub fn named(tag: Tag) -> Self
}

• WrapOption - interface

/// Whether to wrap the added data in a collection.
pub enum WrapOption {
    /// Do not wrap the file or directory.
    NoWrap,
    /// Wrap the file or directory in a colletion.
    Wrap {
        /// Override the filename in the wrapping collection.
        name: Option<String>,
    },
}

impl WrapOption {
    /// Indicate you do not wrap the file or directory.
    pub fn no_wrap() -> Self

    /// Indicate you want to wrap the file or directory in a colletion, with an
    optional name
    pub fn wrap(name: Option<String>) -> Self
}

• BlobAddOutcome - dictionary

/// Outcome of a blob add operation.
pub struct BlobAddOutcome(BlobAddOutcome);

impl BlobAddOutcome {
    /// The hash of the blob
    pub hash: Arc<Hash>,
    /// The format the blob
    pub format: BlobFormat,
    /// The size of the blob
    pub size: u64,
    /// The tag of the blob
    pub  tag: Arc<Tag>,
}

• BlobDownloadRequest - interface

/// A request to the node to download and share the data specified by the hash.
pub struct BlobDownloadRequest(BlobDownloadRequest)
impl BlobDownloadRequest {
    /// Create a BlobDownloadRequest
    pub fn new(
        /// The hash of the data to download and share
        hash: Hash, 
        /// If the format is `BlobFormat::HashSeq`, all the children are downloaded and shared as well
        format: BlobFormat,
        /// Specifies the peer to download the data from
        peer: PeerAddr,
        /// Optional request token that can be used to authorize the download request
        token: Option<RequestToken>,
        /// Tag to tag the data with
        tag: SetTagOption,
        /// The location at which to store the data
        out: DownloadLocation) -> Self
}

• enum DownloadLocation - interface

/// Location to store a downloaded blob at.
pub enum DownloadLocation {
    /// Store in the node's blob storage directory.
    Internal,
    /// Store at the provided path.
    External {
        /// The path to store the data at.
        path: String,
        /// If this flag is true, the data is shared in place, i.e. it is moved to the
        /// out path instead of being copied. The database itself contains only a
        /// reference to the out path of the file.
        ///
        /// If the data is modified in the location specified by the out path,
        /// download attempts for the associated hash will fail.
        in_place: bool,
    },
}

impl DownloadLocation {
    /// Store in the node's blob storage directory.
    pub fn internal() -> Self

    /// Store at the provided path.
    ///
    /// If `in_place` is true, the data is shared in place, i.e. it is moved to the
    /// out path instead of being copied. The database itself contains only a
    /// reference to the out path of the file.
    ///
    /// If the data is modified in the location specified by the out path,
    /// download attempts for the associated hash will fail.
    pub fn external(path: String, in_place: bool) -> Self
}

• BlobListIncompleteResponse - interface

/// A response to a list blobs request
pub struct BlobListIncompleteResponse(BlobListIncompleteResponse);
impl BlobListIncompleteResponse {
    /// The size we got
    pub fn size(&self) -> u64
    /// The size we expect
    pub fn expected_size(&self) -> u64
    /// The hash of the blob
    pub fn hash(&self) -> Arc<Hash>
}

• BlobListCollectionsResponse - interface

/// A response to a list collections request
pub struct BlobListCollectionsResponse(BlobListCollectionsResponse);
impl BlobListCollectionsResponse {
    /// Tag of the collection
    pub fn tag(&self) -> Arc<Tag>
    /// Hash of the collection
    pub fn hash(&self) -> Arc<Hash>
    /// Number of children in the collection
    ///
    /// This is optional, because the data is not always available.
    pub fn total_blobs_count(&self) -> Option<u64>
    /// Total size of the raw data referred to by all links
    ///
    /// This is optional, because the data is not always available.
    pub fn total_blobs_size(&self) -> Option<u64>
}

• ShareMode - enum

/// Intended capability for document share tickets
pub enum ShareMode {
    /// Read-only access
    Read,
    /// Write access
    Write,
}

• OpenState - interface

/// The state for an open replica.
pub struct OpenState(OpenState);
impl OpenState {
    /// Whether to accept sync requests for this replica.
    pub fn sync(&self) -> bool
    /// How many event subscriptions are open
    pub fn subscribers(&self) -> usize
    /// By how many handles to the doc is currently held open
    pub fn handles(&self) -> usize
}

• BlobFormat

/// A format identifier
pub enum BlobFormat {
    /// Raw blob
    #[default]
    Raw,
    /// A sequence of BLAKE3 hashes
    HashSeq,
}

• PublicKey

impl PublicKey {
    fn to_string(&self) -> String
    fn to_bytes(&self) -> Vec<u8>
    fn from_string(&self, str: String) -> Result<PublicKey, IrohError>
    fn from_bytes(&self, bytes: Vec<u8>) -> Result<PublicKey, IrohError>
}

• SocketAddr - interface

/// An internet socket address, either Ipv4 or Ipv6 
enum SocketAddr {
    V4(Ipv4Addr),
    V6(Ipv6Addr),
}

/// IPv4 SocketAddr representation
pub struct Ipv4Addr(Ipv4Addr);

impl Ipv4Addr {
    /// Create a new Ipv4 addr from 4 eight-bit octets
    pub fn new(a:u8, b:u8, c:u8, d:u8) -> Self
    /// Create a new Ipv4 addr from a String
    pub fn from_string(str: String) -> Result<Self, IrohError>
    /// A string representation of an Ipv4Addr
    pub fn to_string(&self) -> String
    /// Get the 4 octets as bytes
    pub fn octets -> [u8;4]
}


/// IPv6 SocketAddr representation
pub struct Ipv6Addr(Ipv6Addr)

impl Ipv6Addr {
    /// Create a new Ipv6 addr from 8 16-bit segments
    pub fn new(a:u16, b:u16, c:u16, d:u16, e:u16, f: u16, g: u16, h: u16) -> Self
    /// Create a new Ipv6 addr from a String
    pub fn from_string(str: String) -> Result<Self, IrohError>
    /// A string representation of an Ipv6Addr
    pub fn to_string(&self) -> String
    /// Get the 8 sixteen-bit segments as an array
    pub fn segments -> [u16;8]
}

/// Type of SocketAddr
enum SocketAddrType {
    V4,
    V6
}

impl SocketAddr {
    /// The type of SocketAddr
    fn r#type(&self) -> SocketAddrType
    /// Get the IPv4 SocketAddr representation
    fn v4(&self) -> Result<Ipv4Addr, IrohError>
    /// Get the IPv6 SocketAddr representation
    fn v6(&self) -> Result<Ipv6Addr, IrohError>
}

• NamespaceId - interface

/// The Namespace's [`PublicKey`] in bytes
pub struct NamespaceId(NamespaceId);
impl NamespaceId {
    fn to_string(&self) -> String
    fn to_bytes(&self) -> Vec<u8>
    fn from_string(&self, str: String) -> Result<NamespaceId, IrohError>
    fn from_bytes(&self, bytes: Vec<u8>) -> Result<NamespaceId, IrohError>
}

• DocTicket - interface

/// Contains both a key (either secret or public) to a document, and a list of peers to join.
pub struct DocTicket(DocTicket);
}
impl DocTicket {
    /// Serialize the ticket to a byte array.
    pub fn to_bytes(&self) -> Result<Vec<u8>, IrohError> {

    /// Parse ticket from a byte array.
    pub fn from_bytes(bytes: Vec<u8>) -> Result<Self, IrohError>

    /// Serialize the ticket to a String
    pub fn to_string(&self) -> Result<String, IrohError>

    /// Parse the ticket from a String
    pub fn from_string(str: String) -> Result<Self, IrohError>
}

• AuthorId - interface

/// The author's [`PublicKey`]
pub struct AuthorId(AuthorId);

impl AuthorId {
    /// Convert to byte array.
    pub fn to_bytes(&self) -> Vec<u8>

    /// Parse the AuthorId from a byte array
    pub fn from_bytes(bytes: Vec<u8> -> Result<Self, IrohError>

    /// Convert to a String
    pub fn to_string(&self) -> String

    /// Parse the AuthorID from a String
    pub fn from_string(str: String) -> Result<Self, IrohError>

    /// Convert to a base32 string limited to the first 10 bytes for a friendly string
    /// representation of the key.
    pub fn fmt_short(&self) -> String
}

• Tag - interface

/// A tag
pub struct Tag(Tag);

impl Tag {
    /// Create a tag from a String
    pub fn from_string(str: String) -> Self

    /// Serialize a tag as a String
    pub fn to_string(&self) -> Result<String, IrohError>

    /// Create a tag from a byte array
    pub fn from_bytes(bytes: Vec<u8>) -> Self

    /// Serialize a tag as a byte array
    pub fn to_bytes(&self) -> Vec<u8>
}

• Hash - interface

pub struct Hash(Hash);

impl Hash {
    /// Calculate the hash of the provide bytes.
    pub fn new(buf: Vec<u8>) -> Self

    /// Bytes of the hash.
    pub fn to_bytes(&self) -> Vec<u8>

    /// Create a `Hash` from its raw bytes representation.
    pub const fn from_bytes(bytes: Vec<u8>) -> Result<Self, IrohError>

    /// Get the cid as bytes.
    pub fn as_cid_bytes(&self) -> Vec<u8>

    /// Try to create a blake3 cid from cid bytes.
    ///
    /// This will only work if the prefix is the following:
    /// - version 1
    /// - raw codec
    /// - blake3 hash function
    /// - 32 byte hash size
    pub fn from_cid_bytes(bytes: Vec<u8>) -> Result<Self, IrohError>

    /// Convert the hash to a hex string.
    pub fn to_hex(&self) -> String
}

• Entry - interface

/// A single entry in a [`Doc`]
///
/// An entry is identified by a key, its [`AuthorId`], and the [`Doc`]'s
/// [`NamespaceId`]. Its value is the 32-byte BLAKE3 [`hash`]
/// of the entry's content data, the size of this content data, and a timestamp.
pub struct Entry(Entry);

impl Entry {
    /// Get the [`NamespaceId`] of this entry.
    pub fn namespace(&self) -> NamespaceId

    /// Get the [`AuthorId`] of this entry.
    pub fn author(&self) -> AuthorId

    /// Get the key of this entry.
    pub fn key(&self) -> Vec<u8>
}

• GetFilter - interface

/// Filter a get query onto a namespace
pub enum GetFilter {
    /// No filter, list all entries
    All,
    /// Filter for exact key match
    Key(Vec<u8>),
    /// Filter for key prefix
    Prefix(Vec<u8>),
    /// Filter by author
    Author(AuthorId),
    /// Filter by key prefix and author
    AuthorAndPrefix(AuthorId, Vec<u8>),
}

impl GetFilter {
    /// Filter by [`AuthorId`] and prefix
    pub fn author_prefix(author: AuthorId, prefix: Vec<u8>) -> Self

    /// No filter, get all entries in a namespace
    pub fn none() -> Self 

    /// Filter by [`AuthorId`]
    pub fn author(author: AuthorId) -> Self

    /// Filter by prefix
    pub fn prefix(prefix: Vec<u8>) -> Self

    /// Filter by an exact key
    pub fn key(key: Vec<u8>) -> Self 
}

• PeerAddr - interface

/// A peer and it's addressing information.
pub struct PeerAddr(PeerAdr);

impl PeerAddr {
    /// Create a new [`PeerAddr`] with empty [`AddrInfo`].
    pub fn new(peer_id: PublicKey) -> Self

    /// Add a derp region to the peer's [`AddrInfo`].
    pub fn with_derp_region(mut self, derp_region: u16) -> Self

    /// Add the given direct addresses to the peer's [`AddrInfo`].
    pub fn with_direct_addresses(
        mut self,
        addresses: Vec<SocketAddr>,
    ) -> Self

    /// Get the direct addresses of this peer.
    pub fn direct_addresses(&self) -> Vec<SocketAddr>

    /// Get the derp region of this peer.
    pub fn derp_region(&self) -> Option<u16>
}

• RequestToken - interface

/// A Request token is an opaque byte sequence associated with a single request.
/// Applications can use request tokens to implement request authorization,
/// user association, etc.
pub struct RequestToken(RequestToken)

impl RequestToken {
    /// Creates a new request token from bytes.
    pub fn new(bytes: Vec<u8>) -> Result<Self, IrohError>

    /// Generate a random 32 byte request token.
    pub fn generate() -> Self

    /// Returns a reference the token bytes.
    pub fn as_bytes(&self) -> Vec<u8>

    /// Create a request token from a string
    pub from_string(str: String) -> Result<Self, IrohError>

    /// Represent a request token as a string
    pub to_string(&self) -> String
}

Callbacks

• AddCallback / AddProgress

pub enum AddProgressType {
    Found,
    Progress,
    Done,
    AllDone,
    Abort
}

/// An AddProgress event indicating an item was found with name `name`, that can be referred to by `id`
pub struct AddProgressFound {
    /// A new unique id for this entry.
    id: u64,
    /// The name of the entry.
    name: String,
    /// The size of the entry in bytes.
    size: u64,
}

/// An AddProgress event indicating we got progress ingesting item `id`.
pub struct AddProgressProgress {
    /// The unique id of the entry.
    id: u64,
    /// The offset of the progress, in bytes.
    offset: u64,
}

/// An AddProgress event indicated we are done with `id` and now have a hash `hash`
pub struct AddProgressDone {
    /// The unique id of the entry.
    id: u64,
    /// The hash of the entry.
    hash: Hash,
}

/// An AddProgress event indicating we are done with the the whole operation
pub struct AddProgressAllDone {
    /// The hash of the created data.
    hash: Hash,
    /// The format of the added data.
    format: BlobFormat,
    /// The tag of the added data.
    tag: Tag,
}

/// An AddProgress event indicating we got an error and need to abort
pub struct AddProgressAbort(IrohError)

/// Progress updates for the add operation.
pub enum AddProgress {
    /// An item was found with name `name`, from now on referred to via `id`
    Found(AddProgressFound),
    /// We got progress ingesting item `id`.
    Progress(AddProgressProgress),
    /// We are done with `id`, and the hash is `hash`.
    Done(AddProgressDone),
    /// We are done with the whole operation.
    AllDone(AddProgressAllDone)
    /// We got an error and need to abort.
    ///
    /// This will be the last message in the stream.
    Abort(AddProgressAbort),
}

impl AddProgress {
    /// Get the type of event
    pub fn r#type(&self) -> AddProgressType
    /// Return the `AddProgressFound` event
    pub fn as_found(&self) -> Result<(), AddProgressFound>
    /// Return the `AddProgressProgress` event
    pub fn as_progress(&self) -> Result<(), AddProgressProgress>
    /// Return the `AddProgressDone` event
    pub fn as_done(&self) -> Result<(), AddProgressDone>
    /// Return the `AddAllDone`
    pub fn as_all_done(&self) -> Result<(), AddProgressAllDone>
    /// Return the `AddProgressAbort`
    pub fn as_abort(self) -> Result<(), AddProgressAbort>
}

• ValidateCallback / ValidateProgress blobs.validate is not yet implemented in core
• SubscribeCallback / LiveEvent
• DownloadCallback / DownloadProgress

use new `Trait` support to remove custom `to_string` and `equal` methods

unable to use `==` in other languages to determine if two objects have the same value

For now, we are implementing an equal method on any necessary structs that allow comparing a struct w/ an arc of that struct

feat: add dart bindings

hello
cute project
you have different sdks for different platforms
just wondering is it possible to use the rust sdk with dart's ffi
dart has ffigen for c headers
can you guys make a c header from your rust code ?

expand on the `DownloadProgressInitialState` struct

The struct contains more fields than what we currently provide. Let's get this on par with iroh core.

`create_collection` and more sophisticated `Collection` api

n0-computer/iroh@1f6153b

Collection api:
https://github.com/n0-computer/iroh/blob/1f6153ba2aeba7b916b9fa352263852498e0e68f/iroh-bytes/src/format/collection.rs

Proposing adding a subset of the Collecton API to ffi:
new - empty collection
push - add a blob to the collection
len - # of blobs
is_empty - if # of blobs == 0
names - Vec
links - Vec

cargo test is broken on main

This really needs to be checked on CI..

running 1 test
test node::tests::test_doc_new ... FAILED

failures:

---- node::tests::test_doc_new stdout ----
id: t57za7c52b65gzogt3aajf55vnovcekzo2k2emsodiewodozarpq
doc_id: dyfjhig53ltoncruvpl4bqm2b5hctylfwbpryshthuenweh6n44a
thread 'node::tests::test_doc_new' panicked at 'assertion failed: `(left == right)`
  left: `[123, 173, 125, 124, 228, 213, 40, 220, 48, 252, 166, 30, 137, 62, 129, 138, 53, 147, 28, 206, 181, 192, 83, 65, 236, 8, 235, 15, 57, 247, 5, 93, 1, 32, 159, 127, 144, 124, 93, 208, 125, 211, 101, 198, 158, 192, 4, 151, 189, 171, 93, 81, 17, 89, 118, 149, 162, 50, 78, 26, 9, 103, 13, 217, 4, 95, 1, 2, 9, 1, 32, 2, 46, 223, 103, 119, 0, 0, 180, 31, 200, 38, 224, 44, 185, 241, 197, 87, 1, 32, 2, 46, 223, 103, 119, 0, 0, 233, 151, 151, 111, 160, 5, 207, 75, 197, 87, 0, 46, 223, 103, 119, 196, 87, 1, 32, 2, 46, 223, 103, 119, 0, 0, 133, 102, 73, 77, 72, 77, 141, 243, 197, 87, 1, 32, 2, 46, 223, 103, 119, 0, 0, 65, 41, 187, 178, 242, 118, 40, 106, 197, 87, 1, 32, 2, 46, 223, 103, 119, 0, 0, 48, 131, 110, 97, 180, 9, 39, 26, 197, 87, 1, 32, 2, 46, 223, 103, 119, 0, 0, 8, 128, 45, 134, 121, 52, 150, 245, 197, 87, 0, 192, 168, 178, 31, 196, 87, 1, 32, 2, 46, 223, 103, 119, 0, 0, 40, 37, 58, 216, 212, 2, 240, 216, 197, 87]`,
 right: `[123, 173, 125, 124, 228, 213, 40, 220, 48, 252, 166, 30, 137, 62, 129, 138, 53, 147, 28, 206, 181, 192, 83, 65, 236, 8, 235, 15, 57, 247, 5, 93, 1, 32, 159, 127, 144, 124, 93, 208, 125, 211, 101, 198, 158, 192, 4, 151, 189, 171, 93, 81, 17, 89, 118, 149, 162, 50, 78, 26, 9, 103, 13, 217, 4, 95, 1, 2, 9, 1, 32, 2, 46, 223, 103, 119, 0, 0, 133, 102, 73, 77, 72, 77, 141, 243, 197, 87, 1, 32, 2, 46, 223, 103, 119, 0, 0, 65, 41, 187, 178, 242, 118, 40, 106, 197, 87, 1, 32, 2, 46, 223, 103, 119, 0, 0, 48, 131, 110, 97, 180, 9, 39, 26, 197, 87, 1, 32, 2, 46, 223, 103, 119, 0, 0, 8, 128, 45, 134, 121, 52, 150, 245, 197, 87, 1, 32, 2, 46, 223, 103, 119, 0, 0, 180, 31, 200, 38, 224, 44, 185, 241, 197, 87, 0, 46, 223, 103, 119, 196, 87, 1, 32, 2, 46, 223, 103, 119, 0, 0, 40, 37, 58, 216, 212, 2, 240, 216, 197, 87, 0, 192, 168, 178, 31, 196, 87, 1, 32, 2, 46, 223, 103, 119, 0, 0, 233, 151, 151, 111, 160, 5, 207, 75, 197, 87]`', src/node.rs:528:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    node::tests::test_doc_new

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.91s

unclear why though

v0.15.0 release

• #126
• #127

test go bindings as part of CI on each push

test(python): write basic python tests

Add Author API

This already exists in iroh, let's bring it into the ffi.

Bindings laundry list

• (python) doc get bytes example can't be completed because it needs to do .get_content_bytes from an entry but .set_bytes returns a Hash and there is no way to get or construct the matching entry except doing .keys and looping over them.
• no doc list support

new bindings: go

Once NordSecurity/uniffi-bindgen-go#12 is resolved it should be straightforward to add

Allow connecting to an existing node instead of start a new one

My use case is for the Python binding but I guess it could be useful for other bindings as well.

Discord chat: https://discord.com/channels/1161119546170687619/1161119546644627528/1199697069686931508

automate python wheel push to pypi

Releases are pushed manually, we should set up pypi tokens and push directly from CI.

publish iroh-node-js

https://github.com/napi-rs/package-template/blob/main/.github/workflows/CI.yml#L420

https://napi.rs/docs/introduction/simple-package#publish-it

CI: add go testing

• regenerate bindings (and check the diff)
• rung go/main.go

Add `ControlMsg` when it becomes public in iroh crate

Fill in missing command API methods in FFI bindings

not all methods listed in our command docs are present in the uniFFI definition. Not all commands make sense, but let's at least audit all of them & either write FFI bindings, or come up with an approach to get the same behaviour without an actual method.

Doc Commands

• doc switch just use a variable, this is console-specific
• doc new - implemented as IrohNode.create_doc
• doc join - _implemented as IrohNode.import_doc
• doc list
• doc share - implemented as doc.share_write
• doc set - implemented as doc.set_bytes
• doc get - implemeted as doc.get_content_bytes
• doc keys - implemented as doc.latest

Author Commands

• author switch just use a variable, this is console-specific
• author list
• author new - implemented as IrohNode.create_author

Blob Commands

• blob get
• blob add
• blob share
• blob list blobs
• blob list incomplete-blobs
• blob list collections
• blob validate

Connection Commands

• node connections
• node connection

bump iroh version

Seems we're a bit behind with iroh @ 0.8.0

test swift bindings as part of CI on each push

Add `ShareTicketOptions::OnlyNodeId` enum

starting with iroh v0.14.0 you will be able to dial using just a node id if both parties have DNS discovery & pkarr publishing enabled.

We can add a ShareTicketOption::OnlyNodeId with will create a ticket that just includes a node_id in the NodeAddr

ci: setup

• Run rust test
• Run clippy
• Run cargo fmt
• building the swift code (running make.sh)
• building the python bindings
• build kotlin bindings