Coder Social home page Coder Social logo

cpp-ipfs-http-client's Introduction

IPFS C++ HTTP API client library

Build CI Status CodeQL Documentation codecov GitHub Issues GitHub pull requests

IPFS C++ HTTP Client Library

Allows C++ applications to communicate with an IPFS node. It implements IPFS API bindings for C++. See the documentation and in partially the Client Class.

See also IPFS on GitHub.

The C++ API is broken up into the following sections (Note: links below go to the js-ipfs project). The following calls are implemented in cpp-ipfs-http-client:

  • Bitswap: all methods are still to-do
  • Block: get(), put(), stat()
  • Bootstrap: all methods are still to-do
  • Config: get(), set(), replace()
  • DAG: all methods are still to-do
  • DHT: findpeer(), findprovs()
  • Files: cat(), add(), ls()
  • Key: gen(), list(), rm(), rename()
  • Miscellaneous: id(), version()
  • Name: all methods are still to-do
  • Object: new(), put(), get(), data(), links(), stat(), patch.addLink(), patch.rmLink(), patch.appendData(), patch.setData()
  • Pin: add(), ls(), rm()
  • PubSub: all methods are still to-do
  • Refs: all methods are still to-do
  • Repo: stat()
  • Stats: bw(), repo() see Repo above
  • Swarm: addrs(), connect(), disconnect(), peers()

As you can see, not all methods are yet implemented.

TODO

  • Implement the missing methods
  • Contributors are welcome!

Dependencies

When building documention, you also need:

Build & Install

git clone https://github.com/vasild/cpp-ipfs-http-client.git
cd cpp-ipfs-http-client
# Configure build
cmake -B build
# Build
cmake --build build -j 8
# Install
sudo cmake --install build

Hint #1: You can also build using Ninja (iso Make), use the following as configure: cmake -GNinja -B build, then use: cmake --build build which will use Ninja, no need for -j anymore.
Hint #2: Build a specific target (eg. ipfs-http-client), use: cmake --build build --target ipfs-http-client -j 8
Hint #3: You could also build the library without tests, use the option: cmake -DBUILD_TESTING=OFF -B build

See the documentation for details.

Run Test cases

Only build & run the test cases, without code coverage:

# Prepare
cmake -B build
# Build & run our tests
cmake --build build --target our_tests -j 8

Build & run Test cases + Code Coverage

Test cases are build by default, but if you want to build with coverage:

cmake -DCMAKE_BUILD_TYPE=Debug -DCOVERAGE:BOOL=ON -DBUILD_SHARED_LIBS:BOOL=ON -B build
# Build & run tests + HTML report
cmake --build build --target ctest_coverage_html -j 8

# Or build & run tests + Cobertura XML file
cmake --build build --target ctest_coverage_xml -j 8

Build Doxygen

Build Doxygen files locally. From the root directory of this project:

cmake -DDOC=ON  -B build
cmake --build build --target doc

Usage

#include <ipfs/client.h>

#include <iostream>
#include <sstream>

int main(int, char**) {
  std::stringstream contents;

  ipfs::Client client("localhost", 5001);

  client.FilesGet("/ipfs/QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG/readme",
                  &contents);

  std::cout << contents.str() << std::endl;

  return 0;
}

More info see: Doxygen Docs - Client Class.

Multi-threading example

The client constructor and destructor are not thread safe. However, all the API IPFS calls are thread safe!

Note: A runtime error will be thrown on the request call (in this example the FilesGet()) when you call the Abort() method, allowing you to stop your code execution inside the thread.

An example of using a thread together with the IPFS Client:

#include <ipfs/client.h>

#include <sstream>
#include <thread>

int main(int, char**) {
  ipfs::Client client("localhost", 5001, "2m");

  // Only start one thread
  std::thread thread([&client]() {
    std::stringstream contents;
    try {
      // File should not exists, takes forever (until time-out)
      client.FilesGet("QmZp1rrtGTictR2rpNcx4673R7qU9Jdr9DQ6Z7F6Wgo2bQ",
                      &contents);
    } catch (const std::runtime_error& e) {
      std::cerr << "Error: " << e.what() << std::endl;
    }
  });

  if (thread.joinable()) {
    client.Abort();  // Abort request
    thread.join();   // Should not be blocking now
    client.Reset();  // Reset internal state
  }
  return 0;
}

See the full multi-threading example on: Doxygen - Examples.

Build via C++ compiler

g++ -std=c++11 -I/path/to/header -L/path/to/lib -lipfs-http-client myprog.cc -o myprog

Build via CMake

Use the C++ IPFS Client inside an existing CMake project. We add the IPFS client inside the lib folder. For example via git submodule (but git clone should also work):

cd your-cmake-project
git submodule add https://github.com/vasild/cpp-ipfs-http-client.git lib/ipfs-http-client

Edit your CMakeLists.txt file to include the C++ IPFS HTTP Client in your build:

add_subdirectory (lib/ipfs-http-client)

Finally, add the C++ IPFS HTTP static library to your target using target_link_libraries()
(in this example ${PROJECT_TARGET} variable is used as target name):

set(PROJECT_TARGET my-app)

target_link_libraries(${PROJECT_TARGET} PRIVATE ipfs-http-client)

Contribute

Feel free to open issues and pull requests. Report vulnerabilities publicly, similar to other non-security issues.

The project adheres to the Google C++ Style Guide. Use clang-format to properly format the code when you submit patches.

Write tests for new code. Changes should not cause the code coverage to go down (ideally up).

License

The code is distributed under the MIT License.

cpp-ipfs-http-client's People

Contributors

daviddias avatar gabrieldougherty avatar geertiebear avatar kenkit avatar linas avatar melroy89 avatar royguo avatar vasild avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cpp-ipfs-http-client's Issues

New IPFS Client Maintainers Group

Hiya :)

In the IPFS org we’ve created a new group for maintainers of client libraries. This will make it easier for us to loop you into issues that may cause problems or necessitate changes in your client library.

If you could let us know the github usernames of all the active maintainers we can add them to the group and note them in the clients table.

Also, we have a great IPFS weekly newsletter. A few days before each edition goes out a new pull request is created in the repo. If you have any announcements related to your client library feed free to log a new issue in the repo or just comment on an upcoming newsletter pull request.

Support for go-ipfs v0.5

This is a notice I'm filling in repo of every HTTP Client I can find.
Feel free to close it if this project already works fine with go-ipfs 0.5

POST-only HTTP API

go-ipfs 0.5 will block GET commands on the API port (ipfs/kubo#7097), requiring every command (RPC) to be sent as HTTP POST request.
See API reference docs: https://docs.ipfs.io/reference/api/http/

This is potentially a breaking change,
double check if this project uses POST for every RPC call.

Testing

Download links for v0.5-rc* are available at ipfs/kubo#7109

You can also test using an ephemeral Docker container:

$ docker run --rm -it --net=host ipfs/go-ipfs:v0.5.0-rc1 

Use infura with cpp-ipfs

Hi,

I am trying to use cpp-ipfs-http-client with Infura. The code works fine with a local ipfs daemon (ipfs::Client client("localhost", 5001), but the port seems closed when using Infura. Here are the diferent configurations I tried:
ipfs::Client client("https://ipfs.infura.io", 5001); ipfs::Client client("ipfs.infura.io", 5001); ipfs::Client client("ipfs.infura.io", 5001, "https");
The terminal issues connection failed errors.
By any chance, did you try using Infura or similar tools smoothly with cpp-ipfs-http-client?

Thanks for your help !

CMake is excessive for this small program

I think better to use Make directly.
Makefile:

#
# IPFS C++ HTTP API client library
#


VER_MAJOR := 0
VER_MINOR := 4
VER_PATCH := 0
VERSION := $(VER_MAJOR).$(VER_MINOR).$(VER_PATCH)

HEADERS := include/ipfs/client.h include/ipfs/http/transport-curl.h include/ipfs/http/transport.h
TESTPROGS := block dht files key object swarm config generic name pin

CXXFLAGS = -Wall -Wpedantic -Wextra -Werror -Os -std=gnu++11 -I./include
LDLIBS = ./libipfs-http-client.a -lcurl

.PHONY: testprogs lib all clean


all: lib testprogs


lib: libipfs-http-client.a  libipfs-http-client.so.$(VERSION)

libipfs-http-client.a: src/client.o src/http/transport-curl.o
        $(AR) rc libipfs-http-client.a  src/client.o src/http/transport-curl.o

libipfs-http-client.so.$(VERSION): src/client.o src/http/transport-curl.o
        $(CXX) $(CXXFLAGS) -shared -Wl,-soname,libipfs-http-client.so.$(VER_MAJOR) -o libipfs-http-client.so.$(VERSION) src/client.o src/http/transport-curl.o -lcurl
        ln -sf libipfs-http-client.so.$(VERSION) libipfs-http-client.so.$(VER_MAJOR)
        ln -sf libipfs-http-client.so.$(VER_MAJOR) libipfs-http-client.so


testprogs: $(addprefix test/,$(TESTPROGS))

$(addprefix test/,$(TESTPROGS)):%:%.cc  include/ipfs/client.h include/ipfs/test/utils.h  libipfs-http-client.a
        $(CXX)  $(CXXFLAGS) $< $(LDLIBS) -o $@


src/client.o: src/client.cc $(HEADERS)
        $(CXX) $(CXXFLAGS) -fPIC -o src/client.o -c src/client.cc

src/http/transport-curl.o: src/http/transport-curl.cc $(HEADERS) include/ipfs/test/utils.h
        $(CXX) $(CXXFLAGS) -fPIC -o src/http/transport-curl.o -c src/http/transport-curl.cc


clean:
        $(RM) src/client.o src/http/transport-curl.o libipfs-http-client.a libipfs-http-client.so* $(addprefix test/,$(TESTPROGS))

Tested in Debian Stretch.

GitHub Actions

Instead of Travis, we could actually start using Github Actions to achieve the same thing.

Maybe it's a better integration with Github.

Let me know if you agree or not Vasil.

Regards,
Melroy van den Berg

Moving to cURL multi API

We are currently using the cURL curl_easy_perform() function within our library. Which works fine and is thread-safe.

However, when using threading to call IPFS http client, you want to be able to cancel those threads. pthread_cancel() should NOT be used, since that is not implemented under Windows (even when using the POSIX thread for win32 - MinGW for example). And thus not cross-platform implemented. Moreover, pthread_cancel under Linux is not guaranteed to stop the thread, depending on the cancellation points/.. and basically on best effort basis.

Anyway, the best way to stop/cancel a running thread in a "native" way is to actually to be able to communicate with the thread using a shared resource. For example using an atomic boolean to stop the thread. The thread should be able to react on the boolean change, allowing the thread to stop itself.

Pseudo code, just a fast sketch:

# Main thread
std::atomic<bool> stop_thread(false);
[...]
// Stop the thread
stop_thread = true
thread.join()
stop_thread = false
# Thread code
while(..) {
  if (stop_thread)
     return;
}

Now the issue with cURL is that the curl_easy_perform() function is a blocking call, and can't be easily cancelled or terminated. Instead I propose to use the cURL Multi API instead. Like the curl_multi_perform(), curl_multi_poll() and curl_multi_remove_handle() functions.

The cURL Multi API is select/poll based, allowing to rewrite the code in such a way you are able to cancel the cURL request within a thread, at any given moment.

See also the FAQ on curl.se - How do I stop an ongoing transfer?

If you are using the multi interface, you can also stop a transfer by removing the particular easy handle from the multi stack at any moment you think the transfer is done or when you wish to abort the transfer.

I underline "or when you wish to abort the transfer".

This solution would automatically solve the cross-platform issue, of canceling Windows Threads as well using for example such an atomic boolean to stop the thread from the main thread, at any given moment.

Personally, I'm facing those issues with my own app using this cpp-ipfs-http-client, which I try to cross-compile towards Windows.

@vasild PELASE, any feedback or ideas are welcome! I know, this requires quite some changes in the code. But I think it's feasible and would really help me out!

#include <nlohmann/adl_serializer.hpp> not installed

Install the project...
-- Install configuration: ""
-- Installing: /usr/local/lib/libipfs-http-client.a
-- Installing: /usr/local/include/ipfs/client.h
-- Installing: /usr/local/include/ipfs/http/transport.h
-- Installing: /usr/local/include/nlohmann/json.hpp

But nlohmann/adl_serializer.hpp is not copied while present in ./build/_deps/json-src/include/nlohmann/adl_serializer.hpp

Therefore this exemple https://github.com/vasild/cpp-ipfs-http-client#usage does not compile:
#include <nlohmann/adl_serializer.hpp> not found

Same for the whole folder.
Edit: -I ./build/_deps/json-src/include/ fixing my compilation issue but this line:

install(FILES ${json_SOURCE_DIR}/include/nlohmann/json.hpp DESTINATION include/nlohmann)

should be fixed

Which methods are missing

Hi,

Not all methods are implemented.

So its stated that there are methods missing in the README. But its not written down which methods are actually missing.

Bottom-line: What is the current state of the project? And much is implemented? And finally, what pieces needs to be done?

Maybe also add a link to the Core API documentation (from JS-IPFS) as a reference: https://github.com/ipfs/js-ipfs/tree/master/docs/core-api

Regards,

Melroy

Could be nice to generate/install pkg-config file

I'm testing your project right now.

Could be nice to create a pkg-config file like in:
https://github.com/sebastiandev/zipper/blob/master/zipper.pc.cmakein

Or for Makefile:
https://github.com/Lecrapouille/MyMakefile/blob/master/Makefile.macros#L165
sudo make install will install it:
https://github.com/Lecrapouille/MyMakefile/blob/master/Makefile.macros#L181
https://github.com/Lecrapouille/MyMakefile/blob/master/Makefile.macros#L63
(not sure this is working well for Mac)

So in your readme instead of:

g++ -std=c++11 -I/path/to/header -L/path/to/lib -lipfs-http-client myprog.cc -o myprog

Could be replaced by:

g++ -std=c++11 myprog.cc -o myprog `pkg-config --cflags --libs ipfs-http-client`

In addition in your command -lipfs-http-client is misplaced and -lcurl is missing. The pkg-config will contain the -lcurl code

Host a local server with this implementation?

Hi,

I just got this library up and running only to realise that it seems to connect to an existing local node. I was expecting it to host a local node also.
Am I correct in this? Do I need to host my own local server then use this library to connect to my local server?

It would be helpful to update the documentation with this information.

Failed initialization: easy handle already used in multi handle

Making multiple calls to client.ObjectPut method throwing up " Failed initialization: easy handle already used in multi handle " error. Looks like transport curl not able to handle multiple calls at a time.

Any idea about this error message ?

FYI: If we make sequential calls it works fine. no issues at all

Missing timeout option/setting

While creating a Client object (or possibly later as well), I would like to set a time-out. Similar to ipfs-http-client:

https://github.com/ipfs/js-ipfs/tree/master/packages/ipfs-http-client#global-timeouts

Luckily there is a common option to set timeout server-side, meaning you don't need to handle time-outs client side.

Let's say you try to access a file that doesn't exist (yet) in IPFS:

curl -X POST "http://127.0.0.1:5001/api/v0/cat?arg=zb2rhe5P4gXftAwvA4eXQ5HJwsER2owDyS9sKaQRRVQPn93bA"

Introducing an additional query string (named timeout with a value of a string, including 's' for seconds for example):

curl -X POST "http://127.0.0.1:5001/api/v0/cat?arg=zb2rhe5P4gXftAwvA4eXQ5HJwsER2owDyS9sKaQRRVQPn93bA&timeout=3s"

Similar behaviour shouldn't be hard to implement in CPP-IPFS client. Just add timeout=<value> to the URL requests.

Regards,

Melroy van den Berg

Discussion: Backwards incompatible interface change

Hi,

I noticed you wrapped the files cat API command with the method name FilesGet:

void Client::FilesGet(const std::string& path, std::iostream* response) {
  http_->Fetch(MakeUrl("cat", {{"arg", path}}), {}, response);
}

Which will get the file via cat and returns the data via iostream as shown above.

However, FilesGet call will conflict with the actual get HTTP API call as well:

https://docs.ipfs.io/reference/http/api/#api-v0-get

Proposal: Fix the misalignment of interface names better sooner than later. Rename FilesGet to FilesCat. And FilesGet can be implemented to download the file directly to disk (proving the arg as path argument).
This will be conform the spec and even in-line with the API CLI.

This change will however impact devs who upgrade to a newer version of cpp-ipfs-http-client, it isn't nice. But again, I rather do this change now then later.

An alternative is: Keep the current names. And add FilesDownload to use the get API call. Which is, as expected, not inline with the spec.

@vasild Please, let me know what you think.

Regards,
Melroy van den Berg

Throw when trying to get links of new object

Try to get links of new object (hash: QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n) with IPFS using curl:
curl "http://localhost:5001/api/v0/object/links?arg=QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n"

IPFS returns:
{"Hash":"QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n"}

There is no "Links" field in the response.
The method Client::ObjectLinks of cpp-ipfs-api throws an exception as it tries to access to the "Links" value

Client crash when copy

#include <ipfs/client.h>

int main()
{
        ipfs::Client ipfs("localhost", 5001);
        ipfs::Client ipfsCopy(ipfs); // Segmentation fault

        return (0);
}

Unable to catch IPFS connection issues

Hi,

When I turn-off IPFS daemon on purpose, it should throw an runtime error as expected.

Nevermind, I should catch the error at client.FilesGet() for example. This client Object creation, client("localhost", ...) , always goes OK, and doesn't check the connection yet.

Regards,
Melroy van den Berg

Transfer progress

Hi, thank you for making this library.
I plan on using it in an app i'm making could not find a way to get transfer progress incase of file download/upload.
Your response will highly appreciated.

Reduce test time of 2 testcases

The following two test cases take up 99% of the execution time:

  • test_dht
  • test_name

Which won't help short development cycles in a DevOps cycle and fast CI/CD.

Let's see if we can make the two mentioned testcases faster.

Console output:

 3/10 Test #63: test_dht .........................      Passed  104.44 sec
 7/10 Test #67: test_name ........................   Passed   80.46 sec

Regards,

Melroy van den Berg

Suspecting downgraded performance

I suspect the following code, is maybe causing down-graded performance:

In FetchAndParseJson():

std::stringstream body;
body.str(); // <--- here

Is a stringstream indeed slow in converting 'large' data to a string?

Another cause could be in ParseJson:

Json::parse(input);

Additional debugging and performance measurements are required!

I see it takes 6ms to execute a simple swarm/peers request. Sure, this specific call could take longer. Yet other API calls are executed (far) below 1ms. And the difference with curl command doesn't show much time differences between those different API calls.

The main difference is the body response size.

Just a thought experiment. Again, does requires follow-up research.

Error when I compile the program as the guidance

This is the command:
c++ -std=c++11 -I ./include -L ./lib -lipfs-http-client myprog.cc -o myprog
output:
myprog.cc:(.text+0x4f): undefined reference to ipfs::Client::Client(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long)' myprog.cc:(.text+0x78): undefined reference to ipfs::Client::~Client()'
collect2: error: ld returned 1 exit status

Please help me!

Issue with client.FilesAdd(...)

Hello,

I am looking for some help concerning the FilesAdd command.

I have installed the cpp-ipfs-http-client and I am able to successfully retrieve the FIlesGet element from the test file: retrieved contents are displayed in the console as expected.

However, the FilesAdd command from the test file fails: no output is displayed.
Here is the code:
ipfs::Client client("localhost", 5001); ipfs::Json add_result; client.FilesAdd( {{"foo.txt", ipfs::http::FileUpload::Type::kFileContents, "abcd"},{"bar.txt", ipfs::http::FileUpload::Type::kFileName, "../compile_commands.json"}}, &add_result); std::cout << "FilesAdd() result:" << std::endl << add_result.dump(2) << std::endl;

Did you already run into this issue? Am I doing something wrong?
Thank you for your help or any tip !

Compilation fails with Release target

When compiling with

cmake -DCMAKE_BUILD_TYPE=Release

the compilation fails with

Scanning dependencies of target ipfs-api
[  5%] Building CXX object CMakeFiles/ipfs-api.dir/src/client.cc.o
[ 10%] Building CXX object CMakeFiles/ipfs-api.dir/src/http/transport-curl.cc.o
/tmp/nix-build-cpp-ipfs-api-2016-11-26.drv-0/cpp-ipfs-api-3b64e6e4520e87f14fee4216b4b5184c6f97a47a-src/src/http/transport-curl.cc: In lambda function:
/tmp/nix-build-cpp-ipfs-api-2016-11-26.drv-0/cpp-ipfs-api-3b64e6e4520e87f14fee4216b4b5184c6f97a47a-src/src/http/transport-curl.cc:293:5: error: 'replace_body' was not declared in this scope
     replace_body = "";
     ^

Background:

This happens with the latest commit of cpp-ipfs-api when building for NixOS (currently an older version is packaged) and the build system uses the CMake default target Release for non-debug builds (Debug being the default for debug builds).

ObjectData function not thread safe

The following code demonstrate that the ObjectData function is not thread safe and throws std::runtime_error exception.

#include <thread>
#include <ipfs/client.h>

int main()
{
        auto hash = "QmcXneAawME7d3vmV3x95p3WVxczeTid7LpJW8DcH4sA5c";
        ipfs::Client ipfs("localhost", 5001);
        std::thread t1([&hash, &ipfs]() { std::string data; ipfs.ObjectData(hash, &data); });
        std::thread t2([&hash, &ipfs]() { std::string data; ipfs.ObjectData(hash, &data); });

        t1.join();
        t2.join();
        return (0);
}

On MacOS:
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Failed initialization

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.