lemourin / libcloudstorage

Official libcloudstorage repository.

libcloudstorage

A C++ library providing access to files located in various cloud services licensed under GNU LGPLv2.1. It is focused on the basic operations on those services.

Supported cloud providers:

• GoogleDrive
• OneDrive
• Dropbox
• box.com
• YandexDisk
• WebDAV
• mega.nz
• AmazonS3
• pCloud
• hubiC
• 4shared
• Google Photos (partial)

Supported operations on files:

• list directory
• download file
• upload file
• get thumbnail
• delete file
• create directory
• move file
• rename file
• fetch direct, preauthenticated url to file

Requirements:

• jsoncpp
• tinyxml2
• libmicrohttpd (optional)
• cURL (with OpenSSL/c-ares, optional)
• libcryptopp (optional, required for AmazonS3)
• mega (optional, required for mega.nz)

Building:

The generic way to build and install it is:

• mkdir build && cd build && cmake .. && make && sudo make install

Optional dependency notes:

• libcryptopp:

  when not found, ICrypto interface needs to be implemented

• libcurl

  when not found, IHttp interface needs to be implemented

• libmicrohttpd

  when not found, IHttpServer interface needs to be implemented

• boost-filesystem

  when found, LocalDrive provider representing local directory will be included

• mega

  when not found, mega cloud provider will not be included

FUSE:

In bin/fuse there is implemented a user space file system using fuse (https://github.com/libfuse/libfuse) library. It will be build when fuse is found. The file system is implemented using libfuse's low level api; however high level api implementation is also provided. The file system supports moving, renaming, creating directories, reading and writing new files. Writing over already present files in cloud provider is not supported. The file system uses asynchronous I/O to its full potency. It doesn't cache files anywhere by itself which implies no local storage overhead. Most cloud providers are fast enough when it comes to watching videos; with mega.nz being the fastest and Google Drive being the slowest.

Windows:

It is possible to run cloudstorage-fuse under Windows using Dokan (https://github.com/dokan-dev/dokany).

Usage:

To add cloud providers to file system, first the cloud providers need to be added. This can be done by calling:

cloudstorage-fuse --add=provider_label

After cloud providers are added, the file system can be mount using:

cloudstorage-fuse mountpoint

Cloud Browser:

In bin/cloudbrowser there is a program which provides easy graphics user interface for all the features implemented in libcloudstorage. It will be built when its dependencies are found.

Cloud Browser dependencies:

• Qt5Core, Qt5Gui, Qt5Quick

• kirigami (https://github.com/KDE/kirigami)

• Qt5WebView

  when found, Cloud Browser will use it to present the authorization scheme

• ffmpeg

  when found, Cloud Browser will generate fallback thumbnails if cloud provider doesn't provide any

• mpv

  when found, Cloud Browser will use mpv-based media player

Screenshot:

Apps that use libcloudstorage:

• https://play.google.com/store/apps/details?id=io.storage.cloudbrowser
• https://www.microsoft.com/en-us/p/cloud-storage-browser/9nbkjrh757k7

TODO:

Implement following cloud providers:

• Apple ICloud
• Asus WebStorage
• Baidu Cloud
• CloudMe
• FileDropper
• Fileserve
• Handy Backup
• IBM Connections
• Jumpshare
• MagicVortex
• MediaFire
• Pogoplug
• SpiderOak
• SugarSync
• Tencent Weiyun
• TitanFile
• Tresorit
• XXL Box

Implement bindings to various languages, notably script languages:

• Obj-C
• python
• ruby
• JavaScript / node
• Java

Integrate in various desktops

• KIO slave
• gvfs implementation

Implement chunked uploads.

libcloudstorage's Issues

desktop file is lacking a Categories= entry

The *.desktop file is lacking a Categories= entry.

According to the menu spec,

By including one of the Main Categories in an application's desktop entry file, the application will be ensured that it will show up in a section of the application menu dedicated to this category. If multiple Main Categories are included in a single desktop entry file, the entry may appear more than once in the menu.

Hence, please add at least one of the following in the Categories= key.

Main Category	Description	Notes
AudioVideo	Application for presenting, creating, or processing multimedia (audio/video)
Audio	An audio application	Desktop entry must include AudioVideo as well
Video	A video application	Desktop entry must include AudioVideo as well
Development	An application for development
Education	Educational software
Game	A game
Graphics	Application for viewing, creating, or processing graphics
Network	Network application such as a web browser
Office	An office type application
Science	Scientific software
Settings	Settings applications	Entries may appear in a separate menu or as part of a "Control Center"
System	System application, "System Tools" such as say a log viewer or network monitor
Utility	Small utility application, "Accessories"

In addition, you could specify one or more from the longer list of Additional Categories.

Please test the result with desktop-file-validate and make sure it passes.

electron-builder offers native support for this since v19.22.1.. Please see the category key at https://www.electron.build/configuration/linux-other.

References:

• AppImage/appimage.github.io#2
• AppImage/appimage.github.io#688

OneDrive link gets 'invalid state'

When linking OneDrive the OAuth flow succeeds, but the application returns invalid state after receiving the callback. Linking does not succeed.

YouTube: live video

Live videos on YouTube don't work.
YouTube support is marked as partial - is it expected behaviour right now?

No http module specified

The error No http module specified is produced after building with the following configuration. I've tried with libcurl4-openssl-dev available but the same error occurs. Which dependency is missing?

FROM ubuntu:19.04

# Install build dependencies
RUN apt-get update -qq && \
  apt-get install -qq \
  autoconf \
  curl \
  g++ \
  git \
  libfuse3-dev \
  libjsoncpp-dev \
  libmicrohttpd-dev \
  libtinyxml2-dev \
  libtool \
  make \
  pkg-config \
  python \
  wget

# Build application
RUN git clone https://github.com/lemourin/libcloudstorage && \
  cd libcloudstorage && \
  ./bootstrap && \
  ./configure && \
  make

libcloudstorage/bin/fuse# ./cloudstorage-fuse --add=dropbox
   _ _ _          _                 _     _                             
  | (_| |        | |               | |   | |                            
  | |_| |__   ___| | ___  _   _  __| |___| |_ ___  _ __ __ _  __ _  ___ 
  | | | '_ \ / __| |/ _ \| | | |/ _` / __| __/ _ \| '__/ _` |/ _` |/ _ \
  | | | |_) | (__| | (_) | |_| | (_| \__ | || (_) | | | (_| | (_| |  __/
  |_|_|_.__/ \___|_|\___/ \__,_|\__,_|___/\__\___/|_|  \__,_|\__, |\___|
                                                              __/ |     
                                                             |___/      

4shared: terminate called after throwing an instance of 'std::runtime_error'
  what():  No http module specified.
Aborted (core dumped)

Hardcoded Secrets

It seems the OAuth app details are hardcoded into the source. Example 1 Example 2.

Can this be changed to use a central config source (like in cloudbrowser), and can that config inherit from environment variables, not hardcoded source values that require a recompile?

Dropbox app frozen pending review

When trying to link dropbox using the following url, the below error is produced. Is this exact clientId required? Could it be substituted with another clientId that is owned by the user?

https://www.dropbox.com/oauth2/authorize?client_id=ktryxp68ae5cicj&redirect_uri=http%3A%2F%2Flocalhost%3A12345&response_type=code&state=dropbox

AppImage does not work on Ubuntu 20.04

Hi,

Unfortunately, if I try to add or download a file, I get this error message:

ERROR: No native FileDialog implementation available.
Qt Labs Platform requires Qt Widgets on this setup.
Add 'QT += widgets' to .pro and create QApplication in main().

Thanks & regards,
Mark

Mounting mega.nz drive doesn't quite work

I have built and installed mega sdk 3.4.7. I have followed the instructions for building and install cloudstorage-fuse, followed the procedure to mount the drive, however, while it was visible in Thunar, it hanged while it was loading contents, and later on it wouldn't let me unmount it, remove the directory, ls showed question marks instead of permissions on the mountpoint, etc. I had to reboot.

Nov 12 19:45:56 smartalex-pc kernel: [  242.466773] INFO: task pool-Thunar:3421 blocked for more than 120 seconds.

AppImage does not run

https://github.com/lemourin/libcloudstorage/blob/a72053cc8f7f57f815809722c1b9344c7bef41da/packaging/appimage/AppRun seems to be looking for cloudbrowser in /usr/bin rather than in the AppDir:

me@host:~$ ./squashfs-root/AppRun 
./squashfs-root/AppRun: 62: ./squashfs-root/AppRun: /usr/bin/cloudbrowser: not found

Do you really need a custom AppRun script?

Reference:
AppImage/appimage.github.io#1016

Web server always started

The OAuth web server is always started regardless of whether an OAuth action is required. This causes future OAuth account setup via cloudstorage-fuse --add= to fail as the https server port is already in use.

Scenario:

# Setup fuse mount
cloudstorage-fuse /mnt/fuse

# Authorise cloud storage provider
cloudstorage-fuse --add=google
Couldn't initialize http server.

Can the OAuth server only be started when the --add command line argument is supplied?