azure / azure-storage-cpplite

Lite version of C++ Client Library for Microsoft Azure Storage

Azure Storage C++ Client Library (Lite) (Deprecated)

For more details on the retirement and alternatives to using this project, visit Retirement notice: The legacy Azure Storage C++ client libraries will be retired on 29 March 2025.

About

The Azure Storage Client Library (Lite) for C++ allows you to build applications against Microsoft Azure Storage's blob service. This is a minimum dependency version that provide basic object storage. For an overview of Azure Storage, see Introduction to Microsoft Azure Storage. If you want to use other services of Azure Storage, or a more comprehensive functionality of Blob service, please see Azure Storage C++ Client Library.

Features

The full supported Azure Storage API can be found in the following list, please be aware that only part of the functionality of some APIs are supported:

• List Containers.
• Create Container.
• Get Container Properties.
• Delete Container.
• List Blobs.
• Put Blob.
• Get Blob.
• Get Blob Properties.
• Delete Blob.
• Copy Blob.
• Put Block.
• Put Block List.
• Get Block List.
• Put Page.
• Get Page Ranges.
• Append Block.

Installation

Supported Platforms

Please be aware that below platforms are tested and verified, but other platforms beyond this list can be working with some modification on the build steps resolving the dependencies. Feel free to test them out and contribute back.

• Ubuntu 16.04 x86_64.
• CentOS 6 x86_64.
• CentOS 7 x86_64.
• macOS Mojave version 10.14.6 x86_64.
• Windows 10 with Visual Studio 2017, x86 or x64.

Build this library on Linux or macOS

Project dependencies:

• GnuTLS (or OpenSSL v1.0.1)
• libcurl v7.35.0
• CMake v3.6
• GNU C++ or Apple clang with C++11 support
• libuuid 2.13.1

Clone the latest code from this repository:

git clone https://github.com/azure/azure-storage-cpplite.git

Install the dependencies, e.g. on Ubuntu:

apt install libssl-dev libcurl4-openssl-dev cmake g++ uuid-dev

Or, on CentOS:

yum install openssl-devel libcurl-devel cmake3 gcc-c++ libuuid-devel

Or, on macOS:

brew install openssl cmake

Please be aware that CentOS 6 comes with GCC version 4.4.7, which does not meet the requirement of this SDK. In order to use this SDK, devtoolset needs to be installed properly. Please be aware that on some Linux distributions, pkg-config is not properly installed that would result in CMake not behaving as expected. Installing pkg-config or updating it will eliminate the potential issue. The tested version of pkg-config is 0.27.1.

Build and install azure-storage-cpplite:

cd azure-storage-cpplite
mkdir build.release
cd build.release
cmake .. -DCMAKE_BUILD_TYPE=Release
# Just build
cmake --build .
# Or build and install
cmake --build . --target install

Use GnuTLS instead of OpenSSL:

Alternatively, you can use GnuTLS instead of OpenSSL. Simply install GnuTLS and add the argument -DUSE_OPENSSL=OFF during CMake configure.

Build this library on Windows

Project dependencies:

• OpenSSL v1.0.2
• libcurl v7.60.0
• CMake v3.6
• Visual Studio 2017

Clone the latest code from this repository:

git clone https://github.com/azure/azure-storage-cpplite.git

Prepare and install the dependencies.

There are two major dependencies on Windows: libcurl and OpenSSL. For the best development experience, we recommend that developers use vcpkg to install dependencies. You can also install your own choice of pre-built binaries.

vcpkg install curl openssl

Build azure-storage-cpplite using CMake in command line

cd azure-storage-cpplite
mkdir build.release
cd build.release
cmake .. -DCMAKE_TOOLCHAIN_FILE=<vcpkg path>/scripts/buildsystems/vcpkg.cmake
cmake --build . --config Release

Specifying customized OpenSSL or libcurl root folder

Default OpenSSL or libcurl root directory may not be applicable for all users. In that case, following parameters can be used to specify the preferred path: -DCURL_INCLUDE_DIR=<libcurl's include directory>, -DCURL_LIBRARY=<libcurl's library path>, -DOPENSSL_ROOT_DIR=<OpenSSL's root directory>.

This applies to both Windows and Unix.

Other build options

There are some advanced options to config the build when using CMake commands:

• -DBUILD_SHARED_LIBS= : specify ON or OFF to control if shared library or static library should be built.
• -DBUILD_TESTS= : specify ON or OFF to control if tests should be built.
• -DBUILD_SAMPLES= : specify ON or OFF to control if samples should be built.
• -A : specify Win32 or x64 to config the generator platform type on Windows.
• -DCMAKE_BUILD_TYPE= : specify Debug or Release to config the build type on Linux or macOS.
• --config : specify Debug or Release to config the build type on Windows.

Usage

Simply include the header files after installing the library, everything is good to go. For a more comprehensive sample, please see sample. To build the sample, add -DBUILD_SAMPLES=ON when building the repository.

#include "storage_credential.h"
#include "storage_account.h"
#include "blob/blob_client.h"

// Your settings
std::string account_name = "YOUR_ACCOUNT_NAME";
std::string account_key = "YOUR_ACCOUNT_KEY";
bool use_https = true;
std::string blob_endpoint = "CUSTOMIZED_BLOB_ENDPOINT";
int connection_count = 2;

// Setup the client
auto credential = std::make_shared<azure::storage_lite::shared_key_credential>(account_name, account_key);
auto storage_account = std::make_shared<azure::storage_lite::storage_account>(account_name, credential, use_https, blob_endpoint);
azure::storage_lite::blob_client client(storage_account, connection_count);

// Start using
auto outcome = client.create_container("YOUR_CONTAINER_NAME").get();

License

This project is licensed under MIT.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repositories using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

When contributing to this client library, there are following ground rules:

1. All source code change should be clearly addressing issues or adding new features, and should be covered with test.
2. Coding style should match with the existing code.
3. Any contribution should not degrade performance or complex functionality.
4. Introducing new dependency should be done with much great caution. Any new dependency should introduce significant performance improvement or unblock critical user scenario.

Build Test

Install Catch2 via package manager, or download Catch2 single header version and specify the location with -DCATCH2_INCLUDE_DIR=<catch2 path> when build.

Add -DBUILD_TESTS=ON when building the repository.

Please modify the connection string here to successfully run the tests. All the tests use standard Azure Storage account.

Please note that in order to run test, a minimum version of g++ 5.1 is required on Linux, and Visual Studio 2017 is required on Windows.

azure-storage-cpplite's Issues

Sample does not link when using shared lib

cmake .. -G "Visual Studio 15 2017" -DCURL_LIBRARIES="D:\source\curl\builds\libcurl-vc14-x64-release-dll-ipv6-sspi-winssl\lib\libcurl.lib" -DCURL_INCLUDE_DIRS="D:\source\curl\builds\libcurl-vc14-x64-release-dll-ipv6-sspi-winssl\include" -DBUILD_SAMPLES=true -DCURL_LINK_TYPE=dll -DCMAKE_GENERATOR_PLATFORM=x64 -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=true -DBUILD_WITH_MT=true

After running the above cmake I issue a build command

MSBuild.exe azurestoragelite.sln /p:Configuration=Release

Although sample project finds the .lib for the sdk, it does not link correctly, I get unresolved symbol messages like :

Severity Code Description Project File Line Suppression State
Error LNK2019 unresolved external symbol "public: bool __cdecl azure::storage_lite::blob_client_wrapper::container_exists(class std::basic_string<char,struct std::char_traits,class std::allocator > const &)" (?container_exists@blob_client_wrapper@storage_lite@azure@@QEAA_NAEBV?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@@z) referenced in function main azure-storage-sample C:\Users\raghu\source\azure-storage-cpplite-shared\build.release\sample\sample.obj 1

Any ideas? Am I missing something in Cmake options?

Thanks
Raghu

Type-limits warnings when building for ARM32

Use case: Implementing Azure Blob Storage on IoT Edge with C++ on ARM32 device, which is now supported using other languages such as Python, Node.

Issue: Trying to build on a 32-bit platform results in warnings due to assumptions about char size (I realize this code hasn't been tested on 32-bit officially):

/app/azure-storage-cpplite/src/blob/blob_client_wrapper.cpp: In member function 'void azure::storage_lite::blob_client_wrapper::upload_file_to_blob(const string&, const string&, std::__cxx11::string, const std::vector<std::pair<std::__cxx11::basic_string, std::__cxx11::basic_string > >&, size_t)':
/app/azure-storage-cpplite/src/blob/blob_client_wrapper.cpp:421:25: error: comparison is always false due to limited range of data type [-Werror=type-limits]
if(fileSize > MAX_BLOB_SIZE)
^
/app/azure-storage-cpplite/src/blob/blob_client_wrapper.cpp:429:25: error: comparison is always false due to limited range of data type [-Werror=type-limits]
if(fileSize > (50000 * MIN_UPLOAD_CHUNK_SIZE))
^
cc1plus: all warnings being treated as errors

request signing failed when accessing Azure Blob Storage on IoT Edge

I was trying to run the SDK on an Embedded Linux device running as IoT leaf to access Azure Blob Storage on IoT Edge. However authorization kept failing. Meanwhile Azure Storage .Net SDK worked just fine.

After some digging, I narrowed down the problem to request signing. The client side (leaf device) signed a different header compared to what sever (Blob Storage on IoT Edge) saw. For example, client signed the header as:

PUT











x-ms-client-request-id:4cb210a3-2560-4da4-85a2-8c76ddd10336
x-ms-date:Mon, 13 May 2019 03:14:11 GMT
x-ms-version:2018-03-28
/mymoxa2/cont1
restype:container

While the server saw:

PUT











x-ms-client-request-id:4cb210a3-2560-4da4-85a2-8c76ddd10336
x-ms-date:Mon, 13 May 2019 03:14:11 GMT
x-ms-version:2018-03-28
/mymoxa2/mymoxa2/cont1
restype:container

The difference was with resource path /mymoxa2/cont1 vs /mymoxa2/mymoxa2/cont1. It tracked down to shared_key_credential::sign_request function in storage_credential.cpp. My temporary hack was to append account name twice. Then it worked. I am not security expert and I bet you'll come up with a better fix.

list_blobs_segmented is not returning directories hierarchically

We are calling list_blobs_segmented from blob_client, we get en-GUIB, en_HI and then en. Shouldn't directory en be returned first?
See below, we get the following error if we ask for only 2 results (en is the third)
drwxrwxrwx 2 azureuser azureuser 4096 Jan 1 1970 ca
drwxrwxrwx 2 azureuser azureuser 4096 Jan 1 1970 ch
d????????? ? ? ? ? ? codeswitch
drwxrwxrwx 2 azureuser azureuser 4096 Jan 1 1970 cs
drwxrwxrwx 2 azureuser azureuser 4096 Jan 1 1970 da
drwxrwxrwx 2 azureuser azureuser 4096 Jan 1 1970 de
drwxrwxrwx 2 azureuser azureuser 4096 Jan 1 1970 el
d????????? ? ? ? ? ? en
drwxrwxrwx 2 azureuser azureuser 4096 Jan 1 1970 en-GBAU
drwxrwxrwx 2 azureuser azureuser 4096 Jan 1 1970 en-IN

put_blob loses errors

blob_client_wrapper::put_blob may lose error codes from its call to m_blobClient->upload_block_blob_from_stream. If that call results in an error then errno is set to the value of the HTTP error code, for example 403. Then put_blob closes the input stream, which will reset errno to 0. The caller of blob_client_wrapper::put_blob then has no indication that the upload failed.

Need create snapshot API for unmanaged disks

Hi,
The storage client library thick version has an API to create a snapshot of a blob.
azure::storage::cloud_page_blob m_blob =
m_blob.create_snapshot()

This API is helpful to create snapshot of unmanaged VHDs attached to a virtual machine.
Does the lite version has a similar API to create snapshots of unmanged VHDs present in a container and attached to a virtual machine?

How to Set Blob Content-Type

When uploading to azure blob an image via stream, it always sets the content type to "application/octet-stream". How do you change the content type when it is uploaded?

I've tried setting the meta content as follows:

vector<pair<string,string>> metadata(1, std::make_pair("Content-Type", "image/jpg"));
client.upload_block_blob_from_stream(AZURE_BLOB_CONTAINER, filename, istream, metadata);

This only adds meta content, but doesn't change the blob properties. I've also tried to directly change the curl request headers:

auto bc = std::make_shared<blob_client>(account, 16);
std::shared_ptr<azure::storage_lite::http_base> http_request = bc->client()->get_handle();
http_request->add_header("Content-Type", "image/jpg");
vector<pair<string,string>> metadata;
bc->upload_block_blob_from_stream(AZURE_BLOB_CONTAINER, filename, istream, metadata);

Any help would be greatly appreciated.

blob_client_wrapper::list_containers_segmented doesn't allow empty prefix

In blob_client_wrapper::list_containers_segmented at https://github.com/Azure/azure-storage-cpplite/blob/master/src/blob/blob_client_wrapper.cpp#L281 there is a early failure check for prefix.length() == 0 even though the List Containers rest definition defines this parameter as optional. This prevents listing all containers in the account.

Proposing the lines 281-285 of blob_client_wrapper.cpp be removed to allow the case of listing all containers

Fix cross-compilation workflow on Windows host with CMake

I am cross-compiling the project on Windows, and had to make some changes in the main CMakeLists file, so that it works and also go around PkgConfig. Please find my fix proposal below.

Original

if (UNIX)
  option(USE_OPENSSL "Use OpenSSL instead of GnuTLS" ON)
  if(USE_OPENSSL)
    target_compile_definitions(azure-storage-lite PRIVATE -DUSE_OPENSSL)
    set(OPENSSL_ROOT_DIR ${OPENSSL_ROOT_DIR} /usr/local/opt/openssl)
    find_package(OpenSSL REQUIRED)
    list(APPEND EXTRA_LIBRARIES OpenSSL::SSL OpenSSL::Crypto)
  else()
    find_package(GnuTLS REQUIRED)
    list(APPEND EXTRA_INCLUDE_DIRS ${GNUTLS_INCLUDE_DIR})
    list(APPEND EXTRA_LIBRARIES ${GNUTLS_LIBRARIES})
  endif()

  if(NOT APPLE)
    find_package(PkgConfig REQUIRED)
    pkg_check_modules(uuid REQUIRED IMPORTED_TARGET uuid)
    list(APPEND EXTRA_LIBRARIES PkgConfig::uuid)
  endif()
elseif(WIN32)
  list(APPEND EXTRA_LIBRARIES rpcrt4 bcrypt)
  target_compile_definitions(azure-storage-lite PRIVATE azure_storage_lite_EXPORTS NOMINMAX)
endif()

Fix

if(WIN32)
  list(APPEND EXTRA_LIBRARIES rpcrt4 bcrypt)
  target_compile_definitions(azure-storage-lite PRIVATE azure_storage_lite_EXPORTS NOMINMAX)
else()
  option(USE_OPENSSL "Use OpenSSL instead of GnuTLS" ON)
  if(USE_OPENSSL)
    target_compile_definitions(azure-storage-lite PRIVATE -DUSE_OPENSSL)
    set(OPENSSL_ROOT_DIR ${OPENSSL_ROOT_DIR} /usr/local/opt/openssl)
    find_package(OpenSSL REQUIRED)
    list(APPEND EXTRA_LIBRARIES OpenSSL::SSL OpenSSL::Crypto)
  else()
    find_package(GnuTLS REQUIRED)
    list(APPEND EXTRA_INCLUDE_DIRS ${GNUTLS_INCLUDE_DIR})
    list(APPEND EXTRA_LIBRARIES ${GNUTLS_LIBRARIES})
  endif()

  if(CMAKE_HOST_SYSTEM_NAME STREQUAL "Linux")
    find_package(PkgConfig REQUIRED)
    pkg_check_modules(uuid REQUIRED IMPORTED_TARGET uuid)
    list(APPEND EXTRA_LIBRARIES PkgConfig::uuid)
  endif()
endif()

Slow parallel upload using blob_client::upload_block_blob_from_buffer

When trying to upload especially smaller files using blob_client::upload_block_blob_from_buffer I find that upload speed is very slow (~500kbit/s). It appears that only a single parallel request is used although the maximum number of concurrent requests was set to 64.

From a cursory glance at the code, a rather large block size is chosen. That is, the minimum block size is set to azure::storage_lite::constants::default_block_size = 8 * 1024 * 1024, which is 8MB. Also, the grain size is set to around 4MB, where it is only 64kB in blob_client::download_blob_to_buffer.

Would it be feasible to set a smaller block size to allow for more blocks being uploaded in parallel even for smaller files?

Thanks for looking into this.

New release

Will there be a new release of the SDK soon? We would like to integrate into our CI/CD and want tie to a specific version.

Given then we would need the new features that you have in place, I was hoping you could cut a new release.

Thank you,

We should add sample of `blob_client` and remove sample of `blob_client_wrapper`

The only sample in this project is for blob_client_wrapper, which is designed for a specific user rather than ordinary users. While it misleads most users into using our cpplite sdk the wrong way.

The interface of blob_client_wrapper has a flaw. It returns void rather than a value indicating whether an error has occurred. So the user must check errno everytime after calling a function. This is counter-intuitive and unfriendly, and also not conventional. In Linux, APIs usually return an int, if the value is -1, error information is stored in errno. Windows APIs work in a similar way, if an API returns a false or INVALID_HANDLE_VALUE, error code is stored in GetLastError(). We have another user complaining about this #31.

In addition, it is error-prone, because errno is very likely to be overwritten by other C/C++ functions, like printf, std::cout or std::fstream, these stuff are implementation-dependent. We also have users reporting these kind of issues, like #47, #67.

How to copy blob to blob and get notification that it was done

Good day,
I'm using the azure-stroage-cpplite library in my project and stuck with the copy blob problem. In the cpplite version, we have only start_copy function that doesn't wait until data will be copied. We don't have any option to place a callback that will be called when copy operation was done. The only way that I found is to call the get_property function until copy_status has pooling status. In the full cpp library, we have the wait_for_copy method that does it the same way. But maybe you could suggest a more elegant and quick way to check that blob was copied successfully without get_prop, sleep, and other additional calls?

Using CNG for hashing on windows

Seems like the only need for OpenSSL on windows is for hashing when using SharedKeys.

Why can't we use Windows CNG (bcrypt) for doing this. This removes the dependency on OpenSSL.

I could update CmakeList.txt and hash.cpp to fix this. What do you guys think?

P.S: This could be considered an extension to the previous issue reported in using SChannel (WinSSL) #42 on windows for TLS. If this is adopted then there is no need to carry the OpenSSL dependency around for the main sdk

Issue in upload APIs.

1. Non-seekable stream cannot be used as input stream for upload APIs, otherwise it would result in a authentication failure.
2. Input stream's length cannot be 0, otherwise a authentication will fail.

api useless without error checking and bool return values

create_container function in blob_client_wrapper should return a bool. Have no idea if this succeeds or not , I guess you can recheck if the container exists after you called create. But it fails and there are no errors to indicate what went wrong. This api is useless if you can't figure out what went wrong , there could be naming issues, permissions, ... Can't even really test if the account connected successfully. Disappointed with the api, should have kept the functions very similar to the non lite version. Right now this is bare bones, would not recommend wasting your time with this.

The value for one of the HTTP headers is not in the correct format

Hi,

I am using the cpp lite version of Azure Storage Client Library.
I am using v0.3 version.
I am connecting to a Dell Azure Stack Hub of version
IdentitySystem : AzureAD
OemVersion : 2.1.1908.3
StampVersion : 1.2002.19.73

Can the v0.3 version connect to the above Azure Stack Hub versions? Am i using the right version of the client library?

std::shared_ptr<storage_credential> credential = std::make_shared<shared_key_credential>(account_name, account_key);
std::shared_ptr<storage_account> account = std::make_shared<storage_account>(account_name, credential, /* use_https */ true, blob_endpoint);
auto bC = std::make_shared<blob_client>(account, 16);
std::string containerName = "fs-magicians-container1";
std::string blobName = "nbu-nagini-unmanaged-disk-v2.ee86dfba-2320-451f-8efd-af7ff9b341c4.screenshot.bmp";
bool exists = true;
blob_client_wrapper bc(bC);
exists = bc.container_exists(containerName);
if (exists)
{
	std::cout << "Success: The container exists";
}
else
{
	std::cout << "Failed to find container" << errno << std::endl;
}

Client request id

How does one provide client request id header ( x-ms-client-request-id )? I know it is optional but would like to know how do applications generate it/provide it and how do I use it with the SDK?

Thank you
~Raghu

curl postfields to request body are not passing thru

The lib_curl_http.h file in the include dir needs an extra setting to allow POSTFields to be passed to the body of the request.
SPN token only works if we post the client_credential params to the body of the message not as params.
In http://github.com/azure/azure-storage-fuse We added this line to get the above working.
check_code(curl_easy_setopt(m_curl, CURLOPT_POSTFIELDS, nullptr)); // CURL won't actually read data on POSTs unless this is explicitly set.

See our working method below.

void set_input_stream(storage_istream s) override {
m_input_stream = s;
check_code(curl_easy_setopt(m_curl, CURLOPT_READFUNCTION, read));
check_code(curl_easy_setopt(m_curl, CURLOPT_READDATA, this));
check_code(curl_easy_setopt(m_curl, CURLOPT_POSTFIELDS, nullptr)); // CURL won't actually read data on POSTs unless this is explicitly set.
}

Segmentation fault during blob_client::upload_block_blob_from_stream

This issue looks similar to this issue from the azure-storage-cpp project: Azure/azure-storage-cpp#265
(in that issue it was mentioned that there were indeed segmentation fault errors in azure-storage-cpp version 3.0 which where fixed later on).

Valgrind output of the error I'm getting:

==8950== Process terminating with default action of signal 11 (SIGSEGV)
==8950== Access not within mapped region at address 0x4008
==8950== at 0xAF4500E: std::istream::read(char*, long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.22)
==8950== by 0x60C7F09: azure::storage_lite::CurlEasyRequest::read(char*, unsigned long, unsigned long, void*) (in /mnt/public/zdevfs2/yoav/src/dev/unittest/CppLinuxUnitTests/builds/libazure-storage-cpplite.so)
==8950== by 0x599F5A2: Curl_fillreadbuffer (in /mnt/public/zdevfs2/yoav/src/dev/unittest/CppLinuxUnitTests/builds/libcurl.so)
==8950== by 0x59A0783: Curl_readwrite (in /mnt/public/zdevfs2/yoav/src/dev/unittest/CppLinuxUnitTests/builds/libcurl.so)
==8950== by 0x59A9885: multi_runsingle (in /mnt/public/zdevfs2/yoav/src/dev/unittest/CppLinuxUnitTests/builds/libcurl.so)
==8950== by 0x59AA260: curl_multi_perform (in /mnt/public/zdevfs2/yoav/src/dev/unittest/CppLinuxUnitTests/builds/libcurl.so)
==8950== by 0x59A1CE9: curl_easy_perform (in /mnt/public/zdevfs2/yoav/src/dev/unittest/CppLinuxUnitTests/builds/libcurl.so)
==8950== by 0x60C66F7: azure::storage_lite::CurlEasyRequest::perform() (in /mnt/public/zdevfs2/yoav/src/dev/unittest/CppLinuxUnitTests/builds/libazure-storage-cpplite.so)
==8950== by 0x60C9021: azure::storage_lite::CurlEasyRequest::submit(std::function<void (int, azure::storage_lite::storage_istream, CURLcode)>, std::chrono::duration<long, std::ratio<1l, 1l> >) (in /mnt/public/zdevfs2/yoav/src/dev/unittest/CppLinuxUnitTests/builds/libazure-storage-cpplite.so)
==8950== by 0x60DDFF1: azure::storage_lite::async_executor::submit_helper(std::shared_ptr<std::promise<azure::storage_lite::storage_outcome > >, std::shared_ptr<azure::storage_lite::storage_outcome >, std::shared_ptrazure::storage_lite::storage_account, std::shared_ptrazure::storage_lite::storage_request_base, std::shared_ptrazure::storage_lite::http_base, std::shared_ptrazure::storage_lite::executor_context, std::shared_ptrazure::storage_lite::retry_context) (in /mnt/public/zdevfs2/yoav/src/dev/unittest/CppLinuxUnitTests/builds/libazure-storage-cpplite.so)
==8950== by 0x60E4CFB: azure::storage_lite::async_executor::submit(std::shared_ptrazure::storage_lite::storage_account, std::shared_ptrazure::storage_lite::storage_request_base, std::shared_ptrazure::storage_lite::http_base, std::shared_ptrazure::storage_lite::executor_context) (in /mnt/public/zdevfs2/yoav/src/dev/unittest/CppLinuxUnitTests/builds/libazure-storage-cpplite.so)
==8950== by 0x60D3100: azure::storage_lite::blob_client::upload_block_blob_from_stream(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::istream&, std::vector<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > > const&, unsigned long) (in /mnt/public/zdevfs2/yoav/src/dev/unittest/CppLinuxUnitTests/builds/libazure-storage-cpplite.so)

Thanks,
Dror

Parallel buffer upload API?

We are evaluating this library (vs azure-storage-cpp), and the smaller dependency tree is appealing. One API that seems to be missing is a parallel put from a buffer/stream of known size. It looks like the blob_client::upload_file_to_blob function could be refactored to take a buffer, or a similar function written which splits over the buffer and then finalizes the block list like upload_file_to_blob does.

Is this an API you would consider adding, or did I miss an existing API? Thanks!

cc @joe-maley @Shelnutt2

Parallel upload/download from/to stream

We have added parallel upload/download from/to buffer for blob_client in release 0.3.0. However, this may not function well if the buffer is larger than usable physical memory, due to inefficient page swapping in/out. If you have demand for uploading/downloading from/to file or stream, you can let us know by commenting in this issue. So that we can evaluate and consider adding this feature in future release.

vcpkg for Windows build

I think using vcpkg is more convinient for resolving dependency on openssl and libcurl on Windows build. You can build both with two lines of command.
Is there a reason why vcpkg is not used?

Move to 'modern' cmake?

Are there current blockers to updating the minimum cmake version to 3.x?

It's something I'm interested in and a lot of nice features in 3.x with target specific setup and additions that could be made to make it easier to integrate this library into another project.

`header_ms_date` is mistakenly following system's locale settings

Using this package here to do some azure blob storage development. Worked fine on my cloud ubuntu VM but received endless 403 responses on my ubuntu laptop.

Turns out get_ms_date in src/utility.cpp is mistakenly following system locale settings. My laptop have locale settings as simplified Chinese, which causes strftime to output Chinese date & time strings. Such date & time information won't be recognized by azure and thus 403.

Workaround: Plz use std::time_put instead of strftime in get_ms_date, locale settings of which can be specified(reference here).

Include files not found in VS 2015 C++ project

I have both VS 2017 and VS 2015 installed on my Windows machines. I'm using VS 2015 for some C++ projects. I've cloned and built the azure-store-cpplite library per the build instructions. Now when I try to include:

#include "storage_credential.h"
#include "storage_account.h"
#include "blob/blob_client.h"

VS does not find these files.

I've run "vcpkg integrate install":

Applied user-wide integration for this vcpkg root.

All MSBuild C++ projects can now #include any installed libraries.
Linking will be handled automatically.
Installing new libraries will make them instantly available.

CMake projects should use: "-DCMAKE_TOOLCHAIN_FILE=C:/projects/vcpkg/scripts/buildsystems/vcpkg.cmake"

But my VS C++ project still doesn't find the include files.

maturity and comparison with azure-storage-cpp

Hello,

We're considering usage of this library (azure-storage-cpplite) for a release that is expected to be G.A. in about 6 months from today. We need to access azure blob storage from a C++ service on Linux servers. We need to understand:

1. Is this library ready for production? is it being used in production by anyone? we're concerned by the "0.3.0" release tag, which might indicate lack of maturity for production.
2. Some comparison with azure-storage-cpp. Based on our tests this library is indeed lighter in terms of memory consumption and number of threads, which are quite important to us, but we'd love a more comprehensive comparison made by anyone.

Build fails on Windows 10 Enterprise

I've followed the build instructions but when in the last build step (cmake --build . --config Release) I get a bunch of "warning C4273: ... inconsistent dll linkage", which are treated as errors and fail the build:

C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\bin\HostX86\x86\CL.exe /c
/I"C:\projects\azure-storage-cpplite\include" /I"C:\projects\vcpkg\installed\x86-windows\include" /nologo /W4 /WX /di
agnostics:classic /O2 /Ob2 /Oy- /D WIN32 /D _WINDOWS /D NDEBUG /D "CMAKE_INTDIR="Release"" /D MBCS /Gm- /EHsc /MD
/GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /GR /Fo"azure-storage-lite.dir\Release\" /Fd"azure-storage-lite.
dir\Release\azure-storage-lite.pdb" /Gd /TP /analyze- /FC /errorReport:queue "C:\projects\azure-storage-cpplite\src\l
ogging.cpp" "C:\projects\azure-storage-cpplite\src\base64.cpp" "C:\projects\azure-storage-cpplite\src\constants.cpp"
"C:\projects\azure-storage-cpplite\src\hash.cpp" "C:\projects\azure-storage-cpplite\src\utility.cpp" "C:\projects\azu
re-storage-cpplite\src\tinyxml2.cpp" "C:\projects\azure-storage-cpplite\src\tinyxml2_parser.cpp" "C:\projects\azure-s
torage-cpplite\src\storage_account.cpp" "C:\projects\azure-storage-cpplite\src\storage_credential.cpp" "C:\projects\a
zure-storage-cpplite\src\storage_url.cpp" "C:\projects\azure-storage-cpplite\src\get_blob_request_base.cpp" "C:\proje
cts\azure-storage-cpplite\src\put_blob_request_base.cpp" "C:\projects\azure-storage-cpplite\src\delete_blob_request_b
ase.cpp" "C:\projects\azure-storage-cpplite\src\copy_blob_request_base.cpp" "C:\projects\azure-storage-cpplite\src\cr
eate_container_request_base.cpp" "C:\projects\azure-storage-cpplite\src\delete_container_request_base.cpp" "C:\projec
ts\azure-storage-cpplite\src\set_container_metadata_request_base.cpp" "C:\projects\azure-storage-cpplite\src\list_con
tainers_request_base.cpp" "C:\projects\azure-storage-cpplite\src\list_blobs_request_base.cpp" "C:\projects\azure-stor
age-cpplite\src\get_blob_property_request_base.cpp" "C:\projects\azure-storage-cpplite\src\set_blob_metadata_request
base.cpp" "C:\projects\azure-storage-cpplite\src\get_block_list_request_base.cpp" "C:\projects\azure-storage-cpplite
src\get_container_property_request_base.cpp" "C:\projects\azure-storage-cpplite\src\put_block_request_base.cpp" "C:\p
rojects\azure-storage-cpplite\src\put_block_list_request_base.cpp" "C:\projects\azure-storage-cpplite\src\append_bloc
k_request_base.cpp" "C:\projects\azure-storage-cpplite\src\put_page_request_base.cpp" "C:\projects\azure-storage-cppl
ite\src\get_page_ranges_request_base.cpp" "C:\projects\azure-storage-cpplite\src\http\libcurl_http_client.cpp" "C:\pr
ojects\azure-storage-cpplite\src\blob\blob_client.cpp" "C:\projects\azure-storage-cpplite\src\blob\blob_client_wrappe
r.cpp"
logging.cpp
c:\projects\azure-storage-cpplite\src\logging.cpp(12): error C2220: warning treated as error - no 'object' file generat
ed [C:\projects\azure-storage-cpplite\build.release\azure-storage-lite.vcxproj]
c:\projects\azure-storage-cpplite\src\logging.cpp(12): warning C4273: 's_logger': inconsistent dll linkage [C:\projects
\azure-storage-cpplite\build.release\azure-storage-lite.vcxproj]
c:\projects\azure-storage-cpplite\include\logging.h(101): note: see previous definition of 'protected: static std::fu
nction<void __cdecl(azure::storage_lite::log_level,std::basic_string<char,std::char_traits,std::allocator

const &)> azure::storage_lite::logger::s_logger'
c:\projects\azure-storage-cpplite\src\logging.cpp(12): error C2491: 'azure::storage_lite::logger::s_logger': definition
of dllimport static data member not allowed [C:\projects\azure-storage-cpplite\build.release\azure-storage-lite.vcxpro
j]
base64.cpp
c:\projects\azure-storage-cpplite\src\base64.cpp(38): error C2220: warning treated as error - no 'object' file generate
d [C:\projects\azure-storage-cpplite\build.release\azure-storage-lite.vcxproj]
c:\projects\azure-storage-cpplite\src\base64.cpp(38): warning C4273: 'azure::storage_lite::to_base64': inconsistent dll
linkage [C:\projects\azure-storage-cpplite\build.release\azure-storage-lite.vcxproj]
c:\projects\azure-storage-cpplite\include\base64.h(10): note: see previous definition of 'to_base64'
c:\projects\azure-storage-cpplite\src\base64.cpp(43): warning C4273: 'azure::storage_lite::to_base64': inconsistent dll
linkage [C:\projects\azure-storage-cpplite\build.release\azure-storage-lite.vcxproj]
c:\projects\azure-storage-cpplite\include\base64.h(11): note: see previous definition of 'to_base64'
c:\projects\azure-storage-cpplite\src\base64.cpp(94): warning C4273: 'azure::storage_lite::from_base64': inconsistent d
ll linkage [C:\projects\azure-storage-cpplite\build.release\azure-storage-lite.vcxproj]
c:\projects\azure-storage-cpplite\include\base64.h(12): note: see previous definition of 'from_base64'
constants.cpp
c:\projects\azure-storage-cpplite\include\constants.dat(1): error C2220: warning treated as error - no 'object' file ge
nerated [C:\projects\azure-storage-cpplite\build.release\azure-storage-lite.vcxproj]
c:\projects\azure-storage-cpplite\include\constants.dat(1): warning C4273: 'azure::storage_lite::constants::http_delete
': inconsistent dll linkage [C:\projects\azure-storage-cpplite\build.release\azure-storage-lite.vcxproj]
c:\projects\azure-storage-cpplite\include\constants.dat(1): note: see previous definition of 'http_delete'
...
217 Warning(s)
34 Error(s)

AZURE_* environment variables

Please consider supporting the AZURE_* environment variables like the azure CLI, especially:

• AZURE_STORAGE_ACCOUNT
• AZURE_STORAGE_KEY

For testing it would also be useful to be able to configure AZURE_STORAGE_ENDPOINT (not currently supported by the CLI, as far as I can tell).

Function to check validity of client

Can't find any function to check if a client connection is valid (accessible and sas token valid), until after trying to upload/download something. Ideally I would like to verify my client before operating on it. Something like:

auto cred = std::make_shared<azure::storage_lite::shared_access_signature_credential>(sas_token);
auto account = std::make_shared<azure::storage_lite::storage_account>(ACCOUNT_NAME, cred, USE_HTTPS);
auto client = std::make_shared<azure::storage_lite::blob_client>(account, MAX_CONCURRENCY);
bool valid = client->valid(); // I'm missing something like this

What do you think?

Running the sample code hangs

I compiled the source and the samples using VS2015. Wen running the sample executable the programs simply hangs in bc.container_exists().

When I use http instead of https the program does not hang. However I get error code 400 and in general the program fails.

Appreciate any insights?

storage response headers

I am able to get blob headers using one call (get_blob_property etc) and download file/stream using another call (download_blob_to_stream etc.).

However I am looking for some simple way to get response headers values (like for status code, response content length, response content range and content-type etc.) and blob itself using a Single call like download_blob_to_stream.

Sample example most welcome.

• Vikas

Improve support for QNX Neutrino

The uuid dependecy relies on the uuid header from the util-linux library, which is not always available. By adding support for the lightweight OSSP uuid, the support of other embedded UNIX-based OS such as QNX can be achieved without minimal modifications in the existing code.

Support for large file upload (>2.1Gb)

Currently azure::storage_lite::get_file_size() returns -1 for files greater than 2.1Gb.. Can we use stat64 here to allow support for large files?

Heap Corruption when using build_request

The following snippet of code causes a heap corruption when calling the destructor:

 CurlEasyRequest::~CurlEasyRequest()
    {
        curl_easy_reset(m_curl);
        m_client->release_handle(m_curl);
        if (m_slist) {
            curl_slist_free_all(m_slist);
        }
    }

at the curl_easy_reset method call. It seems like m_curl gets freed before it gets here.

The code snippet that causes the issue:

std::shared_ptr<storage_account> account = std::make_shared<storage_account>(_accountName, _pcCredentials, /* use_https */ true);
  std::shared_ptr<blob_client > blobClient = std::make_shared<blob_client>(account, 1);  
  
  auto http_request = blobClient->client()->get_handle();
      
   return 0; /// The heap corruption happens after exiting from the method 

The vdsremote_blob_request at this point simply is a child class of get_blob_request_base.

My idea to use build_request is to get to the signed headers which (headers) we will be using internally in our curl requests. Let me know if there is another way to approach this problem and this is not the intended way.

While I understand the reason for curl handle being a raw pointer, I am wondering if it gets freed before all the "references" to it.

Appreciate any inputs.

Azurite authentication fails

I am unable to authenticate with the latest Azurite release (https://hub.docker.com/_/microsoft-azure-storage-azurite) using this cpplite client.

Using the same code that works on the production Azure, but using the storage_account::development_storage_account(), I am able to connect with a locally running Azurite instance but the authentication fails.

Is this supported? Is there a workaround?

debug.log

Empty error response for ill-formatted blob endpoint

When executing blob_client::create_container on an instance constructed with an ill-formatted blob endpoint, the response fails with an empty error message. For example:

std::string blob_endpoint = "https://joetiledb.blob.core.windows.net";

auto account = std::make_shared<azure::storage_lite::storage_account>(
    account_name, credential, use_https, blob_endpoint);
auto client = std::make_shared<azure::storage_lite::blob_client>(account, 2);

std::future<azure::storage_lite::storage_outcome<void>> request =
  client->create_container("garbage");
assert(request.valid());
azure::storage_lite::storage_outcome<void> response = request.get();

if (!response.success()) {
  std::cerr << " response.error().code " << response.error().code  << std::endl;
  std::cerr << "response.error().code_name " << response.error().code_name  << std::endl;
  std::cerr << "response.error().message " << response.error().message  << std::endl;
}

The above prints:

response.error().code 706481520
response.error().code_name
response.error().message

Note that the error code is non-deterministic and I assume to be reading from uninitialized memory. Anyway, the non-obvious issue is that my blob end point is ill-formatted because it includes a https:// prefix. Removing that prefix works as expected. I assume the behavior would be identical with any sort of invalid blob endpoint string. The expected behavior is that the response.error().message would contain some sort of description indicating that the endpoint couldn't be reached, doesn't exist, etc.

Add UUID implementation compatibility with OSSP uuid

The current UUID calls rely on libuuid. Other UUID implementations or existing libraries are not compatible with this Azure utility.

The OSSP uuid is another well-known UUID library for embedded targets, which is currently not supported. Therefore, despite of having an UUID library available, I have to install another dependency (libuuid-dev) to build for my target, or modify the source code.

Would it be possible to add compatibility with OSSP uuid?

Support logging

Right now I cannot enable logging from azure-storage-cpplite and when there is something happened, I cannot figure out what is wrong.

[Survey] Boost dependency?

Hi all,

Currently there are two C++ libraries for Azure Storage Service, this repo manages one and another one is called azure-storage-cpp. Many of you has chosen this library because of the boost dependency in the other one, can you share the reasons why boost library is not acceptable in your scenario? This will be one of the key references when we consider the dependency stories moving both SDKs forward.

If you do not like to disclose your user scenario publicly, feel free to email [email protected] to provide your voice.

Best regards,

Tank

Cannot access Request Headers

I cannot access the request headers. When I use CurlEasyRequest I can get the response headers after performing CurlEasyRequest::perform()
Would it be possible to store the request headers as part of the add_header (in a private member m_request_headers)method and let users of the sdk access them via get_request_headers? I understand this might not be an intended feature of the SDK but would help us tremendously in our use case

Our application performs its own curl requests. We are looking to use this sdk to get the curl request headers.

Would this be possible?

API given error incorrectly even though it could connect and download blob

I tried the sample.cpp provided with a little modification for logging and error checking. Please see below code. I gave correct credentials, the container and blob which i am trying to download exist and i could successfully download blob from blob storage.

But, i get error while creating storage_account, blob_client, blob client wrapper though my credentials are correct. I checked 'errno' variable. Please see the code and guide me if i am using it in a wrong way.

void checkstatus()
{
    if(errno == 0)
    {
        printf("Success\n");
    }
    else
    {
        printf("Failure\n");
    }
}

int main()
{
    std::shared_ptr<storage_credential> cred = nullptr;
    cred = std::make_shared<shared_key_credential>(account_name, account_key);


    printf("\nCreating account : ");
    std::shared_ptr<storage_account> account = std::make_shared<storage_account>(account_name, cred, /* use_https */ true);
    checkstatus();

    if(NULL != account)
    {
    	printf("\nCreating blob client : ");
    	auto bC = std::make_shared<blob_client>(account, 2);
    	checkstatus();

		std::string containerName = "con1";
		std::string blobName = "1.txt";

		bool exists = true;
		printf("\nCreating blob client wrapper : ");
		blob_client_wrapper bc(bC);
		checkstatus();

		printf("\nGet blob property: ");
		bc.get_blob_property(containerName, blobName);
		checkstatus();

		printf("\nChecking does container exist: ");
		exists = bc.container_exists(containerName);
		checkstatus();


		printf("\nChecking does blob exist: ");
		exists = bc.blob_exists(containerName, "1.txt");
		checkstatus();
		if(!exists)
		{
			printf("\n Blob name does not exist");
		}

		time_t last_modified;
		printf("\nDownloading file");
		bc.download_blob_to_file(containerName, blobName, "./13.txt", last_modified);
		std::cout <<"Download Blob done: " << errno << std::endl;
    }
    else
    {
    	std::cout <<"Download Error: " << errno << std::endl;
    }

    return 0;
    
}

Update storage version to use x-ms-error-code in logs

I noticed that storage version 2017-07-29 has "All error responses have an additional response header x-ms-error-code that contains the error code string"

This could be a useful debugging addition to log messages where the client relies heavily on errno

Build failed on macOS Mojave

On macOS Mojave 10.14.4 (18E226) Xcode 10.2.1 (Build version 10E1001)

Build failed with output

-- The C compiler identification is AppleClang 10.0.1.10010046 -- The CXX compiler identification is AppleClang 10.0.1.10010046 -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found CURL: /usr/lib/libcurl.dylib (found version "7.54.0") -- Found OpenSSL: /usr/local/opt/openssl/lib/libcrypto.dylib (found version "1.0.2r") -- Configuring done -- Generating done -- Build files have been written to: /Users/mgukov/dev/vyu/lib_broadcaster_src/azure-storage-cpplite/build.release Scanning dependencies of target azure-storage-lite [ 3%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/logging.cpp.o [ 6%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/base64.cpp.o [ 10%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/constants.cpp.o [ 13%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/hash.cpp.o [ 16%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/utility.cpp.o [ 20%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/tinyxml2.cpp.o [ 23%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/tinyxml2_parser.cpp.o [ 26%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/storage_account.cpp.o [ 30%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/storage_credential.cpp.o [ 33%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/storage_url.cpp.o [ 36%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/get_blob_request_base.cpp.o [ 40%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/put_blob_request_base.cpp.o [ 43%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/delete_blob_request_base.cpp.o [ 46%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/copy_blob_request_base.cpp.o [ 50%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/create_container_request_base.cpp.o [ 53%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/delete_container_request_base.cpp.o [ 56%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/list_containers_request_base.cpp.o [ 60%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/list_blobs_request_base.cpp.o [ 63%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/get_blob_property_request_base.cpp.o [ 66%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/get_block_list_request_base.cpp.o [ 70%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/get_container_property_request_base.cpp.o [ 73%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/put_block_request_base.cpp.o [ 76%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/put_block_list_request_base.cpp.o [ 80%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/append_block_request_base.cpp.o [ 83%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/put_page_request_base.cpp.o [ 86%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/get_page_ranges_request_base.cpp.o [ 90%] Building CXX object CMakeFiles/azure-storage-lite.dir/src/http/libcurl_http_client.cpp.o In file included from /Users/mgukov/dev/vyu/lib_broadcaster_src/azure-storage-cpplite/src/http/libcurl_http_client.cpp:1: In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/sstream:174: In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:138: In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ios:216: In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:15: In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/string:500: In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/string_view:176: In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__string:56: In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/algorithm:644: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/memory:3706:5: error: destructor called on non-final 'azure::storage_lite::CurlEasyRequest' that has virtual functions but non-virtual destructor [-Werror,-Wdelete-non-virtual-dtor] __data_.second().~_Tp(); ^ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/memory:3667:9: note: in instantiation of member function 'std::__1::__shared_ptr_emplace<azure::storage_lite::CurlEasyRequest, std::__1::allocator<azure::storage_lite::CurlEasyRequest> >::__on_zero_shared' requested here __shared_ptr_emplace(_Alloc __a, _Args&& ...__args) ^ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/memory:4327:26: note: in instantiation of function template specialization 'std::__1::__shared_ptr_emplace<azure::storage_lite::CurlEasyRequest, std::__1::allocator<azure::storage_lite::CurlEasyRequest> >::__shared_ptr_emplace<std::__1::shared_ptr<azure::storage_lite::CurlEasyClient>, void *&>' requested here ::new(__hold2.get()) _CntrlBlk(__a2, _VSTD::forward<_Args>(__args)...); ^ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/memory:4706:29: note: in instantiation of function template specialization 'std::__1::shared_ptr<azure::storage_lite::CurlEasyRequest>::make_shared<std::__1::shared_ptr<azure::storage_lite::CurlEasyClient>, void *&>' requested here return shared_ptr<_Tp>::make_shared(_VSTD::forward<_Args>(__args)...); ^ /Users/mgukov/dev/vyu/lib_broadcaster_src/azure-storage-cpplite/include/http/libcurl_http_client.h:305:29: note: in instantiation of function template specialization 'std::__1::make_shared<azure::storage_lite::CurlEasyRequest, std::__1::shared_ptr<azure::storage_lite::CurlEasyClient>, void *&>' requested here auto res = std::make_shared<CurlEasyRequest>(shared_from_this(), m_handles.front()); ^ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/memory:3706:23: note: qualify call to silence this warning __data_.second().~_Tp(); ^ 1 error generated. make[2]: *** [CMakeFiles/azure-storage-lite.dir/src/http/libcurl_http_client.cpp.o] Error 1 make[1]: *** [CMakeFiles/azure-storage-lite.dir/all] Error 2 make: *** [all] Error 2

Fix crosscompiling on Windows by removing PkgConfig dependency

When crosscompiling on Windows, there is currently a dependency to PkgConfig that can be hard to workaround. PkgConfig is used to handle uuid support, and it cannot be skipped despite of providing separately a uuid package.

Crash occurs with container_exists & get_blob_property with invalid credentails

Hello All,

I am just trying to create IoT application with Azure. I am using Blob storage sdk lite and facing some issues as below:
The code at line 4 and 5 sometimes get into infinite loop or crashes (some times) If invalid credential is passed to account.

1. auto bC = std::make_shared<blob_client>(account, 10);
2. bool exists = true;
3. blob_client_wrapper bc(bC);
4. exists = bc.container_exists(strContainerName);
5. get_blob_property

Is there any way to check if valid or any fix available for this?

VS2015 support

Would it be possible to add VS2015 support for the SDK? Our product currently supports VS 2015 and in order to have seamless integration and to pull regular updates it would be important for the SDK to be built in VS 2015.

I have compiled/tested the sdk and there does not seem to be any issue.

Comparison to azure-storage-fuse/azure-storage-cpp-lite

Came across this and was wondering what the difference here is to the source at https://github.com/Azure/azure-storage-fuse/tree/master/azure-storage-cpp-lite ?

I am depending on the storage library source there for an implementation of azure blob support in an upcoming tensorflow/io PR. On the surface it seems like this has been extracted from that original repo and I'd prefer to target this dedicated repo if it's going to stick around as a c++ storage library.

Does this seem appropriate to move the dependency from azure-storage-fuse/azure-storage-cpp-lite source to this?

Configurable Retry Policy

Both class blob_client constructors initialize executor_context blob_client::m_context with the default class retry_policy. The default retry policy retries 26 times with an exponential back-off up to 60 seconds on each retry.

Please support a configurable retry policy. I'd suggest:

1. Allow class retry_policy to be constructed with both a maximum retry count and a retry interval.
2. Allow class blob_client to be constructed with an instance of class retry_policy.

Add support for connecting via https proxy

azure-storage-cpplite currently does not support connecting to a proxy via https (see this issue for more details)

This would be a useful feature to add