boostorg / iostreams

Boost.org iostreams module

Iostreams, part of collection of the Boost C++ Libraries, provides:

• Tools to make it easy to create standard C++ streams and stream buffers for accessing new Sources and Sinks.
• A framework for defining filters and attaching them to standard streams and stream buffers.
• A collection of ready-to-use Filters, Sources and Sinks.
• Utilities to save and restore stream state.

License

Distributed under the Boost Software License, Version 1.0.

Properties

• C++03
• Requires a Link Library

Build Status

Branch	Travis	Appveyor	Coverity Scan	codecov.io	Deps	Docs	Tests
master
develop

Directories

Name	Purpose
doc	documentation
example	examples
include	headers
test	unit tests

More information

• Ask questions
• Report bugs: Be sure to mention Boost version, platform and compiler you're using. A small compilable code sample to reproduce the problem is always good as well.
• Submit your patches as pull requests against develop branch. Note that by submitting patches you agree to license your modifications under the Boost Software License, Version 1.0.
• Discussions about the library are held on the Boost developers mailing list. Be sure to read the discussion policy before posting and add the [iostreams] tag at the beginning of the subject line.

iostreams's Issues

C4706 MSVC warning in zlib.hpp

Happens at least with Boost 1.76 release or lower.
If you build a C++ project which includes Boost's zlib.hpp with /W4 warnings level, you may get a such warning:

boost\iostreams\filter\zlib.hpp(392): warning C4706: assignment within conditional expression

It is caused by this code from the include/boost/iostreams/filter/zlib.hpp:

return !(eof_ = result == zlib::stream_end);

There are at least two possible solutions for this issue:

1. Add 4706 to the list of disabled warnings in this file:

#ifdef BOOST_MSVC
# pragma warning(push)
# pragma warning(disable:4251 4275 4231 4660 4706)
#endif

2. Split this expression:

eof_ = (result == zlib::stream_end);
return !eof_;

No const overloads for getting the underlying device of boost::iostream::stream

There seem to be no const overloads for getting the underlying Device of boost::iostream::stream. It might not be as trivial as by adding the following two lines, because sub-sequential calls do not seem to implement the const overloads either.

Device const & operator*() const { return *this->member; }
Device const * operator->() const { return &*this->member; }

I'm not sure if there is some reason why there are no const overloads for these particular methods. As far as I can see there shouldn't be any issue to add them. Currently I can work around this problem by marking the stream in the example as mutable.

In any case here an example: https://coliru.stacked-crooked.com/a/fe70537e096a2b0b.
Or use the code below:

#include <iostream>
#include <boost/iostreams/categories.hpp>
#include <boost/iostreams/stream.hpp>

struct stream_sink
{
    using char_type = char;
    using category = boost::iostreams::sink_tag;
    
    stream_sink(std::ostream & os) :
        stream(os)
    {}
    
    bool fail() const { return stream.fail(); }
    
    std::streamsize write(const char * s, std::streamsize n) 
    {
        stream.write(s, n);
        return n;
    }    
    
    std::ostream & stream;
};

struct cout_writer
{
    using stream_type = boost::iostreams::stream<stream_sink>;
    
    stream_type stream{std::cout};
    
    operator bool() { return !stream->fail(); }
    
    // Does not compile as there are no const overloads like below:
    // - Device const & operator*() const { ... }
    // - Device const * operator->() const { ... }
    operator bool() const { return !stream->fail(); }
    
    cout_writer & operator<<(std::string_view s)
    {
        stream << s;
        return *this;
    }
};

int main()
{
    cout_writer writer;
    
    writer << "foo\n";
}

Peeking into std::cin through a boost::iostreams::filtering_istream forever blocks

In the below code, I am wrapping std::cin into a boost::iostreams::filtering_istream. Calling peek() on the resulting object seems to block forever since I don't return from peek() whatever the number of characters that I type (pressing enter every now and then to flush).

#include <iostream>
#include <boost/iostreams/filtering_stream.hpp>

int main()
{
  boost::iostreams::filtering_istream in;
  in.push(std::cin); // tried in.push(boost::ref(std::cin))

  if ('a' == std::cin.peek()) std::cout << "ok";
  if ('a' == in.peek()) std::cout << "ok"; // blocks whatever the amount of characters typed, I expected it to behave like the std::cin.

  return 0;
}

I could not find any indication in the documentation that suggests I am not using the library correctly. I tried to wrap std::cin with boost::ref when pushing it to the filtering stream but it did not change the behavior.

Interestingly, the problem only occurs when interactively typing to the standard input. If I provide data to the standard input through a pipeline, it works as expected:

echo "aaaa" | program

I tested the snippet with Visual Studio 2017 (boost 1.75.0 and 1.70.0) and gcc9.3 (boost 1.72.0) and none worked.

Thank you in advance for your help in figuring out what goes wrong.

Please merge the 'develop' branch change into 'master' for the Boost 1.75 release

A fix in the 'develop' branch is for the Embarcadero C++ clang-based compilers.

Deprecation warning with boost 1.73.0 in copy.hpp

iostreams/copy.hpp includes <boost/bind.hpp> which causes a deprecation warning with boost 1.73.0 by simply #including this file.

..\boost\boost/bind.hpp(41): note: The practice of declaring the Bind placeholders (_1, _2, ...)
in the global namespace is deprecated. Please use <boost/bind/bind.hpp> + using
namespace boost::placeholders, or define BOOST_BIND_GLOBAL_PLACEHOLDERS to
retain the current behavior.

As far as I can see iostreams itself does not use the placeholders, so replacing the include with
#include <boost/bind/bind.hpp> will resolve the problem here; but of course with the risk of causing a downstream problem for anyone accidentally relying on _1 etc. at global scope.

Cannot disable zstd with "bjam -sNO_ZSTD" in boost/1.78.0

1. The integration of bzip2 and zlib is much better in iostreams than the integration of lzma and zstd. There is a need to build boost iostreams with sandboxed versions of all the compressors instead of having bjam find and build against the host OS versions of these

2. There is no equivalent of ZLIB_SOURCE/BZIP2_SOURCE for LZMA_SOURCE/ZSTD_SOURCE ??

3. How do I redirect the boost iostreams build to use specific versions of LZMA and ZSTD during boost iostreams build?

4. bjam -sNO_ZSTD does not work
   ./tools/build/src/engine/bjam -d+2 -sNO_ZSTD --debug-configuration variant=debug toolset=gcc address-model=64 visibility=global cxxflags='--std=c++11 -fPIC -ggdb3 -O0' install --prefix=./sles11_x86_64_gcc4.7.2/deb/boost_1_78_0 --stagedir=./sles11_x86_64_gcc4.7.2/deb/boost_1_78_0-build --includedir=./sles11_x86_64_gcc4.7.2/deb/boost_1_78_0/include --libdir=./sles11_x86_64_gcc4.7.2/deb/boost_1_78_0/lib --with-atomic --with-chrono --with-container --with-context --with-contract --with-coroutine --with-date_time --with-exception --with-fiber --with-filesystem --with-graph --with-graph_parallel --with-headers --with-iostreams --with-locale --with-log --with-math --with-mpi --with-program_options --with-random --with-regex --with-serialization --with-stacktrace --with-system --with-test --with-thread --with-timer --with-type_erasure --with-json --with-python
   ....
   notice: [zstd] Using pre-installed library
   notice: [zstd] Condition
   ...

   • zstd : yes
     ...

which leads to the inevitable linker errors

-I-:/nfs/site/itools/em64t_SLES12SP5/pkgs/gcc/4.7.2/.bin/../lib64/gcc/x86_64-suse-linux/4.7.2/../../../../x86_64-suse-linux/bin/ld: warning: libzstd.so.1, needed by
./boost/1.78.0/libs/sles12_x86_64_gcc4.7.2/opt/libboost_iostreams.so, not found (try using -rpath or -rpath-link)
./boost/1.78.0/libs/sles12_x86_64_gcc4.7.2/opt/libboost_iostreams.so: undefined reference to ZSTD_freeCStream' ./boost/1.78.0/libs/sles12_x86_64_gcc4.7.2/opt/libboost_iostreams.so: undefined reference to lzma_stream_decoder@XZ_5.0'
./boost/1.78.0/libs/sles12_x86_64_gcc4.7.2/opt/libboost_iostreams.so: undefined reference to ZSTD_getErrorName' ./boost/1.78.0/libs/sles12_x86_64_gcc4.7.2/opt/libboost_iostreams.so: undefined reference to ZSTD_createDStream'
./boost/1.78.0/libs/sles12_x86_64_gcc4.7.2/opt/libboost_iostreams.so: undefined reference to ZSTD_freeDStream' ./boost/1.78.0/libs/sles12_x86_64_gcc4.7.2/opt/libboost_iostreams.so: undefined reference to ZSTD_compressStream'
./boost/1.78.0/libs/sles12_x86_64_gcc4.7.2/opt/libboost_iostreams.so: undefined reference to ZSTD_decompressStream' ./boost/1.78.0/libs/sles12_x86_64_gcc4.7.2/opt/libboost_iostreams.so: undefined reference to lzma_code@XZ_5.0'
./boost/1.78.0/libs/sles12_x86_64_gcc4.7.2/opt/libboost_iostreams.so: undefined reference to ZSTD_initCStream' ./boost/1.78.0/libs/sles12_x86_64_gcc4.7.2/opt/libboost_iostreams.so: undefined reference to lzma_stream_encoder_mt

Manual close of code_converter device does not flush data

Only when you manualy close boost::iostream::code_converter device the data are not flushed into the mapped device.

When you execute following fragment of the code, there is created empty file log.txt.

void write_text_file()
{

    boost::locale::generator gen;
    typedef io::code_converter<io::file_descriptor_sink> converter_device;
    typedef io::stream<converter_device> converter_stream;

    io::file_descriptor_sink file_sink;
    converter_device dev;
    converter_stream stream;

    dev.imbue(gen("C.UTF-8"));

    file_sink.open("log.txt", std::ios_base::out);

    dev.open(file_sink);
    stream.open(dev);

    stream << L"Write UTF-8 string\n"
           << L"Zapiš UTF-8 řetězec";

    stream.close();
    dev.close();
    file_sink.close();
    
    cout << "file_sink.is_open = " << file_sink.is_open() << "\n"
         << "dev.is_open = " << dev.is_open() << "\n"
         << "stream.is_open = " << stream.is_open();
}

From the following output of the program is see the converter device is still open:

file_sink.is_open = 0
dev.is_open = 1
stream.is_open = 0

The reason is the method close has the input argument which for identification to close input or output stream. The default value is bitwise or of constant in | out.

class code_converter 
    : protected code_converter_base<Device, Codecvt, Alloc>
...
public:
    void close(BOOST_IOS::openmode which = BOOST_IOS::in | BOOST_IOS::out )
    { impl().close(which); }

Private method impl() returns instance of code_converter_impl where the method close is overladed and implemented as follows:

// Contains member data, open/is_open/close and buffer management functions.
template<typename Device, typename Codecvt, typename Alloc>
struct code_converter_impl {
    void close(BOOST_IOS::openmode which)
    {
// BUG: Condition block is called only when which is either in or out but cannot be
// combination of both
        if (which == BOOST_IOS::in && (flags_ & f_input_closed) == 0) {
...
        }
        if (which == BOOST_IOS::out && (flags_ & f_output_closed) == 0) {
    ....
            );
        }
    }

When the which argument has the value which = in | out which is the default value provided by code_convert::close no condition block is called at all.
The result is the device is not closed and data in the buffer are lost.

Cygwin 32-bit failure on AppVeyor tests

On Appveyor tests for cygwin 32-bit I get failure with the message:

[00:03:30] This problem is probably due to using incompatible versions of the cygwin DLL.
[00:03:30] Search for cygwin1.dll using the Windows Start->Find/Search facility
[00:03:30] and delete all but the most recent version. The most recent version should
[00:03:30] reside in x:\cygwin\bin, where 'x' is the drive on which you have
[00:03:30] installed the cygwin distribution. Rebooting is also suggested if you
[00:03:30] are unable to find another cygwin DLL.

This seems like some sort of setup problem for cygwin 32-bit under Appveyor.

GCC suggest-override warnings

GCC 7.5 fires suggest-override warnings in Boost 1.72:

include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:211:6: error: ‘bool boost::iostreams::detail::indirect_streambuf::auto_close() const [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:215:6: error: ‘void boost::iostreams::detail::indirect_streambuf::set_auto_close(bool) [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:319:6: error: ‘bool boost::iostreams::detail::indirect_streambuf::strict_sync() [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:222:6: error: ‘void boost::iostreams::detail::indirect_streambuf::imbue(const std::locale&) [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:234:1: error: ‘boost::iostreams::detail::indirect_streambuf::int_type boost::iostreams::detail::indirect_streambuf::underflow() [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output; boost::iostreams::detail::indirect_streambuf::int_type = int]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:269:1: error: ‘boost::iostreams::detail::indirect_streambuf::int_type boost::iostreams::detail::indirect_streambuf::pbackfail(boost::iostreams::detail::indirect_streambuf::int_type) [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output; boost::iostreams::detail::indirect_streambuf::int_type = int]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:283:1: error: ‘boost::iostreams::detail::indirect_streambuf::int_type boost::iostreams::detail::indirect_streambuf::overflow(boost::iostreams::detail::indirect_streambuf::int_type) [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output; boost::iostreams::detail::indirect_streambuf::int_type = int]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:309:5: error: ‘int boost::iostreams::detail::indirect_streambuf::sync() [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:329:1: error: ‘boost::iostreams::detail::indirect_streambuf::pos_type boost::iostreams::detail::indirect_streambuf::seekoff(boost::iostreams::detail::indirect_streambuf::off_type, std::ios_base::seekdir, std::ios_base::openmode) [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output; boost::iostreams::detail::indirect_streambuf::pos_type = std::fpos<__mbstate_t>; boost::iostreams::detail::indirect_streambuf::off_type = long int; std::ios_base::seekdir = std::_Ios_Seekdir; std::ios_base::openmode = std::_Ios_Openmode]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:335:1: error: ‘boost::iostreams::detail::indirect_streambuf::pos_type boost::iostreams::detail::indirect_streambuf::seekpos(boost::iostreams::detail::indirect_streambuf::pos_type, std::ios_base::openmode) [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output; boost::iostreams::detail::indirect_streambuf::pos_type = std::fpos<__mbstate_t>; std::ios_base::openmode = std::_Ios_Openmode]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:378:13: error: ‘void boost::iostreams::detail::indirect_streambuf::set_next(boost::iostreams::detail::indirect_streambuf::streambuf_type*) [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output; boost::iostreams::detail::indirect_streambuf::streambuf_type = boost::iostreams::detail::linked_streambuf >; typename boost::iostreams::char_type_of::type = char]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:383:13: error: ‘void boost::iostreams::detail::indirect_streambuf::close_impl(std::ios_base::openmode) [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output; std::ios_base::openmode = std::_Ios_Openmode]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:97:34: error: ‘const typeinfo& boost::iostreams::detail::indirect_streambuf::component_type() const [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output; boost::core::typeinfo = std::type_info]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:98:11: error: ‘void* boost::iostreams::detail::indirect_streambuf::component_impl() [with T = boost::iostreams::back_insert_device >; Tr = std::char_traits; Alloc = std::allocator; Mode = boost::iostreams::output]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/direct_streambuf.hpp:63:10: error: ‘bool boost::iostreams::detail::direct_streambuf::auto_close() const [with T = boost::iostreams::basic_array_source; Tr = std::char_traits]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/direct_streambuf.hpp:64:10: error: ‘void boost::iostreams::detail::direct_streambuf::set_auto_close(bool) [with T = boost::iostreams::basic_array_source; Tr = std::char_traits]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/direct_streambuf.hpp:65:10: error: ‘bool boost::iostreams::detail::direct_streambuf::strict_sync() [with T = boost::iostreams::basic_array_source; Tr = std::char_traits]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/direct_streambuf.hpp:202:6: error: ‘void boost::iostreams::detail::direct_streambuf::close_impl(std::ios_base::openmode) [with T = boost::iostreams::basic_array_source; Tr = std::char_traits; std::ios_base::openmode = std::_Ios_Openmode]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/direct_streambuf.hpp:77:34: error: ‘const typeinfo& boost::iostreams::detail::direct_streambuf::component_type() const [with T = boost::iostreams::basic_array_source; Tr = std::char_traits; boost::core::typeinfo = std::type_info]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/direct_streambuf.hpp:78:11: error: ‘void* boost::iostreams::detail::direct_streambuf::component_impl() [with T = boost::iostreams::basic_array_source; Tr = std::char_traits]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/direct_streambuf.hpp:140:1: error: ‘boost::iostreams::detail::direct_streambuf::int_type boost::iostreams::detail::direct_streambuf::underflow() [with T = boost::iostreams::basic_array_source; Tr = std::char_traits; boost::iostreams::detail::direct_streambuf::int_type = int]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/direct_streambuf.hpp:153:1: error: ‘boost::iostreams::detail::direct_streambuf::int_type boost::iostreams::detail::direct_streambuf::pbackfail(boost::iostreams::detail::direct_streambuf::int_type) [with T = boost::iostreams::basic_array_source; Tr = std::char_traits; boost::iostreams::detail::direct_streambuf::int_type = int]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/direct_streambuf.hpp:169:1: error: ‘boost::iostreams::detail::direct_streambuf::int_type boost::iostreams::detail::direct_streambuf::overflow(boost::iostreams::detail::direct_streambuf::int_type) [with T = boost::iostreams::basic_array_source; Tr = std::char_traits; boost::iostreams::detail::direct_streambuf::int_type = int]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/direct_streambuf.hpp:189:1: error: ‘boost::iostreams::detail::direct_streambuf::pos_type boost::iostreams::detail::direct_streambuf::seekoff(boost::iostreams::detail::direct_streambuf::off_type, std::ios_base::seekdir, std::ios_base::openmode) [with T = boost::iostreams::basic_array_source; Tr = std::char_traits; boost::iostreams::detail::direct_streambuf::pos_type = std::fpos<__mbstate_t>; boost::iostreams::detail::direct_streambuf::off_type = long int; std::ios_base::seekdir = std::_Ios_Seekdir; std::ios_base::openmode = std::_Ios_Openmode]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/direct_streambuf.hpp:195:1: error: ‘boost::iostreams::detail::direct_streambuf::pos_type boost::iostreams::detail::direct_streambuf::seekpos(boost::iostreams::detail::direct_streambuf::pos_type, std::ios_base::openmode) [with T = boost::iostreams::basic_array_source; Tr = std::char_traits; boost::iostreams::detail::direct_streambuf::pos_type = std::fpos<__mbstate_t>; std::ios_base::openmode = std::_Ios_Openmode]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/filtering_stream.hpp:98:10: error: ‘void boost::iostreams::detail::filtering_stream_base::notify() [with Chain = boost::iostreams::chain, std::allocator >; Access = boost::iostreams::public_]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/chainbuf.hpp:54:14: error: ‘boost::iostreams::detail::chainbuf::int_type boost::iostreams::detail::chainbuf::underflow() [with Chain = boost::iostreams::chain, std::allocator >; Mode = boost::iostreams::input; Access = boost::iostreams::public_; boost::iostreams::detail::chainbuf::int_type = int]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/chainbuf.hpp:56:14: error: ‘boost::iostreams::detail::chainbuf::int_type boost::iostreams::detail::chainbuf::pbackfail(boost::iostreams::detail::chainbuf::int_type) [with Chain = boost::iostreams::chain, std::allocator >; Mode = boost::iostreams::input; Access = boost::iostreams::public_; boost::iostreams::detail::chainbuf::int_type = int]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/chainbuf.hpp:58:21: error: ‘std::streamsize boost::iostreams::detail::chainbuf::xsgetn(boost::iostreams::detail::chainbuf::char_type*, std::streamsize) [with Chain = boost::iostreams::chain, std::allocator >; Mode = boost::iostreams::input; Access = boost::iostreams::public_; std::streamsize = long int; boost::iostreams::detail::chainbuf::char_type = char]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/chainbuf.hpp:60:14: error: ‘boost::iostreams::detail::chainbuf::int_type boost::iostreams::detail::chainbuf::overflow(boost::iostreams::detail::chainbuf::int_type) [with Chain = boost::iostreams::chain, std::allocator >; Mode = boost::iostreams::input; Access = boost::iostreams::public_; boost::iostreams::detail::chainbuf::int_type = int]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/chainbuf.hpp:62:21: error: ‘std::streamsize boost::iostreams::detail::chainbuf::xsputn(const char_type*, std::streamsize) [with Chain = boost::iostreams::chain, std::allocator >; Mode = boost::iostreams::input; Access = boost::iostreams::public_; std::streamsize = long int; boost::iostreams::detail::chainbuf::char_type = char]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/chainbuf.hpp:64:9: error: ‘int boost::iostreams::detail::chainbuf::sync() [with Chain = boost::iostreams::chain, std::allocator >; Mode = boost::iostreams::input; Access = boost::iostreams::public_]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/chainbuf.hpp:65:14: error: ‘boost::iostreams::detail::chainbuf::pos_type boost::iostreams::detail::chainbuf::seekoff(boost::iostreams::detail::chainbuf::off_type, std::ios_base::seekdir, std::ios_base::openmode) [with Chain = boost::iostreams::chain, std::allocator >; Mode = boost::iostreams::input; Access = boost::iostreams::public_; boost::iostreams::detail::chainbuf::pos_type = std::fpos<__mbstate_t>; boost::iostreams::detail::chainbuf::off_type = long int; std::ios_base::seekdir = std::_Ios_Seekdir; std::ios_base::openmode = std::_Ios_Openmode]’ can be marked override [-Werror=suggest-override]
include/boost/iostreams/detail/streambuf/chainbuf.hpp:69:14: error: ‘boost::iostreams::detail::chainbuf::pos_type boost::iostreams::detail::chainbuf::seekpos(boost::iostreams::detail::chainbuf::pos_type, std::ios_base::openmode) [with Chain = boost::iostreams::chain, std::allocator >; Mode = boost::iostreams::input; Access = boost::iostreams::public_; boost::iostreams::detail::chainbuf::pos_type = std::fpos<__mbstate_t>; std::ios_base::openmode = std::_Ios_Openmode]’ can be marked override [-Werror=suggest-override]

Incomplete documentation regarding zstd filter

The zst filter does not have a documentation page similar to bzip2, gzip, or zlib at
https://www.boost.org/doc/libs/1_70_0/libs/iostreams/doc/classes.

Forward seekable compressed streams

Using boost::iostreams::filtering_istream together with one of the decompression filters, e.g. boost::iostreams::gzip_decompressor results in tellg returning -1 and setting the bad bid and seekg not working.

However, at least forward seeking and telling the number read bytes would be possible and would make working with these streams much easier for certain use cases (e.g. reading custom binary files compressed with these methods).

i.e. this:

#include <boost/iostreams/filtering_stream.hpp>
#include <boost/iostreams/filter/gzip.hpp>
#include <boost/iostreams/device/file.hpp>


namespace io = boost::iostreams;
int main() {
    io::filtering_istream stream;
    stream.push(io::gzip_decompressor());
    stream.push(io::file_source("test.gzip"));
    
    std::cout << stream.tellg() << std::endl; // should print 0, prints -1
    stream.seekg(1000); // skip 1000 bytes
    std::cout << stream.tellg() << std::endl; // should print 1000, prints -1
    
    return 0;
}

See for example gzseek: https://refspecs.linuxbase.org/LSB_3.0.0/LSB-Core-generic/LSB-Core-generic/zlib-gzseek-1.html

and gztell: https://refspecs.linuxbase.org/LSB_3.0.0/LSB-Core-generic/LSB-Core-generic/zlib-gztell-1.html

Forward CFLAGS options to nested bzip2/zlib builds

Hi!

I use CFLAGS/CXXFLAGS="-isysroot <...>" flag to pass system headers location to the compiler. b2 works fine with this and almost all boost libraries are compiled succesfully, except iostreams. When building iostreams with ZLIB/BZIP2, these nested builds fail due to missing isysroot option. How to fix zlib.jam/bzip2.jam to get it work with CFLAGS?

warning: comparison of integer expressions of different signedness

While building 1.68.0 beta 1 on x86_64 linux cross-compiling to Windows 64 bits with mingw-w64 + g++ 8.2.0, I get this warning:

"x86_64-w64-mingw32-g++-8.2.0"   -I/softs/win64-mingw-8.2.0/release/iconv/include -I/softs/win64-mingw-8.2.0/release/gettext/include -I/softs/win64-mingw-8.2.0/release/bzip2/include -I/softs/win64-mingw-8.2.0/release/zlib/include -I/softs/win64-mingw-8.2.0/release/jpeg-turbo/include -I/softs/win64-mingw-8.2.0/release/xz/include -I/softs/win64-mingw-8.2.0/release/tiff/include -I/softs/win64-mingw-8.2.0/release/png/include -std=c++14 -O2 -DNDEBUG -m64 -mthreads -O3 -finline-functions -Wno-inline -Wall  -DBOOST_ALL_NO_LIB=1 -DBOOST_ASIO_NO_DEPRECATED -DBOOST_FILESYSTEM_NO_DEPRECATED -DBOOST_IOSTREAMS_USE_DEPRECATED -DBOOST_LOG_WITHOUT_EVENT_LOG -DNDEBUG  -I"." -I"/softs/win64-mingw-8.2.0/release/bzip2/include" -I"/softs/win64-mingw-8.2.0/release/zlib/include" -c -o "bin.v2/libs/iostreams/build/gcc-8.2.0/release/link-static/target-os-windows/threading-multi/mapped_file.o" "libs/iostreams/src/mapped_file.cpp"
libs/iostreams/src/mapped_file.cpp: In member function 'void boost::iostreams::detail::mapped_file_impl::try_map_file(boost::iostreams::detail::mapped_file_impl::param_type)':
libs/iostreams/src/mapped_file.cpp:324:29: warning: comparison of integer expressions of different signedness: 'boost::iostreams::stream_offset' {aka 'long long int'} and 'boost::iostreams::mapped_file_source::size_type' {aka 'long long unsigned int'} [-Wsign-compare]
             (SIZE_T) (size_ != max_length ? size_ : 0),
                       ~~~~~~^~~~~~~~~~~~~

[Feature request] Request the ability to `sync` single component of chain

As I describe in here. I need flush only the first component of chain, but sync_impl method of indirect_streambuf is private. I can't think of a better way to achieve this.

mapped_file documentation is incomplete

mapped_file documentation doesn't say:

1. if destructor closes file
2. how we should deal with IO errors (std::copy(file.data(), ...) can't throw, I guess?)

One should read source code to get that information:
https://stackoverflow.com/questions/26608264/what-exceptions-does-boost-mapped-file-source-throw
https://stackoverflow.com/questions/7342045/are-boost-mapped-files-raii

iostreams\detail\streambuf\indirect_streambuf.hpp(440) : warning C4702: unreachable code

hi,
i'm compiling my .dll code for release without optimization (/W3 /WX /MD /Od) and msvc reports warning (error in my case):

boost-1.77.0\include\boost\iostreams\detail\streambuf\indirect_streambuf.hpp(440) : error C2220: the following warning is treated as an error
boost-1.77.0\include\boost\iostreams\detail\streambuf\indirect_streambuf.hpp(440) : warning C4702: unreachable code

as a local workaround i've added following pragma to disable_warnings.hpp

pragma warning(disable:4702) // Unreachable code

Documentation example (zlib) is invalid

Hello.

https://www.boost.org/doc/libs/1_69_0/libs/iostreams/doc/classes/zlib.html

Example on the bottom page cannot be compiled.
Similar problem as #79

Inconsistent documentation on return value of write() for SeekableDevice

In the SeekableDevice documentation, the description of write() says:

iostreams/doc/concepts/seekable_device.html

Lines 58 to 60 in 29a8317

	<SPAN CLASS="comment">// Write up to n characters from the buffer
	// s to the output sequence, returning the
	// number of characters written</SPAN>

but the function signature says the return value is void:

iostreams/doc/concepts/seekable_device.html

Line 56 in 29a8317

<SPAN CLASS="keyword">void</SPAN> write(<SPAN CLASS="keyword">const</SPAN> <SPAN CLASS="keyword">char</SPAN>* s, std::streamsize n)

Should that say std::streamsize instead?

compiling boost/iostreams with external lzma links to wrong lzma library

I'm trying to build boost 1.68 on linux (CentOS 6) using the developer toolset 7. Since I'm building zlib, bzip2 and lzma for other parts of the project, I'd like boost to use those during its build. I am using CMake's ExternalProject_Add with a set of configure, buld, and install steps, with the options below. All three libraries are in the LIBRARY_PATH and LD_LIBRARY_PATH.

ExternalProject_Add(boost
	URL  ${CMAKE_CURRENT_SOURCE_DIR}/boost_1_68_0.tar.xz
	URL_MD5 566027ab95e4d5920d84ab9a7efd4afd
	PREFIX ${CMAKE_CURRENT_BINARY_DIR}/boost	
	INSTALL_DIR ${MY_ROOT}
	UPDATE_COMMAND ""		
	CONFIGURE_COMMAND 
		./bootstrap.sh 
		--with-python-version=2.7	
		--with-toolset=gcc 
		--prefix=<INSTALL_DIR>
		--with-libraries=iostreams,system,thread
	BUILD_COMMAND 
		./b2 
		--layout=system
		--build-dir=${CMAKE_CURRENT_BINARY_DIR}/boost/src/boost-build
		--stage-dir=${CMAKE_CURRENT_BINARY_DIR}/boost/src/boost-stage
		variant=release
		link=shared,static
		address-model=64		
		threading=multi
		runtime-link=shared
		-j4 
		--prefix=<INSTALL_DIR>
		cxxflags=${CMAKE_CXX_FLAGS_RELEASE}		
		-sBZIP2_LIBPATH=<INSTALL_DIR>/lib
		-sBZIP2_BINARY=bz2
		-sBZIP2_INCLUDE=<INSTALL_DIR>/include		
		-sLZMA_LIBPATH=<INSTALL_DIR>/lib
		-sLZMA_BINARY=lzma
		-sLZMA_INCLUDE=<INSTALL_DIR>/include
		-sZLIB_LIBPATH=<INSTALL_DIR>/lib
		-sZLIB_BINARY=z
		-sZLIB_INCLUDE=<INSTALL_DIR>/include
		stage
	INSTALL_COMMAND 
		./b2 install
	LOG_UPDATE 1
	LOG_CONFIGURE 1
	LOG_BUILD 1
	LOG_INSTALL 1
	BUILD_IN_SOURCE 1		
)

This finishes fine, and I don't see any problems in the build, however, the output below shows that something must have gone wrong. Any advice?

$ ldd libboost_iostreams.so
	linux-vdso.so.1 =>  (0x00007fffe35ef000)
	librt.so.1 => /lib64/librt.so.1 (0x00007fcd522d0000)
	libbz2.so.1 => /home/tester/lib/libbz2.so.1 (0x00007fcd520b7000)
-->	liblzma.so.0 => /usr/lib64/liblzma.so.0 (0x00007fcd51e96000)   
	libz.so.1 => /home/tester/lib/libz.so.1 (0x00007fcd51c7b000)
	libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007fcd51974000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fcd516f0000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fcd514da000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fcd512bc000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fcd50f28000)
	/lib64/ld-linux-x86-64.so.2 (0x0000555c26f63000)

Output from bootstrap

Building Boost.Build engine with toolset gcc... tools/build/src/engine/bin.linuxx86_64/b2
Unicode/ICU support for Boost.Regex?... /usr
Generating Boost.Build configuration in project-config.jam...

Bootstrapping is done. To build, run:

    ./b2
    
To adjust configuration, edit 'project-config.jam'.
Further information:

   - Command line help:
     ./b2 --help
     
   - Getting started guide: 
     http://www.boost.org/more/getting_started/unix-variants.html
     
   - Boost.Build documentation:
     http://www.boost.org/build/doc/html/index.html

Output from b2 stage

Performing configuration checks

    - default address-model    : 64-bit
    - default architecture     : x86
    - bzip2                    : yes
    - lzma                     : yes
    - zlib                     : yes
    - symlinks supported       : yes
    - lockfree boost::atomic_flag : yes
    - bzip2                    : yes
    - lzma                     : yes
    - zlib                     : yes

Component configuration:

    - atomic                   : not building
    - chrono                   : not building
    - container                : not building
    - context                  : not building
    - contract                 : not building
    - coroutine                : not building
    - date_time                : not building
    - exception                : not building
    - fiber                    : not building
    - filesystem               : not building
    - graph                    : not building
    - graph_parallel           : not building
    - iostreams                : building
    - locale                   : not building
    - log                      : not building
    - math                     : not building
    - mpi                      : not building
    - program_options          : not building
    - python                   : not building
    - random                   : not building
    - regex                    : not building
    - serialization            : not building
    - signals                  : not building
    - stacktrace               : not building
    - system                   : building
    - test                     : not building
    - thread                   : building
    - timer                    : not building
    - type_erasure             : not building
    - wave                     : not building
...(lots more)

Output from b2 install

Performing configuration checks

    - default address-model    : 64-bit
    - default architecture     : x86
    - bzip2                    : yes
    - lzma                     : yes
    - zlib                     : yes
    - symlinks supported       : yes
    - lockfree boost::atomic_flag : yes
    - bzip2                    : yes
    - lzma                     : yes
    - zlib                     : yes

Component configuration:

    - atomic                   : not building
    - chrono                   : not building
    - container                : not building
    - context                  : not building
    - contract                 : not building
    - coroutine                : not building
    - date_time                : not building
    - exception                : not building
    - fiber                    : not building
    - filesystem               : not building
    - graph                    : not building
    - graph_parallel           : not building
    - iostreams                : building
    - locale                   : not building
    - log                      : not building
    - math                     : not building
    - mpi                      : not building
    - program_options          : not building
    - python                   : not building
    - random                   : not building
    - regex                    : not building
    - serialization            : not building
    - signals                  : not building
    - stacktrace               : not building
    - system                   : building
    - test                     : not building
    - thread                   : building
    - timer                    : not building
    - type_erasure             : not building
    - wave                     : not building
... (lots more)

Incorrect base pointer after sync operation

In the else condition of indirect_streambuf::sync_impl(), pbase is not correctly set if data has been left in the out buffer during previous sync operations. I suggest to use setp(pbase() + amt, out().end()); instead.

cannot create an unsigned char bzip2 compressor

https://godbolt.org/z/6Gq67Yd8x

	filtering_stream<output, uint8_t> out;
	out.push(basic_bzip2_compressor<std::allocator<uint8_t>>{});

This is due to bzip2_base::char_type being hard-coded:

iostreams/include/boost/iostreams/filter/bzip2.hpp

Lines 154 to 156 in 9edd46f

	class BOOST_IOSTREAMS_DECL bzip2_base {
	public:
	typedef char char_type;

No way to create empty + appendable files via file_descriptor c-tor

This is perhaps some corner case, but consider that you want to use file_descriptor to manage some log file. Writes to log files are typically write-only-to-the-end, but you may also want to do random reads on logs. Random reads via pread are no problem in Linux (they don't move file pointer), but random reads via ReadFile with offsets specified in OVERLAPPED struct actually move file pointer, so subsequent writes will not be at the end of file.

The problem is fixed by opening a file with O_APPEND/FILE_APPEND_DATA when creating a new file. It would be nice to be able to forcefully open a "fresh" file with std::ios_base::app flag. I imagine this could be done by allowing std::ios_base::trunc | std::ios_base::app combo. The effect should be as follows:

#if defined(BOOST_IOSTREAMS_WINDOWS)
    DWORD dwDesiredAccess;
    if (enable_append) {
        dwDesiredAccess = GENERIC_READ | FILE_APPEND_DATA | FILE_WRITE_ATTRIBUTES | FILE_WRITE_EA |
                          STANDARD_RIGHTS_WRITE | SYNCHRONIZE;
    } else {
        dwDesiredAccess = GENERIC_READ | GENERIC_WRITE;
    }

    HANDLE file_handle =
        ::CreateFileA(file_name.c_str(), dwDesiredAccess, FILE_SHARE_READ | FILE_SHARE_WRITE,
                      NULL, // lpSecurityAttributes
                      CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL,
                      NULL); // hTemplateFile
#else
    // Calculate oflag argument to open.
    int oflag = O_CREAT | O_TRUNC | O_RDWR;
    if (enable_append) {
        oflag |= O_APPEND;
    }
#ifdef _LARGEFILE64_SOURCE
    oflag |= O_LARGEFILE;
#endif

    // Calculate pmode argument to open.
    mode_t pmode = S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH;

    // Open file.
    int file_handle = BOOST_IOSTREAMS_FD_OPEN(file_name.c_str(), oflag, pmode);
#endif

Error reading multi-stream bzip2 files using bzip2_decompressor

Some tools generate multi-stream bzip2 files in which several bzip2 data streams are concatenated end-to-end. Using the bzip2 executable extracts the file correctly. However, using the example code from the docs does not extract the whole content of the file.

The first call to decompress() after the stream is re-initialized (after finishing reading the first stream) returns BZ_DATA_ERROR_MAGIC. Then, the bzip2_error::check(result) throws an exception, and the following output is provided:

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::iostreams::bzip2_error> >'
  what():  bzip2 error: iostream error

The file does contain the magic bytes at the beginning of each stream.

Compile error in iostreams\detail\config\fpos.hpp because std::fpos<>::seekpos() is absent in VS 15.8 Preview 4

packages\jetbrains.boost.include.1.66.0.1\build\native\include\boost\iostreams\positioning.hpp(96): error C2039: 'seekpos': is not a member of 'std::fpos<_Mbstatet>'
1>c:\program files (x86)\microsoft visual studio\preview\enterprise\vc\tools\msvc\14.15.26629\include\iosfwd(132): note: see declaration of 'std::fpos<_Mbstatet>'

p.s. BOOST_IOSTREAMS_HAS_DINKUMWARE_FPOS is defined

UBSAN identified issue in test

https://travis-ci.org/boostorg/iostreams/jobs/448302953#L1170

testing.capture-output ../../bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test.run
====== BEGIN OUTPUT ======
../../libs/iostreams/test/code_converter_test.cpp:205:5: runtime error: member call on address 0x7ffd672455d0 which does not point to an object of type '__codecvt_abstract_base'
0x7ffd672455d0: note: object is of type 'utf8_codecvt_facet<wchar_t, char>'
 00 00 00 00  40 9a 6a 00 00 00 00 00  00 00 00 00 00 00 00 00  00 84 ba 66 bb 7f 00 00  c3 3c dd 66
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for 'utf8_codecvt_facet<wchar_t, char>'
    #0 0x41cc18 in bool codecvt_test1<utf8_codecvt_facet<wchar_t, char> >() (/home/travis/build/boostorg/boost-root/bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test+0x41cc18)
    #1 0x419b54 in bool codecvt_test<utf8_codecvt_facet<wchar_t, char> >() (/home/travis/build/boostorg/boost-root/bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test+0x419b54)
    #2 0x40f02a in code_converter_test() ../../libs/iostreams/test/code_converter_test.cpp:254
    #3 0x43646e in boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) (/home/travis/build/boostorg/boost-root/bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test+0x43646e)
    #4 0x5d1d68 in boost::function0<void>::operator()() const ../../boost/function/function_template.hpp:829
    #5 0x61f1ba in boost::detail::forward::operator()() ../../boost/test/impl/execution_monitor.ipp:1312
    #6 0x624929 in boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) ../../boost/function/function_template.hpp:172
    #7 0x622c7e in boost::function0<int>::operator()() const ../../boost/function/function_template.hpp:830
    #8 0x620870 in int boost::detail::do_invoke<boost::shared_ptr<boost::detail::translator_holder_base>, boost::function<int ()> >(boost::shared_ptr<boost::detail::translator_holder_base> const&, boost::function<int ()> const&) ../../boost/test/impl/execution_monitor.ipp:286
    #9 0x618c7b in boost::execution_monitor::catch_signals(boost::function<int ()> const&) ../../boost/test/impl/execution_monitor.ipp:875
    #10 0x618f6c in boost::execution_monitor::execute(boost::function<int ()> const&) ../../boost/test/impl/execution_monitor.ipp:1214
    #11 0x61afd0 in boost::execution_monitor::vexecute(boost::function<void ()> const&) ../../boost/test/impl/execution_monitor.ipp:1321
    #12 0x60da09 in boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::function<void ()> const&, unsigned int) ../../boost/test/impl/unit_test_monitor.ipp:49
    #13 0x5cf928 in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned int, boost::unit_test::framework::state::random_generator_helper const*) ../../boost/test/impl/framework.ipp:789
    #14 0x5cefc4 in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned int, boost::unit_test::framework::state::random_generator_helper const*) ../../boost/test/impl/framework.ipp:737
    #15 0x5cefc4 in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned int, boost::unit_test::framework::state::random_generator_helper const*) ../../boost/test/impl/framework.ipp:737
    #16 0x5bf706 in boost::unit_test::framework::run(unsigned long, bool) ../../boost/test/impl/framework.ipp:1631
    #17 0x4fbe3d in boost::unit_test::unit_test_main(boost::unit_test::test_suite* (*)(int, char**), int, char**) ../../boost/test/impl/unit_test_main.ipp:247
    #18 0x4fcba0 in main ../../boost/test/impl/unit_test_main.ipp:303
    #19 0x7fbb66809f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44)
    #20 0x40c408  (/home/travis/build/boostorg/boost-root/bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test+0x40c408)
EXIT STATUS: 1
====== END OUTPUT ======
    LD_LIBRARY_PATH="/usr/bin:/usr/lib:/usr/lib32:/usr/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH
    status=0
    if test $status -ne 0 ; then
        echo Skipping test execution due to testing.execute=off
        exit 0
    fi
     "../../bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test"   > "../../bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test.output" 2>&1 < /dev/null
    status=$?
    echo >> "../../bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test.output"
    echo EXIT STATUS: $status >> "../../bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test.output"
    if test $status -eq 0 ; then
        cp "../../bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test.output" "../../bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test.run"
    fi
    verbose=0
    if test $status -ne 0 ; then
        verbose=1
    fi
    if test $verbose -eq 1 ; then
        echo ====== BEGIN OUTPUT ======
        cat "../../bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test.output"
        echo ====== END OUTPUT ======
    fi
    exit $status
...failed testing.capture-output ../../bin.v2/libs/iostreams/test/code_converter_test.test/gcc-7/debug/cxxstd-03-iso/visibility-hidden/code_converter_test.run...

Marking the test as a known failure.

-DBOOST_IOSTREAMS_NO_LIB not documented

If iostreams is built with a prebuilt zlib library, BOOST_IOSTREAMS_NO_LIB needs to be defined to avoid linking errors (missing libboost_zlib...). This should be documented somewhere.

Unable good solution for set filter buffer size

Hello.

I want to make a filter for more beautiful text output. To do this, I need to collect a large amount of data, make beauty and output to std :: cout. Ideally, I should only process data when flush () is used;

I could not do it !!!

Here is an example.

`
#include

// I don't like this
//#define BOOST_IOSTREAMS_DEFAULT_FILTER_BUFFER_SIZE 1024

#include <boost/iostreams/filtering_stream.hpp>

struct toupper_filter
{
typedef char char_type;
typedef boost::iostreams::multichar_output_filter_tag category;

template<typename Sink>
std::streamsize write(Sink& snk, const char* s, std::streamsize n)
{

// point to interest n=128. But i need more large (many kbs). Idealy i need to enter this only after flush()
std::streamsize rest = n;

	for (; rest; --rest, s++)
	{
		int c = toupper(*s);

		bool bOk = boost::iostreams::put(snk, c);

		if (!bOk)
			break;
	}

	return n - rest;
}

};

int main()
{
namespace io = boost::iostreams;

io::filtering_ostream out;

out.push(toupper_filter());
out.push(std::cout);

// This does not work!!!
out.set_filter_buffer_size(1024);

// dont use std::endl
for (size_t x = 0; x < 100; ++x)
{
	out << "something\n";
	out << "anything\n";
}

out.flush();

return 0;

}
`
out.set_filter_buffer_size(1024); - this not work!

Change BOOST_IOSTREAMS_DEFAULT_FILTER_BUFFER_SIZE is work, but it affected to all boost::iostream library. I think it is a bad idea.

Bogus MSVC14.0 C4702 warning

I do not see them from MSVC14.1.

boost\iostreams\detail\streambuf\indirect_streambuf.hpp(257) : warning C4702: unreachable code
boost\iostreams\detail\streambuf\indirect_streambuf.hpp(258) : warning C4702: unreachable code
boost\iostreams\detail\streambuf\indirect_streambuf.hpp(259) : warning C4702: unreachable code
boost\iostreams\detail\streambuf\indirect_streambuf.hpp(261) : warning C4702: unreachable code
boost\iostreams\detail\streambuf\indirect_streambuf.hpp(262) : warning C4702: unreachable code
boost\iostreams\detail\streambuf\indirect_streambuf.hpp(257) : warning C4702: unreachable code
boost\iostreams\detail\streambuf\indirect_streambuf.hpp(258) : warning C4702: unreachable code
boost\iostreams\detail\streambuf\indirect_streambuf.hpp(259) : warning C4702: unreachable code
boost\iostreams\detail\streambuf\indirect_streambuf.hpp(261) : warning C4702: unreachable code
boost\iostreams\detail\streambuf\indirect_streambuf.hpp(262) : warning C4702: unreachable code
boost\iostreams\detail\streambuf\indirect_streambuf.hpp(433) : warning C4702: unreachable code
boost\iostreams\detail\streambuf\indirect_streambuf.hpp(433) : warning C4702: unreachable code

When compiling bzip2/zlib from source, should they be compiled with global visibility?

Hi, as of Boost 1.69 on non-windows systems the default visibility is set to hidden. When requesting boost to build bzip2/zlib from source, it causes libboost_bzip2.so and libboost_zlib.so shared lib symbols to be built with hidden visibility, then they cannot be linked to libboost_iostreams.so

.../libboost_iostreams.so: undefined reference to `BZ2_bzCompressEnd'
.../libboost_iostreams.so: undefined reference to `BZ2_bzDecompressEnd'
.../libboost_iostreams.so: undefined reference to `BZ2_bzCompressInit'
.../libboost_iostreams.so: undefined reference to `BZ2_bzDecompress'
.../libboost_iostreams.so: undefined reference to `BZ2_bzDecompressInit'
.../libboost_iostreams.so: undefined reference to `BZ2_bzCompress'

Is there any workaround for this besides building all of boost with global visibility? or building bzip2/zlib externally? Can I patch the Jamfile or rebuild just bzip2/zlip/iostreams with global visibility?

The gzip.html example from the website doesn't compile

Example from the bottom of this page: https://www.boost.org/doc/libs/1_69_0/libs/iostreams/doc/classes/gzip.html fails to compile:

$ c++ -o gunzip gunzip.cpp -I /usr/local/include/
gunzip.cpp:12:5: error: no template named 'filtering_streambuf'; did you mean 'boost::iostreams::filtering_streambuf'?
    filtering_streambuf<input> in;
    ^~~~~~~~~~~~~~~~~~~
    boost::iostreams::filtering_streambuf
/usr/local/include/boost/iostreams/filtering_streambuf.hpp:60:41: note: 'boost::iostreams::filtering_streambuf' declared here
BOOST_IOSTREAMS_DEFINE_FILTER_STREAMBUF(filtering_streambuf, boost::iostreams::chain, char)
                                        ^

The example looks like it should produce an executable.

boost-libs-1.69.0_1 on FreeBSD 11.2

Boost 1.58.0 built by GCC-4.9 32 bit Linux missing iostreams

iostreams is available when I try to build in 64 bit. Here is command used to build in 32 bit :

--toolset=gcc --layout=versioned --build-type=complete address-model=32 architecture=x86 cflags=-m32 cxxflags=-m32 instruction-set=i686 threading=multi install

1.58.0 is downloaded here : https://www.boost.org/users/history/version_1_58_0.html

Test case fails on cygwin - lack of wide path support

On cygwin, boost::filesystem uses wide character paths.
The mapped_file implementation in iostreams only supports this on Windows proper.
In cygwin it falls back to open() and there is no conversion code from the wide string that allows it to work. Since the built-in path object supports wide and not wide, mapped_file currently cannot file any file if the path is wide because the path string is empty.

https://github.com/boostorg/iostreams/blob/develop/src/mapped_file.cpp#L264

I disabled one of the mapped file tests only for CYGWIN to work around this. I was going to hook that code into boost::locale to leverage from_utf, but that creates a dependency that isn't currently there so I decided to open an issue for it instead.

file_descriptor_impl::write() fails silently on Windows for writes of lengths that don't fit in a DWORD

iostreams/src/file_descriptor.cpp

Line 301 in 29a8317

std::streamsize file_descriptor_impl::write(const char* s, std::streamsize n)

that static cast overflows when n doesn't fit in a DWORD, but then the un-overflowed value is returned. pass it 5GB to write and it'll write 1GB and cheerfully tell the caller it wrote the whole 5GB.

Problem when cross-compiling with libc++ in mingw

Hello,
I am getting the following error from my compiler, in a relatively special build configuration:

/opt/mingw-llvm-toolchain/llvm-mingw/boost_1_67_0/boost/iostreams/positioning.hpp:52:10: error: no matching constructor for initialization of 'std::streampos' (aka 'fpos<int>')
{ return std::streampos(std::mbstate_t(), off); }
         ^              ~~~~~~~~~~~~~~~~~~~~~
/opt/mingw-llvm-toolchain/llvm-mingw/x86_64-w64-mingw32/include/c++/v1/string:538:31: note: candidate constructor not viable: allows at most single argument '__off', but 2 arguments were provided
    _LIBCPP_INLINE_VISIBILITY fpos(streamoff __off = streamoff()) : __st_(), __off_(__off) {}
                              ^
/opt/mingw-llvm-toolchain/llvm-mingw/x86_64-w64-mingw32/include/c++/v1/string:532:28: note: candidate constructor (the implicit copy constructor) not viable: requires 1 argument, but 2 were provided
class _LIBCPP_TEMPLATE_VIS fpos
                           ^
/opt/mingw-llvm-toolchain/llvm-mingw/x86_64-w64-mingw32/include/c++/v1/string:532:28: note: candidate constructor (the implicit move constructor) not viable: requires 1 argument, but 2 were provided

...

/opt/mingw-llvm-toolchain/llvm-mingw/boost_1_67_0/boost/iostreams/positioning.hpp:96:16: error: no member named 'seekpos' in 'std::__1::fpos<int>'
    return pos.seekpos();
           ~~~ ^

My toolchain is as follows :

• Linux host
• clang / libc++ MinGW cross-compiler (e.G. clang is a linux binary which creates windows executables with the mingw API) : basically this : https://github.com/mstorsjo/llvm-mingw

Any idea of what's going on ? I suppose that BOOST_IOSTREAMS_HAS_DINKUMWARE_FPOS is defined when it shouldn't ?

restrict doesn't work on istringstream

I'm wanting to use boost::iostreams::restrict to present parts of a file given as an istream as if they were first class istreams.

My first test case which works on an ifstream works fine. The second test case which is identical except it uses an istringstream instead fails - calling read() reads from the start of the content rather than from the offset position.

I realise this may be a flaw in the stringstream implementation, rather than a bug in iostreams. Can anyone shed light on this, and if so, suggest an alternative to using stringstreams?

osx CI build job fails with iostreams related test errors

This job was newly added. Looks like there are some issues to clean up. Making it an allowed failure for now.

Failing build example:

https://travis-ci.org/boostorg/iostreams/jobs/525323051

Failing test output:

testing.capture-output ../../bin.v2/libs/iostreams/test/stream_state_test.test/clang-darwin-9.1/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.run
====== BEGIN OUTPUT ======
Running 12 test cases...
../../libs/iostreams/test/stream_state_test.cpp:89: error: in "stream state test/wrap_nothrow <&test_seekp>__execute": unexpected exception thrown by function(stream)
../../libs/iostreams/test/stream_state_test.cpp:72: error: in "stream state test/wrap_nothrow <&test_seekp>__execute": stream still good
../../libs/iostreams/test/stream_state_test.cpp:74: error: in "stream state test/wrap_nothrow <&test_seekp>__execute": stream did not set badbit
../../libs/iostreams/test/stream_state_test.cpp:75: error: in "stream state test/wrap_nothrow <&test_seekp>__execute": stream did not fail
../../libs/iostreams/test/stream_state_test.cpp:77: error: in "stream state test/wrap_nothrow <&test_seekp>__execute": stream does not report failure by operator !
../../libs/iostreams/test/stream_state_test.cpp:79: error: in "stream state test/wrap_nothrow <&test_seekp>__execute": stream does not report failure by operator void* or bool
../../libs/iostreams/test/stream_state_test.cpp:72: error: in "stream state test/wrap_throw <&test_seekp>__execute": stream still good
../../libs/iostreams/test/stream_state_test.cpp:74: error: in "stream state test/wrap_throw <&test_seekp>__execute": stream did not set badbit
../../libs/iostreams/test/stream_state_test.cpp:75: error: in "stream state test/wrap_throw <&test_seekp>__execute": stream did not fail
../../libs/iostreams/test/stream_state_test.cpp:77: error: in "stream state test/wrap_throw <&test_seekp>__execute": stream does not report failure by operator !
../../libs/iostreams/test/stream_state_test.cpp:79: error: in "stream state test/wrap_throw <&test_seekp>__execute": stream does not report failure by operator void* or bool
unknown location:0: fatal error: in "stream state test/wrap_throw_delayed<&test_seekp>__execute": std::__1::ios_base::failure: bad seek: unspecified iostream_category error
*** 12 failures are detected in the test module "Master Test Suite"
EXIT STATUS: 201
====== END OUTPUT ======
    
    status=0
    if test $status -ne 0 ; then
        echo Skipping test execution due to testing.execute=off
        exit 0
    fi
     "../../bin.v2/libs/iostreams/test/stream_state_test.test/clang-darwin-9.1/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test"   > "../../bin.v2/libs/iostreams/test/stream_state_test.test/clang-darwin-9.1/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.output" 2>&1 < /dev/null
    status=$?
    echo >> "../../bin.v2/libs/iostreams/test/stream_state_test.test/clang-darwin-9.1/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.output"
    echo EXIT STATUS: $status >> "../../bin.v2/libs/iostreams/test/stream_state_test.test/clang-darwin-9.1/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.output"
    if test $status -eq 0 ; then
        cp "../../bin.v2/libs/iostreams/test/stream_state_test.test/clang-darwin-9.1/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.output" "../../bin.v2/libs/iostreams/test/stream_state_test.test/clang-darwin-9.1/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.run"
    fi
    verbose=0
    if test $status -ne 0 ; then
        verbose=1
    fi
    if test $verbose -eq 1 ; then
        echo ====== BEGIN OUTPUT ======
        cat "../../bin.v2/libs/iostreams/test/stream_state_test.test/clang-darwin-9.1/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.output"
        echo ====== END OUTPUT ======
    fi
    exit $status
...failed testing.capture-output ../../bin.v2/libs/iostreams/test/stream_state_test.test/clang-darwin-9.1/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.run...

MSVC warning when using iostreams as a dynamic lib

I have Iostreams 1.70 build using vcpkg as a dynamic lib, and when I use it in our project this warning shows up:

Warning	C4275	 non dll-interface class 'std::ios_base::failure' used as base for dll-interface class 'boost::iostreams::bzip2_error'	wesnoth	C:\Users\\Documents\vcpkg\installed\x64-windows\include\boost\iostreams\filter\bzip2.hpp	112	

stream_state_test.cpp failed in clang on Windows

I tried to use clang-cl on Windows(VS2019) to run the tests, and found stream_state_test.cpp failed with the following error message.

ksh-3.2$ ./stream_state_test.exe
Running 12 test cases...
stream_state_test.cpp(89): error: in "stream state test/wrap_nothrow <&test_seekg>__execute": unexpected exception thrown by function(stream)
stream_state_test.cpp(72): error: in "stream state test/wrap_nothrow <&test_seekg>__execute": stream still good
stream_state_test.cpp(74): error: in "stream state test/wrap_nothrow <&test_seekg>__execute": stream did not set badbit
stream_state_test.cpp(75): error: in "stream state test/wrap_nothrow <&test_seekg>__execute": stream did not fail
stream_state_test.cpp(77): error: in "stream state test/wrap_nothrow <&test_seekg>__execute": stream does not report failure by operator !
stream_state_test.cpp(79): error: in "stream state test/wrap_nothrow <&test_seekg>__execute": stream does not report failure by operator void* or bool
stream_state_test.cpp(72): error: in "stream state test/wrap_throw <&test_seekg>__execute": stream still good
stream_state_test.cpp(74): error: in "stream state test/wrap_throw <&test_seekg>__execute": stream did not set badbit
stream_state_test.cpp(75): error: in "stream state test/wrap_throw <&test_seekg>__execute": stream did not fail
stream_state_test.cpp(77): error: in "stream state test/wrap_throw <&test_seekg>__execute": stream does not report failure by operator !
stream_state_test.cpp(79): error: in "stream state test/wrap_throw <&test_seekg>__execute": stream does not report failure by operator void* or bool
unknown location(0): fatal error: in "stream state test/wrap_throw_delayed<&test_seekg>__execute": class std::ios_base::failure: bad seek: iostream stream error
stream_state_test.cpp(89): error: in "stream state test/wrap_nothrow <&test_seekp>__execute": unexpected exception thrown by function(stream)
stream_state_test.cpp(72): error: in "stream state test/wrap_nothrow <&test_seekp>__execute": stream still good
stream_state_test.cpp(74): error: in "stream state test/wrap_nothrow <&test_seekp>__execute": stream did not set badbit
stream_state_test.cpp(75): error: in "stream state test/wrap_nothrow <&test_seekp>__execute": stream did not fail
stream_state_test.cpp(77): error: in "stream state test/wrap_nothrow <&test_seekp>__execute": stream does not report failure by operator !
stream_state_test.cpp(79): error: in "stream state test/wrap_nothrow <&test_seekp>__execute": stream does not report failure by operator void* or bool
stream_state_test.cpp(72): error: in "stream state test/wrap_throw <&test_seekp>__execute": stream still good
stream_state_test.cpp(74): error: in "stream state test/wrap_throw <&test_seekp>__execute": stream did not set badbit
stream_state_test.cpp(75): error: in "stream state test/wrap_throw <&test_seekp>__execute": stream did not fail
stream_state_test.cpp(77): error: in "stream state test/wrap_throw <&test_seekp>__execute": stream does not report failure by operator !
stream_state_test.cpp(79): error: in "stream state test/wrap_throw <&test_seekp>__execute": stream does not report failure by operator void* or bool
unknown location(0): fatal error: in "stream state test/wrap_throw_delayed<&test_seekp>__execute": class std::ios_base::failure: bad seek: iostream stream error

*** 24 failures are detected in the test module "Master Test Suite"

Seems that clang on Windows does not behave properly on the wrap_nothrow, wrap_throw and throw_delayed cases.

Iostreams doesn't build on Mac

The Mac build on Travis fails with a link error caused by PR #95, but the job is marked as allowed failure due to an unrelated issue #91.

Whether liblzma contains the necessary MT functions should be checked by the Jamfile using check-target-builds. (Add a source file has_lzma_cputhreads.cpp, then add [ check-target-builds has_lzma_cputhreads : : define=BOOST_IOSTREAMS_LZMA_NO_MULTITHREADED=1 ] to lzma-requirements.

Hang forever when reset boost::iostreams::filtering_ostream with a broken sink device

boost version: 1.78
compiler: gcc version 11.2.0

typical stack trace:

#0  boost::iostreams::non_blocking_adapter<boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> > >::write (n=4096,
    s=0x556a785d1000 "R`G\312 X\355\231r\220\256\306\257\022\202\065%W\261\260\342\236\245\277\277\364\332\240\320\253\253\037d\256\256V\036\256v.Ko\315w\326\360\261?\027\304\276\346\066\367\347\364>\257\177-\037+\354S\001$K\320g?\342\353 XmE=\004k\354l`a\357\177{\303\323\235\210}R(\373$\252\004/\302\344\337D\261\350\\rF\370\203\255\335\210}\006\215C_\035\264y\343y\207\v\367\371\265\035.\253,\003\220[\\D\226s\311\371G\217\060\347\342\312\346|\326\321\210\340\263\237\034}\354V\324 \334\246\303|\237~\267\300ns\335D\026\327x\\\207\335{?~\345b\304\264\200\337\345\315\303\214W\264\027\037tq\233\373\256"..., this=0x7ffc25e00910) at /usr/local/include/boost/iostreams/detail/adapter/non_blocking_adapter.hpp:41
#1  boost::iostreams::detail::write_device_impl<boost::iostreams::output>::write<boost::iostreams::non_blocking_adapter<boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> > > > (n=<optimized out>, s=<optimized out>, t=..., t=...,
    s=<optimized out>, n=<optimized out>) at /usr/local/include/boost/iostreams/write.hpp:117
#2  boost::iostreams::write<boost::iostreams::non_blocking_adapter<boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> > > > (n=<optimized out>, s=<optimized out>, t=..., t=..., s=<optimized out>, n=<optimized out>)
    at /usr/local/include/boost/iostreams/write.hpp:49
#3  boost::iostreams::symmetric_filter<boost::iostreams::detail::zlib_compressor_impl<std::allocator<char> >, std::allocator<char> >::flush<boost::iostreams::non_blocking_adapter<boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> > > > (snk=...,
    this=<optimized out>) at /usr/local/include/boost/iostreams/filter/symmetric.hpp:231
#4  boost::iostreams::symmetric_filter<boost::iostreams::detail::zlib_compressor_impl<std::allocator<char> >, std::allocator<char> >::flush<boost::iostreams::non_blocking_adapter<boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> > > > (snk=...,
    this=<optimized out>) at /usr/local/include/boost/iostreams/filter/symmetric.hpp:222
#5  boost::iostreams::symmetric_filter<boost::iostreams::detail::zlib_compressor_impl<std::allocator<char> >, std::allocator<char> >::close<boost::iostreams::non_blocking_adapter<boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> > > > (
    mode=<optimized out>, snk=..., this=<optimized out>) at /usr/local/include/boost/iostreams/filter/symmetric.hpp:177
#6  boost::iostreams::basic_gzip_compressor<std::allocator<char> >::close<boost::iostreams::non_blocking_adapter<boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> > > > (this=0x556a78638048, snk=..., m=<optimized out>)
    at /usr/local/include/boost/iostreams/filter/gzip.hpp:258
#7  0x0000556a7744218d in boost::iostreams::detail::close_impl<boost::iostreams::detail::two_sequence>::close<boost::iostreams::basic_gzip_compressor<std::allocator<char> >, boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> > > (
    which=std::_S_out, snk=warning: RTTI symbol not found for class 'boost::iostreams::stream_buffer<boost::iostreams::detail::mode_adapter<boost::iostreams::output, std::ostream>, std::char_traits<char>, std::allocator<char>, boost::iostreams::output>'
..., t=...) at /usr/local/include/boost/iostreams/close.hpp:243
#8  boost::iostreams::close<boost::iostreams::basic_gzip_compressor<std::allocator<char> >, boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> > > (which=std::_S_out, snk=warning: RTTI symbol not found for class 'boost::iostreams::stream_buffer<boost::iostreams::detail::mode_adapter<boost::iostreams::output, std::ostream>, std::char_traits<char>, std::allocator<char>, boost::iostreams::output>'
..., t=...) at /usr/local/include/boost/iostreams/close.hpp:117
#9  boost::iostreams::detail::flt_wrapper_impl<boost::iostreams::any_tag>::close<boost::iostreams::basic_gzip_compressor<std::allocator<char> >, boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> > > (which=std::_S_out, dev=0x556a785d8080, f=...)
    at /usr/local/include/boost/iostreams/detail/adapter/concept_adapter.hpp:243
#10 boost::iostreams::detail::concept_adapter<boost::iostreams::basic_gzip_compressor<std::allocator<char> > >::close<boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> > > (dev=0x556a785d8080, which=std::_S_out, this=<optimized out>)
    at /usr/local/include/boost/iostreams/detail/adapter/concept_adapter.hpp:104
#11 boost::iostreams::detail::indirect_streambuf<boost::iostreams::basic_gzip_compressor<std::allocator<char> >, std::char_traits<char>, std::allocator<char>, boost::iostreams::output>::close_impl (which=std::_S_out, this=<optimized out>)
    at /usr/local/include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:400
#12 boost::iostreams::detail::indirect_streambuf<boost::iostreams::basic_gzip_compressor<std::allocator<char> >, std::char_traits<char>, std::allocator<char>, boost::iostreams::output>::close_impl (this=<optimized out>, which=std::_S_out)
    at /usr/local/include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:383
#13 0x0000556a774a7b5e in boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> >::close (which=<optimized out>, this=0x556a78638000) at /usr/local/include/boost/iostreams/detail/streambuf/linked_streambuf.hpp:82
#14 boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> >::close (which=std::_S_out, this=0x556a78638000) at /usr/local/include/boost/iostreams/detail/streambuf/linked_streambuf.hpp:70
#15 boost::iostreams::detail::chain_base<boost::iostreams::chain<boost::iostreams::output, char, std::char_traits<char>, std::allocator<char> >, char, std::char_traits<char>, std::allocator<char>, boost::iostreams::output>::close (m=std::_S_out, b=0x556a78638000)
    at /usr/local/include/boost/iostreams/chain.hpp:285
#16 boost::iostreams::detail::chain_base<boost::iostreams::chain<boost::iostreams::output, char, std::char_traits<char>, std::allocator<char> >, char, std::char_traits<char>, std::allocator<char>, boost::iostreams::output>::closer::operator() (b=0x556a78638000,
    this=<synthetic pointer>) at /usr/local/include/boost/iostreams/chain.hpp:300
#17 boost::iostreams::detail::execute_foreach<std::_List_iterator<boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> >*>, boost::iostreams::detail::chain_base<boost::iostreams::chain<boost::iostreams::output, char, std::char_traits<char>, std::allocator<char> >, char, std::char_traits<char>, std::allocator<char>, boost::iostreams::output>::closer>(std::_List_iterator<boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> >*>, std::_List_iterator<boost::iostreams::detail::linked_streambuf<char, std::char_traits<char> >*>, boost::iostreams::detail::chain_base<boost::iostreams::chain<boost::iostreams::output, char, std::char_traits<char>, std::allocator<char> >, char, std::char_traits<char>, std::allocator<char>, boost::iostreams::output>::closer) [clone .constprop.0] (first=0x556a78638000, last=last@entry=0x2, op=..., op@entry=...) at /usr/local/include/boost/iostreams/detail/execute.hpp:121
#18 0x0000556a774405c9 in boost::iostreams::detail::chain_base<boost::iostreams::chain<boost::iostreams::output, char, std::char_traits<char>, std::allocator<char> >, char, std::char_traits<char>, std::allocator<char>, boost::iostreams::output>::chain_impl::close (
    this=0x556a785a1140) at /usr/local/include/boost/iostreams/chain.hpp:348
#19 0x0000556a77444e02 in boost::iostreams::detail::chain_base<boost::iostreams::chain<boost::iostreams::output, char, std::char_traits<char>, std::allocator<char> >, char, std::char_traits<char>, std::allocator<char>, boost::iostreams::output>::reset (
    this=0x556a785ce430) at /usr/local/include/boost/smart_ptr/shared_ptr.hpp:784
#20 boost::iostreams::detail::chain_client<boost::iostreams::chain<boost::iostreams::output, char, std::char_traits<char>, std::allocator<char> > >::reset (this=0x556a785ce418) at /usr/local/include/boost/iostreams/chain.hpp:488

At boost/iostreams/detail/adapter/non_blocking_adapter.hpp:43 I found the iostreams::write keeps return zero (due to the broken sink device), and it makes the loop starting at line 41 running forever.

gcc-4.8 on xenial test failure

This did not happen on gcc 4.8 on trusty.
It is happening on gcc 4.8 that comes with xenial.
It seems similar to boostorg/dynamic_bitset#41
Both issues seem to revolve around basic_ios clear handling.

Here is a job where it fails:

https://travis-ci.org/boostorg/iostreams/jobs/525323038

Here is the output:

gcc.compile.c++ ../../bin.v2/libs/iostreams/test/stream_state_test.test/gcc-4.8/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.o
gcc.link ../../bin.v2/libs/iostreams/test/stream_state_test.test/gcc-4.8/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test
testing.capture-output ../../bin.v2/libs/iostreams/test/stream_state_test.test/gcc-4.8/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.run
====== BEGIN OUTPUT ======
Running 12 test cases...
unknown location(0): fatal error: in "stream state test/wrap_throw_delayed<&test_read>__execute": std::ios_base::failure[abi:cxx11]: basic_ios::clear: iostream error
../../libs/iostreams/test/stream_state_test.cpp(119): last checkpoint
unknown location(0): fatal error: in "stream state test/wrap_throw_delayed<&test_write>__execute": std::ios_base::failure[abi:cxx11]: basic_ios::clear: iostream error
../../libs/iostreams/test/stream_state_test.cpp(119): last checkpoint
unknown location(0): fatal error: in "stream state test/wrap_throw_delayed<&test_seekg>__execute": std::ios_base::failure[abi:cxx11]: basic_ios::clear: iostream error
../../libs/iostreams/test/stream_state_test.cpp(119): last checkpoint
unknown location(0): fatal error: in "stream state test/wrap_throw_delayed<&test_seekp>__execute": std::ios_base::failure[abi:cxx11]: basic_ios::clear: iostream error
../../libs/iostreams/test/stream_state_test.cpp(119): last checkpoint
*** 4 failures are detected in the test module "Master Test Suite"
EXIT STATUS: 201
====== END OUTPUT ======
    LD_LIBRARY_PATH="/usr/bin:/usr/lib:/usr/lib32:/usr/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH
    status=0
    if test $status -ne 0 ; then
        echo Skipping test execution due to testing.execute=off
        exit 0
    fi
     "../../bin.v2/libs/iostreams/test/stream_state_test.test/gcc-4.8/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test"   > "../../bin.v2/libs/iostreams/test/stream_state_test.test/gcc-4.8/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.output" 2>&1 < /dev/null
    status=$?
    echo >> "../../bin.v2/libs/iostreams/test/stream_state_test.test/gcc-4.8/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.output"
    echo EXIT STATUS: $status >> "../../bin.v2/libs/iostreams/test/stream_state_test.test/gcc-4.8/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.output"
    if test $status -eq 0 ; then
        cp "../../bin.v2/libs/iostreams/test/stream_state_test.test/gcc-4.8/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.output" "../../bin.v2/libs/iostreams/test/stream_state_test.test/gcc-4.8/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.run"
    fi
    verbose=0
    if test $status -ne 0 ; then
        verbose=1
    fi
    if test $verbose -eq 1 ; then
        echo ====== BEGIN OUTPUT ======
        cat "../../bin.v2/libs/iostreams/test/stream_state_test.test/gcc-4.8/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.output"
        echo ====== END OUTPUT ======
    fi
    exit $status
...failed testing.capture-output ../../bin.v2/libs/iostreams/test/stream_state_test.test/gcc-4.8/release/cxxstd-03-iso/threading-multi/visibility-hidden/stream_state_test.run...

Given this does not happen on gcc 4.8 under travis but does under xenial, and does not happen on any other gcc we test in CI, it looks like an environmental issue or a bug in a specific release of the libstdc++ or compiler there.

Reading file in chunks using boost::iostreams::mapped_file_source crashes on Windows

I wasted a few hours wondering and trying to fix a bug which caused my application to crash the moment I tried to read the 2nd chunk of a file. The same code runs just fine on Linux but does not on Windows.

Here is the code snippet I'm using:

boost::iostreams::mapped_file_source read_bytes(const char *file_path,
unsigned_value offset, unsigned_value length) {
	iostreams::mapped_file_params parameters;
	parameters.path = file_path;
	parameters.length = static_cast<size_t>(length);
	parameters.flags = iostreams::mapped_file::mapmode::readonly;
	parameters.offset = static_cast<boost::iostreams::stream_offset>(offset);

	boost::iostreams::mapped_file_source file;

	file.open(parameters);

	if (file.is_open()) {
		return file;
	}

	printf("Failed to open file\n");
	exit(EXIT_FAILURE);
}

This is the exception Visual Studio showed me:

Unhandled exception at 0x00007FFD7AFBA388 in PointerSearcher.exe: Microsoft C++ exception: boost::wrapexcept<std::ios_base::failure> at memory location 0x0000003034FCF270.

I'm pretty sure I didn't mess anything up since the behavior of Boost should be identical on all platforms and I'm using an offset and a length to read chunks from a file.

Thank you.

include/boost/iostreams/device/mapped_file.hpp 349 visibility problems

For time reason I cound not do more testing, but in boost 1.75.0, look like there is a conflict in visibility.

/home/i-bird/openfpm_dependencies_clang/BOOST/include/boost/iostreams/device/mapped_file.hpp:349:53: error: 'safe_bool_helper' is a private member of 'boost::iostreams::mapped_file_source'
using mapped_file::operator int mapped_file_source::safe_bool_helper::*;
^
/home/i-bird/openfpm_dependencies_clang/BOOST/include/boost/iostreams/device/mapped_file.hpp:155:8: note: implicitly declared private here
struct safe_bool_helper { int x; };

... Looking at the code as reported by the compiler ... I have the tendency to give right to the compiler. On another note I have to admit I compiled BOOST with many compilers gcc nvcc clang and this is the first time I saw this error. It happen with the combo nvcc 11.2 + clang with underlying compiler

The warkaround was to move the line under
typedef int safe_bool_helper::* safe_bool;

into the public section

boost::iostream::mapped_file_impl::map_file Cannot retry mapping with hint under Windows

Code in mapped_file.cpp

void mapped_file_impl::map_file(param_type& p)
{
    try {
        try_map_file(p);
    } catch (const std::exception&) {
        if (p.hint) {
            p.hint = 0;
            try_map_file(p);
        } else {
            throw;
        }
    }
}

As I see, function should try secondary mapping without hint if mapping with hint is failed.
Windows version has two handles ( handle_ for file and mapped_handle_ for mapping). Function cleanup_and_throw will be called If mapping failed. It will close both handles. But It is necessary to close only mapping handle for second try of mapping.
So Second try always fails under Windows if nonzero hint is passed.
*nix version has only one handle and it works.

C4706: assignment within conditional expression

boost\iostreams\filter\zlib.hpp(392): warning C4706: assignment within conditional expression

This warning is seen from MSVC 15.9.8 / 15.9.9 during linking of a release build.
Which is odd. Debug builds are fine.

Offending code is:
return !(eof_ = result == zlib::stream_end);

std::allocator<> with VS 15.5 and /std:c++latest

The VS 15.5 compiler (/std:c++latest ) complains about zlib.hpp because of the deprecated std::allocator<>

error C4996: 'std::allocator<char>::rebind<char>::other': warning STL4010: Various members of std::allocator are deprecated in C++17. Use std::allocator_traits instead of accessing these members directly. You can define _SILENCE_CXX17_OLD_ALLOCATOR_MEMBERS_DEPRECATION_WARNING or _SILENCE_ALL_CXX17_DEPRECATION_WARNINGS to acknowledge that you have received this warning.

File: zlib.hpp

template<typename Alloc>
struct zlib_allocator_traits {
#ifndef BOOST_NO_STD_ALLOCATOR
    typedef typename Alloc::template rebind<char>::other type;
...

template<typename Alloc = std::allocator<char> >
class zlib_compressor_impl : public zlib_base, public zlib_allocator<Alloc> {

etc 

error: prototype for ‘int boost::iostreams::detail::bzip2_base::end(bool, std::nothrow_t)’ does not match any in class ‘boost::iostreams::detail::bzip2_base’

Ubuntu 18.04. Compilation log:

username@username:~/iostreams/build$ b2
warning: No toolsets are configured.
warning: Configuring default toolset "gcc".
warning: If the default is wrong, your build may not work correctly.
warning: Use the "toolset=xxxxx" option to override our guess.
warning: For more configuration options, please consult
warning: http://boost.org/boost-build2/doc/html/bbv2/advanced/configuration.html
Performing configuration checks

    - bzip2                    : yes (cached)
    - lzma                     : no  (cached)
    - zstd                     : no  (cached)
    - zlib                     : yes (cached)
...found 47 targets...
...updating 2 targets...
gcc.compile.c++ bin/gcc-7.3.0/debug/bzip2.o
../src/bzip2.cpp:114:5: error: prototype for ‘int boost::iostreams::detail::bzip2_base::end(bool, std::nothrow_t)’ does not match any in class ‘boost::iostreams::detail::bzip2_base’
 int bzip2_base::end(bool compress, std::nothrow_t)
     ^~~~~~~~~~
In file included from ../src/bzip2.cpp:19:0:
/usr/include/boost/iostreams/filter/bzip2.hpp:178:10: error: candidate is: void boost::iostreams::detail::bzip2_base::end(bool)
     void end(bool compress);
          ^~~
../src/bzip2.cpp: In member function ‘void boost::iostreams::detail::bzip2_base::end(bool)’:
../src/bzip2.cpp:125:35: error: no matching function for call to ‘boost::iostreams::detail::bzip2_base::end(bool&, const std::nothrow_t&)’
         end(compress, std::nothrow)
                                   ^
../src/bzip2.cpp:122:6: note: candidate: void boost::iostreams::detail::bzip2_base::end(bool)
 void bzip2_base::end(bool compress)
      ^~~~~~~~~~
../src/bzip2.cpp:122:6: note:   candidate expects 1 argument, 2 provided

    "g++"   -fPIC -O0 -fno-inline -Wall -g  -DBOOST_IOSTREAMS_DYN_LINK=1 -DBOOST_IOSTREAMS_USE_DEPRECATED   -c -o "bin/gcc-7.3.0/debug/bzip2.o" "../src/bzip2.cpp"

...failed gcc.compile.c++ bin/gcc-7.3.0/debug/bzip2.o...
...skipped <pbin/gcc-7.3.0/debug>libboost_iostreams.so for lack of <pbin/gcc-7.3.0/debug>bzip2.o...
...failed updating 1 target...
...skipped 1 target...

Using Boost.Iostreams and Bzip2 with link-time optimization can lead to a segmentation fault

Configuration: boost 1_176_0 compiled with Mingw-w64 v7 and g++ 10.2 on Windows.

The problem is discussed here: https://stackoverflow.com/questions/68723476/why-does-link-time-optimization-cause-a-segmentation-fault

best_compression for zst filter should be increased from 19 to 22

zstd compression allows up to 22, instead of 19. Tested to be working when changed manually. The maximum compression level was changed from 19 in zstd version 0.8.1 ( facebook/zstd@be6180c ) in 2016, which practically nobody will be using at this point.

boost::iostreams::copy - sink - ENOSPC (No space left on device) error handling

Copying the issue reported on legacy Trac system
https://svn.boost.org/trac10/ticket/9478

Platform: GCC 7 on Linux with boost v1.66 (originally reported on older version; confirmed by re-testing)
In the code fragment below, is there a way to handle ENOSPC?

#include <fstream>
#include <iostream>
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/bzip2.hpp>

// open input file stream of the bzip2 file
std::ifstream ifs("file.bz2");

// open output stream to the "full" device
// full device is a "utility-device" to check how applications handle ENOSPC
// more details in "man full"
std::ofstream ofs("/dev/full");

// Setup the iostreams filter
boost::iostreams::filtering_streambuf<boost::iostreams::output> filters;
filters.push(boost::iostreams::bzip2_decompressor());
filters.push(ofs);

// "run" the filter
boost::iostreams::copy(ifs, filters);

If I do strace of the compiled binary, the code seem to infinitely call writev() with the same data and returns ENOSPC error.

writev(4, [{NULL, 0}, {"DATA DATA "..., 4096}], 2) = -1 ENOSPC (No space left on device)

How can this error be handled or made thrown as an error from boost::iostreams::copy()

Is it possible to set appropriate exceptions() on the ofstream object? I tried ofs.exceptions(std::ios::badbit | std::ios::failbit) but it didn't make any difference.

More detailed discussion on SO: ​http://stackoverflow.com/q/20414577/83005

A shorter code to reproduce the issue.

#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/device/file.hpp>

int main()
{
    using namespace boost::iostreams;
    file_sink   ofs("/dev/full");
    file_source ifs("/dev/zero");

    filtering_streambuf<output> filters(ofs);
    copy(ifs, filters);
}