boostorg / serialization Goto Github PK
View Code? Open in Web Editor NEWBoost.org serialization module
Home Page: http://boost.org/libs/serialization
serialization's People
Contributors
Stargazers
Watchers
Forkers
dk1978 sanderbog insideoutclub marcelraad danieljames faithandbrave tempoz mirams apolukhin akumta bjoe aminiussi drivehappy redboltz limbolily jeffdiclemente nasailja fiesh trel joachim-faulhaber shima07 jzmaddock samanalysis ya1gaurav crossbuild praetorian20 roflincopter pfligersdorffer rick68 alfc xantares pepsiman ohgamekillers autodesk-forks danielae xltools romain-geissler-1a el-bart olologin njlr jsonn iamrameshkumar yuri-kilochek vtnerd jeking3 masterleinad youngxiang bazelboost naudhr res2k tobias-loew fft001 seregey oxygene myestro ricardocosme devindusoft fromeworld flamefire mike-devel kojoley pauldreik alvin-leung buckaroo-pm yssource lastique dkoes chhtz boostandroid sdebionne maximli hdembinski mosjin stevykimpe samcday lasersson mgehre glenfe cosurgi taweitu muffgaga kevle jinhang2008 andygogo15 zhaohui0923 joaquintides h00ly666 loulansuiye lumaghost f3rm4rf3r vgvassilev nekto89 adammajer giomasce-throwaway samd2 chmodawk sdarwin yutakasi634 opensourceeda fieryswampshireserialization's Issues
Stop auto registring of archives
When some one tries to create an archive class derived from the existing classes like xml_iarchive_impl etc.. the corresponding concrete class eg xml_iarchive automatically registered through BOOST_SERIALIZATION_REGISTER_ARCHIVE macro which should be an unwanted behavior because if the derived class has some aditional members then it wont be found in other archive classes.
Consider allowing serialization of classes with user-defined operator&
BS uses operator& as a way to obtain addresses (probably to track identical objects), and therefore the intent is clearly to take the address and not some fancy handler resulting from and hypothetical return type of the user defined operator&.
If the user has a custom operator& that returns something different than a pointer, serialization fails to compile. If it returns a fake pointer value the program may crash.
The only change necessary, is to replace &t by boost::addressof(t).
boost::addressof is warrantied to return the real address of the object in a decltype(t)*.
Yes, this is repeat of
#41
I made a pull request (explained in the issue) but it was at a time when I didn't know how use git/github
https://github.com/boostorg/serialization/pull/41/commits and I am afraid the changes didn't go anywhere.
In part I had a hard time making the changes and testing because BS is not a header-only library :)
#71
Support json formula?
Would it support Json format in the future version?
Undefined behavior in xml_oarchive_impl constructor
====== BEGIN OUTPUT ======
boost/archive/detail/interface_oarchive.hpp:47:16: runtime error: downcast of address 0x7ffdfddb9c00 which does not point to an object of type 'boost::archive::xml_oarchive'
0x7ffdfddb9c00: note: object is of type 'boost::archive::xml_oarchive_impl<boost::archive::xml_oarchive>'
26 7f 00 00 18 e5 62 4a 26 7f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for 'boost::archive::xml_oarchive_impl<boost::archive::xml_oarchive>'
#0 0x7f264a1b33ba in boost::archive::detail::interface_oarchive<boost::archive::xml_oarchive>::This() /home/travis/build/boostorg/boost-root/./boost/archive/detail/interface_oarchive.hpp:47:16
#1 0x7f264a1b3452 in boost::archive::basic_xml_oarchive<boost::archive::xml_oarchive>::init() /home/travis/build/boostorg/boost-root/./boost/archive/impl/basic_xml_oarchive.ipp:240:11
#2 0x7f264a1b71fe in boost::archive::xml_oarchive_impl<boost::archive::xml_oarchive>::xml_oarchive_impl(std::ostream&, unsigned int) /home/travis/build/boostorg/boost-root/./boost/archive/impl/xml_oarchive_impl.ipp:114:15
#3 0x429e49 in boost::archive::xml_oarchive::xml_oarchive(std::ostream&, unsigned int) /home/travis/build/boostorg/boost-root/./boost/archive/xml_oarchive.hpp:122:9
In the constructor of xml_oarchive_impl<Archive>, the object is still of type xml_oarchive_impl, the derived class Archive does not exist yet.
It then calls init, which uses This(), which static_casts this to Archive*, which is an error.
heap_allocation for a class without a specific new operator calls the class' destructor
Since the commit 69ecae6 , the method doesnt_have_new_operator::invoke_delete(T * t) not only frees memory, but also calls the destructor of class T.
This causes compilation errors when using BS with blitz++ arrays, because the destructor of blitz::MemoryBlock is protected.
What is the rationale behind the commit? Please note that the commit did not change the behaviour of the class if DONT_USE_HAS_NEW_OPERATOR is defined.
Same concerns were expressed by @qingl97 in his comment under the discussed commit.
Null pointers lead to segfault using clang
If I understand correctly, Boost Serialization is supposed to handle null pointers internally (handling them with a specific value and not calling serialize() on the dereferenced pointer).
I found out that this doesn't work if I compile my program with clang.
Consider the following program:
#include <iostream>
#include <cstdlib>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
struct Node
{
std::string name;
Node* next;
Node(const std::string& name)
: name (name), next(NULL)
{ }
template <typename Archive>
void serialize(Archive& ar, unsigned /* version */)
{
ar & name;
ar & next;
}
};
int main()
{
Node* first = new Node("first");
Node* second = new Node("second");
Node* last = new Node("last");
first->next = second;
second->next = last;
boost::archive::text_oarchive oa { std::cout };
oa << *first;
delete first;
delete second;
delete last;
return EXIT_SUCCESS;
};If I compile this with g++ (version 6.4.0), I get the following output:
22 serialization::archive 15 1 0
0 5 first 0
1 6 second 0
2 4 last -1Which makes sense. If I compile the same code using clang (version 6.0.0), then I get the following output/error:
22 serialization::archive 15 1 0
0 5 first 0
1 6 second 0
2 4 last 0
zsh: segmentation fault (core dumped) ./serialize_bugIt seems that the NULL pointer is not handled well and that it is dereferenced (leading to the segfault). The bug is still here if I use nullptr instead of NULL.
The bug disappears for clang if I deactivate the optimizations (so I'm not exactly sure if this is a Boost bug or a CLang bug).
Serialization of boost::optional should use load_construct_data
It currently isn't possible to serialize a boost::optional<T> where T doesn't have a default constructor. This used to work, but was changed in 632df7a. However, this is an important use case. While the workaround is easy, there is usually a good reason not to allow a default constructor, for example there isn't a reasonable default value. In other cases, it may not even be possible to add a default constructor, for example when adding serialization to 3rd party classes.
I can't think for a good reason for this limitation, as the serialization framework defines the load_construct_data hook specifically for this use case. Since boost::optional can be considered a container with either 0 or 1 items, I would therefore expect it to also be serialized in a similar way to the STL container, and call load_construct_data (using use stack_construct<T> I assume).
Inconsistency in the serialization of fixed sized arrays T[N]
I think there is an inconsistency in the library design, which is that the serialization of T[N] stores the value N in the archive. I think this violates some design principles of the library because it is stored to no purpose.
The code is contained ultimately in boost::archive::save_array_type::invoke in the line ar << BOOST_SERIALIZATION_NVP(count);.
Later the count is read in boost::archive::load_array_type::invoke but it is not part of the state of the object, it is only used as a "sanity" check (if(static_cast<std::size_t>(count) > current_count) throw ...).
This original code could be there for two reasons, to allow saving and loading to differently sized static arrays T[N1] and T[N2] respectively (when N2 >= N1), or as a runtime "type" check.
But I think in both cases it violates the philosophy of Boost.Serialization by adding type information in the serialization instead of relying in static types and assuming correctness of compiled code.
I detected this problem when trying to serialize std::array<T, N> to a custom archive and instead of getting N entries I got N+1, which was something that my custom archive relied on for it to be useful.
A change like this (removing the count entry) for T[N] (and indirectly to std::array and boost::array) may require a breaking change or including some conditional code depending on the library version.
A more pragmatic change could be to do this at the level of "true value types" such as a std::array<T, N> (I think B.S shouldn't worry to much about non-value types such T[N]), and implement it as:
template <class Archive, class T, std::size_t N>
void serialize(Archive& ar, std::array<T,N>& a, const unsigned int /* version */){
ar & boost::serialization::make_nvp(
"elems",
boost::serialization::make_array(a.data()), N) // was *static_cast<T (*)[N]>(static_cast<void *>(a.data()))
);
}
Finally a workaround, if one controls the archive type, is to specialize a couple of cases:
template<class It, class T, std::size_t N>
numeric_oarchive<It>& operator<<(numeric_oarchive<It>& noa, std::array<T, N> const& a){
return noa & boost::serialization::make_array(a.data(), N);
}
template<class It, class T, std::size_t N>
numeric_iarchive<It>& operator>>(numeric_iarchive<It>& nia, std::array<T, N>& a){
return nia & boost::serialization::make_array(a.data(), N);
}
However then one must repeat this for boost::array<T, N> and T[N].
Undefined behavior due to use of *_impl pointers
Compiling with -fsanitize=undefined gives a lot of errors similar to
testing.capture-output ../../../bin.v2/libs/serialization/test/test_class_info_save_xml_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/ test_class_info_save_xml_warchive.run
====== BEGIN OUTPUT ======
../../../boost/archive/detail/interface_oarchive.hpp:47:16: runtime error: downcast of address 0x7ffd66f64220 which does not point to an object of type 'xml_woarchive'
0x7ffd66f64220: note: object is of type 'boost::archive::xml_woarchive_impl<boost::archive::xml_woarchive>'
00 00 00 00 50 12 c7 4f f2 7f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for 'boost::archive::xml_woarchive_impl<boost::archive::xml_woarchive>'
../../../boost/archive/iterators/wchar_from_mb.hpp:168:31: runtime error: member call on address 0x7ffd66f63cf0 which does not point to an object of type '__codecvt_abstract_base'
0x7ffd66f63cf0: note: object is of type 'boost::archive::detail::utf8_codecvt_facet'
f2 7f 00 00 48 25 8d 4f f2 7f 00 00 00 00 00 00 00 00 00 00 60 1b b8 4d f2 7f 00 00 00 00 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for 'boost::archive::detail::utf8_codecvt_facet'
/opt/compiler/gcc-7.2.0/include/c++/7.2.0/bits/codecvt.h:202:32: runtime error: member call on address 0x7ffd66f63cf0 which does not point to an object of type '__codecvt_abstract_base'
0x7ffd66f63cf0: note: object is of type 'boost::archive::detail::utf8_codecvt_facet'
f2 7f 00 00 48 25 8d 4f f2 7f 00 00 00 00 00 00 00 00 00 00 60 1b b8 4d f2 7f 00 00 00 00 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for 'boost::archive::detail::utf8_codecvt_facet'
a.count=2
b.count=1
No errors detected.
feature request: add boost::circular_buffer serialization
For possible implementation and benchmarking please see here.
Boost depreciation of BOOST_*_ENDIAN
I'm trying other ways to run graph tests, and the following came up.
clang-linux.compile.c++.without-pch ../../../bin.v2/libs/serialization/build/clang-linux-8.0.0/debug/threading-multi/visibility-hidden/binary_iarchive.o
In file included from ../../../libs/serialization/src/binary_iarchive.cpp:20:
In file included from ../../../boost/archive/impl/basic_binary_iarchive.ipp:25:
In file included from ../../../boost/detail/endian.hpp:9:
../../../boost/predef/detail/endian_compat.h:11:9: warning: The use of BOOST_*_ENDIAN and BOOST_BYTE_ORDER is deprecated. Please include <boost/predef/other/endian.h> and use BOOST_ENDIAN_*_BYTE instead [-W#pragma-messages]
#pragma message("The use of BOOST_*_ENDIAN and BOOST_BYTE_ORDER is deprecated. Please include <boost/predef/other/endian.h> and use BOOST_ENDIAN_*_BYTE instead")
Boost optional<T> where T has no default constructor could be serialized just fine
I've planed to use boost::serialization in our RPC and discovered that serialization of the optional where T has no default constructor is forbidden by the static assert in optional.hpp. Comment says that it is related to some sort of inherent limitation, but I can not understand what is that limitation.
If I remove the static assert then code provided below works just fine:
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/optional.hpp>
#include <boost/optional.hpp>
#include <iostream>
#include <sstream>
struct Bar
{
Bar(int m) : m{m} {}//no default c-tor
int m;
};
struct Foo
{
boost::optional<Bar> bar;
};
namespace boost::serialization
{
template<class Archive>
void serialize(Archive& ar, Foo& foo, const unsigned int)
{
ar & foo.bar;
}
template<class Archive>
void serialize(Archive&, Bar&, const unsigned int)
{/*save_construct_data and load_construct_data provided instead*/}
template<class Archive>
inline void save_construct_data(Archive & ar, const Bar * v, const unsigned int)
{
ar << v->m;
}
template<class Archive>
inline void load_construct_data(Archive & ar, Bar * p, const unsigned int)
{
int m = {};
ar >> m;
::new(p)Bar{m};
}
}//namespace boost::serialization
int main()
{
std::stringstream ss;
{
Foo foo{ Bar{123} };
std::cout << "data before serialization is: " << foo.bar->m << std::endl;
boost::archive::text_oarchive oa(ss);
oa << foo;
}
{
Foo foo;
boost::archive::text_iarchive ia(ss);
ia >> foo;
std::cout << "data after deserialization is: " << foo.bar->m << std::endl;
}
return 0;
}@robertramey Could you please explain this limitation in documentation or explain it to me so I could add pull request with documentation update? Or, if there no reason for this static assert to exist, I would be really happy if we can remove it.
Make name-value-pair-serialization concept based to avoid coupling
One of the best features of Boost Serialization is that it defines a serialization protocol that is independent of Boost.Serialization itself.
This makes possible to implement serialization of classes non-invasively and without #including any of the Boost Serialization headers.
This has an exception however, serialization over named-tag archives (XML) requires the explicit use of the type boost::serialization::nvp (or the macro), forcing the inclusion of boost/serialization/nvp.hpp and increasing the coupling with Boost.Serialization.
One workaround would be that Boost.Serialization accepts (for XML serialization at least) any name-value pair generic class, (as long as it has the members ::name() -> const char* and ::value() (and ::const_value()?) and not just the explicit boost::serialization::nvp type.
(Another workaround would be to allow XML archives to invent a unique name of the tags if that it is not specified in some way by means of an name-value wrapper).
~xml_iarchive throws exception when input stream is almost empty
Call stack:
boost::archive::basic_xml_grammar<char>::my_parse
boost::archive::basic_xml_grammar<char>::windup
boost::archive::xml_iarchive_impl<boost::archive::xml_iarchive>::~xml_iarchive_impl<boost::archive::xml_iarchive>
boost::archive::xml_iarchive::~xml_iarchive
When the destructor is called, my input stream contains "\r\n".
std::basic_stream::get() sets eofbit and failbit, basic_xml_grammar::my_parse() checks failbit before eofbit and throws the exception: https://github.com/boostorg/serialization/blob/develop/src/basic_xml_grammar.ipp#L193
This behavior was introduced in boost 1.66 (commit 64dc620).
Checking eofbit first fixed the problem for me. My application stopped crashing because of exception thrown from ~xml_iarchive_impl() which is implicitly declared as noexcept in VS2017.
However I'm not sure what the correct fix should be. I don't think windup() should throw any exception as it's called from noexcept destructor.
memory leak in singleton
After commit 7d216b4 (where singleton_wrapper is now allocated on the heap (include/boost/serialization/singleton.hpp:138)) Visual Studio reports memory leaks for those allocated objects.
Boost Serialization is not forward compatible
We have just learned, that boost serialization text archives are not forward compatible - that is, it is generally not possible to de-serialize text-archives from a newer boost version with an older one.
The documentation calls this out in the exceptions description at unsupported_version, and also in the ToDo list ("Back Versioning"), but fails to mention it in any conceptual/introduction/feature part.
I have created this issue:
- To ask whether this could be called out more explicitly in the docs.
- To highlight that the version in
basic_archive.cppseems to be constantly updated, namely we used to use v1.44 and that's version 7, while the current version in 1.66 is v16 - but even with Boost v1.62, that is not even 1,5 years old, the version has been upped twice from 14 to 16, where given the comments, I'm not even sure they created any incompatibility(??) - To ask whether there is any workaround (for text archives) to allow old versions reading newer archives, or whether we should just abandon Boost.Serialization for tasks that require different application to talk to each other. (Because we cannot keep the boost version here in sync currently.)
Thanks!
How to determine the class_id?
GCC doesn't properly deserialize from shared library base class pointers
This issue started around release 1.61 - this has held us back from updating Boost for sometime, however I never had time to report it since it seemed fairly complex.
I was able to narrow this down to a simple reproduce project, attached here: reproduce.zip
To reproduce:
$ make
$ LD_LIBRARY_PATH=. ./main
main: main.cpp:36: int main(): Assertion `basePtrA->test() == basePtrB->test()' failed.
Aborted (core dumped)
This issue appears to be related to the combination of the following:
- Executable serializes through a base class pointer, and then deserializes into another base class pointer. This also occurs with shared_ptr serialization.
- GCC with no optimizations (-O0), I suspect this is actually caused by linker optimizations around symbols. Building under Clang/MSVC I couldn't reproduce this. This same behavior has been observed through GCC 5.x to 7.x.
- Shared library that contains the abstract base class and derived class definitions, each class is marked as exported (visibility="default").
-fvisibility=hiddencompile flag set for the executable. Setting this to "default" seemed to allow it to work.- Derived class deserialized through
boost::archive::binary_iarchivepresence seems to trigger this behavior. I suspect this causes a symbol to be emitted which actually triggers the root problem.
I suspect this is in an issue with symbol visibility, where compiling/linking under -O1 or higher optimizes out symbols. I'm unsure why this resolves the issue however and at this point I can't seem to root cause the problem.
After playing around with BOOST_SYMBOL_VISIBLE in archive/serialization, adding it to the following appears to allow this to work correctly:
class BOOST_SYMBOL_VISIBLE singleton(boost/serialization/singleton.hpp)class BOOST_SYMBOL_VISIBLE iserializer : public basic_iserializer(boost/archive/detail/iserializer.hpp)
When I initially investigated this ~2 years ago I had bisected this problem down to the commit in 61b81fad. I noticed there had been several symbol changes since, so I'm not certain if this is actually the problem anymore.
Due to the nature of it touching singleton and symbols, this might be related to #104.
VS 2017 warns about use of strerror, wants strerror_s instead
On AppVeyor with the Visual Studio 2017 image, I get the following warning:
c:\projects\boost\libs\serialization\src\basic_xml_grammar.ipp(197): warning C4996: 'strerror': This function or variable may be unsafe. Consider using strerror_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
I propose to add this define to build/Jamfile.v2 to avoid the warning, which seems unwarranted here as far as I can see.
Warning "unused variable" in archive/iterators (develop branch)
Could you please remove this?
./boost/archive/iterators/wchar_from_mb.hpp:168:31: warning: unused variable โrโ [-Wunused-variable]
s/&*/boost::addressof incomplete
Hi Robert,
I see you've replaced occurrences of raw &*t with boost::addressof(t), presumably to cope with types whose operator& is overloaded. This replacement is still not done at
(at least). Thank you,
Consider making the library header only
The library always need to be linked against -lboost_serialization.
It would be great to make the library header only, or optionally header only (like Boost.ASIO).
For certain uses (e.g. not using default archives) I made it work as header only, by #including some .cpp from the source. Which shows that with some changes it would be possible to make it header only, and in particular if one wants to use the serialization infrastructure.
What are the obstacles to make the library header only?
specs for the on-disk binary archive layout
hi there,
for interoperability with C++ programs that produce/consume Boost serialization archives, I had the need to implement a reader+writer, from Go.
reverse engineering the on-disk data format got me this far:
but, I'd sleep better if there were proper specs for how the binary data is laid out on disk.
could such a thing be devised?
link failure with intel19
Hi,
On branch develop, on linux with inte2019 (icc version 19.0.0.117 (gcc version 4.8.5 compatibility)) my test build is failing with:
intel-linux.link.dll ../../../bin.v2/libs/serialization/test/intel-linux/debug/visibility-hidden/libpolymorphic_base.so.1.69.0
intel-linux.archive ../../../bin.v2/libs/serialization/build/intel-linux/debug/link-static/visibility-hidden/libboost_serialization.a
intel-linux.link.dll ../../../bin.v2/libs/serialization/test/intel-linux/debug/visibility-hidden/libmulti_shared_1.so.1.69.0
ld: ../../../bin.v2/libs/serialization/build/intel-linux/debug/link-static/visibility-hidden/libboost_serialization.a(basic_oarchive.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
../../../bin.v2/libs/serialization/build/intel-linux/debug/link-static/visibility-hidden/libboost_serialization.a: error adding symbols: Bad value
"icpc" -o "../../../bin.v2/libs/serialization/test/intel-linux/debug/visibility-hidden/libmulti_shared_1.so.1.69.0" -Wl,-soname -Wl,libmulti_shared_1.so.1.69.0 -shared "../../../bin.v2/libs/serialization/test/intel-linux/debug/visibility-hidden/multi_shared1.o" "../../../bin.v2/libs/serialization/build/intel-linux/debug/link-static/visibility-hidden/libboost_serialization.a" "../../../bin.v2/libs/filesystem/build/intel-linux/debug/visibility-hidden/libboost_filesystem.so.1.69.0" "../../../bin.v2/libs/system/build/intel-linux/debug/visibility-hidden/libboost_system.so.1.69.0" -fPIC -m64 -g -fvisibility=hidden -fvisibility-inlines-hidden
...failed intel-linux.link.dll ../../../bin.v2/libs/serialization/test/intel-linux/debug/visibility-hidden/libmulti_shared_1.so.1.69.0...
intel-linux.link.dll ../../../bin.v2/libs/serialization/test/intel-linux/debug/visibility-hidden/libmulti_shared_2.so.1.69.0
ld: ../../../bin.v2/libs/serialization/build/intel-linux/debug/link-static/visibility-hidden/libboost_serialization.a(basic_oarchive.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
../../../bin.v2/libs/serialization/build/intel-linux/debug/link-static/visibility-hidden/libboost_serialization.a: error adding symbols: Bad value
"icpc" -o "../../../bin.v2/libs/serialization/test/intel-linux/debug/visibility-hidden/libmulti_shared_2.so.1.69.0" -Wl,-soname -Wl,libmulti_shared_2.so.1.69.0 -shared "../../../bin.v2/libs/serialization/test/intel-linux/debug/visibility-hidden/multi_shared2.o" "../../../bin.v2/libs/serialization/build/intel-linux/debug/link-static/visibility-hidden/libboost_serialization.a" "../../../bin.v2/libs/filesystem/build/intel-linux/debug/visibility-hidden/libboost_filesystem.so.1.69.0" "../../../bin.v2/libs/system/build/intel-linux/debug/visibility-hidden/libboost_system.so.1.69.0" -fPIC -m64 -g -fvisibility=hidden -fvisibility-inlines-hidden
...failed intel-linux.link.dll ../../../bin.v2/libs/serialization/test/intel-linux/debug/visibility-hidden/libmulti_shared_2.so.1.69.0...
...failed updating 3 targets...
...skipped 1 target...
Before that, I get the following test failure:
./../../bin.v2/libs/serialization/test/test_dll_exported.test/intel-linux/debug/visibility-hidden/test_dll_exported.output" 2>&1 < /dev/null
====== BEGIN OUTPUT ======
No errors detected.
EXIT STATUS: 139
====== END OUTPUT ======
LD_LIBRARY_PATH="/beegfs/SCRATCH/alainm/views/boost/bin.v2/libs/filesystem/build/intel-linux/debug/visibility-hidden:/beegfs/SCRATCH/alainm/views/boost/bin.v2/libs/serialization/build/intel-linux/debug/visibility-hidden:/beegfs/SCRATCH/alainm/views/boost/bin.v2/libs/serialization/test/intel-linux/debug/visibility-hidden:/beegfs/SCRATCH/alainm/views/boost/bin.v2/libs/system/build/intel-linux/debug/visibility-hidden:/trinity/shared/apps/170135/x86_64/intel-2019.0.045/compilers_and_libraries_2019.0.117/linux/bin/lib:/trinity/shared/apps/170135/x86_64/intel-2019.0.045/compilers_and_libraries_2019.0.117/linux/lib/intel64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH
status=0
if test $status -ne 0 ; then
echo Skipping test execution due to testing.execute=off
exit 0
fi
"../../../bin.v2/libs/serialization/test/test_dll_exported.test/intel-linux/debug/visibility-hidden/test_dll_exported" > "../../../bin.v2/libs/serialization/test/test_dll_exported.test/intel-linux/debug/visibility-hidden/test_dll_exported.output" 2>&1 < /dev/null
status=$?
echo >> "../../../bin.v2/libs/serialization/test/test_dll_exported.test/intel-linux/debug/visibility-hidden/test_dll_exported.output"
echo EXIT STATUS: $status >> "../../../bin.v2/libs/serialization/test/test_dll_exported.test/intel-linux/debug/visibility-hidden/test_dll_exported.output"
if test $status -eq 0 ; then
cp "../../../bin.v2/libs/serialization/test/test_dll_exported.test/intel-linux/debug/visibility-hidden/test_dll_exported.output" "../../../bin.v2/libs/serialization/test/test_dll_exported.test/intel-linux/debug/visibility-hidden/test_dll_exported.run"
fi
verbose=0
if test $status -ne 0 ; then
verbose=1
fi
if test $verbose -eq 1 ; then
echo ====== BEGIN OUTPUT ======
cat "../../../bin.v2/libs/serialization/test/test_dll_exported.test/intel-linux/debug/visibility-hidden/test_dll_exported.output"
echo ====== END OUTPUT ======
fi
exit $status
...failed testing.capture-output ../../../bin.v2/libs/serialization/test/test_dll_exported.test/intel-linux/debug/visibility-hidden/test_dll_exported.run...
Regards
xml_iarchive destructor calls abort()
In the following Unit Test from VS2017 15.5.2 with Boost 1.66.0, boost::archive::xml_iarchive() is successfully used to load an integer. vcpkg and Boost.org distributions show the same error. When it goes out of scope, its destructor calls abort(). Boost 1.65.1 works with no error.
#include "stdafx.h"
#include "CppUnitTest.h"
using namespace Microsoft::VisualStudio::CppUnitTestFramework;
#include <boost/archive/archive_exception.hpp>
#include <boost/serialization/variant.hpp>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/archive/xml_oarchive.hpp>
#include <boost/serialization/split_member.hpp>
namespace UtilityLibTest
{
class XmlArchiveTest
{
public:
XmlArchiveTest() { m_value = 4; }
int Value() const { return m_value; }
void Value(const int v) { m_value = v; }
private:
friend class boost::serialization::access;
void load(boost::archive::xml_iarchive & ar, unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(m_value);
}
void save(boost::archive::xml_oarchive & ar, unsigned int version) const
{
ar & BOOST_SERIALIZATION_NVP(m_value);
}
BOOST_SERIALIZATION_SPLIT_MEMBER()
protected:
int m_value;
};
TEST_CLASS(UnitTest1)
{
public:
TEST_METHOD(XmlArchive_SaveLoad)
{
XmlArchiveTest store;
// save block
std::stringstream xml(std::stringstream::out);
boost::archive::xml_oarchive archive(xml);
archive & BOOST_SERIALIZATION_NVP(store);
xml.flush();
auto xml1 = xml.str();
store.Value(234);
// load block
std::stringstream xml2;
xml2 << xml1;
boost::archive::xml_iarchive archive2(xml2);
archive2 & BOOST_SERIALIZATION_NVP(store);
Assert::AreEqual(4, store.Value(), L"4 != store.Value");
}
};
}
Call stack
> ucrtbased.dll!issue_debug_notification(const wchar_t * const message) Line 28 C++ Non-user code. Symbols loaded.
ucrtbased.dll!__acrt_report_runtime_error(const wchar_t * message) Line 154 C++ Non-user code. Symbols loaded.
ucrtbased.dll!abort() Line 51 C++ Non-user code. Symbols loaded.
ucrtbased.dll!terminate() Line 59 C++ Non-user code. Symbols loaded.
vcruntime140d.dll!FindHandler(EHExceptionRecord * pExcept, EHRegistrationNode * pRN, _CONTEXT * pContext, void * pDC, const _s_FuncInfo * pFuncInfo, unsigned char recursive, int CatchDepth, EHRegistrationNode * pMarkerRN) Line 627 C++ Non-user code. Symbols loaded.
vcruntime140d.dll!__InternalCxxFrameHandler(EHExceptionRecord * pExcept, EHRegistrationNode * pRN, _CONTEXT * pContext, void * pDC, const _s_FuncInfo * pFuncInfo, int CatchDepth, EHRegistrationNode * pMarkerRN, unsigned char recursive) Line 347 C++ Non-user code. Symbols loaded.
vcruntime140d.dll!__CxxFrameHandler(EHExceptionRecord * pExcept, EHRegistrationNode * pRN, void * pContext, void * pDC) Line 219 C++ Non-user code. Symbols loaded.
ntdll.dll!ExecuteHandler2@20๏ฟฝ() Unknown Non-user code. Symbols loaded.
ntdll.dll!ExecuteHandler@20๏ฟฝ() Unknown Non-user code. Symbols loaded.
ntdll.dll!_KiUserExceptionDispatcher@8๏ฟฝ() Unknown Non-user code. Symbols loaded.
KernelBase.dll!_RaiseException@16๏ฟฝ() Unknown Non-user code. Symbols loaded.
vcruntime140d.dll!_CxxThrowException(void * pExceptionObject, const _s__ThrowInfo * pThrowInfo) Line 136 C++ Non-user code. Symbols loaded.
UnitTestsUtilityLib.dll!boost::serialization::throw_exception<boost::archive::archive_exception>(const boost::archive::archive_exception & e) Line 37 C++ Symbols loaded.
UnitTestsUtilityLib.dll!boost::archive::basic_xml_grammar<char>::my_parse(std::basic_istream<char,std::char_traits<char> > & is, const boost::spirit::classic::rule<boost::spirit::classic::scanner<std::_String_iterator<std::_String_val<std::_Simple_types<char> > >,boost::spirit::classic::scanner_policies<boost::spirit::classic::iteration_policy,boost::spirit::classic::match_policy,boost::spirit::classic::action_policy> >,boost::spirit::classic::nil_t,boost::spirit::classic::nil_t> & rule_, const char delimiter) Line 194 C++ Symbols loaded.
UnitTestsUtilityLib.dll!boost::archive::basic_xml_grammar<char>::windup(std::basic_istream<char,std::char_traits<char> > & is) Line 465 C++ Symbols loaded.
UnitTestsUtilityLib.dll!boost::archive::xml_iarchive_impl<boost::archive::xml_iarchive>::~xml_iarchive_impl<boost::archive::xml_iarchive>() Line 197 C++ Symbols loaded.
UnitTestsUtilityLib.dll!boost::archive::xml_iarchive::~xml_iarchive() Line 129 C++ Symbols loaded.
UnitTestsUtilityLib.dll!UtilityLibTest::UnitTest1::XmlArchive_SaveLoad() Line 72 C++ Symbols loaded.
(This is a copy of #13354 in the Boost Trac, I wasn't sure if it would be better reported here.)
develop build broken
Hi Robert,
Something just broke the develop build on all msvc versions prior to 14.1, my appveyor CI tests are all failing in:
[00:10:40] compile-c-c++ ..\..\..\bin.v2\libs\serialization\build\9218ae764f8240b7321ebd1ea2d19264\xml_grammar.obj
[00:10:40] xml_grammar.cpp
[00:10:40] ..\..\..\boost/archive/iterators/xml_unescape.hpp(114) : fatal error C1063: compiler limit : compiler stack overflow
[00:10:40] Internal Compiler Error in C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\BIN\cl.exe. You will be prompted to send an error report to Microsoft later.
[00:10:41]
[00:10:41] call "..\..\..\bin.v2\standalone\msvc\0b00851a9d61d40fad804f9053579ad8\msvc-setup.bat" x86 >nul
[00:10:41] cl /Zm800 -nologo @"..\..\..\bin.v2\libs\serialization\build\9218ae764f8240b7321ebd1ea2d19264\xml_grammar.obj.rsp"
UBSAN: downcast of address in interface_oarchive
When building Boost.Serialization with UBSAN enabled I am seeing this error. I am using the docker build environment in boostorg/boost#184 to make the environment, and then you can see the command below and the issue it finds:
boost@9423c57fd56c:/boost/libs/serialization/test$ UBSAN_OPTIONS=print_stacktrace=1 ../../../b2 toolset=gcc-7 cxxstd=03 cxxflags=-fno-omit-frame-pointer cxxflags=-fsanitize=undefined cxxflags=-fno-sanitize-recover=undefined linkflags=-fsanitize=undefined linkflags=-fno-sanitize-recover=undefined linkflags=-fuse-ld=gold variant=debug -j3 test_bitset_xml_archive
Performing configuration checks
- default address-model : 64-bit (cached)
- default architecture : x86 (cached)
- symlinks supported : yes (cached)
...patience...
...found 2246 targets...
...updating 2 targets...
testing.capture-output ../../../bin.v2/libs/serialization/test/test_bitset_xml_archive.test/gcc-7/debug/cxxstd-03-iso/test_bitset_xml_archive.run
====== BEGIN OUTPUT ======
../../../boost/archive/detail/interface_oarchive.hpp:47:16: runtime error: downcast of address 0x7ffff8978910 which does not point to an object of type 'xml_oarchive'
0x7ffff8978910: note: object is of type 'boost::archive::xml_oarchive_impl<boost::archive::xml_oarchive>'
c2 fb bf ab 00 38 5e 27 e9 7f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for 'boost::archive::xml_oarchive_impl<boost::archive::xml_oarchive>'
#0 0x7fe9275a29d0 in boost::archive::detail::interface_oarchive<boost::archive::xml_oarchive>::This() (/boost/bin.v2/libs/serialization/build/gcc-7/debug/cxxstd-03-iso/libboost_serialization.so.1.68.0+0x1859d0)
#1 0x7fe92759ead0 in boost::archive::basic_xml_oarchive<boost::archive::xml_oarchive>::init() (/boost/bin.v2/libs/serialization/build/gcc-7/debug/cxxstd-03-iso/libboost_serialization.so.1.68.0+0x181ad0)
#2 0x7fe9275a1e76 in boost::archive::xml_oarchive_impl<boost::archive::xml_oarchive>::xml_oarchive_impl(std::ostream&, unsigned int) (/boost/bin.v2/libs/serialization/build/gcc-7/debug/cxxstd-03-iso/libboost_serialization.so.1.68.0+0x184e76)
#3 0x55f563efb2f3 in boost::archive::xml_oarchive::xml_oarchive(std::ostream&, unsigned int) ../../../boost/archive/xml_oarchive.hpp:122
#4 0x55f563ef8944 in test_main(int, char**) /boost/libs/serialization/test/test_bitset.cpp:47
#5 0x55f563ef85be in main /boost/libs/serialization/test/test_tools.hpp:200
#6 0x7fe925d86b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
#7 0x55f563ef82c9 in _start (/boost/bin.v2/libs/serialization/test/test_bitset_xml_archive.test/gcc-7/debug/cxxstd-03-iso/test_bitset_xml_archive+0xe2c9)
EXIT STATUS: 1
====== END OUTPUT ======
LD_LIBRARY_PATH="/boost/bin.v2/libs/filesystem/build/gcc-7/debug/cxxstd-03-iso:/boost/bin.v2/libs/serialization/build/gcc-7/debug/cxxstd-03-iso:/boost/bin.v2/libs/system/build/gcc-7/debug/cxxstd-03-iso:/usr/bin:/usr/lib:/usr/lib32:/usr/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH
status=0
if test $status -ne 0 ; then
echo Skipping test execution due to testing.execute=off
exit 0
fi
"../../../bin.v2/libs/serialization/test/test_bitset_xml_archive.test/gcc-7/debug/cxxstd-03-iso/test_bitset_xml_archive" > "../../../bin.v2/libs/serialization/test/test_bitset_xml_archive.test/gcc-7/debug/cxxstd-03-iso/test_bitset_xml_archive.output" 2>&1 < /dev/null
status=$?
echo >> "../../../bin.v2/libs/serialization/test/test_bitset_xml_archive.test/gcc-7/debug/cxxstd-03-iso/test_bitset_xml_archive.output"
echo EXIT STATUS: $status >> "../../../bin.v2/libs/serialization/test/test_bitset_xml_archive.test/gcc-7/debug/cxxstd-03-iso/test_bitset_xml_archive.output"
if test $status -eq 0 ; then
cp "../../../bin.v2/libs/serialization/test/test_bitset_xml_archive.test/gcc-7/debug/cxxstd-03-iso/test_bitset_xml_archive.output" "../../../bin.v2/libs/serialization/test/test_bitset_xml_archive.test/gcc-7/debug/cxxstd-03-iso/test_bitset_xml_archive.run"
fi
verbose=0
if test $status -ne 0 ; then
verbose=1
fi
if test $verbose -eq 1 ; then
echo ====== BEGIN OUTPUT ======
cat "../../../bin.v2/libs/serialization/test/test_bitset_xml_archive.test/gcc-7/debug/cxxstd-03-iso/test_bitset_xml_archive.output"
echo ====== END OUTPUT ======
fi
exit $status
...failed testing.capture-output ../../../bin.v2/libs/serialization/test/test_bitset_xml_archive.test/gcc-7/debug/cxxstd-03-iso/test_bitset_xml_archive.run...
...failed updating 1 target...
...skipped 1 target...
Build issues (polymorphic_base) on MSVC 2017, 2015
Seeing this on develop:
commit efcd46eb1be621e212b3d31fae12cdd7230cdfe3 (HEAD -> develop, origin/develop, origin/HEAD)
Author: Robert Ramey <[email protected]>
Date: Sat Nov 3 06:23:22 2018 -0700
Just discovered that registering type via EXPORT doesn't work with polymorphic archives !!!
Corrected this.
c:\boost\libs\serialization\test>..\..\..\b2.exe toolset=msvc-14.1 address-model=64 cxxflags=-permissive- cxxstd=latest
Performing configuration checks
- default address-model : 32-bit (cached)
- default architecture : x86 (cached)
- symlinks supported : no (cached)
- junctions supported : yes (cached)
- hardlinks supported : yes (cached)
- Boost.Config Feature Check: cxx11_hdr_array : yes (cached)
- Boost.Config Feature Check: std_wstreambuf : yes (cached)
- Boost.Config Feature Check: cxx11_hdr_forward_list : yes (cached)
- Boost.Config Feature Check: hash : no (cached)
- Boost.Config Feature Check: cxx11_hdr_unordered_map : yes (cached)
- Boost.Config Feature Check: cxx11_hdr_unordered_set : yes (cached)
- Boost.Config Feature Check: slist : no (cached)
- Boost.Config Feature Check: auto_ptr : no (cached)
...patience...
...patience...
...found 7148 targets...
...updating 7 targets...
msvc.link.dll ..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.dll
Creating library ..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.lib and object ..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.exp
polymorphic_derived2.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: virtual __cdecl polymorphic_base::~polymorphic_base(void)" (__imp_??1polymorphic_base@@UEAA@XZ) referenced in function "public: virtual __cdecl polymorphic_derived2::~polymorphic_derived2(void)" (??1polymorphic_derived2@@UEAA@XZ)
polymorphic_derived2.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: __cdecl polymorphic_base::polymorphic_base(void)" (__imp_??0polymorphic_base@@QEAA@XZ) referenced in function "public: __cdecl polymorphic_derived2::polymorphic_derived2(void)" (??0polymorphic_derived2@@QEAA@XZ)
polymorphic_derived2.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: __cdecl polymorphic_base::polymorphic_base(class polymorphic_base const &)" (__imp_??0polymorphic_base@@QEAA@AEBV0@@Z) referenced in function "public: __cdecl polymorphic_derived2::polymorphic_derived2(class polymorphic_derived2 const &)" (??0polymorphic_derived2@@QEAA@AEBV0@@Z)
polymorphic_derived2.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: class polymorphic_base & __cdecl polymorphic_base::operator=(class polymorphic_base const &)" (__imp_??4polymorphic_base@@QEAAAEAV0@AEBV0@@Z) referenced in function "public: class polymorphic_derived2 & __cdecl polymorphic_derived2::operator=(class polymorphic_derived2 const &)" (??4polymorphic_derived2@@QEAAAEAV0@AEBV0@@Z)
..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.dll : fatal error LNK1120: 4 unresolved externals
call "..\..\..\bin.v2\standalone\msvc\msvc-14.1\address-model-64\architecture-x86\msvc-setup.bat" >nul
link /NOLOGO /INCREMENTAL:NO /DLL /DEBUG /MACHINE:X64 /MANIFEST /subsystem:console /out:"..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.dll" /IMPLIB:"..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.lib" @"..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.dll.rsp"
if %ERRORLEVEL% NEQ 0 EXIT %ERRORLEVEL%
...failed msvc.link.dll ..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.dll ..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.lib ..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.pdb ..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.pdb...
...removing ..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.lib
...removing ..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi\dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.pdb
...skipped <p..\..\..\bin.v2\libs\serialization\test\test_dll_exported.test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi>test_dll_exported.exe for lack of <p..\..\..\bin.v2\libs\serialization\test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi>dll_polymorphic_derived2-vc141-mt-gd-x64-1_69.lib...
...skipped <p..\..\..\bin.v2\libs\serialization\test\test_dll_exported.test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi>test_dll_exported.run for lack of <p..\..\..\bin.v2\libs\serialization\test\test_dll_exported.test\msvc-14.1\debug\address-model-64\cxxstd-latest-iso\threading-multi>test_dll_exported.exe...
...failed updating 3 targets...
...skipped 3 targets...
Will try HEAD~1 to see if that isolates it.
Don't infect user code with aggressive inlining pragmas
Some headers (listed below) in this library has the following very aggressive inlining pragmas specified:
#if defined(_MSC_VER)
# pragma once
#pragma inline_depth(511)
#pragma inline_recursion(on)
#endif
First of all, 511 is not a valid value for inline_depth according to the documentation of recent MSVC versions.
Secondly, and more importantly, these pragmas will leak into anything that directly or indirectly include these headers and affect the compilation of other code. This applies to any code, even code that does not interface with boost::serialization in any way. This is a very unexpected side-effect from using this library.
For example, I've seen compile times of a single function go from milliseconds to over 30 seconds just because I include one of the headers.
The affected files I managed to find by a simple search:
- include/boost/archive/detail/check.hpp
- include/boost/archive/detail/iserializer.hpp
- include/boost/archive/detail/oserializer.hpp
Std container binary serialization doesn't use fixed width integer
For example, i serialize a std::string( std::vector and other containers have the same problem ) on x64 environment, and deserialize it on x86 environment always get a error result or exception.
Both use binary_i/oarchive.
This is caused by "size_t", which is the type of the container size.
Why not use a fixed width integer, like std::uint32_t?
compile problem on Android ndk r16 beta 1
Hello, in attempt to build boost for android with more or less reasonable strict settings (-Werror) I uncovered some bugs.
clang-darwin.compile.c++ android-build/boost/bin.v2/
libs/serialization/build/clang-darwin-5.0~x86/debug/
address-model-32/link-static/target-os-android/threading-
multi/polymorphic_iarchive.o
In file included from libs/serialization/src/polymorphic_iarchive.cpp:20:
In file included from ./boost/archive/polymorphic_iarchive.hpp:32:
./boost/archive/detail/iserializer.hpp:69:7: error: macro expansion
producing 'defined' has undefined behavior [-Werror,-Wexpansion-to-defined]
#if ! DONT_USE_HAS_NEW_OPERATOR
^
./boost/archive/detail/iserializer.hpp:63:12: note: expanded from macro
'DONT_USE_HAS_NEW_OPERATOR'
|| defined(__SUNPRO_CC) && (__SUNPRO_CC < 0x590) \
^
./boost/archive/detail/iserializer.hpp:212:9: error: macro expansion
producing 'defined' has undefined behavior [-Werror,-Wexpansion-to-defined]
#if DONT_USE_HAS_NEW_OPERATOR
^
./boost/archive/detail/iserializer.hpp:63:12: note: expanded from macro
'DONT_USE_HAS_NEW_OPERATOR'
|| defined(__SUNPRO_CC) && (__SUNPRO_CC < 0x590) \
^
2 errors generated.
"/usr/local/opt/android-ndk/android-ndk-r16-beta1//
toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++"
"-DBOOST_AC_USE_PTHREADS" "-DBOOST_SP_USE_PTHREADS" "-fvisibility=hidden"
"-fvisibility-inlines-hidden" "-Wno-unused-local-typedef" -x c++
-ftemplate-depth-255 -fvisibility=hidden -fvisibility-inlines-hidden
-std=c++14 -O0 -g -O0 -fno-inline -Wall -g --target=i686-none-linux-android
--gcc-toolchain=/usr/local/opt/android-ndk/android-ndk-
r16-beta1//toolchains/x86-4.9/prebuilt/darwin-x86_64
--sysroot=/usr/local/opt/android-ndk/android-ndk-r16-beta1//sysroot
-isystem /usr/local/opt/android-ndk/android-ndk-r16-beta1//
sources/cxx-stl/llvm-libc++/include -isystem /usr/local/opt/android-ndk/
android-ndk-r16-beta1//sources/cxx-stl/llvm-libc++abi/include -isystem
/usr/local/opt/android-ndk/android-ndk-r16-beta1//sources/android/support/include
-isystem /usr/local/opt/android-ndk/android-ndk-r16-beta1//sysroot/usr/include
-isystem /usr/local/opt/android-ndk/android-ndk-r16-beta1//
sysroot/usr/include/i686-linux-android -DANDROID -D__ANDROID_API__=21
-ffunction-sections -funwind-tables -fstack-protector-strong
-fno-limit-debug-info -fPIC -no-canonical-prefixes -mstackrealign
-Wa,--noexecstack -Wformat -Werror=format-security -Wall -Werror -Wshadow
-march=i686 -DBOOST_ALL_NO_LIB=1 -D_LITTLE_ENDIAN -I"." -c -o
"android-build/boost/bin.v2/libs/serialization/build/
clang-darwin-5.0~x86/debug/address-model-32/link-static/
target-os-android/threading-multi/polymorphic_iarchive.o"
"libs/serialization/src/polymorphic_iarchive.cpp"
This is using clang toolchain on Android NDK r16 beta1.
Feature request: detect incorrect usage of filter iterators
Consider the following example:
#include <string>
#include <boost/archive/iterators/binary_from_base64.hpp>
#include <boost/archive/iterators/transform_width.hpp>
#include <boost/iterator/filter_iterator.hpp>
struct cleanup_base64_input {
bool operator()(unsigned char t) {
return !(t == ' ' || t == '\n' || t == '\t' || t == '=');
}
};
typedef boost::archive::iterators::transform_width<
boost::archive::iterators::binary_from_base64<
boost::filter_iterator<
cleanup_base64_input, std::string::const_iterator
>
>,
8, 6
> decoder;
std::string decode(const std::string& data) {
return std::string(decoder(data.begin()), decoder(data.end()));
}
int main() {
try {
const int res = static_cast<int>(decode("gq5E5CI4e5uAjheO1AX4W").size());
return res; // Must not get here!
} catch(...) {
return 0;
}
}
Building that example with -fsanitize=address and running it gives the following output:
==24139==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60300000eff6 at pc 0x0000004028ff bp 0x7ffcb165e450 sp 0x7ffcb165e440
READ of size 1 at 0x60300000eff6 thread T0
#0 0x4028fe in boost::iterators::filter_iterator<cleanup_base64_input, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::satisfy_predicate() ../../../boost/iterator/filter_iterator.hpp:100
#1 0x403f43 in boost::iterators::filter_iterator<cleanup_base64_input, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::increment() ../../../boost/iterator/filter_iterator.hpp:90
#2 0x403e7d in void boost::iterators::iterator_core_access::increment<boost::iterators::filter_iterator<cleanup_base64_input, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >(boost::iterators::filter_iterator<cleanup_base64_input, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) ../../../boost/iterator/iterator_facade.hpp:556
That's because the filter iterator is meant to be initialized with two iterators - to the begin and to the end of the range. Which does not happen here, and the iterator is only initilized with begin iterator.
We'd appreciate a way to initialize the filtering iterator with additional parameters, or it's automatical initialization with additional iterator, or some kind of static_assert on incorrect usage (if that's possible).
Unable to track pointer instances correctly
The following simple example shows a bug in de-serializing a vector of a simple struct, involving a mixture of pointers and strings.
I attach the cpp-reproducer as a txt file because the code insertion seems to be broken here...
enum serialization
During the serialization enumerations are considering as a special case and serializing as int now. There are some drawbacks following this way
- a shorter enums occupy 4 bytes anyway for binary serialization;
- input/output operators are ignored during XML serialization.
It will be helpful to have an option to serialize enumerations as is. May be a new macro introduction (smth like BOOST_SERIALIZATION_KEEP_ENUM) could be a solution
- compilation w/o macro in old way to support backward compatibility
- recompilation with defined macro in a bright new way.
Thanks
add generic container serialization
I noticed that serialization code is missing for boost::container::vector and boost::container::string and probably more of the Boost containers. Boost should be able to serialize ideally all or at least most of its own containers.
The straight forward thing would be to implement serialization code for each individual container, but that's a maintenance burden and it does not exploit the fact that many containers adhere to a standard interface.
With a bit of concept checking a templated generic serialization code for containers could be written that adhere to the vector interface; similar for string, list, and map. The current code for std::vector, std::string, std::map, std::list could be turned into such a generic code. The concept checking would implement a form of duck-typing.
Serializing by reference and deserializing by pointer causes segfault on deserialization
xml_iarchive in broken state after archive_exception is thrown
The following example works with
#include <iostream>
#include <sstream>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/archive/xml_oarchive.hpp>
int main()
{
std::ostringstream os;
{
boost::archive::xml_oarchive archive(os);
int a = 42;
archive & boost::serialization::make_nvp("foo", a);
std::cout << " Wrote a=" << a << " to archive\n";
}
std::cout << "archive xml: " << os.str() << "\n";
try {
std::istringstream is(os.str());
boost::archive::xml_iarchive archive(is);
int a = 0;
archive & boost::serialization::make_nvp("foo", a);
try {
archive & boost::serialization::make_nvp("not_there", a);
} catch (const boost::archive::archive_exception &) {
std::cout << "exception 1\n";
}
std::cout << " Read a=" << a << " from archive\n" << std::endl;
} catch (...) {
std::cout << "exception 2\n";
}
}
In a centos:7 docker container with the distro boost-serialization-1.53.0 this outputs what I would expect:
[root@ea27e53f5ba4 ~]# g++ -std=c++14 -Wall -g -lboost_serialization -fPIC -o boost-serialization-example boost-serialization-example.cc && ./boost-serialization-example
g++: error: unrecognized command line option '-std=c++14'
[root@ea27e53f5ba4 ~]# g++ -std=c++11 -Wall -g -lboost_serialization -fPIC -o boost-serialization-example boost-serialization-example.cc && ./boost-serialization-example
Wrote a=42 to archive
archive xml: <?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE boost_serialization>
<boost_serialization signature="serialization::archive" version="10">
<foo>42</foo>
</boost_serialization>
exception 1
Read a=42 from archive
On fedora 28 with the distro boost-serialization-1.66.0 the output is this:
Wrote a=42 to archive
archive xml: <?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE boost_serialization>
<boost_serialization signature="serialization::archive" version="16">
<foo>42</foo>
</boost_serialization>
exception 1
Read a=42 from archive
terminate called after throwing an instance of 'boost::archive::archive_exception'
what(): input stream error-Success
Aborted (core dumped)
Note that the second exception is not caught by the outer catch (...) indicating that the process is in a corrupted state. In other words it is not just that the xml_iarchive::~xml_iarchive() also throws an exception, because that exception should be caught by the outer block.
Also note that the xml does have the closing </boost_serialization> tag so this is not a duplicate of #82.
I guess the special thing here is is that the exception is caught, but the xml_archive instance stays in scope. If the xml_archive is destroyed while the exception is being thrown the destructor does nothing and the problem goes away:
template<class Archive>
BOOST_ARCHIVE_DECL
xml_iarchive_impl<Archive>::~xml_iarchive_impl(){
if(std::uncaught_exception())
return;
if(0 == (this->get_flags() & no_header)){
gimpl->windup(is);
}
}
Stack Overflow
Recently I used this library for a project, and I works on mac and ubuntu very well. However, when I transplant it to Windows, it failed when storing data into disk. So I use Visual Stdio 2017 to track the bug. And Visual Stdio told me that the failure was due to stack overflow.
So I use mingw/g++ and change the stack size to 16mb, and it worked.
Here is the screen shot.
basic_text_oprimitive.hpp, function:save_impl.
Seeing random serialization failures in unit tests
I can re-run a build job that fails and it disappears. It's happening around 5% of the time on the jobs I've seen. This is new behavior as of the last 24 hours or so.
Example:
https://travis-ci.org/boostorg/ptr_container/jobs/448820164#L1935
I'm going to refresh my CI enhancement pull request for this repo to get more coverage.
Minor CMake path issue
Does the path at https://github.com/boostorg/serialization/blob/develop/CMake/CMakeLists.txt#L103 is missing an extra ..?
I mean, if I use CMake project to build Boost.Serialization inside Boost superproject, I get BOOST_ROOT not found, but extra .. fixes the problem.
Singleton destruction check completely broken leading to memory leaks or crashs
The commit b0a794d introduced a change that completely breaks the singleton implementation.
The problem is: singleton<T> is NOT a singleton! It is NEVER instantiated. Instead it instantiates a class singleton_wrapper derived from T. That instance is then returned in the get_instance method.
Because it is never instantiated it is also never destroyed. This leads to the destroyed flag never set. It only happens to be false because it is default (or value?) initialized.
This was made even worse with commit 7d216b4 where the singleton instance is not a static variable with automatic, thread-safe construction/destruction but a pointer which is leaked.
The whole implementation is against the singleton pattern: Currently is is completely possible to create a singleton<Foo> instance. Even worse: Currently you would have to do this EXACTLY once to have correct code for EVERY singleton. If you don't do this you get a memory leak or corrupted memory depending on the version. If you do this more than once you yet a double-free error which cannot be caught.
My suggestion: Make the singleton a real singleton (private ctor, no assignment etc.), inherit from T directly and let the instance variable be singleton instead of singleton_wrapper (get rid of that, but check with http://tinyurl.com/ljdp8 that this does not cause a regression, if it does, put the members ( destroyed...) into singleton_wrapper like before), check the setting of the destroyed flag which must rely on the actual destruction of the "real"(!!!) singleton instance.
Memory leaks in tests
Compiling with -fsanitize=address reports memory leaks similar to
Direct leak of 160 byte(s) in 1 object(s) allocated from:
#0 0x7f29c482d9c0 in operator new(unsigned long) ../../.././libsanitizer/asan/asan_new_delete.cc:80
#1 0x466293 in boost::archive::detail::heap_allocation<J>::doesnt_have_new_operator::invoke_new() (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/ test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x466293)
#2 0x465202 in boost::archive::detail::heap_allocation<J>::invoke_new() (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/ test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x465202)
#3 0x463876 in boost::archive::detail::heap_allocation<J>::heap_allocation() (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/ test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x463876)
#4 0x46247b in boost::archive::detail::pointer_iserializer<boost::archive::text_wiarchive, J>::heap_allocation() const (/mnt/data/darndt/boost-develop/bin.v2/libs/ serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x46247b)
#5 0x7f29c4079b92 in boost::archive::detail::basic_iarchive_impl::load_pointer(boost::archive::detail::basic_iarchive&, void*&, boost::archive::detail:: basic_pointer_iserializer const*, boost::archive::detail::basic_pointer_iserializer const* (*)(boost::serialization::extended_type_info const&)) (/mnt/data/darndt/ boost-develop/bin.v2/libs/serialization/build/gcc-gnu-7.2.0/debug/threadapi-pthread/libboost_serialization.so.1.66.0+0xa0b92)
#6 0x7f29c4078655 in boost::archive::detail::basic_iarchive::load_pointer(void*&, boost::archive::detail::basic_pointer_iserializer const*, boost::archive:: detail::basic_pointer_iserializer const* (*)(boost::serialization::extended_type_info const&)) ../../../libs/serialization/src/basic_iarchive.cpp:573
#7 0x4242c6 in void boost::archive::detail::load_pointer_type<boost::archive::text_wiarchive>::invoke<J*>(boost::archive::text_wiarchive&, J*&) (/mnt/data/darndt/ boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x4242c6)
#8 0x423a69 in void boost::archive::load<boost::archive::text_wiarchive, J*>(boost::archive::text_wiarchive&, J*&) (/mnt/data/darndt/boost-develop/bin.v2/libs/ serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x423a69)
#9 0x423694 in void boost::archive::detail::common_iarchive<boost::archive::text_wiarchive>::load_override<J*>(J*&) (/mnt/data/darndt/boost-develop/bin.v2/libs/ serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x423694)
#10 0x4232c7 in void boost::archive::basic_text_iarchive<boost::archive::text_wiarchive>::load_override<J*>(J*&) (/mnt/data/darndt/boost-develop/bin.v2/libs/ serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x4232c7)
#11 0x422e89 in void boost::archive::text_wiarchive_impl<boost::archive::text_wiarchive>::load_override<J*>(J*&) (/mnt/data/darndt/boost-develop/bin.v2/libs/ serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x422e89)
#12 0x4227f4 in boost::archive::text_wiarchive& boost::archive::detail::interface_iarchive<boost::archive::text_wiarchive>::operator>><J*>(J*&) (/mnt/data/darndt/ boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x4227f4)
#13 0x421f1d in void boost::serialization::nvp<J*>::load<boost::archive::text_wiarchive>(boost::archive::text_wiarchive&, unsigned int) (/mnt/data/darndt/boost- develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x421f1d)
#14 0x421918 in void boost::serialization::access::member_load<boost::archive::text_wiarchive, boost::serialization::nvp<J*> >(boost::archive::text_wiarchive&, boost::serialization::nvp<J*>&, unsigned int) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/ threadapi-pthread/test_cyclic_ptrs_text_warchive+0x421918)
#15 0x421717 in boost::serialization::detail::member_loader<boost::archive::text_wiarchive, boost::serialization::nvp<J*> >::invoke(boost::archive:: text_wiarchive&, boost::serialization::nvp<J*>&, unsigned int) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc- gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x421717)
#16 0x4215c5 in void boost::serialization::split_member<boost::archive::text_wiarchive, boost::serialization::nvp<J*> >(boost::archive::text_wiarchive&, boost:: serialization::nvp<J*>&, unsigned int) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/ threadapi-pthread/test_cyclic_ptrs_text_warchive+0x4215c5)
#17 0x421473 in void boost::serialization::nvp<J*>::serialize<boost::archive::text_wiarchive>(boost::archive::text_wiarchive&, unsigned int) (/mnt/data/darndt/ boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x421473)
#18 0x42128c in void boost::serialization::access::serialize<boost::archive::text_wiarchive, boost::serialization::nvp<J*> >(boost::archive::text_wiarchive&, boost::serialization::nvp<J*>&, unsigned int) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/ threadapi-pthread/test_cyclic_ptrs_text_warchive+0x42128c)
#19 0x4210cb in void boost::serialization::serialize<boost::archive::text_wiarchive, boost::serialization::nvp<J*> >(boost::archive::text_wiarchive&, boost:: serialization::nvp<J*>&, unsigned int) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/ threadapi-pthread/test_cyclic_ptrs_text_warchive+0x4210cb)
#20 0x420dd3 in void boost::serialization::serialize_adl<boost::archive::text_wiarchive, boost::serialization::nvp<J*> >(boost::archive::text_wiarchive&, boost:: serialization::nvp<J*>&, unsigned int) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/ threadapi-pthread/test_cyclic_ptrs_text_warchive+0x420dd3)
#21 0x4208e3 in void boost::archive::detail::load_non_pointer_type<boost::archive::text_wiarchive>::load_only::invoke<boost::serialization::nvp<J*> >(boost:: archive::text_wiarchive&, boost::serialization::nvp<J*> const&) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc- gnu-7.2.0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x4208e3)
#22 0x420493 in void boost::archive::detail::load_non_pointer_type<boost::archive::text_wiarchive>::invoke<boost::serialization::nvp<J*> const>(boost::archive:: text_wiarchive&, boost::serialization::nvp<J*> const&) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2. 0/debug/threadapi-pthread/test_cyclic_ptrs_text_warchive+0x420493)
#23 0x420321 in void boost::archive::load<boost::archive::text_wiarchive, boost::serialization::nvp<J*> const>(boost::archive::text_wiarchive&, boost:: serialization::nvp<J*> const&) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi- pthread/test_cyclic_ptrs_text_warchive+0x420321)
#24 0x4201a2 in void boost::archive::detail::common_iarchive<boost::archive::text_wiarchive>::load_override<boost::serialization::nvp<J*> const>(boost:: serialization::nvp<J*> const&) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi- pthread/test_cyclic_ptrs_text_warchive+0x4201a2)
#25 0x41fc6b in void boost::archive::basic_text_iarchive<boost::archive::text_wiarchive>::load_override<boost::serialization::nvp<J*> const>(boost::serialization:: nvp<J*> const&) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/ test_cyclic_ptrs_text_warchive+0x41fc6b)
#26 0x41f0a7 in void boost::archive::text_wiarchive_impl<boost::archive::text_wiarchive>::load_override<boost::serialization::nvp<J*> const>(boost::serialization:: nvp<J*> const&) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/threadapi-pthread/ test_cyclic_ptrs_text_warchive+0x41f0a7)
#27 0x41ddee in boost::archive::text_wiarchive& boost::archive::detail::interface_iarchive<boost::archive::text_wiarchive>::operator>><boost::serialization::nvp<J* > const>(boost::serialization::nvp<J*> const&) (/mnt/data/darndt/boost-develop/bin.v2/libs/serialization/test/test_cyclic_ptrs_text_warchive.test/gcc-gnu-7.2.0/debug/ threadapi-pthread/test_cyclic_ptrs_text_warchive+0x41ddee)
#28 0x40e591 in test2() /mnt/data/darndt/boost-develop/libs/serialization/test/test_cyclic_ptrs.cpp:143
#29 0x4105c0 in test_main(int, char**) /mnt/data/darndt/boost-develop/libs/serialization/test/test_cyclic_ptrs.cpp:199
The 8bbcca5341fa3eabb6d0ab83c5f655ccceabf41f breaks library for mingw(-64)/gcc
add seralization for std::optional and std::variant
We need a serialization function for std::optional and std::variant.
Maybe they can be placed in boost/serialization/optional.hpp, the file for boost::optional and boost/serialization/variant.hpp for boost::variant, or new header files?
Bug: consistency: missing BOOST_NORETURN of serialization-specific throw_exception() causes warning-as-error C6011 (NULL ptr access) [MSVC2015]
Hello all,
using Boost 1.61.0.0 (HEAD appears to still have this issue!), we are getting
...\extern\boost\1.61.0.0\out\v140-x64\include\boost-1_61\boost\serialization\shared_ptr_helper.hpp(160):
error C6011: Dereferencing NULL pointer 'true_type'.
This happens despite boost/serialization/shared_ptr_helper.hpp already explicitly containing a true_type NULL ptr check!:
template<class T>
void reset(SPT< T > & s, T * t){
if(NULL == t){
s.reset();
return;
}
const boost::serialization::extended_type_info * this_type
= & boost::serialization::type_info_implementation< T >::type
::get_const_instance();
// get pointer to the most derived object's eti. This is effectively
// the object type identifer
typedef typename mpl::if_<
is_polymorphic< T >,
polymorphic,
non_polymorphic
>::type type;
const boost::serialization::extended_type_info * true_type
= type::get_object_type(*t);
// note:if this exception is thrown, be sure that derived pointern
// is either registered or exported.
if(NULL == true_type)
boost::serialization::throw_exception(
boost::archive::archive_exception(
boost::archive::archive_exception::unregistered_class,
this_type->get_debug_info()
)
);
// get void pointer to the most derived type
// this uniquely identifies the object referred to
// oid = "object identifier"
const void * oid = void_downcast(
*true_type,
*this_type,
t
);
I then relatively quickly realized that
this is due to
boost/serialization/throw_exception.hpp missing
a noreturn function attribute (BOOST_NORETURN), which
(consistency!!!?!?) is used in some other similar helpers, namely:
- boost/throw_exception.hpp
Given this missing noreturn attribute, the helper-internal "throw" (which disrupts code flow, i.e. is a properly valid cancellation point prior to execution of the NULL ptr deref!) will
NOT be visible / shadowed to MSVC Code Analysis (as opposed to a directly code-inline "throw"), thus
noreturn is required, else Code-Analysis-induced C6011 diagnosis.
Note that
some other locations might be affected by this kind of problem, too. Specifically:
- asio/detail/throw_exception.hpp
// Declare the throw_exception function for all targets.
template
void throw_exception(const Exception& e);
// Only define the throw_exception function when exceptions are enabled.
// Otherwise, it is up to the application to provide a definition of this
// function.
if !defined(BOOST_ASIO_NO_EXCEPTIONS)
template
void throw_exception(const Exception& e)
{
throw e;
}
See my code consistency bug fix patch in attachment
boost_serialization_throw_exception.hpp.diff.txt
which managed to get rid of this warning-as-error build fail here, in my serialization-restricted use case (my patch does NOT treat asio and potentially existing other locations yet).
HTH,
Andreas Mohr
Template Instantiation Missing for basic_xml_grammar<char>
Hello,
On Ubuntu 18.04 the template instantiation for basic_xml_grammar is missing from the dynamic library, but is present in the static library. Specifically look at the following output where I am checking the existence of the symbols for two functions load_start and parse_start_tag:
~$ nm -D /usr/lib/x86_64-linux-gnu/libboost_serialization.so.1.62.0 | grep load_start
000000000002d100 W _ZN5boost7archive18basic_xml_iarchiveINS0_12xml_iarchiveEE10load_startEPKc
~$ nm -D /usr/lib/x86_64-linux-gnu/libboost_serialization.so.1.62.0 | grep parse_start_tag
~$ nm /usr/lib/x86_64-linux-gnu/libboost_serialization.a | grep load_start
nm: stl_port.o: no symbols
0000000000000000 W _ZN5boost7archive18basic_xml_iarchiveINS0_12xml_iarchiveEE10load_startEPKc
~$ nm /usr/lib/x86_64-linux-gnu/libboost_serialization.a | grep parse_start_tag
nm: stl_port.o: no symbols
0000000000000000 W _ZN5boost7archive17basic_xml_grammarIcE15parse_start_tagERSi
U _ZN5boost7archive17basic_xml_grammarIcE15parse_start_tagERSi
This appears to be related to the fact that the instantiation is in file xml_grammar.cpp which includes file basic_xml_grammar.ipp. Both of those are in libs/serialization/src, so they are accessible only during the compilation of boost itself, but not during compilation/linking of any software against boost. I will venture to guess that an optimizing compiler/linker has for some reason removed the unused instantiation in the dynamic library, but I could not verify that because I was unable to find the flags used for the compilation of the officially distributed library (by Ubuntu).
I was able to copy basic_xml_grammar.ipp into my project together with the relevant instantiation code from xml_grammar.cpp (and the code of function init_chset() ). This causes the missing instantiation to appear in the object code of my own software and as a result everything compiles and links properly.
I am raising this question with you rather than with Ubuntu at this time because I could not see the reason why basic_xml_grammar.ipp and the instantiation are in boost serialization rather than boost archive and why they are in the implementation of the library rather than the header files.
If you are pondering why I need these in my code: because I implement my own archive xml_iarchive_nan which inherits everything from your standard xml_iarchive, but has special code to read nan in doubles because the standard library unfortunately does not read what it writes when it comes to doubles whose values are nan or inf.
Let me know if you need a code sample and I will spend the time to generate minimal example that shows the issue. Also let me know if this is absolutely not relevant to you and you prefer me to have discussion with Ubuntu about their compile/link flags that have lost the relevant instantiation.
Thank you.
Serialization of Boost MultiArray
Hi, I have a serialization code for boost::multi_array, and related types that I would like to contribute to the code for example as `boost/serialization/multi_array.hpp". Is it better if I contribute to Boost.MultiArray instead? That library doesn't seem to be actively maintained.
Deprecation warning from Boost.Predef in binary_iarchive
gcc.compile.c++ ../../bin.v2/libs/serialization/build/gcc-7.3.0/debug/visibility-hidden/binary_iarchive.o
In file included from ../../boost/detail/endian.hpp:9:0,
from ../../boost/archive/impl/basic_binary_iarchive.ipp:25,
from ../../libs/serialization/src/binary_iarchive.cpp:20:
../../boost/predef/detail/endian_compat.h:11:161: note: #pragma message: The use of BOOST_*_ENDIAN and BOOST_BYTE_ORDER is deprecated. Please include <boost/predef/other/endian.h> and use BOOST_ENDIAN_*_BYTE instead
#pragma message("The use of BOOST_*_ENDIAN and BOOST_BYTE_ORDER is deprecated. Please include <boost/predef/other/endian.h> and use BOOST_ENDIAN_*_BYTE instead")
^
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
###C++ CODE(study_boost_serialization.cpp)###
#include <iostream>
#include <fstream>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/format.hpp>
class gps_position {
friend class boost::serialization::access;
int degrees;
int minutes;
float seconds;
std::vector<int> ints;
template<class Archive>
void serialize(Archive &ar, const unsigned int /* file_version */) {
ar & BOOST_SERIALIZATION_NVP(degrees);
ar & BOOST_SERIALIZATION_NVP(minutes);
ar & BOOST_SERIALIZATION_NVP(seconds);
ar & BOOST_SERIALIZATION_NVP(ints);
}
public:
gps_position() : degrees(1), minutes(2), seconds(3.f), ints() {
ints.push_back(4);
ints.push_back(5);
};
void reset() {
degrees = 0;
minutes = 0;
seconds = 0.f;
ints.resize(0);
}
void say() {
std::cout << boost::format("gps_position{degrees=%1%,minutes=%2%,seconds=%3%,ints.size=%4%}\n") % degrees %
minutes % seconds % ints.size();
}
};
class location : public gps_position {
friend class boost::serialization::access;
std::string name;
template<class Archive>
void serialize(Archive &ar, const unsigned int /* file_version */) {
ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(gps_position);
ar & BOOST_SERIALIZATION_NVP(name);
}
public:
location() : name("location") {}
void reset() {
gps_position::reset();
name.resize(0);
}
void say() {
gps_position::say();
std::cout << boost::format("location{name=%1%}\n") % name;
}
};
class FZ {
friend boost::serialization::access;
gps_position *l;
template<class Archive>
void serialize(Archive &ar, const unsigned int /* file_version */) {
ar & BOOST_SERIALIZATION_NVP(l);
}
public:
FZ() {
l = new location;
}
void ref(FZ &right) {
delete l;
l = right.l;
}
void reset() {
delete l;
l = nullptr;
}
void say() {
std::cout << boost::format("FZ{l=@%1%}\n") % l;
if (l)
l->say();
}
};
using namespace std;
int main() {
// create class instance
FZ fz;
FZ fz2;
fz2.ref(fz);
#if SAVE
ofstream ofs("./test.xml");
// save data to archive
{
boost::archive::xml_oarchive oa(ofs);
// write class instance to archive
oa << BOOST_SERIALIZATION_NVP(fz);
oa << BOOST_SERIALIZATION_NVP(fz2);
// archive and stream closed when destructors are called
}
//
//// save data to archive
//{
// boost::archive::text_oarchive oa(std::cout);
// // write class instance to archive
// oa << BOOST_SERIALIZATION_NVP(l);
// // archive and stream closed when destructors are called
//}
#endif
#if LOAD
fz.reset();
fz2.reset();
ifstream ifs("./test.xml");
{
boost::archive::xml_iarchive ia(ifs);
ia >> BOOST_SERIALIZATION_NVP(fz);
ia >> BOOST_SERIALIZATION_NVP(fz2);
}
fz.say();
fz2.say();
#endif
return 0;
}
################
###CMAKE CODE###
add_executable(study_boost_serialization_load study_boost_serialization.cpp)
target_link_libraries(study_boost_serialization_load boost_serialization)
target_compile_definitions(study_boost_serialization_load PRIVATE -DLOAD)
add_executable(study_boost_serialization_save study_boost_serialization.cpp)
target_link_libraries(study_boost_serialization_save boost_serialization)
target_compile_definitions(study_boost_serialization_save PRIVATE -DSAVE)
###########################################
###CONSOLE OUTPUT(study_boost_serialization_load)###
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
###############
###DEBUG func stack###
##study_boost_serialization.cpp##
ifstream ifs("./test.xml");
{
boost::archive::xml_iarchive ia(ifs);//error line
ia >> BOOST_SERIALIZATION_NVP(fz);
ia >> BOOST_SERIALIZATION_NVP(fz2);
}
##xml_iarchive.hpp##
class BOOST_SYMBOL_VISIBLE xml_iarchive :
public xml_iarchive_impl<xml_iarchive>{
public:
xml_iarchive(std::istream & is, unsigned int flags = 0) :
xml_iarchive_impl<xml_iarchive>(is, flags)//error line
{}
~xml_iarchive(){};
};
Plea to revert recent changes to singleton.hpp
This is a plea to revert all the changes recently done to singleton.hpp. The reasons for this plea are
- having to call destructors for singlons to prevent memory-leaks is very unusual and counter-intuitive
- using
if (!t)
t = new singleton_wrapper;
in singleton::get_instance() is not thread-safe (the former code using a non-pointer static variable was thread-safe by design) - The reason for all those changes is a non-standard-conforning implementation detail of the gcc-compiler. The so-called fix for the gcc-error was not a fix at all, but simple a memory-leak as the problematik pointer was never(!) deleted (and therefore the problem seemde to be fixed).
If not removed at all, then those changes should definitively be guarded by some gcc-workaround macro, that all other compilers use the old - and totally correct - code.
In my opinion those changes should be reverted as soon as possible and hopefully make it into Boost 1.67.
Tobias
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.