Py3 std::string is not converted to 'bytes' about python

[stefanseefeld]

I agree this is an issue, though I'm not yet sure what the right fix is. std::wstring is defined as std::basic_string<wchar_t>, with wchar_t being of non-portable compiler-dependent width. So it isn't even clear what data it can hold, much less what encoding the user choses to store in it.

I think it would be best to not provide a default conversion for it, rather than second-guessing what the user may want / need.

[pavelschon]

I think the issue is not with std::wstring, but std::string.

The following workaround works for me:

#if PY_MAJOR_VERSION >= 3
object getString()
{
    return object( handle<>( PyBytes_FromString( "string" ) ) );
}
#else /* python 2.x */
std::string getString()
{
    return "string";
}
#endif

Then it produces expected results:

Python 3.4.2 (default, Oct  8 2014, 10:45:20) 
[GCC 4.9.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from str_test import getString, getWString
>>> type(getString())
<class 'bytes'>
>>> type(getWString())
<class 'str'>

[stefanseefeld]

OK, fair enough. :-)
Still, I think the handling of getWString() is wrong, too, i.e. Boost.Python shouldn't provide a default converter for std::wstring.

[tadeu]

I think the handling is already correct in both cases, because std::string and std::wstring are both supposed to handle text, not bytes. std::wstring is supposed to be text encoded in UTF-16 when sizeof(wchar_t) == 2 (i.e., in Windows) and in UTF-32 when sizeof(wchar_t) == 4 (i.e., in Unix with default compiler flags). By extension, the universally accepted encoding for std::string would be UTF-8.

In Python 3 philosophy, bytes is supposed to be just that, an array of bytes, not text. Sometimes this array of bytes may be holding an encoded representation of text, but this is just a special case.