Comments (16)
note this issue is with pypy only - with cPython I seem to be connecting
successfully.
Original comment by [email protected]
on 5 Aug 2013 at 6:10
from pypyodbc.
oh and it happens on cPython3.3 also. So only cPython2.7 is working.
Original comment by [email protected]
on 5 Aug 2013 at 6:14
from pypyodbc.
Hi Mike,
Good to see you in my project's comment areas. I like your MAKO template's
speed :)
Because I don't have an OSX environment, it's hard for me to reproduce the
problem. However, I did research the doc about the problem.
The truth is, the return value is -1, but it's SQL_ERROR instead of
SQL_NULL_DATA.
From MSDN, the reason why SQLGetDiagRec returns a SQL_ERROR can be caused by 3
possibilities:
RecNumber was negative or 0.
(which is number_errors in my script)
BufferLength was less than zero.
(which is len(Message) in my script)
When using asynchronous notification, the asynchronous operation on the handle
was not complete.
(Which I'm not sure if you were using asynchronous operation?)
So can you do some value tracking about the 3 possible reasons if you can
reproduce the bug?
Original comment by [email protected]
on 11 Aug 2013 at 5:02
- Changed state: Accepted
from pypyodbc.
I will first apply your patch to the development branch in GitHub
Original comment by [email protected]
on 11 Aug 2013 at 5:04
from pypyodbc.
doesn't seem like any of those things. I doubt the MS docs are enough here as
we are using non-MS ODBC client libraries on OSX (iODBC):
Python 3.3.2 (default, Jun 30 2013, 23:23:11)
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.24)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pypyodbc
>>> c = pypyodbc.connect("DSN=ms2005;UID=scott;PWD=tiger")
>
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/py
pyodbc.py(945)ctrl_err()
-> raise ProgrammingError('', 'SQL_NULL_DATA')
(Pdb) locals()
{'pdb': <module 'pdb' from
'/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/pdb.py'>,
'ansi': False, 'Buffer_len': c_short(109), 'ret': -1, 'ht': 2, 'err_list':
[('I', '[', 0)], 'h': c_void_p(140644749023424), 'state':
<ctypes.c_char_Array_22 object at 0x1102cfd40>, 'val_ret': -1, 'raw_s': <class
'str'>, 'number_errors': 2, 'ODBC_func': <_FuncPtr object at 0x110496c80>,
'Message': <ctypes.c_char_Array_10240 object at 0x1102cfe60>, 'NativeError':
c_int(0)}
(Pdb) len(Message)
10240
(Pdb) state.value
b'I'
(Pdb) Message.value
b'['
(Pdb) number_errors
2
(Pdb)
Original comment by [email protected]
on 11 Aug 2013 at 6:06
from pypyodbc.
Hi, I found one API error usage, thought it had no impact on Windows and Linux,
but I'm not sure if it's causing this problem. Can you test it?
Line 912, change the below:
NativeError, Message, len(Message), ADDR(Buffer_len))
to
ADDR(NativeError), Message, len(Message), ADDR(Buffer_len))
Thanks.
Original comment by [email protected]
on 14 Aug 2013 at 2:07
from pypyodbc.
pretty much the same
Python 3.3.2 (default, Jun 30 2013, 23:23:11)
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.24)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pypyodbc
>>> try:
... pypyodbc.connect("DSN=ms2005;PWD=tiger;UID=scott")
... except:
... import pdb
... pdb.post_mortem()
...
>
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/py
pyodbc.py(945)ctrl_err()
-> raise ProgrammingError('', 'SQL_NULL_DATA')
(Pdb) locals()
{'Buffer_len': c_short(109), 'Message': <ctypes.c_char_Array_10240 object at
0x10171ae60>, 'ht': 2, 'val_ret': -1, 'ansi': False, 'err_list': [('I', '[',
0)], 'state': <ctypes.c_char_Array_22 object at 0x10171ad40>, 'number_errors':
2, 'ODBC_func': <_FuncPtr object at 0x101821120>, 'NativeError': c_int(0), 'h':
c_void_p(140189483713840), 'ret': -1, 'raw_s': <class 'str'>}
(Pdb) number_errors
2
(Pdb) state.value
b'I'
(Pdb) Message.value
b'['
(Pdb)
Original comment by [email protected]
on 14 Aug 2013 at 4:13
from pypyodbc.
From the incomplete err_list, your OS's internal encoding might use UCS-4
instead of UCS-2. And this might related to why SQL_ERROR (-1) was returned.
I'm attaching a modified pypyodbc.py, which contains debug information and will
print out the raw encoded string of variable state, so we could see what is the
encoding used.
Can you try it and post the pdb info?
Original comment by [email protected]
on 17 Aug 2013 at 9:14
Attachments:
from pypyodbc.
Python 3.3.2 (default, Jun 30 2013, 23:23:11)
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.24)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pypyodbc
>>> try:
... pypyodbc.connect("DSN=ms2005;PWD=tiger;UID=scott")
... except:
... import pdb
... pdb.post_mortem()
...
b'I\x00\x00\x00M\x00\x00\x000\x00\x00\x000\x00\x00\x002\x00\x00\x00\x00\x00'
b'I\x00\x00\x00M\x00\x00\x000\x00\x00\x000\x00\x00\x002\x00\x00\x00\x00\x00'
>
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/py
pyodbc.py(945)ctrl_err()
-> raise ProgrammingError('', 'SQL_ERROR')
(Pdb) locals()
{'ODBC_func': <_FuncPtr object at 0x109753120>, 'err_list': [('I', '[', 0)],
'h': c_void_p(140387085764016), 'val_ret': -1, 'ret': -1, 'Message':
<ctypes.c_char_Array_4096 object at 0x10964de60>, 'raw_s': <class 'str'>,
'ansi': False, 'NativeError': c_int(0), 'Buffer_len': c_short(109), 'ht': 2,
'debug':
b'I\x00\x00\x00M\x00\x00\x000\x00\x00\x000\x00\x00\x002\x00\x00\x00\x00\x00',
'number_errors': 2, 'state': <ctypes.c_char_Array_22 object at 0x10964dd40>}
(Pdb) state.value
b'I'
(Pdb) Message.value
b'['
(Pdb)
Original comment by [email protected]
on 17 Aug 2013 at 2:58
from pypyodbc.
OK, confirmed your OS's internal encoding use UCS-4. So I'm attaching a
pypyodbc.py for UCS4, can you run it and see if it makes a difference.
However, even it works, problems remains how to detect iODBC returns string in
UCS2 or UCS4.
But let's first see if the attached works or not.
Thanks!
Original comment by [email protected]
on 18 Aug 2013 at 2:00
Attachments:
from pypyodbc.
Python 3.3.2 (default, Jun 30 2013, 23:23:11)
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.24)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pypyodbc
>>> try:
... pypyodbc.connect("DSN=ms2005;PWD=tiger;UID=scott")
... except:
... import pdb
... pdb.post_mortem()
...
>
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/py
pyodbc.py(462)ucs2_dec()
-> uchar = buffer.raw[i:i + 4].decode('utf_32')
(Pdb) locals()
{'i': 20, 'uchar': '2', 'buffer': <ctypes.c_char_Array_22 object at
0x104fc9d40>, 'uchars': ['I', 'M', '0', '0', '2']}
(Pdb) where
<stdin>(2)<module>()
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pypyodbc.py(2287)__init__()
-> self.connect(connectString, autocommit, ansi, timeout, unicode_results,
readonly)
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pypyodbc.py(2335)connect()
-> check_success(self, ret)
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pypyodbc.py(953)check_success()
-> ctrl_err(SQL_HANDLE_DBC, ODBC_obj.dbc_h, ret, ODBC_obj.ansi)
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pypyodbc.py(940)ctrl_err()
-> err_list.append((from_buffer_u(state), from_buffer_u(Message),
NativeError.value))
>
/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/py
pyodbc.py(462)ucs2_dec()
-> uchar = buffer.raw[i:i + 4].decode('utf_32')
(Pdb)
Original comment by [email protected]
on 20 Aug 2013 at 6:21
from pypyodbc.
here's the full traceback:
(Pdb) cont
>>> pypyodbc.connect("DSN=ms2005;PWD=tiger;UID=scott")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pypyodbc.py", line 2287, in __init__
self.connect(connectString, autocommit, ansi, timeout, unicode_results, readonly)
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pypyodbc.py", line 2335, in connect
check_success(self, ret)
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pypyodbc.py", line 953, in check_success
ctrl_err(SQL_HANDLE_DBC, ODBC_obj.dbc_h, ret, ODBC_obj.ansi)
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pypyodbc.py", line 940, in ctrl_err
err_list.append((from_buffer_u(state), from_buffer_u(Message), NativeError.value))
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pypyodbc.py", line 462, in ucs2_dec
uchar = buffer.raw[i:i + 4].decode('utf_32')
UnicodeDecodeError: 'utf32' codec can't decode bytes in position 0-1: truncated
data
Original comment by [email protected]
on 20 Aug 2013 at 6:21
from pypyodbc.
Ah, truncated data, not enough buffer. Can you try this one?
Original comment by [email protected]
on 22 Aug 2013 at 2:33
Attachments:
from pypyodbc.
that fixed it, nice work
Original comment by [email protected]
on 22 Aug 2013 at 3:28
from pypyodbc.
The reason why pypyodbc got broken on OSX, is that OSX's iODBC provide data to
applications in UCS4(UTF32), which is special compared with MS ODBC and
unixODBC.
The common practice for MS ODBC, and unixODBC is providing data to applications
in UCS2(UTF16).
So I would revise pypyodbc to switch the buffer interface for UCS4 when
detecting that iODBC is the underlying ODBC driver manager. The way to detect
that is by the name of the dll library loaded. iODBC's dll should be
"libiodbc.dylib"
Did you change anything about DLL loading part of pypyodbc so pypyodbc can load
iODBC's library on OSX, or you just used the version I provided without any
changes?
I will add a detection logic, then pypyodbc would be able to run under
iODBC/unixODBC without any changes needed.
Original comment by [email protected]
on 23 Aug 2013 at 5:00
from pypyodbc.
Version 1.2.0 has been uploaded and contains the fix. The new version judges if
iODBC is been used and switch to use UCS4/UTF-32 to decode and encoding data
from the ODBC library.
Original comment by [email protected]
on 21 Sep 2013 at 11:17
- Changed state: Fixed
from pypyodbc.
Related Issues (20)
- Description attribute returns only first char of name (as byte) using Oracle ODBC & Linux HOT 2
- Trimmed varchar results.
- Column names get wonky on 64-bit system using iSeries DB2 driver HOT 3
- Oracle and Decimal_cvt issue
- IronPython 2.7 StopIteration exception raised when retrieving SQL data set
- "invalid literal" exceptions querying empty numeric fields when using LinuxODBC with FreeTDS
- use mbdtools,chinese character encoded by gb charset can't access correctly
- Passing datetime.date returns TypeError: Wrong Type HOT 2
- Mapping bigint in sqlserver 2008 into float, instead of int in python3
- Bad cursor description (mixed bytes and strings) on Ubuntu 14.04 HOT 2
- Row objects don't raise KeyError for invalid fields
- Data source name not found on linux + unixodbc + as400 iseries driver HOT 1
- Cannot access column names by attribute HOT 4
- pypyodbc doesn't support the MSSQL datetimeoffset column type.
- state = err_list[0][0] Index out of range
- UnicodeDecodeError because UTF-16 surrogate pairs are not handled HOT 1
- Enhancement: Support for lists as SQL parameters
- executemany is very slow
- Sage Line 100 Odbc connect KO HOT 1
- Not working on Windows 7 64 bit 3.4.3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pypyodbc.