Comments (13)
not sure about discovering the thrift version, but in hbase/lib I have a libthrift-0.8.0.jar if that helps
from happybase.
the same with happybase git master
from happybase.
HappyBase does not use the framed transport for the Thrift connection. Do you have any particular reason to use that transport instead if the other ones?
— Wouter
Travis Parker [email protected] schreef:
happybase 0.3 with hbase 0.94.0 is getting its connection closed
consistently:import happybase
conn = happybase.Connection("localhost")
t = conn.table("test")
list(t.scan())
Traceback (most recent call last):
File "", line 1, in
File "happybase/api.py", line 535, in scan
scan_id = client.scannerOpenWithScan(self.name, scan)
File "happybase/hbase/Hbase.py", line 1716, in scannerOpenWithScan
return self.recv_scannerOpenWithScan()
File "happybase/hbase/Hbase.py", line 1728, in recv_scannerOpenWithScan
(fname, mtype, rseqid) = self._iprot.readMessageBegin()
File "thrift/protocol/TBinaryProtocol.py", line 126, in
readMessageBegin
sz = self.readI32()
File "thrift/protocol/TBinaryProtocol.py", line 203, in readI32
buff = self.trans.readAll(4)
File "thrift/transport/TTransport.py", line 58, in readAll
chunk = self.read(sz-have)
File "thrift/transport/TTransport.py", line 160, in read
self.__rbuf = StringIO(self.__trans.read(max(sz, self.__rbuf_size)))
File "thrift/transport/TSocket.py", line 94, in read
buff = self.handle.recv(sz)
socket.error: [Errno 104] Connection reset by peerand in the thrift server log:
2012-06-29 13:06:01,330 ERROR
org.apache.thrift.server.TNonblockingServer: Read an invalid frame size
of -2147418111. Are you using TFramedTransport on the client side?
Reply to this email directly or view it on GitHub:
#6
from happybase.
Do you have any particular reason to use that transport instead if the other ones?
No, I'm running exactly the python code I posted above, not doing anything special. If happybase isn't using the framed transport then I guess the thrift server's log message is mis-diagnosing the issue.
from happybase.
Maybe the default server side transport type has changed in 0.94. I guess it's a flag to the start-daemon thrift script.
— Wouter
Travis Parker [email protected] schreef:
Do you have any particular reason to use that transport instead if
the other ones?No, I'm running exactly the python code I posted above, not doing
anything special. If happybase isn't using the framed transport then I
guess the thrift server's log message is mis-diagnosing the issue.
Reply to this email directly or view it on GitHub:
#6 (comment)
from happybase.
starting the thrift server in the foreground with bin/hbase thrift start -c -threadpool
I see this error on the console running the server:
ERROR thrift.TBoundedThreadPoolServer: Thrift error occurred during processing of message.
org.apache.thrift.protocol.TProtocolException: Expected protocol id ffffff82 but got ffffff80
at org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:445)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:22)
at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:287)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
and get this in the python client:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "happybase/api.py", line 546, in scan
File "happybase/hbase/Hbase.py", line 1716, in scannerOpenWithScan
File "happybase/hbase/Hbase.py", line 1728, in recv_scannerOpenWithScan
File "thrift/protocol/TBinaryProtocol.py", line 126, in readMessageBegin
sz = self.readI32()
File "thrift/protocol/TBinaryProtocol.py", line 203, in readI32
buff = self.trans.readAll(4)
File "thrift/transport/TTransport.py", line 58, in readAll
chunk = self.read(sz-have)
File "thrift/transport/TTransport.py", line 160, in read
self.__rbuf = StringIO(self.__trans.read(max(sz, self.__rbuf_size)))
File "thrift/transport/TSocket.py", line 108, in read
raise TTransportException(type=TTransportException.END_OF_FILE, message='TSocket read 0 bytes')
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes
for -hsha
, -nonblocking
, and -threadedselector
the help output says "this implies the framed transport" and indeed using any of those produces the error in the OP.
from happybase.
ah HAH. the -c
was wrong (I had thought that was a negation of -f
for framed transport).
the thrift server must be started with bin/hbase thrift start -threadpool
. The help output says of -threadpool
that "this is the default", but it requires that one of [-hsha, -nonblocking, -threadpool, -threadedselector]
be explicitly used.
starting it with bin/hbase-daemon.sh start thrift
was picking one of the framed transport modes.
this is worth documenting imho
from happybase.
Ah, this should definitely end up in the docs. Maybe HappyBase can work with other Thrift transports as well. Reopening so that I don't forget about this.
from happybase.
Okay, this issues seems fixed now. I've also updated the tutorial and the API docs. Could you please have a look at http://happybase.readthedocs.org/en/latest/tutorial.html and tell me what you think? Thanks!
from happybase.
looks great, thanks!
from happybase.
I still see these errors on using latest code.
from happybase.
@bilkulbekar Please provide more details. This usually only happens when the Thrift server speaks a different protocol than the Thrift client (HappyBase).
from happybase.
This is still not resolved
from happybase.
Related Issues (20)
- How does happyhbase fuzzy query rowkey HOT 3
- Support Pre-Split when create table ?
- HBase remotely connecting to python project HOT 7
- Support gevent?
- suppot count table rows? HOT 1
- org.apache.hadoop.hbase.NamespaceNotFoundException: HOT 4
- [feature] Support reconnect host when connectionpool raise error HOT 2
- thriftpy2.protocol.exc.TProtocolException: Bad protocol id in the message: 72 HOT 4
- happybase 1.2.0 supports hbase 2.2.5 ? HOT 1
- How can I do a query for specific columns by regex-statement HOT 2
- TTransportException: TTransportException(type=4, message='TSocket read 0 bytes') HOT 4
- TypeError: __str__ returned non-string (type bytes) hbase HOT 6
- import happybase error HOT 1
- Does it support snapshot management? HOT 1
- Is there a way to update / modify TTL (time to leave) on already created table ? HOT 1
- Hbase compatibility
- Table Put - How Do We Assign And Use A Variable For the 'Row Key' HOT 8
- compact_table major no work
- 使用happybase创建连接池并扫描整张表一段时间以后报错 HOT 5
- Support for 'cacheBlocks' in TScan HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from happybase.