what is the fast way for writing? about happybase

[wbolster]

Have you profiled your program to see where it spends the most time? cProfile or other profiling tools may help you here.

If it's inside your application's Python code, you might want to optimize that, and if that turns out not to be good enough, you may look into running multiple processes (e.g. using the multiprocessing module), since CPython only runs a single thread at a time (due to the GIL).

On the other hand, if most of the time is spent writing to sockets and waiting for server responses, threads or gevent (with monkey patching so that Thrift uses gevent-aware sockets) may be a better option.

So, the anwer to your question "is there a better way" is "it depends". :)

[saintthor]

thank you.

[wbolster]

Oh, and if you decide to use multiple threads or processes, running multiple Thrift servers on different nodes may also help increase performance. http://chase-seibert.github.io/blog/2013/05/24/why-you-should-not-write-your-own-load-balancer.html might be an interesting read in that case.

[saintthor]

thanks. but i don't understand how to run multiple Thrift servers in one hadoop group.

[wbolster]

Just run a Thrift server on each node.

[saintthor]

thank you. it is wonderful.