Is there a way to run a system process asynchronously? about tornado

[greut]

web.by fires up a thread; I've done it by using an external worker.

[lucky]

You probably want subprocess.

or celery

[yashh]

The better way to handle this is using a message queue. You push a job onto a queue and a worker picks it up and process it. Check out RabbitMQ, pyres, gearman, bealstalkd etc.

[fictorial]

+1 for yashh

you can find an example at http://brianglass.wordpress.com/2009/11/29/asynchronous-shell-commands-with-tornado/

[parente]

The multiprocessing package in Python 2.6 std lib works well with Tornado's IO loop.

[alexdong]

I wrote a module for tornado similar to the AsyncHTTPClient just for this purpose.
http://github.com/alexdong/tornado/blob/master/tornado/async_process.py

[yashh]

Is there a way to run a set of mysql queries in parallel. Bret Taylor's post on mysql explains that they query all mysql shards in parallel. Can we do this in tornado ?

[alexdong]

Well, you might want to leverage the async_process and pipe the result into tornado.

[mark-kubacki]

Put the data (e.g. filename of the image in question) in a queue Q and process that queue in another process. Python comes already with everything you need:
http://docs.python.org/library/multiprocessing.html

If you want to do that distributed, check out ZeroMQ http://www.zeromq.org

[bdarnell]

Tornado 3.0 introduced tornado.process.Subprocess: http://www.tornadoweb.org/en/stable/process.html#tornado.process.Subprocess