Controlling workers with supervisord about bccresquebundle

[ruudk]

I'm using Supervisor and don't have any problems. I don't use the Symfony2 command but I directly call the bin/resque file like this:

[program:app:campfire]
directory=/home/myapp/current
command=php vendor/bcc/resque-bundle/BCC/ResqueBundle/bin/resque
numprocs=1
stdout_logfile=/home/myapp/shared/app/logs/workers/campfire.log
stderr_logfile_maxbytes=10MB
redirect_stderr=true
autostart=true
autorestart=true
environment=ENVIRONMENT='prod',QUEUE='campfire',PREFIX='app',BLOCKING=1,INTERVAL=5,APP_INCLUDE='/home/myapp/current/app/bootstrap.php.cache'
user=app
stopsignal=QUIT

[t3chn0r]

Thanks @ruudk, I was able to get the scheduler running and I'm able now to start and stop it using supervisorctl. The weird thing was that the directory bin under ResqueBundle was not there, it seems like composer doesn't download that folder at all.

With that said, I'm now having problems with resque, when I try to start the service I get the following error...

PHP Stack trace:
PHP   1. {main}() .../vendor/bcc/resque-bundle/BCC/ResqueBundle/bin/resque:0
PHP Fatal error:  Class 'Resque_Log' not found in .../vendor/bcc/resque-bundle/BCC/ResqueBundle/bin/resque on line 43

Any ideas? I checked the entire bcc structure and I couldn't find that class. I wonder is composer is also failing to install other files from this bundle?

Regards,
t3chn0r

[ruudk]

Did composer install chrisboulton/resque correctly? And what is your supervisorctl config?

On 15 dec. 2013, at 20:35, t3chn0r [email protected] wrote:

Thanks @ruudk, I was able to get the scheduler running and I'm able now to start and stop it using supervisorctl. The weird thing was that the directory bin under ResqueBundle was not there, it seems like composer doesn't download that folder at all.

With that said, I'm now having problems with resque, when I try to start the service I get the following error...

PHP Stack trace:
PHP 1. {main}() .../vendor/bcc/resque-bundle/BCC/ResqueBundle/bin/resque:0
PHP Fatal error: Class 'Resque_Log' not found in .../vendor/bcc/resque-bundle/BCC/ResqueBundle/bin/resque on line 43
Any ideas? I checked the entire bcc structure and I couldn't find that class. I wonder is composer is also failing to install other files from this bundle?

Regards,
t3chn0r

—
Reply to this email directly or view it on GitHub.

[t3chn0r]

Well, it is installed but I guess it's not complete. The Resque_Log seems to be in that package in a file named Log.php but I don't have that file. Maybe it's related to this comment...

The supervisorctl config is exactly the same as yours but with different paths for my app...

What do you have in your composer.json? Do you also have all the packages or do you only have this bundle? And which version do you have? Or do you have dev-master?

Thanks a lot again for your help!

[ruudk]

This is my composer file:

"chrisboulton/php-resque": "dev-master",
"chrisboulton/php-resque-scheduler": "dev-master",
"bcc/resque-bundle": "dev-master"

Do you see the vendor/chrisboulton directory?

[t3chn0r]

Yes, that did it! I'm having another problem now but I reopened an old thread instead so we can get the issue solve. Thanks a lot @ruudk ! 👍

[ruudk]

The problem is with the versioning of PHP-Resque. The last release was a year ago. If they release a new version, we can require that version so you only have to require bcc/resque-bundle. For now, you should do it this (ugly) way :(

[epicwhale]

@ruudk @t3chn0r

Correct me if I'm wrong here.. but for every worker I want to run and monitor with supervisord, I need to create a new [program:..] entry in the config file? And NOT use the --count=x parameter?

[ruudk]

Yes. You can set num_procs with supervisord.

[epicwhale]

@ruudk thanks, got it working.

Please note, its numprocs (not num_procs). This worked for me.
numprocs=5 process_name=%(program_name)s_%(process_num)02d

[ruudk]

Great!

[epicwhale]

@ruudk also, shouldn't the supervisor configuration have a stopsignal=QUIT.

Else won't stopping the process/group cause on-going jobs to be shut in an intermediate state?

[ruudk]

I don't know, I run 10 workers in 1 group and when I stop the group it takes like 5 seconds to stop all the workers. So I think the default will be good already? Or am I wrong?

[epicwhale]

The php-resque documentation mentions that it supports SIGNALS just like the Ruby version

QUIT - Wait for job to finish processing then exit
TERM / INT - Immediately kill job then exit
USR1 - Immediately kill job but don't exit
USR2 - Pause worker, no new jobs will be processed
CONT - Resume worker.
https://github.com/chrisboulton/php-resque

I believe by default, supervisor uses the 'TERM' signal which means it will immediately kill the job? Isn't that undesirable? (vs QUIT which waits for on-going jobs to finish processing and then exits gracefully)

[ruudk]

Could you please test it and let me know :)? If you are right then we should also update the docs and I should update my supervisord.conf :D

[epicwhale]

I still have to test it (not sure how to do it full-proof), but its a safe bet to believe it will work based on how people have been configuring Ruby's Resque with Supervisor. If you look at this article - http://ryanmckern.com/2013/10/finally-resque-supervisor-have-learned-to-get-along/ - there's a clear stopsignal=QUIT

[ruudk]

Ok, I think you are correct. I updated my example above, and will also add this to the repo readme.

[PATROMO]

The solution is stopasgroup=true:

If true, the flag causes supervisor to send the stop signal to the whole process group and implies killasgroup is true. This is useful for programs, such as Flask in debug mode, that do not propagate stop signals to their children, leaving them orphaned.