Home > Uncategorized > Challenge: daemonizing a paster script spawned in a virtual environment

Challenge: daemonizing a paster script spawned in a virtual environment

I digress occasionally during project roll outs , marginally becoming obsessed with a semi-trivial aspect, mostly because I know it can be done.

This is my research reflecting investigation of daemonizing a paster script, specifically a celeryd “configurator” and executor written by marcin for the rhodecode project.

Conclusion is that since celeryd is executed via the celeryd paster script authored by marcin, it will be difficult to daemonize the spawned celeryd.

You could spawn celeryd_detach.py from deep in the /bin/ of you virtualenv, but then you wouldn’t be pulling the celeryd config out of the paste conf file production.ini.

marcin states that he did tweak, explicitly, the celery-paste py scripts for his own paste config interpreter.

Some other options, including how to daemonize paster process itself.

python -m celery.bin.celeryd_detach does daemonize the process, but it isn’t paster.

https://bitbucket.org/twillis/celery-paste/ is this the class that rhodecode uses?

No, marcin did take these scripts and tweak them for his own paste script, celeryd.

virtualenvs simply prepend the path of the virtualenv to the $PATH, so you’re always running the python, etc, that is located in that path. This is what the script ./venv/bin/activate does.

starting celeryd

#source /var/www/rhodecode-venv/bin/activate
/var/www/rhodecode-venv/bin/paster celeryd /var/www/rhodecode/production.ini --pidfile=/var/run/celeryd.pid

start rhodecode

 #source /var/www/rhodecode-venv/bin/activate
 /var/www/rhodecode/bin/paster serve /var/www/rhodecode/production.ini --daemon --user=root --pid-file=/var/run/rhodecode.pid

stop a paster serve daemon spawn

 /var/www/rhodecode/bin/paster serve --stop-daemon --pid-file=/var/run/rhodecode.pid

It seems that the paster script must be integrated when an egg is installed.
This, to me, means it’s quite likely that paster celeryd is installed by the rhodecode egg.

marcin states:

 [14:59] <branded> marcinkuzminski: looking further into paste scripts, did you write your own script to spawn celery?
 [15:01] <branded> it doesn't look like you chose to leverage celery-paste (https://bitbucket.org/twillis/celery-paste/)
 [15:09] <marcinkuzminski> branded, my code is based on that, but i patched few things
[15:10] <branded> marcinkuzminski: thanks for replying... interesting, I have been doing research to daemonize the celeryd process... this option isn't present in your script, but has been present in the celeryd project for a bit
 [15:10] <branded> is this something that can be changed? or is there another method to daemonize?
 [15:11] <branded> ...with reference to https://github.com/ask/celery/issues/373
 [15:12] <marcinkuzminski> branded, i know people used to demonized it somehow, i think someone posted an answer on discussion group
[15:12] <branded> may i log an enhancement request for this option?
 [15:14] <branded> it seems that i can likely run celery configured separately, and daemonize that process as i see fit
 [15:28] <marcinkuzminski> branded, i didn't had requests for that, but there are many outside tools for daemonizing things
[15:15] <branded> is it for convenience that you've opted to pull the config from a paste ini?
 [15:15] <branded> or is it critical?
 [15:16] <marcinkuzminski> branded, yes it's critical

By setting up the environment, as if I was to launch a virtualenvironment:

[root@alchemy-clone ~]# echo $PATH
 [root@alchemy-clone ~]# PATH=/var/www/rhodecode-venv/bin/:$PATH
 /var/www/rhodecode-venv/bin/paster celeryd /var/www/rhodecode/production.ini --pidfile=/var/run/celeryd.pid
 paster celeryd /var/www/rhodecode/production.ini --pidfile=/var/run/celeryd.pid

reported path of the celeryd:

/var/www/rhodecode-venv/bin/python /var/www/rhodecode-venv/lib/python2.6/site-packages/celery-2.2.7-py2.6.egg/celery/bin/celeryd.py --help

Works but there’s no config obviously…

/var/www/rhodecode-venv/bin/python /var/www/rhodecode-venv/lib/python2.6/site-packages/celery-2.2.7-py2.6.egg/celery/bin/celeryd_detach.py --pidfile=/var/run/celeryd_detach.pid

What we’ll have to do is hack away, we’ll daemonize celeryd by configuring it to run with the same settings that are in our production.ini

EVERY TIME production.ini is changed, the celery run time configuration must be changed as well.

production.ini’s celery config is referenced by rhodecode related code as well.

What we’re concerned with is the CELERY configuration section of production.ini

 use_celery = true
 broker.host = localhost
 broker.vhost = celeryvhost
 broker.port = 5672
 broker.user = celeryuser
 broker.password = password
 celery.imports = rhodecode.lib.celerylib.tasks
 celery.result.backend = amqp
 celery.result.dburi = amqp://
 celery.result.serialier = json
 #celery.send.task.error.emails = true
 #celery.amqp.task.result.expires = 18000
 celeryd.concurrency = 2
 #celeryd.log.file = celeryd.log
 celeryd.log.level = debug
 celeryd.max.tasks.per.child = 1
 #tasks will never be sent to the queue, but executed locally instead.
 celery.always.eager = false


And here I stopped, knowing that if I decided to write my own paste config interpreting celeryd paste script, that I’d be compromising stability in the future.

However, I’m opting for using the & forker.


After some other troubleshooting, I have determined that python needs to have the HOME environmental variable set for it to execute without an error.

  1. December 28, 2011 at 4:43 pm

    The Rhodecode beta branch now has upstart scripts that I created for both Rhodecode and Celeryd that runs both as daemons without hacking apart any code.

  2. December 28, 2011 at 4:54 pm

    Thanks Matt! I had written some, but they are not great! I use the & to drop the processes to the background.


    It looks like the upstart scripts you speak of are in your fork of rhodecode on your site: http://code.mattzuba.com/rhodecode/src/58df0b3ed377/init.d


  3. December 29, 2011 at 12:31 am

    I put them in my fork, but also sent a pull request back to marcin’s bitbucket repo which he was kind enough to accept, so they are also in the development line on Rhodecode :)

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: