Part 2: Install Rhodecode

UPDATE: March 21, 2012
Oh, hai.
I’ll be migrating this to mod_wsgi and changing this entire page. to reflect that.
Why? Because I’m also going to migrate a moinmoin instance to be hosted on the same apache instance. Also, having the repos accessible by apache:apache will make it easy to deal with.

---

This is a rabbit eating lettuce, which is green like celery.

Install python setuptools
Go to http://pypi.python.org/pypi/setuptools#downloads and find the latest release for python 2.6

cd
wget http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg#md5=bfa92100bd772d5a213eedd356d64086
sh setuptools-0.6c11-py2.6.egg
rm -f setuptools-0.6c11-py2.6.egg

Configure a python virtual environment

easy_install virtualenv virtualenvwrapper

use virtualenvs for our paste user:

su paste
mkdir /home/paste/.virtualenvs
exit

Modify the startup script for each shell instance for paste

su paste
echo >> /home/paste/.bashrc
echo export WORKON_HOME=\$HOME/.virtualenvs >> /home/paste/.bashrc
echo source /usr/bin/virtualenvwrapper.sh >> /home/paste/.bashrc
source /home/paste/.bashrc
#some virtual environment should be created /home/paste/.virtualenvs
ls -l -a /home/paste/.virtualenvs/
exit

Rhodecode Install

We are now moving onto the Rhodecode Installation

mkdir -p /var/www/rhodecode-venv /var/www/rhodecode
chown paste:paste /var/www/rhodecode-venv /var/www/rhodecode
su paste
cd #this is super critical :D
virtualenv --no-site-packages /var/www/rhodecode-venv #create the virtual environment for rhodecode
/var/www/rhodecode-venv/bin/easy_install --prefix=/var/www/rhodecode-venv rhodecode #install rhodecode accessible through the virtualenv
exit

tip: Check out Quick Primer: python virtual environments to learn more.

Installing celery and rabbitmq
Installing celery and rabbitmq will improve performance, but all will work without that. The suggestion is: “if you plan to use RhodeCode for say 7 to 10 repositories, RhodeCode will perform perfectly well without celery running.”

Whatever that vague statement means, disregard. Let’s go ahead and install celery along with rabbitmq.  Remember “if you make the decision to run RhodeCode with celery make sure you run celeryd using paster and message broker together with the application.”  This is because the paste script celeryd pulls the celeryd config from the paste config file.  We’ll get to this later.

Install celery
Following the celery install process

/var/www/rhodecode-venv/bin/easy_install --prefix=/var/www/rhodecode-venv Celery #install celery into the virtualenv

Install RabbitMQ server
Onto RabbitMQ installation: in relation to celery broker installation

Following rabbitmq server installation information

Install Erlang prerequisite for rabbitmq-server
According to the RabbitMQ docs

echo [fedora_el6] >> /etc/yum.repos.d/fedora_el6.repo #allow yum access to the fedora repo
echo name=fedora_el6 >> /etc/yum.repos.d/fedora_el6.repo
echo baseurl=http://download1.fedora.redhat.com/pub/epel/\$releasever/\$basearch/ >> /etc/yum.repos.d/fedora_el6.repo #the url might have changed
echo enabled=1 >> /etc/yum.repos.d/fedora_el6.repo
echo skip_if_unavailable=1 >> /etc/yum.repos.d/fedora_el6.repo
echo gpgcheck=0 >> /etc/yum.repos.d/fedora_el6.repo
yum -y install erlang #install erlang
sed 's/^enabled=1/enabled=0/' -i /etc/yum.repos.d/fedora_el6.repo && rm -f /etc/yum.repos.d/fedora_el6.repo # Disable fedora repo

Install rabbitmq-server
Use the RPM for Fedora: (Thanks Hussain http://hussainanjar.com/2011/04/installing-rabbitmq-on-centos5/)

cd
wget http://www.rabbitmq.com/releases/rabbitmq-server/v2.6.1/rabbitmq-server-2.6.1-1.noarch.rpm
yum -y --nogpgcheck localinstall ./rabbitmq-server-2.6.1-1.noarch.rpm
rm -f rabbitmq-server-2.6.1-1.noarch.rpm

Add rabbitmq-server to chkconfig db so that it came be easily managed to start on system boot

chkconfig --add rabbitmq-server
chkconfig --level 345 rabbitmq-server on
service rabbitmq-server start

check some things:

service rabbitmq-server status
netstat -apn | grep $(cat /var/run/rabbitmq/pid)

Configuring RabbitMQ server

rabbitmqctl add_user celeryuser RANDOMPASSWORD
rabbitmqctl add_vhost celeryvhost
rabbitmqctl set_permissions -p celeryvhost celeryuser ".*" ".*" ".*"

There is a note to NEVER use kill to stop RabbitMQ server, so use:

service rabbitmq-server stop

Finally on to RhodeCode
With reference to the Rhodecode setup documentation.

Install some python modules, including python-ldap, into your virtualenv
To have Rhodecode leverage LDAP for authentication, we’ll install python-ldap.

yum -y install openldap openldap-clients openldap-devel openssl-devel #horray, openldap-2.4.19 is already installed!
/var/www/rhodecode-venv/bin/easy_install --prefix=/var/www/rhodecode-venv pyasn1 pyasn1-modules
/var/www/rhodecode-venv/bin/easy_install --prefix=/var/www/rhodecode-venv python-ldap #ignore the various (not found) errors, a long as it builds "Instaled ...python_ldap...egg"

LDAP configuration takes place in the Rhodecode web UI, which will be covered later.

Configure RhodeCode and celery

su paste
/var/www/rhodecode-venv/bin/paster make-config RhodeCode /var/www/rhodecode-venv/production.ini
vim /var/www/rhodecode-venv/production.ini

set the following values in the paste config file:

email_to
error_email_from
app_email_from
smtp_server
force_https = false # we'll deal with this through apache ([11:36]  [the force_https] flag is for redirections and url for cloning)
use_celery = true
broker.vhost = celeryvhost
broker.user = celeryuser
broker.password = RANDOMPASSWORD
celery.send.task.error.emails = true
#celeryd.log.file = celeryd.log #we'll take care of logging via stdout of the paste instance that executes celeryd
celeryd.log.level = debug #leave this value as is, we'll set the value to WARNING at run time

Then exit vim and back to root:

exit

If you’re super interested, check out info on python paste config files here:
Paste: config file
paste.httpserver

Configure the root for the repos

mkdir /reporoot #make a directory to your repo root
chown paste:paste /reporoot/

You can copy existing repos over to this location, and rhodecode will learn of their existence and add them to it’s database when it starts.

Setup your sql database and admin user
The following is the database configuration script.

su paste
/var/www/rhodecode-venv/bin/paster setup-app -q /var/www/rhodecode-venv/production.ini
# Destory the DB
# enter /reporoot as the repo root
# enter an admin username, password
# enter the admin email etc.
exit #to get back to root

Create an init script for paster script celeryd

To support chkconfig properly, you must change the chkconfig line in the following init script.
the syntax is as follows:

chkconfig [run levels] [execution order at startup, S] [kill order at shutdown, K]

1) We’ll be using this at runlevel 5.
2) We’ll want to start it after rabbitmq-server starts.
3) We’ll kill it before rabbitmq-server dies.

cat /etc/init.d/rabbitmq-server | grep chkconf
find / -iname K05rabbitmq-server
find / -iname S80rabbitmq-server

From that info, here is a init script for paste_script_celeryd:
I’m very much interested in using the daemon() function of /etc/init.d/functions; but won’t be doing that for now.

vim /etc/init.d/paste-script-celeryd

#!/bin/sh
#
#
#
#
# chkconfig: 345 81 04
# description:
#init script for paste script celeryd, which was written by
#branded on freenode
#mbrownnyc everywhere else
#hacky init.d script for celeryd paste script
#

### BEGIN INIT INFO
# Provides:          paste-script-celeryd
# Required-Start:    $remote_fs $network
# Required-Stop:     $remote_fs $network
# Description:       Paste Script to launch celeryd reading config out of a paste config file
### END INIT INFO

  start()
  {
        if [ ! -f /var/run/paste/celeryd.pid ]; then #process isn't running probably
                echo Attempting to start celeryd paste script...  forking it up with \&
                su -c '/var/www/rhodecode-venv/bin/paster celeryd /var/www/rhodecode-venv/production.ini --pidfile=/var/run/paste/celeryd.pid -f /var/log/celerydpaster.log -l WARNING -q &' paste
                while [ ! -f /var/run/paste/celeryd.pid ]; do sleep 1 && echo .; done
                echo Done. PID is $(cat /var/run/paste/celeryd.pid).
        else
                echo The celeryd paste script has filed a paste PID $(cat /var/run/paste/celeryd.pid)
                echo It\'s likely started already at that PID.
        fi
  }
  stop()
  {
         if [ -f /var/run/paste/celeryd.pid ]; then #process is running probably
                echo sending SIGINT to the process $(cat /var/run/paste/celeryd.pid) to trigger a "Warm shutdown"
                pypasteceleryPID=$(cat /var/run/paste/celeryd.pid)
                kill -s INT $pypasteceleryPID
                echo Waiting for process to die...
                #umm... yea! http://stackoverflow.com/questions/696839/how-do-i-write-a-bash-script-to-restart-a-process-if-it-dies
                while [ -f /var/run/paste/celeryd.pid ]; do sleep 1 && echo .; done
                echo Truncating /var/log/celerydpaster.log to entries from last run only.
                sed -i '1,'$(expr $(grep -n -e "^----\s\*\*\*\*\s-----" /var/log/celerydpaster.log | cut -d':' -f1 | awk '$1 > max { max=$1 }; END { print max }') - 1)'d' /var/log/celerydpaster.log
        else
                echo /var/run/paste/celeryd.pid doesn\'t exist, so the celeryd paste script was not used to execute celeryd if celeryd is actually running.
                echo Otherwise it isn\'t running.
        fi

  }
  restart()
  {
        stop
        start
  }
  status()
  {
       if [ -f /var/run/paste/celeryd.pid ]; then
                echo celeryd paste script is running as PID $(cat /var/run/paste/celeryd.pid)
                tail -n 20 /var/log/celerydpaster.log
        else
                echo celeryd paste script isn\'t running.
        fi

  }

case "$1" in
    start)
                start
    ;;
    stop)
                stop
    ;;
    restart)
                stop
                start
    ;;
    status)
                status
    ;;
    *)
                echo $"Usage: $0 {start|stop|restart|status}"
        ;;
esac
exit 1
[esc]wq![enter]

Allow the script to be executable:

chmod +x /etc/init.d/paste-script-celeryd

Then add it to the chkconfig db and check status

chkconfig --add paste-script-celeryd
chkconfig --level 345 paste-script-celeryd on
mkdir -p /var/run/paste/
touch /var/log/celerydpaster.log
chown paste:paste /var/log/celerydpaster.log /var/run/paste/

find / -iname S81paste-script-celeryd
find / -iname K04paste-script-celeryd

service rabbitmq-server start #start rabbitmq
service paste-script-celeryd start #start celeryd
ls -l /proc/ | grep $(cat /var/run/paste/celeryd.pid) # the owner should be

Create an init script for paste serve

vim /etc/init.d/paste-serve-rhodecode

#!/bin/sh
#
#
#
#
# chkconfig: 345 82 03
# description:
#init script for paste serve rhodecode, which was written by
#branded on freenode
#mbrownnyc everywhere else
#

### BEGIN INIT INFO
# Provides:          paste-serve-rhodecode
# Required-Start:    $remote_fs $network
# Required-Stop:     $remote_fs $network
# Description:       Paste httpserver to serve rhodecode over http
### END INIT INFO

  start()
  {
        if [ ! -f /var/run/paste/rhodecode.pid ]; then #process isn't running probably
                echo Attempting to start paste serve rhodecode...  daemonizing
                su -c '/var/www/rhodecode-venv/bin/paster serve /var/www/rhodecode-venv/production.ini --daemon --user=paste --pid-file=/var/run/paste/rhodecode.pid --log-file=/var/log/pasteserve.log' paste
                while [ ! -f /var/run/paste/rhodecode.pid ]; do sleep 3 && echo .; done
                echo Done. PID is $(cat /var/run/paste/rhodecode.pid).
        else
                echo The paste serve rhodecode has filed a paste PID $(cat /var/run/paste/rhodecode.pid)
                echo It\'s likely started already at that PID.
        fi
  }
  stop()
  {
        if [ -f /var/run/paste/rhodecode.pid ]; then #process is running probably
                echo Telling daemonized paste httpserve process to die
                su -c '/var/www/rhodecode-venv/bin/paster serve --stop-daemon --pid-file=/var/run/paste/rhodecode.pid' paste
                echo Waiting for process to die...
                while [ -f /var/run/paste/rhodecode.pid ]; do sleep 1 && echo .; done
        else
                echo /var/run/paste/rhodecode.pid doesn\'t exist, so the paste httpserve process was not used to executed using the proper config, if paste is running.
                echo Otherwise it isn\'t running.
        fi
  }
  restart()
  {
        stop
        start
  }
  status()
  {
        if [ -f /var/run/paste/rhodecode.pid ]; then
                echo paste httpserve process is running as PID $(cat /var/run/paste/rhodecode.pid)
                su -c '/var/www/rhodecode-venv/bin/paster serve /var/www/rhodecode-venv/production.ini --status --pid-file=/var/run/paste/rhodecode.pid' paste
        else
                echo paste serve rhodecode isn\'t running.
        fi

  }

case "$1" in
    start)
                start
    ;;
    stop)
                stop
    ;;
    restart)
                stop
                start
    ;;
    status)
                status
    ;;
    *)
                echo $"Usage: $0 {start|stop|restart|status}"
        ;;
esac
exit 1
[esc]wq![enter]

Allow the script to be executable:

chmod +x /etc/init.d/paste-serve-rhodecode

Then add it to the chkconfig db:

chkconfig --add paste-serve-rhodecode
chkconfig --level 345 paste-serve-rhodecode on

Then touch and chown the log, start the service and review status:

touch /var/log/pasteserve.log
chown paste:paste /var/log/pasteserve.log
service paste-serve-rhodecode start
ls -l /proc/ | grep $(cat /var/run/paste/rhodecode.pid) # the owner should be paste