HOWTO: run Django on Apache and mod_wsgi

sean

UPDATE: We now have an application installer which provides build scripts for apache, mod_Wsgi, and django on python2.7. It is named 'Django 1.8 LTS' in the new application type list, it coverts the process outlined here into an automated process.

Opalstack's built-in Django installer sets up Django running behind a uwsgi server.

If you prefer to use Apache and mod_wsgi instead of uwsgi, here's how:

First create a proxy port app, make a note of the app name and port, and attach the app to a site.

In the rest of this example, the app is myapp and the port is 55555.

Next, install Apache HTTPD and mod_wsgi:

export APP=myapp
cd ~/apps/$APP
mkdir -p src tmp
export TMPDIR=$PWD/tmp
cd src
wget http://archive.apache.org/dist/httpd/httpd-2.4.41.tar.gz
tar zxf httpd-2.4.41.tar.gz
cd httpd-2.4.41
./configure --prefix=$HOME/apps/$APP/apache2 --enable-mods-shared=all --enable-mpms-shared=all --with-mpm=prefork
make
make install
cd ..
wget https://github.com/GrahamDumpleton/mod_wsgi/archive/4.7.0.tar.gz
tar zxf 4.7.0.tar.gz
cd mod_wsgi-4.7.0
./configure --with-python=/usr/bin/python3.6 --with-apxs=$HOME/apps/$APP/apache2/bin/apxs
make
make install
cd ../..
cp apache2/conf/httpd.conf apache2/conf/httpd.conf.original

Next, make a virtual environment for your project. In this example I'll install Django and make a new project:

cd $HOME/apps/$APP
python3.6 -m venv env
source env/bin/activate
pip install django
django-admin startproject myproject
sed -r -i "s/^ALLOWED_HOSTS = \[\]/ALLOWED_HOSTS = \['\*'\]/" myproject/myproject/settings.py
sed -r -i "/^DATABASES =/, /^}$/ s/^/#/" myproject/myproject/settings.py

Next, configure Apache to listen on your assigned port and serve your project. To do so, edit the app's apache2/conf/httpd.conf to look like this, changing the first four variables to match your user and app:

# change the next four values for your shell user and app
Define OPAL_USER myuser
Define APP_NAME myapp
Define APP_PORT 55555
Define PROJ_NAME myproject

# you usually don't need to edit anything below this line.
Define APP_ROOT /home/${OPAL_USER}/apps/${APP_NAME}
Define VIRT_ENV ${APP_ROOT}/env
Define PROJ_ROOT ${APP_ROOT}/${PROJ_NAME}
Define LOG_ROOT /home/${OPAL_USER}/logs/apps/${APP_NAME}

ServerRoot ${APP_ROOT}/apache2

LoadModule mpm_worker_module modules/mod_mpm_worker.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule dir_module        modules/mod_dir.so
LoadModule env_module        modules/mod_env.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule mime_module       modules/mod_mime.so
LoadModule rewrite_module    modules/mod_rewrite.so
LoadModule setenvif_module   modules/mod_setenvif.so
LoadModule wsgi_module       modules/mod_wsgi.so
LoadModule unixd_module      modules/mod_unixd.so

LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog ${LOG_ROOT}/access_${APP_NAME}.log combined
ErrorLog ${LOG_ROOT}/error_${APP_NAME}.log

DirectoryIndex index.py
DocumentRoot ${APP_ROOT}/apache2/htdocs

Listen ${APP_PORT}
KeepAlive Off
SetEnvIf X-Forwarded-SSL on HTTPS=1
ServerLimit 1
StartServers 1
MaxRequestWorkers 5
MinSpareThreads 1
MaxSpareThreads 3
ThreadsPerChild 5

# python-home = path to your virtualenv
# python-path = path to your project directory
# this is usually all of the python path config that you need.
WSGIDaemonProcess ${APP_NAME} processes=2 threads=12 python-home=${VIRT_ENV} python-path=${PROJ_ROOT}
WSGIProcessGroup ${APP_NAME}
WSGIRestrictEmbedded On
WSGILazyInitialization On
WSGIScriptAlias / ${PROJ_ROOT}/${PROJ_NAME}/wsgi.py

Finally, start the app:

~/apps/myapp/apache2/bin/apachectl start

And that's it - you now have Django up and running on Apache + mod_wsgi with Python 3.6!

stefan

Thanks for this very useful tutorial, Sean! It helped me understand a thing or two about how Apache works.

One thing that's different from Webfaction though (since we'd install Apache ourselves) is that we can't write logs to ~/logs as non-root users. Would OpalStack give us access to that folder? For now I decided to simply put the logs inside the APP_ROOT.

sean

stefan We can't give you direct access to ~/logs, but you should be able to put your application logs or whatever in ~/logs/appname.

If that's not working for you then please email support so that we can fix it.

If you keep your logs anywhere else then you'll need to manage the log rotation yourself to avoid using all of your disk space.

stefan

sean Oh, I see now where the problem was. In your tutorial you should change

Define LOG_ROOT /home/${OPAL_USER}/logs/${APP_NAME}

Define LOG_ROOT /home/${OPAL_USER}/logs/apps/${APP_NAME}

Apache couldn't write to ~/logs/, but can write to ~/logs/apps/.

sean

stefan my bad! Fixed 🙂

tofias

A quirk of Apache url handling breaks the top script. You need to point to the archived version if you want a stable URL e.g., http://archive.apache.org/dist/httpd/httpd-2.4.41.tar.gz I don't see any affordance of linking to a "latest" build, but I must admit I didn't look very carefully.

sean

tofias thanks, fixed!

joshuccr

Thanks a lot, I have a question.

Our company have a lot of Django projects that runs with apache and mod_wsgi.
A lot of them use Django 1.X and python 2.7
We tried to setup 3 django apps with this tutorial changed some things, but We haven't achived to run yet.

First, we change this:
./configure --with-python=/usr/bin/python3.6 --with-apxs=$HOME/apps/$APP/apache2/bin/apxs

For that
./configure --with-python=/usr/bin/python2.7 --with-apxs=$HOME/apps/$APP/apache2/bin/apxs

And replace python3.6 for python2.7 using that:

export PATH=$HOME/.local/bin:$PATH
pip2.7 install --user -U pip
pip2.7 install --user virtualenv
cd ~/apps/appname
mv env env.old
virtualenv -p /bin/python2.7 env
source env/bin/activate

When we tried to install some dependances with pip we have some problems.

Can you help us in how to set up Django projects with python 2.7 for deploying with Apache?

Thanks, a lot.

sean

joshuccr If you're migrating a reasonably modern Django+mod_wsgi app from WF then there's a good chance that it will work without building a new Apache yourself.

In most cases you can get them running with a few changes:

Make a Python 2.7 environment for your Python dependencies: Python Virtual Environments | Opalstack Support Center
Install your Python dependencies into the new environment.
Adjust your various scripts and config to adapt them to Opalstack:
- Change webapps to apps in start, stop, restart, and httpd.conf
- Change your WF username to your Opalstack shell username in start, stop, restart, and httpd.conf (if the shell username is different).
- Change the Listen port assignment in httpd.conf to your new app's port assignment.
- Change logs/user to logs/apps/appname in httpd.conf.
- Change your various Python paths in httpd.conf and anywhere else you set them to point to your new Python environment.

I've done about a dozen of these and 90% of the time these steps will get you 90% of the way. Any remaining issues are usually no different than setting up a new app from scratch. Give it a try!

joshuccr

sean Thanks a lot Sean, we are trying it.

joshuccr

sean We trying without building a new Apache, We imported the app from Webfaction and we made the changes.

The structure that App had in Webfaction was:

~/webapps/project:
  --~/webapps/project/apache2
  -- ~/webapps/project/bin
   -- ~/webapps/project/lib
    -- ~/webapps/project/django_project1
    -- ~/webapps/project/django_project1

When we ran the follow commands we got this:

~/apps/myapp/apache2/bin/ start  -bash:: command not found
 -bash:  ~/apps/myapp/apache2/bin/: Is a directory
-bash: -bash:: command not found
 ~/apps/myapp/apache2/bin/apachectl start
-bash: : command not found
  ~/apps/myapp/apache2/bin/apachectl: No such file or directory
-bash: -bash:: command not found
~/apps/myapp/apache2/bin/apachectl: No such file or directory

Our wefaction mod_wsgi =3.4.

Sorry, but we don't know how to migrate correctly.

sean

joshuccr you've got a space in your command before start: ~/apps/myapp/apache2/bin/ start

Try it like this: ~/apps/myapp/apache2/bin/start

joshuccr

sean I tried and obtain this:

-bash: /home/opaluser/apps/myapp/apache2/bin/start: Permission denied

sean

joshuccr do chmod u+x ~/apps/myapp/apache2/bin/* and try again.

joshuccr

sean Thank you very much Sean.
Apache server is running, but we checked and we obtained a 500 error.

When we checked logs we saw this:

[Tue Dec 15 23:05:13.939600 2020] [wsgi:error] [pid 129026:tid 140655410722560] [remote ::1:40646] from zinnia.models.entry import Entry
[Tue Dec 15 23:05:13.939604 2020] [wsgi:error] [pid 129026:tid 140655410722560] [remote ::1:40646] File "/home/{OpalUser}/apps/{myapp}/lib/python2.7/django_blog_zinnia-0.12.3-py2.7.egg/zinnia/models/entry.py", line 24, in <module>
[Tue Dec 15 23:05:13.939612 2020] [wsgi:error] [pid 129026:tid 140655410722560] [remote ::1:40646] from zinnia.models.category import Category
[Tue Dec 15 23:05:13.939616 2020] [wsgi:error] [pid 129026:tid 140655410722560] [remote ::1:40646] File "/home/{OpalUser}/apps/{myapp}/lib/python2.7/django_blog_zinnia-0.12.3-py2.7.egg/zinnia/models/category.py", line 5, in <module>
[Tue Dec 15 23:05:13.939624 2020] [wsgi:error] [pid 129026:tid 140655410722560] [remote ::1:40646] from mptt.models import MPTTModel
[Tue Dec 15 23:05:13.939635 2020] [wsgi:error] [pid 129026:tid 140655410722560] [remote ::1:40646] ImportError: No module named mptt.models

When we checked the dependances at
/home/{OpalUser}/apps/{myapp}/lib/python2.7
we saw that we have that dependances:

django_blog_zinnia-0.12.3-py2.7.egg
django_mptt-0.5.4-py2.7.egg-info

PD: Sorry for my explanation I don't use to write in english, my mother tongue is spanish.

sean

joshuccr WebFaction did a lot of weird non-standard stuff with Python search paths so your default library setup isn't going to work here.

The first two steps in the procedure are intended to cover that:

Make a Python 2.7 environment for your Python dependencies: Python Virtual Environments | Opalstack Support Center

Install your Python dependencies into the new environment.

You'll need to install any packages that were in your appname/lib/python2.7 and in your ~/lib/python2.7 directory at WF.

joshuccr

sean Thanks a lot. It is very hard for us to migrate all django projects, because have the same Webfaction Structure in all of them.
Sean, we have already created python 2.7 virtual environment.
Is it possible to move with filezilla all dependances that we have in appname/lib/python2.7 and ~/lib/python2.7, directly in _env/lib/python2.7/site-packages?

The projects don't have a requirements.txt file for use pip.

sean

joshuccr no, sorry - they must be installed with pip. If you're not sure how to do this feel free to email support so that we can look at exactly what you have.

joshuccr

sean Sorry, Sean I checked the backups again and we have two requiriments files :
/appname/lib/python2.7
~/lib/python2.7
We are going to try to install using pip.

And again, Thank you a lot.

sean

joshuccr You're very welcome!

joshuccr

sean Thanks 😃

Well we created a virtual env and install all dependances of both projects and we obtained this error when we run Apache2 (we changed in httpd.conf this:
/home/{OpalUser}/apps/{myapp}/lib/python2.7 -> /home/{OpalUser}/apps/{myapp}/env)

[Wed Dec 16 01:01:53.723176 2020] [wsgi:error] [pid 240845:tid 140655410722560] [remote ::1:41240] from django.core.wsgi import get_wsgi_application
[Wed Dec 16 01:01:53.723196 2020] [wsgi:error] [pid 240845:tid 140655410722560] [remote ::1:41240] ImportError: No module named django.core.wsgi
[Wed Dec 16 01:01:53.816825 2020] [wsgi:error] [pid 240845:tid 140655263209216] [remote 127.0.0.1:49356] mod_wsgi (pid=240845): Target WSGI script '/home/ccr/apps/django_ccr_apache/ccr-web/wsgi.py' cannot be loaded as Python module.
[Wed Dec 16 01:01:53.816866 2020] [wsgi:error] [pid 240845:tid 140655263209216] [remote 127.0.0.1:49356] mod_wsgi (pid=240845): Exception occurred processing WSGI script '/home/{OpalUser}/apps/{myapp}/{myproject}/wsgi.py'.
[Wed Dec 16 01:01:53.816915 2020] [wsgi:error] [pid 240845:tid 140655263209216] [remote 127.0.0.1:49356] Traceback (most recent call last):
[Wed Dec 16 01:01:53.816964 2020] [wsgi:error] [pid 240845:tid 140655263209216] [remote 127.0.0.1:49356] File "/home/ccr/apps/django_ccr_apache/ccr-web/wsgi.py", line 23, in <module>
[Wed Dec 16 01:01:53.817029 2020] [wsgi:error] [pid 240845:tid 140655263209216] [remote 127.0.0.1:49356] from django.core.wsgi import get_wsgi_application
[Wed Dec 16 01:01:53.817071 2020] [wsgi:error] [pid 240845:tid 140655263209216] [remote 127.0.0.1:49356] ImportError: No module named django.core.wsgi

We didn't install this dependance in our requiriment file.