WF to Opalstack (Django and Postgres). Where to begin?

cjrw

Hello, we have around 25 servers hosting 2-8 apps each and are looking to migrate by the deadline. We have limited personnel to dedicate to this task and need to find the most efficient method possible. We're mainly a PHP shop now and haven't had any experience migrating Django apps, only maintaining.

Should we begin with a new server setup and test the migration script when it has been prepared? Should we look at deploying from our repository? Should we export the Postgres db using phpPgAdmin over dumping data, which has often been problematic in my experience? How long will the Python 2.7 apps continue to run?

So many questions, so little time. Thank you.

sean

Manually migrating a Django app isn't difficult. We'll have most of it automated soon, but the basic steps are:

Create a Django app here at Opalstack: https://help.opalstack.com/article/61/installing-django
Create your database here at Opalstack: https://help.opalstack.com/article/51/managing-databases
Export your database from the old host and import it here (I recommend using the command line method): https://help.opalstack.com/article/82/-exporting-and-importing-postgresql-databases
Install your app's dependencies into the app's virtualenv: https://help.opalstack.com/article/60/pythonuwsgi-applications#installing-python-dependencies-for-your-project
Don't forget your database libraries: https://help.opalstack.com/article/68/using-databases-with-django
Copy your Django project directory (or pull it from your repo) into the app directory and configure uwsgi to use it: https://help.opalstack.com/article/66/serving-your-own-django-project
Set up your static content: https://help.opalstack.com/article/67/serving-static-content-for-your-django-project
Hook the Django app and static app up to a site: https://help.opalstack.com/article/52/managing-sites

If you need the app to use Python 2.7 then you can convert the virtualenv over to Python 2.7 by running the following commands before performing step 4 above:

export PATH=$HOME/.local/bin:$PATH
pip2.7 install --user -U pip==20.3.4
pip2.7 install --user  virtualenv
cd ~/apps/appname
mv env env.old
virtualenv --python=python2.7 env
source env/bin/activate
pip install uwsgi django 
# specify the django version if you need an older version, eg django==X.Y
# older versions need to be installed like this:
# pip install --ignore-installed --no-binary django Django==X.Y
./stop
./start

We will have Python 2.7 installed system-wide for as long is it is available in our OS package repositories. If it's going to be removed, then we'll give you plenty of notice so that you can either upgrade (come on, do it already 😆 ) or build your own install of Python 2.7 to keep your app running.

kwhelan

sean On your script install the "pip install uwsgi django" does it make sense if I need django 1.76 to do the install as "pip install uwsgi django==1.7.6" the command seems to work fine

cjrw

Wow Sean! Thank you for providing that information. We appreciate it.

sean

kwhelan yes, if you need a specific version of Django and that version is available in pip, then specifying the version when you run pip is the correct thing to do.

Parthian

Is there a auto-migrate process for Django yet. Don't see it here - https://community.opalstack.com/d/155-howto-migrate-stuff-automatically-from-webfaction

Nearly done with the migrates. Worked well (although I've still to do the Domain DNS and some sites are hard to test).
Will probably do my small Django site manually. Lots of PHP sites, Emails (did the dragndrop approach) and a Django on another account. Can't test my MODx sites, tried the subdomain approach test.username.opalstacked.com but no joy, Probably a bit like the WP Config Host path etc. Will just wait until I change the nameservers over.

sean

Parthian No Django auto-migrations yet. Those are going to be tricky for a few reasons:

Django's changed a lot over the years and there are some really old Django sites still running at WF.
Django's a framework with a suggested project layout, but a lot of people abandon those suggestions and do their own thing so we have no way to predict what someone is doing without getting hands-on.
Probably many other reasons 🙂

But we'll do what we can!

sean

Parthian Can't test my MODx sites, tried the subdomain approach test.username.opalstacked.com but no joy

Not sure why that wouldn't work, feel free to reach out to support with the details if you like.

Parthian

Thanks Sean, it's all going great.
RE- MODx. I abandoned the subdomain approach, as I was ready to update the DNS to go live anyway (lots of sites on the same domain plus email). All ducks have to be in a row, kind of thing. Reasonably smooth after an evening to propogate.
What I've found is that many of my PHP/Static type sites didn't work until I added LetsEncrypt. Probably due to htaccess re-directing http to https. All going very well now.

Yes, Django is tough. Lots of uwsgi and settings stuff. The help Guide is handy (pink ?).

I've just discovered an oddity. I'm a GeoDjango user, and I don't see a way on Adminer to add PostGIS.
And it looks like the Migration Tool doesn't copy over the PostGIS functions and tables. Webfaction creation of a Postgresql database has buttons for tsearch and postgis.
I note from another Post that users have to contact Support with the Database name, I'll do that but I think any table that references PostGIS Points etc doesn't get copied over with the migrate tool. This is just a guess, but the only table missing from my migrate is one with geo-points.
May be worth updating the Migrate forum post that Django isn't included and that GeoDjango users will need to take additional steps 1. contact support to add PostGIS and 2. manually copy over Geo enabled Tables - probably after support has added PostGIS.
Ta.

sean

Parthian migrating PostGIS and other extensions is a planned update.

For now, yes if you have PostGIS or some other extension installed then you'll need to re-import that DB manually after the migration, eg:

Export your DB at WF and copy it over to your Opalstack server.
Delete and re-create the DB on our end.
Contact support to have the extensions enabled.
Import your data on our end.

The migration procedure above will be moved to its own HOWTO as soon as I get a chance, and I'll be sure to mention the GeoDjango stuff if we don't have it solved in the migrator by then. Thanks for pointing it out!

Parthian

Smashing. Adminer doesn't seem to give access to features like Functions/Views/Triggers. Any plans to use phpPGAdmin or similar?

sean

Parthian Nope, but if you'd like to install your own phpPgAdmin as a custom PHP app you can do that.

Adminer does let you enter arbitrary SQL, so you should be able to do anything your DB user has permission to do.

You also have access to the psql command-line client and access for desktop GUI sql clients via SSH tunnel.

sean

Just wanted to share a recent experience: today I helped a customer who copied their Apache+mod_wsgi Django app from WebFaction get their app running with very little difficulty!

All that was needed was:

Create a proxy port app to get a port assignment.
Edited the apps httpd.conf:
- to set the port number to the new port assignment
- to set the various paths for logs, python path, and WSGI handler to match the filesystem paths on Opalstack
Edited the start/stop scripts to match the filesystem paths on Opalstack

Once that was done, the migrated Apache started up and the app was live!

WF's various versions of Django and mod_wsgi use a lot of different bundles of pre-compiled software, and I'm certain that some of them won't run on our platform. I doubt that every WF Django app can be copied over and run this easily, but if you're migrating and you're keen to stay on your familiar Apache+mod_wsgi then please give it a shot. 🙂

sdegrace

I'm using a new VPS, when I try and do the

virtualenv -p /bin/python2.7 env

step, I get an error:

ImportError: No module named zipp

I am also continually getting warnings like this form pip:

You are using pip version 8.1.2, however version 20.2.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

But that command gives me "permission denied," and my shell user cannot sudo and I don't see a way to give them that ability.

I feel like if I can figure it out for this one Django site, the rest should come over more easily!

sean

sdegrace I've updated the instructions above with the latest steps available in our official docs.

sdegrace

sean
Your response completely fixed my problem, thank you!

If I would have just one suggestion, you will have to install the mysql or postgres client as a necessary step as part of installing the packages you are using. If you added the instructions for that to the tutorial, it would remove a smidge of additional learning curve. Overall, though, this has been a great experience and I'm glad I chose Opalstack to replace Webfaction as my host!

sean

sdegrace I added a step with a link to the relevant docs.