End-of-life for Opalstack API v0 and the WebFaction migration tool

‼️ If your WebFaction account is now closed then the automatic migration procedures below are not going to work for you. You'll need to create your apps, databases, domains etc manually via your Opalstack dashboard and set up your sites manually.

As we've mentioned on @opalstack and elsewhere in the forum here we've been building a suite of migration tools to help incoming WF customers move their stuff over to our service.

I'm happy to say that we've deployed the migration stuff across the fleet and the following migrations can be performed automatically!

Wordpress sites (including the app, DB, DB user, and domains)
Generic PHP apps
Generic port listeners (basically anything that listens on a port at WF: Django, Rails, "custom app listening on port", websockets etc)
MySQL databases and users (MariaDB on our side)
PostgreSQL databases and users
Email addresses and mailboxes (more info)

The migration tool is wrapped up in a command called wfmigrate.

Example: Migrating a Wordpress site

Here's a quick example of how to use it to migrate a WP site:

Create a shell user via your Opalstack dashboard if you don't have one already: https://docs.opalstack.com/user-guide/shell-users/
Go to https://my.opalstack.com/tokens/ and create a new API token if you don't have one already. Make a note of the token key (not its name) as you will need that in your migration config later. The key is the long sequence of characters in the second column on the token page in the dashboard.
SSH to your Opalstack server as your shell user: https://docs.opalstack.com/user-guide/server-access/#ssh-access
Make a copy of our boilerplate config file for the WP migrator (Do this every time to ensure that you're using the latest configuration required by the migrator):
```
cp /opt/app_migrators/wf_wordpress/config-template.ini ./config.ini
```
The boilerplate config files are located at:
- /opt/app_migrators/wf_generic_dns/config-template.ini
- /opt/app_migrators/wf_generic_domains/config-template.ini
- /opt/app_migrators/wf_generic_email/config-template.ini
- /opt/app_migrators/wf_generic_maildir/config-template.ini
- /opt/app_migrators/wf_generic_maria/config-template.ini
- /opt/app_migrators/wf_generic_php/config-template.ini
- /opt/app_migrators/wf_generic_php_symlink/config-template.ini
- /opt/app_migrators/wf_generic_port/config-template.ini
- /opt/app_migrators/wf_generic_psql/config-template.ini
- /opt/app_migrators/wf_generic_staticonly/config-template.ini
- /opt/app_migrators/wf_generic_staticonly_symlink/config-template.ini
- /opt/app_migrators/wf_wordpress/config-template.ini
Edit the config file config.ini that you just created to plug in your Opalstack info and WF info. Note the following:
- The value for apitoken is the key (not the name) of the token you created in in step 2.
- The server for your WebFaction info is the short server hostname, like webNNN or wf-NN-NN-NN-NN. Do not include ".webfaction.com" in the server name.

Run a simulation to do a pre-migration check. This will ensure that you don't have any conflicting items already created on Opalstack:

wfmigrate wordpress ./config.ini --mode simulate

You'll see a lot of output go by as wfmigrate inventories all of your items and simulates the migration. There are couple of things to look out for:
- WARNING messages will appear if the simulation encounters an item on your Opalstack server that has the same name as the item on your WebFaction server. Such items will be deleted and replaced when you run the live migration in the next step, so pay attention here! If there's something you don't want to delete, back it up before you run the live migration.
- CRITICAL messages will appear if the simulation runs into a problem that prevents it from performing the migration, like a bad password, a SSH key that can't be set, etc. The simulation will terminate at that point.

Once you've run a successful simulation you can perform the live migration with the do mode option:

wfmigrate wordpress ./config.ini --mode do

wfmigrate will then run to create all of the necessary items in your Opalstack account and move everything from WF over to our side. All of the above info about WARNING and CRITICAL messages applies to this step as well.

That's pretty much it!

Tip: if you notice a database connection error after migrating WP, then check your DB_PASS in wp-config.php. If the migrated password has multiple sets of quotation marks around it then correct that, eg change `""password123"" to "password123". If you think the password might be incorrect, then reset it and put the correct password in your wp-config.php.

The other items work similarly, and you can run wfmigrate --help to see all available options:

$ wfmigrate --help
usage: wfmigrate [-h]
                 {account,domains,dns,maria,psql,email,email-data,php,php-symlink,port,staticonly,staticonly-symlink,wordpress}
                 ...

WF -> Opalstack Migrator

positional arguments:
  {account,domains,dns,maria,psql,email,email-data,php,php-symlink,port,staticonly,staticonly-symlink,wordpress}
                        Migration Component
    account             Account (config builder)
    domains             Domains
    dns                 DNS Overrides
    maria               MariaDB (for MySQL databases)
    psql                PostgreSQL (for PostgreSQL databases)
    email               Email
    email-data          Email Data (Import & Deduplication)
    php                 Generic PHP
    php-symlink         Generic Symlink to PHP
    port                Generic Proxy-Port (custom app listening on port)
    staticonly          Generic Static-Only (no .htaccess)
    staticonly-symlink  Generic Symlink to Static-Only (no .htaccess)
    wordpress           Wordpress

optional arguments:
  -h, --help            show this help message and exit

You can also get help on a specific migrator, eg:

$ wfmigrate wordpress --help 
usage: wfmigrate wordpress [-h] --mode {simulate,do}
                           [--loglevel {debug,info,warning,error,critical}]
                           config_path

positional arguments:
  config_path           Path to config.ini

optional arguments:
  -h, --help            show this help message and exit
  --mode {simulate,do}  Mode of operation
  --loglevel {debug,info,warning,error,critical}
                        Logging verbosity level

Caveats

⚠️ Copy the configuration template every time. We're always updating wfmigrate so you need to have the correct configuration every time you run it.

⚠️ When you migrate a MySQL db from WebFaction, the password for the MariaDB user created on our end will not be the same. You'll need to either change the DB password to match your old one from WF, or update your application settings to use the new DB password. You can find the new DB password in your Opalstack account notifications.

⚠️ The maria and psql migration options do not migrate databases from WebFaction private database instances. It can only migrate MySQL databases hosted on WebFaction's shared MySQL created via WebFaction's dashboard.

⚠️ When you migrate a port listener the migrator will create a matching app on Opalstack and copy the app files over from WebFaction, but the app will almost certainly not work right away. Once the app has been copied you'll need to adjust whatever configuration it uses, for example filesystem paths are different, server IPs, maybe your shell user name, etc. Binaries like Apache and Nginx that are copied over might need to be completely rebuilt from source.

⚠️ READ THE WARNINGS when you run the simulation step. If you see something like "htaccess file contains incompatible directive: SetHandler" or similar then you will need to edit your .htaccess after the migration those directives. If you don't do this then your site visitors will see your site's PHP code instead of the rendered HTML.

⚠️ At this time PostGIS databases cannot be migrated automatically. Instead you'll need to do the following:
1. Export the database at WF and copy the exports to Opalstack.
2. Create new PostgreSQL databases on Opalstack.
3. Contact support to have PostGIS enabled.
4. Import your data.

⚠️ We only support PHP versions 5.6, 7.3, 7.4, and 8.0. If you're migrating a WP or PHP app then set the the version to the closest match.

Migrating multiple apps, sites, and databases

The migrator has an account option that you can use to generate migration templates and commands to migrate all of your apps or databases in a single run.

To migrate multiple apps:

Copy the account migration template:

mkdir -p ~/migrations/apps
cd ~/migrations/apps
cp /opt/app_migrators/wf_account/config-template.ini account.ini

Edit account.ini to provide your Opalstack and WF account info.

Run the migrator to generate your app migration config:

wfmigrate account ./account.ini --buildconfigs=apps

The migrator will run and generate the following items:
- A ini file for each app to be migrated
- A file migration_commands.txt containing the commands to run for each app migration. These are initially set to run as simulations.
Edit migration_commands.txt to remove any apps that you don't want to migrate.

Run the simulations:

bash migration_commands.txt

After the simulation is done, correct any problems that were noted (especially incompatible PHP versions as noted above and htaccess directives) and then edit migration_commands.txt to change the simulate option to do.

Run the live migration:

bash migration_commands.txt

The migrator will then run to migrate each application.

**The multiple app migration does not migrate your sites, only your apps. Proceed to the next step to migrate your sites.

To migrate multiple sites:

Copy the account migration template:

mkdir -p ~/migrations/sites
cd ~/migrations/sites
cp /opt/app_migrators/wf_account/config-template.ini account.ini

Edit account.ini to provide your Opalstack and WF account info.

Run the migrator to generate your app migration config:

wfmigrate account ./account.ini --buildconfigs=sites

The migrator will run and generate the following items:
- A ini file for each site to be migrated
- A file migration_commands.txt containing the commands to run for each site migration. These are initially set to run as simulations.
Edit migration_commands.txt to remove any sites that you don't want to migrate. If you have HTTP>HTTPS redirect sites at WebFaction then leave those out of the migration. You can enable HTTPS redirection in your Opalstack dashboard after the sites are migrated.

Run the simulations:

bash migration_commands.txt

After the simulation is done, correct any problems that were noted and then edit migration_commands.txt to change the simulate option to do.

Run the live migration:

bash migration_commands.txt

The migrator will then run to migrate each site.

To migrate multiple databases:

Copy the account migration template:

mkdir -p ~/migrations/dbs
cd ~/migrations/dbs
cp /opt/app_migrators/wf_account/config-template.ini account.ini

Edit account.ini to provide your Opalstack and WF account info.

Run the migrator to generate your app migration config:

wfmigrate account ./account.ini --buildconfigs=dbs

The migrator will run and generate the following items:
- A ini file for each app to be migrated
- A file migration_commands.txt containing the commands to run for each app migration. These are initially set to run as simulations.
Edit migration_commands.txt to remove any databases that you don't want to migrate. ‼️If you've already migrated your Wordpress apps then remove your WP databases from the file.
Edit each ini file to include your WF database credentials. ‼️Don't forget this step!

Run the simulations:

bash migration_commands.txt

After the simulation is done, correct any problems that were noted and then edit migration_commands.txt to change the simulate option to do.

Run the live migration:

bash migration_commands.txt

The migrator will then run to migrate each database.