HOWTO migrate stuff automatically from WebFaction!

sasa_buklijas

Just some feedback from my side.
I had no issue with wfmigrate wordpress, it worked perfectly and quickly, around 1-2 minute.

After migration I found out that some plugins (crayon-syntax-highlighter, web4pro-about-me) are making WordPress crash. To be clear this is not problem with OpalStack, but because PHP was not updated on WebFaction (it was running depreciated version), these outdated plugins still worked, if WebFaction did update I would have same problem there.

This Wordpress site is my personal development/coder blog and I have no plans to publish more content via WordPress, WordPress is just not good option for developer/coder blog. I have a plan(but not time) to switch to static site generator. Duo to these errors I decide to switch is to static site. I have used this tool https://www.youtube.com/watch?v=UKFCWMCNWLM&ab_channel=RedServerHost. Process is done and all is working fine.

I posted this because maybe somebody is in similar situation, have WordPress that will not be updated anymore, one option is to convert it to static site.

P.S. I did kept original WordPress on OpalStack, just in case I change my mind, but it is not available via site routes.

sean

vkbr If your migration is complete you can delete the config and any of the files that it created. We leave them there in case post-migration troubleshooting is needed.

sean

webscience dmeehan we have not been able to reproduce the password issue that you've reported.

If you're still hitting this problem then please email support with to let us know the location of the config that you're trying to use. We'll also need your permission to actually run the migrator with that config.

webscience

sean Dear Sean, i found the problem. It was a stupid mistake from my side, the client changed the server pw and kept the ssh pw the same. However, now i have another problem. The wordpress install is in de app directory but i still get this:
Traceback (most recent call last): File "/usr/local/bin/wfmigrate", line 66, in <module> migrator = __import__(COMPONENTS[args.component]['module']).Migrator(cfg, args) File "/opt/app_migrators-1.4.0/wf_wordpress/migrator.py", line 32, in __init__ self.wordpress_cfg = WordpressCfg(cfg['wordpress']) File "/opt/app_migrators-1.4.0/wf_wordpress/migrator.py", line 22, in __init__ self.wordpress_path = data['wordpress_path'] File "/usr/lib64/python3.6/configparser.py", line 1233, in __getitem__ raise KeyError(key) KeyError: 'wordpress_path'

aa11

Hi,

Something small to add about wfmigrate.

I ran it on 2020-11-04 (UTC) and it seems to have run perfectly for very standard PHP add/domain/route. Thanks!!

A funny thing I experienced:

tl;dr: Watch out for how you specify the php_version number (and probably any x_version parameter in config.ini.

Setup "config.ini"
Set: php_version = 8 (<-- Spoiler alert: This is wrong)
Run wfmigrate php ./config.ini --mode simulate
Result: ✅ "Simulation started/finished" (no errors)
Run wfmigrate php ./config.ini --mode do
Result: ✅ Lots of DEBUG output (which I like) and in the end everything was completed without error.
Check the App in the Control Panel: https://my.opalstack.com/applications/
See that Type is set to PHP 0.8 😂
In the Control Panel, I was able to easily Edit that app and switch the PHP Version dropdown to PHP 8.0.
Everything seems to be ok.

However I see the following also:

If I upload a file with <?php phpinfo() ?>, I see:
- PHP Version 8.0.0RC2
If I ssh into that apps directory and run php -v, I see:
- PHP 7.3.23 (cli) (built: Sep 29 2020 08:33:03) ( NTS )

But maybe that's expected?

Running wfmigrate with php_version = 8.0 in "config.ini" generated the following

Traceback (most recent call last):
  File "/usr/local/bin/wfmigrate", line 66, in <module>
    migrator = __import__(COMPONENTS[args.component]['module']).Migrator(cfg, args)
  File "/opt/app_migrators-1.3.2/wf_generic_php/migrator.py", line 17, in __init__
    self.php_cfg = PhpCfg(cfg['php'])
  File "/opt/app_migrators-1.3.2/lib/config_types/php_cfg.py", line 3, in __init__
    self.php_version = int(data['php_version'])
ValueError: invalid literal for int() with base 10: '8.0'

So the correct value, to get PHP 8.0 is: php_version = 80 (Which looks a little weird)

But I'm still not sure why the Control Panel would show 0.8, since that's not a valid version of PHP. Shouldn't it, at least, get the version number from what's running on the server?

I'm not sure which is the best way to make this better:

Update wfmigrate to use dot notation for the version number?
Add some logic to account for a single digit major version number (and auto-interpret that as x.0)?
Leave wfmigrate as is and just add some wording to the docs?

I think any of those seem fine, but require different amounts of work. (Translation: Just update the docs ;-)

In any case, it was quick and easy to fix and also funny, so not much of a problem :-)

I hope this helps.

Thanks again!!

sean

aa11 thanks for the report, definitely a bug we need to fix.

playdo

@sean I'm running into a similar issue as others in this thread - my db user owns multiple databases, and the second db migration removed the user. I thought the permissions might automatically resolve, but turns out my dbuser only has permissions on the latest migrated database. I can't figure how to grant the user privileges to the other databases again. Is there a way to do this, or what would the recommended solution be?
Thanks!
EDIT: I deleted the previously migrated databases, created new ones assigned to the same user and then did a manual export/import -- I'm wondering if there is any caveats to be aware of with this approach (coming from mysql)? Things look fine as far as I can tell, but curious if the wfmigrate script was doing any additional steps one should be aware of. Thanks!

sean

playdo known bug, we're working to fix it.

sean

Updated the OP to match the latest wfmigrate, mainly the addition of the "port" migration option.

pjrobertson

Just signed up for Opal and am starting to use wfmigrate. A few bits of feedback:

EDIT: I've sorted these into 'high priority' and 'low priority'. Since wfmigrate only has a shelf life of 30 more days, there are some that need prioritising.

High Priority

It seems like the ssh password doesn't support certain characters (e.g. `) My original ssh_pass = v5`h1j1M`L didn't work.
wfmigrate didn't seem to migrate over my HTTPS site route, only the HTTP site route. It'd be nice if it could do this automatically. Or at the very least if I could define in the config.ini my Opal front end settings (e.g. Shared Certificate/LetsEncrypt, redirect HTTP to HTTPS etc.)
wfmigrate wordpress doesn't seem to work for cases where the Wordpress install isn't in the base directory (e.g. if wordpress is installed in a subdirectory as explained by wordpress here). It'd be nice if there was an option called wordpress_path that could be used to specify a subdirectory for the wordpress install, and so that the script knows where to find wp-config.php. (I assume my only option now is to use the php and maria migrations separately).

Low Priority

It'd be good if the tool supported SSH Keys for logging in to the WF servers (e.g. if ssh_pass = is left intentionally blank, then the tool would try logging in with SSH). Saves having to write SSH passes in the config, and makes it easier to maintain multiple configs.
I just set up wfmigrate for a simple php app (wfmigrate php) from WF. Running --mode simulate gave just: Simulation Started / Simulation Complete messages. But when I ran with --mode do I got a lot more debug messages. Looks like the simulate didn't work properly? I can confirm that the migrate did work when using --mode do though. EDIT: Looks like --mode simulate runs with loglevel INFO but --mode do runs with DEBUG, hence the confusion.
It'd be nice if the type of transfer (wordpress, php, maria, psql) could be defined in the ini file as opposed to having to define it on the command line. E.g. at the start of each ini file you insert a block [php] or [wordpress] or [maria] or [psql]. Couple of benefits: a) it helps with documentation: each config file is self explanatory about what type of transfer is. b) could help with automating wfmigrate. No need for wfmigrate php config.ini, just find . -name *.ini -exec wfmigrate {} \;
The ability to define multiple apps in one config would be nice. E.g.
```
[app]
appname = foo

[app]
appname = bar
```
1. Any way to define the Webfaction/Opalstack login details just once? I'm migrating 10-20 apps over. It's a bit painful having to copy/paste the login details over to each ini file. Not very DRY. Having a separate accounts.ini file where I can define accounts just once would be really useful. Something like wfmigrate php CONFIG.ini --accounts accounts.ini where the --accounts flag would overwrite any info already found in the CONFIG.ini file
I'm still playing with wfmigrate, so I'll report back on more issues soon. Keep up the good work, We have just 30 days to migrate from WF now. Thanks to all you're doing!

pjrobertson

I think I found a workaround for issue #3 above:

pjrobertson wfmigrate wordpress doesn't seem to work for cases where the Wordpress install isn't in the base directory

To use wfmigrate in this case, on your Webfaction server, copy wp-config.php from its wordpress sub-folder to the app's root directory. Then re-run wfmigrate. Once the migration has been complete, on your Opalstack server, move wp-config.php from the root folder back to the wordpress sub-folder.

sean

pjrobertson wfmigrate wordpress doesn't seem to work for cases where the Wordpress install isn't in the base directory

Our WP migrator is designed to work with the standard WP install that WF provided.

debiprasad

Migrating WordPress websites were working fine 2 days ago. But now it's throwing an error:

2020-11-08 14:49:22 : DEBUG : Got stdout: "", stderr: "mysqldump: Got error: 1045: Access denied for user 'debiprasad_oriya'@'localhost' (using password: NO) when trying to connect", retcode: 2

dolomiti

debiprasad @sean
I ran in the same issue. In my case the WF mysql password started with a "#" char, and I suppose that the Opalstack script ignored it supposing it was a comment (hence the message "using password: NO").
Changing the password on the WF side before migrating the worpress app to Opalstack, avoiding the presence of a "#" char at the beginning solved my problem. I wrote my workaround here in case somebody will hit the same issue in the future.

debiprasad

Another error in another WordPress migration:

2020-11-08 14:58:42 : INFO : Extracting DB User credentials from /home/debiprasad/apps/ost_staging/wp-config.php
...
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3205: ordinal not in range(128)

sean

debiprasad Another error in another WordPress migration:

Please email support to let us know the location of the config file that you're using so that we can investigate this.

tinygod

My last two WordPress migrations ended like this:

2020-11-08 16:48:08 : INFO : Checking .htaccess for incompatible directives
Traceback (most recent call last):
  File "/usr/local/bin/wfmigrate", line 67, in <module>
    migrator.run()
  File "/opt/app_migrators-1.3.2/wf_wordpress/migrator.py", line 184, in run
    self.migrate()
  File "/opt/app_migrators-1.3.2/wf_wordpress/migrator.py", line 166, in migrate
    self.wf_migrator_php.check_htaccess()
  File "/opt/app_migrators-1.3.2/lib/wf_migrator_php.py", line 136, in check_htaccess
    for line in fp:
  File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 87: ordinal not in range(128)

Please advise.

sean

tinygod Thanks for the report, we're looking into it! In the meantime please email our support address to let us know the location of the config file you were using when this happened. Please don't email the file to us since it contains passwords etc.

pjrobertson

Great to see the addition of more options in the Maria/psql migration configs!

Having the ability to now specify the database name, username and password (if the username already exists) on the Opalstack side is great! Keep up the good work ❤️

saebel

OMG! That was amazing. I was able to migrate some of my simpler apps in no time flat.

Now, of course, I have some weird stuff with Laravel which I had to install separately and then use a symbolic link to point to the public directory. I don't get the impression that will be a simple thing to migrate. So I'm guessing I'll need to manually do those sites after installing Laravel, Composer, etc?

Also, I have private MySQL instances. Can those be migrated using the tool? I'm guessing no because it doesn't follow the standard patterns (as in there are no databases or database users in the Webfaction control panel).

sean

webscience and all: we are constantly updating the wfmigrate tool with new features and configuration options.

The instructions above include a step to copy the config template prior to running the migrator.

That step isn't optional - be sure to do it every time to ensure that you have the correct config template.

Thanks!

webscience

sean Thank you Sean, this works!

« Previous Page Next Page »