HOWTO migrate stuff automatically from WebFaction!

sean

aa11 thanks for the report, definitely a bug we need to fix.

playdo

@sean I'm running into a similar issue as others in this thread - my db user owns multiple databases, and the second db migration removed the user. I thought the permissions might automatically resolve, but turns out my dbuser only has permissions on the latest migrated database. I can't figure how to grant the user privileges to the other databases again. Is there a way to do this, or what would the recommended solution be?
Thanks!
EDIT: I deleted the previously migrated databases, created new ones assigned to the same user and then did a manual export/import -- I'm wondering if there is any caveats to be aware of with this approach (coming from mysql)? Things look fine as far as I can tell, but curious if the wfmigrate script was doing any additional steps one should be aware of. Thanks!

sean

playdo known bug, we're working to fix it.

sean

Updated the OP to match the latest wfmigrate, mainly the addition of the "port" migration option.

pjrobertson

Just signed up for Opal and am starting to use wfmigrate. A few bits of feedback:

EDIT: I've sorted these into 'high priority' and 'low priority'. Since wfmigrate only has a shelf life of 30 more days, there are some that need prioritising.

High Priority

It seems like the ssh password doesn't support certain characters (e.g. `) My original ssh_pass = v5`h1j1M`L didn't work.
wfmigrate didn't seem to migrate over my HTTPS site route, only the HTTP site route. It'd be nice if it could do this automatically. Or at the very least if I could define in the config.ini my Opal front end settings (e.g. Shared Certificate/LetsEncrypt, redirect HTTP to HTTPS etc.)
wfmigrate wordpress doesn't seem to work for cases where the Wordpress install isn't in the base directory (e.g. if wordpress is installed in a subdirectory as explained by wordpress here). It'd be nice if there was an option called wordpress_path that could be used to specify a subdirectory for the wordpress install, and so that the script knows where to find wp-config.php. (I assume my only option now is to use the php and maria migrations separately).

Low Priority

It'd be good if the tool supported SSH Keys for logging in to the WF servers (e.g. if ssh_pass = is left intentionally blank, then the tool would try logging in with SSH). Saves having to write SSH passes in the config, and makes it easier to maintain multiple configs.
I just set up wfmigrate for a simple php app (wfmigrate php) from WF. Running --mode simulate gave just: Simulation Started / Simulation Complete messages. But when I ran with --mode do I got a lot more debug messages. Looks like the simulate didn't work properly? I can confirm that the migrate did work when using --mode do though. EDIT: Looks like --mode simulate runs with loglevel INFO but --mode do runs with DEBUG, hence the confusion.
It'd be nice if the type of transfer (wordpress, php, maria, psql) could be defined in the ini file as opposed to having to define it on the command line. E.g. at the start of each ini file you insert a block [php] or [wordpress] or [maria] or [psql]. Couple of benefits: a) it helps with documentation: each config file is self explanatory about what type of transfer is. b) could help with automating wfmigrate. No need for wfmigrate php config.ini, just find . -name *.ini -exec wfmigrate {} \;
The ability to define multiple apps in one config would be nice. E.g.
```
[app]
appname = foo

[app]
appname = bar
```
1. Any way to define the Webfaction/Opalstack login details just once? I'm migrating 10-20 apps over. It's a bit painful having to copy/paste the login details over to each ini file. Not very DRY. Having a separate accounts.ini file where I can define accounts just once would be really useful. Something like wfmigrate php CONFIG.ini --accounts accounts.ini where the --accounts flag would overwrite any info already found in the CONFIG.ini file
I'm still playing with wfmigrate, so I'll report back on more issues soon. Keep up the good work, We have just 30 days to migrate from WF now. Thanks to all you're doing!

pjrobertson

I think I found a workaround for issue #3 above:

pjrobertson wfmigrate wordpress doesn't seem to work for cases where the Wordpress install isn't in the base directory

To use wfmigrate in this case, on your Webfaction server, copy wp-config.php from its wordpress sub-folder to the app's root directory. Then re-run wfmigrate. Once the migration has been complete, on your Opalstack server, move wp-config.php from the root folder back to the wordpress sub-folder.

sean

pjrobertson wfmigrate wordpress doesn't seem to work for cases where the Wordpress install isn't in the base directory

Our WP migrator is designed to work with the standard WP install that WF provided.

debiprasad

Migrating WordPress websites were working fine 2 days ago. But now it's throwing an error:

2020-11-08 14:49:22 : DEBUG : Got stdout: "", stderr: "mysqldump: Got error: 1045: Access denied for user 'debiprasad_oriya'@'localhost' (using password: NO) when trying to connect", retcode: 2

dolomiti

debiprasad @sean
I ran in the same issue. In my case the WF mysql password started with a "#" char, and I suppose that the Opalstack script ignored it supposing it was a comment (hence the message "using password: NO").
Changing the password on the WF side before migrating the worpress app to Opalstack, avoiding the presence of a "#" char at the beginning solved my problem. I wrote my workaround here in case somebody will hit the same issue in the future.

debiprasad

Another error in another WordPress migration:

2020-11-08 14:58:42 : INFO : Extracting DB User credentials from /home/debiprasad/apps/ost_staging/wp-config.php
...
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3205: ordinal not in range(128)

sean

debiprasad Another error in another WordPress migration:

Please email support to let us know the location of the config file that you're using so that we can investigate this.

tinygod

My last two WordPress migrations ended like this:

2020-11-08 16:48:08 : INFO : Checking .htaccess for incompatible directives
Traceback (most recent call last):
  File "/usr/local/bin/wfmigrate", line 67, in <module>
    migrator.run()
  File "/opt/app_migrators-1.3.2/wf_wordpress/migrator.py", line 184, in run
    self.migrate()
  File "/opt/app_migrators-1.3.2/wf_wordpress/migrator.py", line 166, in migrate
    self.wf_migrator_php.check_htaccess()
  File "/opt/app_migrators-1.3.2/lib/wf_migrator_php.py", line 136, in check_htaccess
    for line in fp:
  File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 87: ordinal not in range(128)

Please advise.

sean

tinygod Thanks for the report, we're looking into it! In the meantime please email our support address to let us know the location of the config file you were using when this happened. Please don't email the file to us since it contains passwords etc.

pjrobertson

Great to see the addition of more options in the Maria/psql migration configs!

Having the ability to now specify the database name, username and password (if the username already exists) on the Opalstack side is great! Keep up the good work ❤️

saebel

OMG! That was amazing. I was able to migrate some of my simpler apps in no time flat.

Now, of course, I have some weird stuff with Laravel which I had to install separately and then use a symbolic link to point to the public directory. I don't get the impression that will be a simple thing to migrate. So I'm guessing I'll need to manually do those sites after installing Laravel, Composer, etc?

Also, I have private MySQL instances. Can those be migrated using the tool? I'm guessing no because it doesn't follow the standard patterns (as in there are no databases or database users in the Webfaction control panel).

webscience

sean Dear Sean, i found the problem. It was a stupid mistake from my side, the client changed the server pw and kept the ssh pw the same. However, now i have another problem. The wordpress install is in de app directory but i still get this:
Traceback (most recent call last): File "/usr/local/bin/wfmigrate", line 66, in <module> migrator = __import__(COMPONENTS[args.component]['module']).Migrator(cfg, args) File "/opt/app_migrators-1.4.0/wf_wordpress/migrator.py", line 32, in __init__ self.wordpress_cfg = WordpressCfg(cfg['wordpress']) File "/opt/app_migrators-1.4.0/wf_wordpress/migrator.py", line 22, in __init__ self.wordpress_path = data['wordpress_path'] File "/usr/lib64/python3.6/configparser.py", line 1233, in __getitem__ raise KeyError(key) KeyError: 'wordpress_path'

sean

webscience and all: we are constantly updating the wfmigrate tool with new features and configuration options.

The instructions above include a step to copy the config template prior to running the migrator.

That step isn't optional - be sure to do it every time to ensure that you have the correct config template.

Thanks!

webscience

sean Thank you Sean, this works!

Sleckie

Sean,
What about web2py apps? As far as migrating?

sean

Sleckie use the "port" app option to migrate your web2py apps.

After the app is migrated you'll need to create a Python env in the app directory, pip install uwsgi into that, and then configure uwsgi to serve the app.

You'll need to migrate any related databases separately.

« Previous Page Next Page »