HOWTO migrate stuff automatically from WebFaction!

OrangyTang

Hello! I was just trying this and the simulate went ok (albeit with a .htaccess SetHandler warning that I'll fix later). However when doing the actual migration I got a permission denied error:


2020-11-16 21:24:15 : INFO : Writing sqlpasswd file to jcampbell_ud2wp.opalmigration-local.passwd
2020-11-16 21:24:15 : INFO : Changing app config DB credentials in /home/jcampbell/apps/ud2wp/wp-config.php
Traceback (most recent call last):
  File "/usr/local/bin/wfmigrate", line 71, in <module>
    migrator.run()
  File "/opt/app_migrators-1.5.2/wf_wordpress/migrator.py", line 210, in run
    self.migrate()
  File "/opt/app_migrators-1.5.2/wf_wordpress/migrator.py", line 185, in migrate
    self.set_app_db_creds()
  File "/opt/app_migrators-1.5.2/wf_wordpress/migrator.py", line 136, in set_app_db_creds
    with open(self.opal_wp_conf_filepath, 'wb') as fp: fp.write(b''.join(new_config_lines))
PermissionError: [Errno 13] Permission denied: '/home/jcampbell/apps/ud2wp/wp-config.php'

There was nothing at that location before running the script, and it does appear to have copied the wp-config.php file, so I'm not sure why the permission problem. The file itself has permissions 444 (read-read-read) which seems not what I was expecting.

Thanks.

sean

OrangyTang I'll pass this along to the migration developer, thanks for the report!

OrangyTang

Answering my own question - turns out a Wordpress plugin on the old host had changed the file permisions for several files (like wp-config) which seems to trip up the migration script. After fixing them on the WebFaction end, the migration script gets past that bit - might be nice for the migration script to set the permission manually rather than erroring though.

I've got another error with access-denied for a DB user, but I suspect that's something I've configured wrong elsewhere 🙂

OrangyTang

Ok I think I've ruled out a mis-config, and I'm not sure why I'm getting this error now:

2020-11-16 22:11:22 : INFO : Extracting DB User password from jcampbell_ud2wp.opalmigration-local.passwd 2020-11-16 22:11:22 : INFO : Writing sqlpasswd file to jcampbell_ud2wp.opalmigration-local.passwd 2020-11-16 22:11:22 : INFO : Changing app config DB credentials in /home/jcampbell/apps/ud2wp/wp-config.php 2020-11-16 22:11:22 : INFO : Importing DB using sqlpasswd file jcampbell_ud2wp.opalmigration-local.passwd ERROR 1045 (28000): Access denied for user 'jcampbell_ud2wp'@'localhost' (using password: YES) Traceback (most recent call last): File "/usr/local/bin/wfmigrate", line 71, in <module> migrator.run() File "/opt/app_migrators-1.5.2/wf_wordpress/migrator.py", line 210, in run self.migrate() File "/opt/app_migrators-1.5.2/wf_wordpress/migrator.py", line 186, in migrate self.wf_migrator_maria.import_db() File "/opt/app_migrators-1.5.2/lib/wf_migrator_maria.py", line 157, in import_db raise RuntimeError(f'Command "{cmd}" exited with status {result.returncode}') RuntimeError: Command "['mysql', '--defaults-extra-file=jcampbell_ud2wp.opalmigration-local.passwd', '-u', 'jcampbell_ud2wp', 'jcampbell_ud2wp']" exited with status 1

The DB has been created, as has the DB user, and the password generated ok and inserted into the .opalmigration-local.passwd file. However inside that file the password entry looks suspect. If the password is ABCD then the actual file is:

[mysql] host=localhost port=3306 user=jcampbell_ud2wp password='"ABCD"'

With both 'single' and "double" quotes around the password, is that correct?

I also tried by copying the created DB name/user/password into the config.ini and re-running, but this results in the same '"double"' quotes and the same error. (Logs taken from the second run which is why it's password: YES).

Any hints what could be wrong?

Thanks.

sean

OrangyTang Any hints what could be wrong?

Probably a bug in the migrator.

OrangyTang

Darn. I've tried manually fixing the quotes in the password file, but the migrator script recreates it each time, which means I'm at bit stuck atm. 🙁

sean

OrangyTang I've already let the dev know, hopeful for a fix in the next few hours.

sean

OrangyTang should all be fixed as of several days ago.

sdegrace

It blows my mind how well this worked for WordPress. Thank you for doing this! If I would have one suggestion it would be to clearly specify not to use fully qualified domains in the config.ini file, because that was my one bit of learning curve. It didn't take all that much time to figure out, but a bit of forewarning on that point would remove the only piece of friction I found. Well done! Now to migrate the rest of the giant pile of sites from WebFaction 😛.

saebel

sean
Thanks a bunch! It took some experimenting, but I finally figured out the entire process for migrating a private instance on webfaction over to a private instance on Opalstack. I've documented all the steps, so if you are interested in sharing/publishing them, I'm happy to email them to you for your review/use.

sean

saebel feel free to share them as a tutorial here in the forum, just make a new post and tag it as Tutorial 🙂

saebel

Question: Is there a way to migrate just a folder and all of its contents? I've got Laravel, and I've already installed the base components (composer, laravel, etc.), but I'm not looking forward to downloading every single folder/subfolder/file from one website locally and then uploading it to the other. Is there a way to do that through a migrate template?

sean

saebel we don't have a migrator for arbitrary directories, but you can do that yourself with rsync like so:

rsync -Paz username@webNNN.webfaction.com:/home/username/whatever/directory ~/wherever/you/want/the/directory

Jorgeoa

Hi,
A feature request here...
I have an application called "admin" and the migration tool can't import it because it seems to be a reserved word:
{'name': ['The name you entered is reserved, please select something else.']}
Would be grate if we can indicate the opalstack app name in the config file to allow "rename" the application from webfaction to opal.
Regards,
Jorge

sean

Jorgeoa naimrahidin ok, I've logged a feature request for that.

naimrahidin

I agree with Jorgeoa,

Would be best if we can migrate and renaming the application name.

Else we would need to create a new application in webfaction and CP the content to the new application and then only migrate.

snaushads

@sean I created an account and followed through steps and looked at the config.ini for wf-migrator. I have a few questions to see if my understanding is correct.

Contents of migration config file

[opalstack]
account = my_opalstack_account_name
apitoken = my_opalstack_api_token
web_server = opal1.opalstack.com
ip = 1.2.3.4

[webfaction]
account = my_webfaction_account_name
account_pass = my_webfaction_account_password
ssh_pass = my_webfaction_ssh_password
server = web1

[app]
appname = name_of_app_to_migrate

[wordpress]
auto_site_url = True
wordpress_path = /

[php]
php_version = 74
fpm_max_children = 5
fpm_max_requests = 20

[local_db]
db_host = localhost
db_port = 3306
db_user = opalstack_side-name_of_dbuser-blank_if_not_yet_exists
db_name = opalstack_side-name_of_db-blank_if_not_yet_exists
db_pass = opalstack_side-db_user_password-blank_if_not_yet_exists

I can:

Create multiple ssh users and import the sites in few of those separately.
I can ssh using each user and create a different config file for each site (with apt appname that i need to import.
Lastly, i can create db_user, db_name and db_pass by myself for each site to import and the respective data would be imported in the target (OpalStack) db

Kindly revert

sean

snaushads Create multiple ssh users and import the sites in few of those separately.

yes!

snaushads I can ssh using each user and create a different config file for each site (with apt appname that i need to import.

yes again!

snaushads Lastly, i can create db_user, db_name and db_pass by myself for each site to import and the respective data would be imported in the target (OpalStack) db

yes again! 🙂

snaushads

sean thanks much, such a relief.

snaushads

sean I am getting an error when running the simulation. Changed password to keep them simple as well. Able to login to WF panel and ssh with those passwords.

Error
Traceback (most recent call last): File "/usr/local/bin/wfmigrate", line 70, in <module> migrator = __import__(COMPONENTS[args.component]['module']).Migrator(cfg, args) File "/opt/app_migrators-1.6.0/wf_wordpress/migrator.py", line 27, in __init__ self.wf_cfg = WfCfg(cfg['webfaction']) File "/opt/app_migrators-1.6.0/lib/config_types/wf_cfg.py", line 12, in __init__ self.update_derived_attrs() File "/opt/app_migrators-1.6.0/lib/config_types/wf_cfg.py", line 16, in update_derived_attrs self.wf_session_id, self.wf_account_cfg = self.wf_api.login(self.wf_account, self.wf_password, self.wf_server.capitalize(), 2) File "/usr/lib64/python3.6/xmlrpc/client.py", line 1112, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python3.6/xmlrpc/client.py", line 1452, in __request verbose=self.__verbose File "/usr/lib64/python3.6/xmlrpc/client.py", line 1154, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python3.6/xmlrpc/client.py", line 1170, in single_request return self.parse_response(resp) File "/usr/lib64/python3.6/xmlrpc/client.py", line 1342, in parse_response return u.close() File "/usr/lib64/python3.6/xmlrpc/client.py", line 656, in close raise Fault(**self._stack[0]) xmlrpc.client.Fault: <Fault 1: 'Not allowed to login to machine "Web461"'>

cworrell

I just noticed that the wf_generic_php migration sets up our app as php-fpm+nginx instead of php+apache as before. We use .htaccess for caching and rewrites, so I've been creating another app after migrating and moving the files manually.

sean

cworrell I just noticed that the wf_generic_php migration sets up our app as php-fpm+nginx instead of php+apache as before.

We can flip the app type for you from our end if you need us to.

cworrell Would it be possible to add another parameter too for the app name on opalstack's side?

Logged that feature request earlier today.

« Previous Page Next Page »