How to clone a WordPress website with WP CLI and rsync

As part of the day job (or even when I’m working on my own website), I need to clone a live website to my local development environment so that I know I have a current version of all the data and files to work on. There are many migration plugins which allow me to do this, but many of them throw a fit when dealing with large amounts of data. So I use a couple of simple command-line scripts to automate the process.

My current website contains over 25 Gb of data and so the server can often run out of memory when a plugin attempts to create a downloadable package. That lead me to write a couple of scripts, which allow me to bring the local development version up-to-date with the live version.

First of all, I’m running a local development version of the website on my own computer using Local: an installable app which allows me to create and run WordPress on my own machine. I can’t recommend this strongly enough, as it’s installed in minutes. Setting up a new website using Local – including a local database, a WordPress installation, a local domain and a free SSL certificate – takes less than a minute.

Once the local site is up and running, I right-click on the website in Local and choose Open Site Shell. This opens an SSH connection to the running website and I can then run the following commands on the command line.

Make sure that these commands are run in the webroot folder, or the files will be copied to the wrong directory.

The first command is to get the database. This assumes that WP CLI – the command line interface for WordPress – is installed on the server. (In the example, at ­/bin/wp.) If you’re using Cyon hosting in Switzerland, there are instructions for installing WP CLI in this (German language) blog post.

(In these examples, replace the uppercase words with the values which are appropriate to your site and server.)

ssh $USER@$SERVER -A "cd $WEBROOT && ­/bin/wp db export - | gzip" > live.sql.gz && gunzip live.sql.gz && wp db import live.sql && wp search-replace $LIVEDOMAIN $DEVDOMAIN && rm live.sql

Once you have the database, you will also need to copy the files before trying to open the website in a browser. By using rsync, only files which are new (or newer) than those already existing locally will be copied down. This saves a lot of time if you’ve run the commands before.

Make sure that the ending slashes are correct as per this example, or the files will be copied to the wrong place. Local will have already created local themes, plugins and uploads folders, but you might need to create the languages folder manually before you start trying to copy files down.

rsync -azP -e "ssh" $USER@$SERVER:$WEBROOT/wp-content/themes/ ./wp-content/themes -delete

rsync -azP -e "ssh" $USER@$SERVER:$WEBROOT/wp-content/plugins/ ./wp-content/plugins -delete

rsync -azP -e "ssh" $USER@$SERVER:$WEBROOT/wp-content/languages/ ./wp-content/languages -delete

If your site is using “must-use” plugins, then you’ll need to copy them down…

rsync -azP -e "ssh" $USER@$SERVER:$WEBROOT/wp-content/mu-plugins/ ./wp-content/mu-plugins -delete

…and you’ll obviously need the uploads.

rsync -azP -e "ssh" $USER@$SERVER:$WEBROOT/wp-content/uploads/ ./wp-content/uploads -delete

The number of files in the uploads folder can be very large the first time you run the command, depending on how large your website is. If you don’t want to copy all of the files down, you can modify the paths accordingly. You might need to manually create the target folder locally (e.g. wp-content/uploads/2020) if it doesn’t exist.

rsync -azP -e "ssh" $USER@$SERVER:$WEBROOT/wp-content/uploads/2020/ ./wp-content/uploads/2020 -delete

Once you’ve run these commands, your local version will be a precise copy of the live website and you can start work.

A final note: if you’re familiar with bash scripts, then it’s easy to pack these commands into a single file fromlive.sh and run them from the command line in one go using the command sh fromlive.sh.

#!/bin/bash

USER="myuser"
SERVER="myserver.ch"
WEBROOT="­/public_html"
WPCLI="­/bin/wp"
LIVEDOMAIN="example.org"
DEVDOMAIN="example.local"

ssh $USER@$SERVER -A "cd $WEBROOT && $WPCLI db export - | gzip" > live.sql.gz && gunzip live.sql.gz && wp db import live.sql && wp search-replace $LIVEDOMAIN $DEVDOMAIN && rm live.sql
rsync -azP -e "ssh" $USER@$SERVER:$WEBROOT/wp-content/themes/ ./wp-content/themes -delete
rsync -azP -e "ssh" $USER@SERVER:$WEBROOT/wp-content/plugins/ ./wp-content/plugins -delete
rsync -azP -e "ssh" $USER@$SERVER:$WEBROOT/wp-content/languages/ ./wp-content/languages -delete
rsync -azP -e "ssh" $USER@$SERVER:$WEBROOT/wp-content/mu-plugins/ ./wp-content/mu-plugins -delete
rsync -azP -e "ssh" $USER@$SERVER:$WEBROOT/wp-content/uploads/ ./wp-content/uploads -delete