From 84ca56cc830f5ba6f17fba33c331486dd48cf078 Mon Sep 17 00:00:00 2001 From: Heather Guthrie Date: Fri, 22 Jan 2021 18:13:30 -0600 Subject: [PATCH 1/5] Add the correct cli commands for migrating files --- src/cloud/live/stage-prod-migrate.md | 67 ++++++++++++++++++++-------- 1 file changed, 48 insertions(+), 19 deletions(-) diff --git a/src/cloud/live/stage-prod-migrate.md b/src/cloud/live/stage-prod-migrate.md index 4b9201de73e..28241c0be23 100644 --- a/src/cloud/live/stage-prod-migrate.md +++ b/src/cloud/live/stage-prod-migrate.md @@ -14,7 +14,7 @@ Previous step To migrate your database and static files to Staging and Production: - [Deploy code](#code) -- [Migrate static files](#cloud-live-migrate-static) +- [Migrate static files](#migrate-static-files) - [Migrate the database](#cloud-live-migrate-db) If you encounter errors or need to change project configuration, complete the required updates in your local environment. Then, push the code changes to the Integration environment to deploy and test before pushing to Staging and Production environments. @@ -27,7 +27,8 @@ You can use the [Project Web Interface](#interface) or [SSH and CLI commands](#s The Project Web Interface provides features to create, manage, and deploy code in Integration, Staging, and Production environments for Starter and Pro plans. -For Pro projects, deploy the Integration branch you created to Staging and Production: +{:.procedure} +For Pro projects, deploy the Integration branch to Staging and Production: 1. [Log in](https://accounts.magento.cloud) to your project. 1. Select the Integration branch. @@ -35,13 +36,14 @@ For Pro projects, deploy the Integration branch you created to Staging and Produ 1. Select the Staging branch. 1. Select the **Merge** option to deploy to Production. -For Starter, deploy the development branch you created to Staging and Production: +{:.procedure} +For Starter, deploy the development branch you created to Staging and Production (master): 1. [Log in](https://accounts.magento.cloud) to your project. 1. Select the prepared code branch. 1. Select the **Merge** option to deploy to Staging. Complete all testing. 1. Select the Staging branch. -1. Select the **Merge** option to deploy to Production. +1. Select the **Merge** option to deploy to Master. ![Use the merge option to deploy]({{ site.baseurl }}/common/images/cloud/cloud_project-merge.png) @@ -54,7 +56,7 @@ You can use the [Magento Cloud CLI commands]({{ site.baseurl }}/cloud/reference/ - [Build and deploy on local]({{ site.baseurl }}/cloud/live/live-sanity-check.html) - [Prepare to deploy to Staging and Production]({{ site.baseurl }}/cloud/live/stage-prod-migrate-prereq.html) -#### Step 1: Deploy and test the Integration environment: +#### Step 1: Deploy and test the Integration environment 1. After logging into the project, check out the Integration environment: @@ -112,7 +114,7 @@ You can use the [Magento Cloud CLI commands]({{ site.baseurl }}/cloud/reference/ 1. Complete site testing. -#### Step 3: Deploy to Production: +#### Step 3: Deploy to Production 1. Check out, synchronize, and create a snapshot of your local Production environment. @@ -124,13 +126,46 @@ You can use the [Magento Cloud CLI commands]({{ site.baseurl }}/cloud/reference/ 1. Complete site testing. -## Migrate static files {#cloud-live-migrate-static} +## Migrate static files + +[Static files](https://glossary.magento.com/static-files) are stored in `mounts`. You can list the mounts available in your project using the `magento-cloud` CLI: + +```bash +magento-cloud mount:list +``` + +There are two methods for migrating files from a source mount location, such as your local environment, to a destination mount location. Both methods use the `rsync` utility, but we recommend using the `magento-cloud` CLI. + +### Migrate files using the CLI + +You can use the `mount:upload` and `mount:download` CLI commands to migrate files. Both commands use the `rsync` utility, but the CLI provides options and prompts tailored to the {{site.data.var.ece}} environment. + +The following example uploads files from a local `pub/media/` folder to the remote `pub/media/` folder for the current environment: + +```bash +magento-cloud mount:upload --source /path/to/project/pub/media/ --mount pub/media/ +``` + +Sample response: + +```terminal +Uploading files from pub/media to the remote mount pub/media + +Are you sure you want to continue? [Y/n] Y -You migrate [static files](https://glossary.magento.com/static-files) from your `pub/media` directory to Staging or Production. + building file list ... done + ./ + sample-file.jpeg + + sent 8.43K bytes received 48 bytes 3.39K bytes/sec + total size is 154.57K speedup is 18.23 +``` + +Use the `--help` option for the `mount:upload` and `mount:download` commands to see more options. For example, there is a `--delete` option to remove extraneous files during the migration. -We recommend using the Linux remote synchronization and file transfer command [`rsync`](https://en.wikipedia.org/wiki/Rsync). The rsync utility uses an algorithm that minimizes the amount of data by moving only the portions of files that have changed. Rsync also supports compression. +### Migrate files using rsync -Use the following command to migrate files: +Alternatively, you can use the `rsync` utility to migrate files. ```bash rsync -azvP @@ -143,19 +178,13 @@ This command uses the following options: - `v`–verbose - `P`–partial progress -For additional options, see the [rsync man page](http://linux.die.net/man/1/rsync). - -To migrate static files from your local machine, use the `rsync` command to copy the `pub/media` directory from your local Magento server to staging or production: - -```bash -rsync -azvP local_machine/pub/media/ :pub/media/ -``` +See the [rsync man page](http://linux.die.net/man/1/rsync). {:.procedure} To migrate static files from remote-to-remote environments directly (fast approach): {:.bs-callout-info} -To transfer media from remote-to-remote environments directly, you must enable ssh agent forwarding, see [GitHub guidance](https://developer.github.com/v3/guides/using-ssh-agent-forwarding/). +To transfer media from remote-to-remote environments directly, you must enable SSH agent forwarding, see [GitHub guidance](https://developer.github.com/v3/guides/using-ssh-agent-forwarding/). 1. [Open an SSH connection]({{ site.baseurl }}/cloud/env/environments-ssh.html#ssh) to the source environment. @@ -165,7 +194,7 @@ To transfer media from remote-to-remote environments directly, you must enable s ssh -A ``` -1. Use the `rsync` command to copy the `pub/media` directory from your current environment to another remote environment: +1. Use the `rsync` command to copy the `pub/media` directory from your current environment to a different remote environment: ```bash rsync -azvP pub/media/ :pub/media/ From f523daf8a8608a8c0c5f1639a880df8bdfe577db Mon Sep 17 00:00:00 2001 From: Heather Guthrie Date: Fri, 22 Jan 2021 18:34:41 -0600 Subject: [PATCH 2/5] fix problems from test --- src/cloud/live/stage-prod-migrate.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/cloud/live/stage-prod-migrate.md b/src/cloud/live/stage-prod-migrate.md index 28241c0be23..ff3d95d7b61 100644 --- a/src/cloud/live/stage-prod-migrate.md +++ b/src/cloud/live/stage-prod-migrate.md @@ -156,7 +156,7 @@ Are you sure you want to continue? [Y/n] Y building file list ... done ./ sample-file.jpeg - + sent 8.43K bytes received 48 bytes 3.39K bytes/sec total size is 154.57K speedup is 18.23 ``` From c3cb5b6fdb3e2c7e71c087c87dfc06e690efe5fa Mon Sep 17 00:00:00 2001 From: Heather Guthrie Date: Mon, 25 Jan 2021 16:07:04 -0600 Subject: [PATCH 3/5] Improved rsync remote to remote, added better example to cli --- src/cloud/live/stage-prod-migrate.md | 41 +++++++++++++++++++++------- 1 file changed, 31 insertions(+), 10 deletions(-) diff --git a/src/cloud/live/stage-prod-migrate.md b/src/cloud/live/stage-prod-migrate.md index ff3d95d7b61..b3c40858dfa 100644 --- a/src/cloud/live/stage-prod-migrate.md +++ b/src/cloud/live/stage-prod-migrate.md @@ -128,19 +128,36 @@ You can use the [Magento Cloud CLI commands]({{ site.baseurl }}/cloud/reference/ ## Migrate static files -[Static files](https://glossary.magento.com/static-files) are stored in `mounts`. You can list the mounts available in your project using the `magento-cloud` CLI: +[Static files](https://glossary.magento.com/static-files) are stored in `mounts`. There are two methods for migrating files from a source mount location, such as your local environment, to a destination mount location. Both methods use the `rsync` utility, but we recommend using the `magento-cloud` CLI for moving files between the local and remote environment. + +### Migrate files using the CLI + +You can use the `mount:upload` and `mount:download` CLI commands to migrate files. Both commands use the `rsync` utility, but the CLI provides options and prompts tailored to the {{site.data.var.ece}} environment. For example, if you use the simple command, the CLI prompts you to select which mount or mounts to upload or download. ```bash -magento-cloud mount:list +magento-cloud mount:download ``` -There are two methods for migrating files from a source mount location, such as your local environment, to a destination mount location. Both methods use the `rsync` utility, but we recommend using the `magento-cloud` CLI. +Sample response: -### Migrate files using the CLI +```terminal +Enter a number to choose a mount to download from: + [0] app/etc + [1] pub/static + [2] var + [3] pub/media + [4] All mounts + > 3 -You can use the `mount:upload` and `mount:download` CLI commands to migrate files. Both commands use the `rsync` utility, but the CLI provides options and prompts tailored to the {{site.data.var.ece}} environment. +Target directory: ~/pub/media/ -The following example uploads files from a local `pub/media/` folder to the remote `pub/media/` folder for the current environment: +Downloading files from the remote mount pub/media to pub/media + +Are you sure you want to continue? [Y/n] Y +``` + +{:.procedure} +To upload files from a local `pub/media/` folder to the remote `pub/media/` folder for the current environment: ```bash magento-cloud mount:upload --source /path/to/project/pub/media/ --mount pub/media/ @@ -174,7 +191,7 @@ rsync -azvP This command uses the following options: - `a`–archive -- `z`–compress +- `z`–compress files during the migration - `v`–verbose - `P`–partial progress @@ -186,20 +203,24 @@ To migrate static files from remote-to-remote environments directly (fast approa {:.bs-callout-info} To transfer media from remote-to-remote environments directly, you must enable SSH agent forwarding, see [GitHub guidance](https://developer.github.com/v3/guides/using-ssh-agent-forwarding/). -1. [Open an SSH connection]({{ site.baseurl }}/cloud/env/environments-ssh.html#ssh) to the source environment. +1. Use SSH to log in to the source environment. Do not use the `magento-cloud` CLI. - To find the **SSH access** link in your Project Web Interface, select the environment and click **Access Site**. The syntax for the SSH command is as follows: + To find the **SSH access** link in your Project Web Interface, select the environment and click **Access Site**. ```bash ssh -A ``` -1. Use the `rsync` command to copy the `pub/media` directory from your current environment to a different remote environment: + Using the `-A` option is very important because it enables forwarding of the authentication agent connection. + +1. Use the `rsync` command to copy the `pub/media` directory from your source environment to a different remote environment. ```bash rsync -azvP pub/media/ :pub/media/ ``` +1. Log in to the other remote environment to verify the files migrated successfully. + ## Migrate the database {#cloud-live-migrate-db} {%include cloud/note-db-import-export-warning.md%} From 6182904541e2628ef48bde262e5e5f85da86763d Mon Sep 17 00:00:00 2001 From: Heather Guthrie Date: Mon, 25 Jan 2021 16:09:44 -0600 Subject: [PATCH 4/5] add remote to remote recommendation --- src/cloud/live/stage-prod-migrate.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/cloud/live/stage-prod-migrate.md b/src/cloud/live/stage-prod-migrate.md index b3c40858dfa..bf2615622eb 100644 --- a/src/cloud/live/stage-prod-migrate.md +++ b/src/cloud/live/stage-prod-migrate.md @@ -128,7 +128,7 @@ You can use the [Magento Cloud CLI commands]({{ site.baseurl }}/cloud/reference/ ## Migrate static files -[Static files](https://glossary.magento.com/static-files) are stored in `mounts`. There are two methods for migrating files from a source mount location, such as your local environment, to a destination mount location. Both methods use the `rsync` utility, but we recommend using the `magento-cloud` CLI for moving files between the local and remote environment. +[Static files](https://glossary.magento.com/static-files) are stored in `mounts`. There are two methods for migrating files from a source mount location, such as your local environment, to a destination mount location. Both methods use the `rsync` utility, but we recommend using the `magento-cloud` CLI for moving files between the local and remote environment. And we recommend using the `rsync` method when moving files from a remote source to a different remote location. ### Migrate files using the CLI From 14feefe13b5035e4d55d2e1799344206f1856062 Mon Sep 17 00:00:00 2001 From: Heather Guthrie Date: Mon, 25 Jan 2021 16:22:32 -0600 Subject: [PATCH 5/5] some fixes for the notes --- src/cloud/live/stage-prod-migrate.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/src/cloud/live/stage-prod-migrate.md b/src/cloud/live/stage-prod-migrate.md index bf2615622eb..1fdf14527cc 100644 --- a/src/cloud/live/stage-prod-migrate.md +++ b/src/cloud/live/stage-prod-migrate.md @@ -132,7 +132,7 @@ You can use the [Magento Cloud CLI commands]({{ site.baseurl }}/cloud/reference/ ### Migrate files using the CLI -You can use the `mount:upload` and `mount:download` CLI commands to migrate files. Both commands use the `rsync` utility, but the CLI provides options and prompts tailored to the {{site.data.var.ece}} environment. For example, if you use the simple command, the CLI prompts you to select which mount or mounts to upload or download. +You can use the `mount:upload` and `mount:download` CLI commands to migrate files between the local and remote environment. Both commands use the `rsync` utility, but the CLI commands provide options and prompts tailored to the {{site.data.var.ece}} environment. For example, if you use the simple command with no options, the CLI prompts you to select which mount or mounts to upload or download. ```bash magento-cloud mount:download @@ -197,22 +197,21 @@ This command uses the following options: See the [rsync man page](http://linux.die.net/man/1/rsync). -{:.procedure} -To migrate static files from remote-to-remote environments directly (fast approach): - {:.bs-callout-info} To transfer media from remote-to-remote environments directly, you must enable SSH agent forwarding, see [GitHub guidance](https://developer.github.com/v3/guides/using-ssh-agent-forwarding/). -1. Use SSH to log in to the source environment. Do not use the `magento-cloud` CLI. +{:.procedure} +To migrate static files from remote-to-remote environments directly (fast approach): + +1. Use SSH to log in to the source environment. Do not use the `magento-cloud` CLI. Using the `-A` option is very important because it enables forwarding of the authentication agent connection. + {:.bs-callout-tip} To find the **SSH access** link in your Project Web Interface, select the environment and click **Access Site**. ```bash ssh -A ``` - Using the `-A` option is very important because it enables forwarding of the authentication agent connection. - 1. Use the `rsync` command to copy the `pub/media` directory from your source environment to a different remote environment. ```bash