Here I am going to show you how we automated the AEM In-Place Upgrade from version 6.2 to 6.4 Service Pack 3 with AEM OpenCloud and its CI/CD tools.
I’m going to use AEM-OpenCloud Manager to provision Jenkins. AEM-OpenCloud Manager helps you to manage your AEM OpenCloud environment by providing ready to use CI/CD pipelines
But before we can even start, we need to make sure the following prerequisites are met:
AEM installation including repository as AWS snapshots
Generate machine images with Packer-AEM
# AEM upgrade configuration profile
aem:
# Enable repository reconfiguration
enable_reconfiguration: true
# Install AEM Upgrade tools
enable_upgrade_tools: true
# Enable cleanup of *.bak files from the repository
# older than 30 days during offline-compaction
enable_bak_files_cleanup: true
bak_files_cleanup_max_age_in_days: 30
# Repository reconfiguration parameters
reconfiguration:
# Remove existing AEM Global Truststore
enable_truststore_removal: true
# Enabling SSL communication for AEM
# These parameters are similar to the one used by Packer-AEM
certs_base: s3://aem-opencloud/certificates
ssl_keystore_password: changeit
# Enable the creation of the AEM OpenCloud system users
enable_create_system_users: true
Generate machine images with Packer-AEM
- AEM Version (AEM 6.2 SP1)
- AEM Version (AEM 6.4 SP3)
- AEM SSL Certificate
- AEM 6.4 Jar File
- AEM 6.4 Service Pack 3
- Public certificate of the SAML Service Provider for enabling SAML authentication
Configuration profiles
The first thing we need to set up is two configuration profiles for our AEM stacks. One will be used to create the upgrade environment and the other one to create our AEM 6.4 SP3 environment. I’m going to use the configuration profiles aem-upgrade-aem62-aem64 and aem-consolidated-rhel7-aem64 from the AEM OpenCloud repository aem-helloworld-config and will explain you the necessary parameters for the upgrade.
We will use the config profile aem-upgrade-aem62-aem64 for the upgraded stack. This config profile enables the installation of the AEM upgrade tools and the repository reconfiguration. Additionally, it will remove the existing AEM Global Truststore and all *.bak files older than 30 days from the repository.
The first thing we need to set up is two configuration profiles for our AEM stacks. One will be used to create the upgrade environment and the other one to create our AEM 6.4 SP3 environment. I’m going to use the configuration profiles aem-upgrade-aem62-aem64 and aem-consolidated-rhel7-aem64 from the AEM OpenCloud repository aem-helloworld-config and will explain you the necessary parameters for the upgrade.
We will use the config profile aem-upgrade-aem62-aem64 for the upgraded stack. This config profile enables the installation of the AEM upgrade tools and the repository reconfiguration. Additionally, it will remove the existing AEM Global Truststore and all *.bak files older than 30 days from the repository.
aem:
# Enable repository reconfiguration
enable_reconfiguration: true
# Install AEM Upgrade tools
enable_upgrade_tools: true
# Enable cleanup of *.bak files from the repository
# older than 30 days during offline-compaction
enable_bak_files_cleanup: true
bak_files_cleanup_max_age_in_days: 30
# Repository reconfiguration parameters
reconfiguration:
# Remove existing AEM Global Truststore
enable_truststore_removal: true
# Enabling SSL communication for AEM
# These parameters are similar to the one used by Packer-AEM
certs_base: s3://aem-opencloud/certificates
ssl_keystore_password: changeit
# Enable the creation of the AEM OpenCloud system users
enable_create_system_users: true
The configuration profile aem-consolidated-rhel7-aem64 will be used with the default settings to create our first upgraded AEM 6.4 SP3 Stack.
Upgrade steps
Now that all requirements have been met, let’s take a look at the in-place upgrade steps. For those who want to know more about this upgrade process, I recommend you to read the official Adobe AEM Upgrade guideline. But In a nutshell, the upgrade steps are:
Preparation of the AEM Quickstart jar file
Content Repository Migration
Performing The Upgrade
The automation of these steps with AEM OpenCloud required us to develop the automation tools, new InSpec tests for the InSpec profile InSpec-aem and new Stack Manager events for AEM Stack Manager to trigger the upgrade steps in Jenkins.
Now that all requirements have been met, let’s take a look at the in-place upgrade steps. For those who want to know more about this upgrade process, I recommend you to read the official Adobe AEM Upgrade guideline. But In a nutshell, the upgrade steps are:
Preparation of the AEM Quickstart jar file
Content Repository Migration
Performing The Upgrade
The automation of these steps with AEM OpenCloud required us to develop the automation tools, new InSpec tests for the InSpec profile InSpec-aem and new Stack Manager events for AEM Stack Manager to trigger the upgrade steps in Jenkins.
Preparation of the AEM Quickstart jar file
The upgrade step preparation of the AEM Quickstart jar file backups the current crx-quickstart directory prepares the instance for the upgrade and unpacks the AEM 6.4 jar file.
The upgrade step preparation of the AEM Quickstart jar file backups the current crx-quickstart directory prepares the instance for the upgrade and unpacks the AEM 6.4 jar file.
# java -Xmx4096m -jar aem-quickstart.jar -unpack
After the preparation, ensure that the AEM 6.4 quickstart jar file exists at crx-quickstart/app/cq-quickstart-6.4.0-standalone-quickstart.jar
After the preparation, ensure that the AEM 6.4 quickstart jar file exists at crx-quickstart/app/cq-quickstart-6.4.0-standalone-quickstart.jar
Content Repository Migration
For the content repository migration. The crx2oak migration tool, defined in the Jenkins parameter source_crx2oak, needs to get prepared. Afterwards, we can trigger the content repository migration:
For the content repository migration. The crx2oak migration tool, defined in the Jenkins parameter source_crx2oak, needs to get prepared. Afterwards, we can trigger the content repository migration:
# java -Xmx4096m -jar cq-quickstart-6.4.0-standalone-quickstart.jar -v -x crx2oak -xargs -- --load-profile segment-no-ds
Once the migration is done the logfile crx-quickstart/logs/upgrade.log gets checked for the following keywords:
Performing The Upgrade
Performing the AEM upgrade is the most complicated and critical part of the upgrade process.
This example performs the AEM 6.4 upgrade on the author instance:
# java -server -Xmx4096m -Dcom.adobe.upgrade.forcemigration=true \
-Djava.awt.headless=true -Dsling.run.modes=author,crx3,crx3tar \
-jar crx-quickstart/app/cq-quickstart-6.4.0-standalone-quickstart.jar start -c crx-quickstart -i launchpad \
-p 4502 -Dsling.properties=crx-quickstart/conf/sling.properties
Unlike the previous upgrade commands, this command does not exit by itself, therefore we are waiting ten minutes before killing the command.# sleep 600
Our experience shows that a wait time of 10 minutes is enough to finish the AEM upgrade on an AWS m4.4xlarge instance with a repository size of ~30 Gigabyte. This value can be overwritten if necessary.
Afterward the logfile crx-quickstart/logs/upgrade.log gets checked again for the following keywords:
All these tests are giving us enough confidence that the AEM upgrade was successful.
Once the migration is done the logfile crx-quickstart/logs/upgrade.log gets checked for the following keywords:
- does it contain migration completed ?
- does it not contain WARN ?
- does it not contain ERROR ?
Performing The Upgrade
Performing the AEM upgrade is the most complicated and critical part of the upgrade process.
This example performs the AEM 6.4 upgrade on the author instance:
# java -server -Xmx4096m -Dcom.adobe.upgrade.forcemigration=true \
-Djava.awt.headless=true -Dsling.run.modes=author,crx3,crx3tar \
-jar crx-quickstart/app/cq-quickstart-6.4.0-standalone-quickstart.jar start -c crx-quickstart -i launchpad \
-p 4502 -Dsling.properties=crx-quickstart/conf/sling.properties
Unlike the previous upgrade commands, this command does not exit by itself, therefore we are waiting ten minutes before killing the command.# sleep 600
Our experience shows that a wait time of 10 minutes is enough to finish the AEM upgrade on an AWS m4.4xlarge instance with a repository size of ~30 Gigabyte. This value can be overwritten if necessary.
Afterward the logfile crx-quickstart/logs/upgrade.log gets checked again for the following keywords:
- does it contain UPGRADE TASK DONE
- does it contain UPGRADE FINISHED
- does it not contain ERROR
All these tests are giving us enough confidence that the AEM upgrade was successful.
AEM Upgrade pipeline
How do we integrate the upgrade steps in our automation process? Since we already have a process for creating an AEM environment, we only need to extend this process with our upgrade steps.
As you can see we are installing the service pack separately on the author and publish instances, and after the installation we are waiting 900 seconds before proceeding with the next step. Based on our experience this should give AEM enough time to install the service pack and be ready afterward.
Now that we have enough knowledge about the upgrade process and the automation process, let’s start upgrading AEM 6.2 to AEM 6.4 SP3.
How do we integrate the upgrade steps in our automation process? Since we already have a process for creating an AEM environment, we only need to extend this process with our upgrade steps.
- Create an AEM environment
- Run a readiness check
- Pre-deployment of custom code
- Run Offline-Compaction Snapshot
- Preparation of the AEM Quickstart jar file
- Content Repository Migration
- Performing The Upgrade
- Post-deployment of custom code compatible to AEM 6.4
- Deploy AEM6.4 Service Pack 3
- Take offline-snapshot
- Delete the environment
There are two important points. Firstly the offline-compaction will delete all *.bak files from the repository older than 30 days, as we configure this in the configuration-profile earlier. The second important point is we are using the deploy-artifacts feature to deploy three different deployment-descriptor. The first one is deploying custom code compatible for AEM 6.2. The second one is deploying the custom code compatible with AEM 6.4 and maybe some further packages e.g. trigger a repository reindexing. The third one will install AEM 6.4 Service Pack 3.
Here is an example of the AEM Service-Pack 3 deployment-descriptor:
Here is an example of the AEM Service-Pack 3 deployment-descriptor:
{
"author-publish-dispatcher": {
"packages": [
{
"source": "s3://aem-opencloud/adobeaemcloud/AEM-6.4.3.0-6.4.3.zip",
"group": "adobe/cq640/servicepack",
"name": "aem-service-pkg",
"version": "6.4.3",
"replicate": false,
"activate": false,
"force": false,
"sleep_seconds": "900"
},
{
"source": "s3://aem-opencloud/adobeaemcloud/AEM-6.4.3.0-6.4.3.zip",
"group": "adobe/cq640/servicepack",
"name": "aem-service-pkg",
"version": "6.4.3",
"replicate": false,
"activate": false,
"force": false,
"aem_id": "publish",
"sleep_seconds": "900"
}
],
"artifacts": [
]
}
}
As you can see we are installing the service pack separately on the author and publish instances, and after the installation we are waiting 900 seconds before proceeding with the next step. Based on our experience this should give AEM enough time to install the service pack and be ready afterward.
Now that we have enough knowledge about the upgrade process and the automation process, let’s start upgrading AEM 6.2 to AEM 6.4 SP3.
Upgrading AEM
Assuming aem-opencloud-manager was already successfully provisioned and all configuration profiles are prepared we can start the upgrade by running the pipeline aem-upgrade-in-place which can be found in the category migration, and fill out all necessary parameters.
Assuming aem-opencloud-manager was already successfully provisioned and all configuration profiles are prepared we can start the upgrade by running the pipeline aem-upgrade-in-place which can be found in the category migration, and fill out all necessary parameters.
AEM upgrade pipeline parameters
Define the snapshot ids of the repository with SOURCE_AUTHOR_SNAPSHOT_ID & SOURCE_PUBLISH_SNAPSHOT_ID or if the repository snapshots are already AEM-OpenCloud compatible you only need to define SOURCE_SNAPSHOT_STACK_PREFIX. The parameter STACK_S3_BUCKET is the bucket name of the AEM Stack and is needed for uploading all three the deployment descriptor files PRE_UPGRADE_DEPLOYMENT_DESCRIPTOR_URL, POST_UPGRADE_DEPLOYMENT_DESCRIPTOR_URL and AEM_UPDATE_DEPLOYMENT_DESCRIPTOR_URL to the right location.
Once the pipeline is triggered it may take 2 to 3 hours until the AEM upgrade is completed. On a stack where the AEM Hello World package is installed it took ~1h 35min.
Define the snapshot ids of the repository with SOURCE_AUTHOR_SNAPSHOT_ID & SOURCE_PUBLISH_SNAPSHOT_ID or if the repository snapshots are already AEM-OpenCloud compatible you only need to define SOURCE_SNAPSHOT_STACK_PREFIX. The parameter STACK_S3_BUCKET is the bucket name of the AEM Stack and is needed for uploading all three the deployment descriptor files PRE_UPGRADE_DEPLOYMENT_DESCRIPTOR_URL, POST_UPGRADE_DEPLOYMENT_DESCRIPTOR_URL and AEM_UPDATE_DEPLOYMENT_DESCRIPTOR_URL to the right location.
Once the pipeline is triggered it may take 2 to 3 hours until the AEM upgrade is completed. On a stack where the AEM Hello World package is installed it took ~1h 35min.
AEM 6.4 SP3 upgrade pipeline run
Once the pipeline finishes successfully, we will have new snapshots which can be used for all new AEM 6.4 environments that we are going to create.
Since we want to upgrade to AEM 6.5, we need to replace the AEM_UPGRADE_VERSION with 6.5, and because I couldn’t find a newer version of crx2oak, I will leave this property empty. The repository migration will then fallback to the default crx2oak tool which is provided by the 6.5 Jar file.
Once triggered it takes approximately 1h until the AEM upgrade to AEM 6.5 is finished. For those who are wondering why it’s faster. We aren’t installing any hotfixes or service packs.
Create an upgraded AEM stack
When creating a new AEM 6.4 stack we just need to define the parameter SOURCE_SNAPSHOT_STACK_PREFIX with the stack prefix of the automation pipeline e.g. AEMUPGRADE-CON. This will lookup the latest snapshots for this stack and attaches them to the new stack. After the stack creation, we should be able to access our upgraded repository and when we navigate to the AEM welcome page we see that AEM is running on version 6.4.3.0
When creating a new AEM 6.4 stack we just need to define the parameter SOURCE_SNAPSHOT_STACK_PREFIX with the stack prefix of the automation pipeline e.g. AEMUPGRADE-CON. This will lookup the latest snapshots for this stack and attaches them to the new stack. After the stack creation, we should be able to access our upgraded repository and when we navigate to the AEM welcome page we see that AEM is running on version 6.4.3.0
Successfully upgraded to AEM 6.4 SP3
AEM 6.5 Upgrade
At the time when I started writing this article, AEM 6.5 was not available. But now that AEM 6.5 is available, I’m curious how it works with AEM 6.5.
So I ran the upgrade automation for upgrading my AEM 6.2 instance to AEM 6.5.
The prerequisites are the same as for the AEM6.4 SP3 upgrade and the parameters I supplied to Jenkins are very similar to the ones I provided for the AEM Upgrade to AEM 6.4 SP3 as well. The ones I’ve overwritten are:
At the time when I started writing this article, AEM 6.5 was not available. But now that AEM 6.5 is available, I’m curious how it works with AEM 6.5.
So I ran the upgrade automation for upgrading my AEM 6.2 instance to AEM 6.5.
The prerequisites are the same as for the AEM6.4 SP3 upgrade and the parameters I supplied to Jenkins are very similar to the ones I provided for the AEM Upgrade to AEM 6.4 SP3 as well. The ones I’ve overwritten are:
AEM_UPGRADE_VERSION: 6.5
SOURCE_CRX2OAK:
Since we want to upgrade to AEM 6.5, we need to replace the AEM_UPGRADE_VERSION with 6.5, and because I couldn’t find a newer version of crx2oak, I will leave this property empty. The repository migration will then fallback to the default crx2oak tool which is provided by the 6.5 Jar file.
Once triggered it takes approximately 1h until the AEM upgrade to AEM 6.5 is finished. For those who are wondering why it’s faster. We aren’t installing any hotfixes or service packs.
AEM 6.5 upgrade pipeline run
After the successful AEM upgrade, we can check that we are now running on AEM 6.5.
After the successful AEM upgrade, we can check that we are now running on AEM 6.5.
Successfully upgraded to AEM 6.5.0
Conclusion
Besides the already implemented upgrade scenario of content export/import, we have also implemented the in-place migration. Thanks to the automation processes the AEM upgrade becomes less of a hassle and as you can see, we can easily upgrade to the latest available AEM version. But what does this mean in a real-life scenario?
In the real world, you are now able to create automation processes that allow you to improve the code upgrade process by testing the latest code commitments from your developers on an upgraded AEM environment from your production system without any human interactions. Further your developers can determine and fix bugs in a shorter period of time which leads to more efficiency as your developers will have more time they can invest for code quality and improvements. On top of that, your application team does not have to deal with how-to perform the AEM upgrade in much detail which also leads to saving resources.
To use this pipeline for a production release I would recommend adding some minor additional tasks depending on your need e.g. performing the Pre-Upgrade Maintenance Tasks or some manual tasks. But this should give you an easier understanding of the AEM In-Place upgrade and how an AEM In-Place upgrade can be automated for a production release.
Besides the already implemented upgrade scenario of content export/import, we have also implemented the in-place migration. Thanks to the automation processes the AEM upgrade becomes less of a hassle and as you can see, we can easily upgrade to the latest available AEM version. But what does this mean in a real-life scenario?
In the real world, you are now able to create automation processes that allow you to improve the code upgrade process by testing the latest code commitments from your developers on an upgraded AEM environment from your production system without any human interactions. Further your developers can determine and fix bugs in a shorter period of time which leads to more efficiency as your developers will have more time they can invest for code quality and improvements. On top of that, your application team does not have to deal with how-to perform the AEM upgrade in much detail which also leads to saving resources.
To use this pipeline for a production release I would recommend adding some minor additional tasks depending on your need e.g. performing the Pre-Upgrade Maintenance Tasks or some manual tasks. But this should give you an easier understanding of the AEM In-Place upgrade and how an AEM In-Place upgrade can be automated for a production release.
No comments:
Post a Comment
If you have any doubts or questions, please let us know.