Post Upgrade Checks and Troubleshooting post-upgrade-checks-and-troubleshooting
Post Upgrade Checks post-upgrade-checks
Following the In-Place Upgrade the following activities should be executed to finalize the upgrade. It is assumed AEM has been started with the AEM 6.5 LTS jar and that the upgraded code base has been deployed.
Verify logs for Upgrade Success verify-logs-for-upgrade-success
upgrade.log
In the past, inspecting the post upgrade state of your instance required careful inspection of various log files, parts of the repository and the launchpad. Generating a post upgrade report can help detect defective upgrades before going live.
The main purpose of this feature is to reduce the need for manual interpretation or complex parsing logic across multiple endpoints required to qualify the success of an upgrade. The solution aims to provide unambiguous information for external automation systems to react on the success or the identified failure of an update.
More specifically, it ensures that:
- Upgrade failures detected by the upgrade framework are centralized in a single upgrade report.
- The upgrade report includes indicators about necessary manual intervention.
To accommodate this, changes have been made in the way logs are generated in the upgrade.log file.
error.log
The error.log should be carefully reviewed during and following the startup of AEM using the target version jar. Any warnings or errors should be reviewed. In general, it is best to look for issues at the beginning of the log. Errors that occur later in the log may actually be side-effects of a root cause that is called out early in the file. If repeated errors and warnings occur see below for Analyzing Issues with the Upgrade.
Verify OSGi Bundles verify-osgi-bundles
Navigate to the OSGi console /system/console/bundles and look to see if any bundles are not started. If any bundles are in an installed state, consult the error.log to determine the root issue.
Verify Oak Version verify-oak-version
Following the upgrade, you should see that Oak version has been updated to 1.68.x. To verify the Oak version, navigate to the OSGi console and look at the version associated with Oak bundles: Oak Core, Oak Commons, Oak Segment Tar.
Initial Validation of Pages initial-validation-of-pages
Perform an initial validation against several pages in AEM. If upgrading an Author environment open the Start page and Welcome page ( /aem/start.html, /libs/cq/core/content/welcome.html). On both Author and Publish environments, open a few application pages and smoke test that they render correctly. If any issues occur consult the error.log to troubleshoot.
Verify Scheduled Maintenance Configurations verify-scheduled-maintenance-configurations
Enable Data Store Garbage Collection enable-data-store-garbage-collection
If using a File Data Store, ensure that the Data Store Garbage Collection task is enabled and added to the Weekly Maintenance list. Instructions are outlined under Revision Cleanup.
Enable Online Revision Cleanup enable-online-revision-cleanup
If using MongoMK or the new TarMK segment format, ensure that the Revision Clean Up task is enabled and added to the Daily Maintenance list. Instructions are outlined under Revision Cleanup.
Enable Replication Agents enable-replication-agents
Once publish environment has been fully upgraded and validated, enable replication agents on the Author Environment. Verify that agents are able to connect to respective Publish instances. See Upgrade Procedure for more details on order of events.
Enable Custom Scheduled Jobs enable-custom-scheduled-jobs
Any scheduled jobs as part of the code base can be enabled at this point.
Execute Test Plan execute-test-plan
Execute detailed test plan as defined in Upgrading Code and Customizations under the Testing Procedure section.
Analyzing Issues With The Upgrade analyzing-issues-with-the-upgrade
This section contains some issue scenarios one might face along the upgrade procedure to AEM 6.5 LTS.
Packages and Bundles Fail to Update packages-and-bundles-fail-to-update
In case packages fail to install during the upgrade, the bundles they contain will not be updated either. This category of issues is caused by data store misconfiguration. They will also appear as ERROR and WARN messages in the error.log. Since in most of these cases the default login may fail to work, you can use CRXDE directly to inspect and find the configuration problems.