Monthly Offline Maintenance of the AEM Repository

Now that Adobe has fixed many of the problems with On-Line RevisionGC() and the associated On-Line tar Compaction in AEM 6.3 onwards, I notice that administrators are beginning to go easy on repository maintenance.

Note that Adobe still recommends monthly off-line maintenance. This monthly planned downtime is a very good idea for two reasons:

1) Off-Line tar Compaction is still better at re-claiming wasted storage than its On-Line counterpart

2) The stop/start cycle will expose other repository integrity issues that crop up only when AEM is re-started (at the most inopportune time!).

So here are steps you can follow (you may end up adding others).

Make sure AEM is shut down.

On Linux, make sure you are running as the same user as the one that owns the AEM files. If you run as root, remember to change file ownership back afterwards.

Make sure the oak-run.jar you use matches the version of Jackrabbit Oak AEM is on.

See this and this for more information.

Some of these steps can take many hours or even days.

1) Stop AEM
2) Re-index some (or all) key indexes with

java -Doak.index.ramBufferSizeMB=4096 -jar oak-run-1.8.9.jar index --reindex --index-paths=/oak:index/lucene,/oak:index/cqPageLucene,/oak:index/damAssetLucene,/oak:index/versionStoreIndex,/oak:index/workflowDataLucene --read-write --fds-path=G:\AEM_6.4\author\crx-quickstart\repository\datastore G:\AEM_6.4\author\crx-quickstart\repository\segmentstore

3) Start AEM for the new indexes to be downloaded and cached locally

4) Stop AEM

5) Check the consistency of the Oak repository node store

java -jar oak-run-1.8.9.jar check --bin G:\AEM_6.4\author\crx-quickstart\repository\segmentstore

6) Remove any unreferenced checkpoints

java -jar oak-run-1.8.9.jar checkpoints G:\AEM_6.4\author\crx-quickstart\repository\segmentstore rm-unreferenced

7) tar Compaction

java -Dtar.memoryMapped=true -Xmx8g -jar oak-run-1.8.9.jar compact G:\AEM_6.4\author\crx-quickstart\repository\segmentstore

8) Move any .tar.bak and .tar.ro.bak files out of the /segmentstore folder to an archive folder somewhere (can be deleted later)

9) Check the consistency of the Oak repository blob store, make sure no missing blobs are reported

java -jar oak-run-1.8.9.jar datastorecheck --consistency --repoHome G:\AEM_6.4\author\crx-quickstart\repository --store G:\AEM_6.4\author\crx-quickstart\repository\segmentstore --fds G:\AEM_6.4\author\crx-quickstart\install\org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore.config

10) If you run the above commands as root, change file ownership back with
chown aem:aem -R /mnt/aem/author

10) Start AEM