Oak TarMK Compaction

If we are using Tar files as the storage, it tends to grow in size and starts claiming disk space every time when data is created or updated as data in tar files are never overwritten rather it keeps adding new versions. To mitigate the same, AEM has garbage collection mechanism which is known as ‘Tar Compaction’ to remove the unused data and reclaim the disk space.

We can be triggered from ‘Revision Clean Up’ in the Operations Dashboard (http://<host>:<port>/libs/granite/operations/content/maintenance.html). However, it can done faster via offline compaction. Follow the below steps to do the same

Step1: Download the ‘Oak-run’, Tar Compaction Tool – http://mvnrepository.com/artifact/org.apache.jackrabbit/oak-run/

Make sure you download the right version of the tool based on your oak repository version. Latest version of oak with AEM 6.1 and SP1 is 1.2.11



Step2: Shutdown the AEM instance. Take the backupjava –jar oak-run-*.jar backup <aem-folder>/crx-quickstart/repository/segmentstore <backup>

Note:
<aem-folder> – is the path where the aem jar file resides
<backup> – is the path where the backup has to be stored

Step3: Find all the checkpoints using the oak-run tool with the below command. This finds all the unreferenced pointsjava -jar oak-run-*.jar checkpoints <aem-folder>/crx-quickstart/repository/segmentstore
checkpoints

Step4: Remove all the unreferenced checkpointsjava -jar oak-run-*.jar checkpoints <aem-folder>/crx-quickstart/repository/segmentstore rm-unreferenced

You will see how many checkpoints get removed, if you had any!



Step4: Now, run the compaction of the tar filesjava -jar oak-run-*.jar compact <aem-folder>/crx-quickstart/repository/segmentstore



Step5: Check your repository file size. It would have reduced drastically!compact completion

We can also create a script combining all the above steps to automate the same

#!/bin/bash

today = "$(date +'%d-%m-%Y')"

logfile = "tarcompact-$today.log"

aeminstallfolder = "/opt/data/aem"

aemfolder = "$aeminstallfolder/crx-quickstart"

oakrun = "$aeminstallfolder/help/oak-run-1.2.11.jar"

## Shutdown AEM

printf "Shutting down AEM.\n"

$aemfolder / bin / stop

now = "$(date)"

echo "AEM Shutdown at: $now" >> $aeminstallfolder / help / logs / $logfile

## Find old checkpoints

printf "Finding old checkpoints.\n"

java - jar $oakrun checkpoints $aemfolder / repository / segmentstore >> $aeminstallfolder / help / logs / $logfile

## Delete unreferenced checkpoints

printf "Deleting unreferenced checkpoints.\n"

java - jar $oakrun checkpoints $aemfolder / repository / segmentstore rm - unreferenced >> $aeminstallfolder / help / logs / $logfile

## Run compaction

printf "Running compaction. This may take a while.\n"

java - jar $oakrun compact $aemfolder / repository / segmentstore >> $aeminstallfolder / help / logs / $logfile

## Report Completed

printf "Compaction complete. Please check the log at:\n"

printf "$aeminstallfolder/help/logs/$logfile\n"

## Start AEM back up

now = "$(date)"

printf "Starting up AEM.\n"

$aemfolder / bin / start

echo "AEM Startup at: $now" >> $aeminstallfolder / help / logs / $logfile