Performance Tuning: Adobe CQ5/Adobe AEM

This post explains the approach to tune the Adobe CQ5/Adobe AEM (The version referred here is a 5.6.1) platform.

Enable JMX Remote monitoring:
Remote JMX monitoring will help us to monitor the server resources (e.g. memory usage and CPU usage) through JMX clients such as VisualVM.

Open <<AEM_HOME>>/crx-quikstart/bin/start.sh and append the following configurations in CQ_JVM_OPTS.

CQ_JVM_OPTS='-server -XX:PermSize=768m -XX:MaxPermSize=768m -Xms6g -Xmx6g - XX:NewRatio=1-XX:SurvivorRatio=12-XX:+UseParallelGC -XX:+UseParallelOldGC- XX:ParallelGCThreads=8 -XX:+HeapDumpOnOutOfMemoryError - XX:HeapDumpPath=/app/aem/crash/aem_java.hprof -Xloggc:/app/aem/crash/aem_gc.log - XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintHeapAtGC - Djava.awt.headless=true -Dcom.sun.management.jmxremote.port=50055 - Dcom.sun.management.jmxremote.authenticate=false - Dcom.sun.management.jmxremote.ssl=false'

The memory usage of the server also can be monitored from the system console. Login to ConfigMgr (http://localhost:4502/system/console/configMgr) Main--> Memory Usage



JVM Tuning:
The default JVM parameters are not optimal for running large applications. So the parameters (e.g. memory and GC) should be tuned for optimal performance of the application.
Create a folder crash under <<AEM_HOME>>
Open <<AEM_HOME>>/crx-quikstart/bin/start.sh and replace the variable CQ_JVM_OPTS with
the following data.

CQ_JVM_OPTS='-server -XX:PermSize=768m -XX:MaxPermSize=768m -Xms6g -Xmx6g - XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC -XX:+UseCompressedOops - XX:+UseCMSCompactAtFullCollection -XX:CMSFullGCsBeforeCompaction=0 - XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/app/aem/crash/aem_java.hprof -Xloggc:/app/aem/crash/aem_gc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps - XX:+PrintGCDateStamps -XX:+PrintHeapAtGC -Djava.awt.headless=true '

•   Apply the required load to the system – Apache JMeter can be used to load the request to the system.
  
•   Monitor the system resources like CPU, Memory, and threads using VisualVM while loading is applied to the system - the memory usage can also be monitored through AEM ConfigMgr.
  
•   Change the memory parameters and GC parameters based on the above test result.
  

If you provide too little memory to an application it will run out of memory. The JVM will not be able to free up memory space at the rate that your application needs it. In this scenario, JVM will throw an OutOfMemoryError and shut down completely
The above three steps should be repeated to achieve the optimum system performance by applying different values for the parameters.

Thread Dumps:
Thread Dump is a snapshot taken at a given time which provides a complete listing of all created Java Threads. The thread dump can be analyzed to determine the bottleneck or blocking threads.
Different tools can be used to analyze the thread dumps e.g. Samurai and IBM Thread and Monitor Dump Analyzer for Java.

Identify The AEM PID:

threaddump.sh
#!/bin/sh #

• #  Takes the JBoss PID as an argument. #
  
• #  Make sure you set JAVA_HOME #
  
• #  Create thread dumps a specified number of times (i.e. LOOP) and INTERVAL. #
  
• #  Thread dumps will be collected in the same directory from where this script is been executed. #
  
• #  Usage: sh ./threaddump.sh <AEM_PID> #
  
• #  Number of times to collect data.
  

LOOP=10
# Interval in seconds between data points.
INTERVAL=5
# Setting the Java Home, by giving the path where your JDK is kept JAVA_HOME=/usr/local/java/jdk1.7.0_67
for ((i=1; i <= $LOOP; i++)) do
$JAVA_HOME/bin/jstack -l -J-d64 -m $1 >> jstack.$pid.$(date +%H%M%S.%N) echo "thread dump #" $i
if [ $i -lt $LOOP ]; then
echo "sleeping..."
sleep $INTERVAL fi
done
Execute the above script to take the thread dump a specified number of cycle and interval: sh ./threaddump.sh 5475
The application can be tuned based on the thread dump analysis.

AEM profiler:
Sometimes the process will be very slow and it will take more time to complete. This issue will happen most of the time, if there is some issue with the executed code.
The inbuilt AEM Profiler can be used to identify the issue - There are different profiler tools available to perform the same. E.g YourKit
Login to ConfigMgr (http://localhost:4502/system/console/configMgr) 
Main -->Profiler

Click on start collecting by applying the load to the server

Stop the collection after a particular interval



This can be used to determine which JVM threads are consuming more CPU and the associated packages and classes.
After identifying the class that is taking more time can be tuned to resolve the issue.

HTTP Service Thread Count:
Login to system/console and go to Configuration Click on Day CQSE HTTP Service
Change the Maximum Number of threads

Tuning the components timing:
Sometimes there is a possibility the individual components will take more time, follow the below steps to identify the component that is taking more time.
Open /libs/foundation/components/timing/timing.jsp
Uncomment the following code:-
/* uncomment the following to get more timing details in the page out.println("\nRaw RequestProgressTracker data:");
StringBuilder mb = new StringBuilder();
Iterator<String> it = t.getMessages();
while(it.hasNext()) { mb.append(it.next());
}
out.print(mb.toString()); out.println("\nChartData dump:");

for(ChartBar d : chartData) { out.print(d.start); out.print(' '); out.print(d.fullname); out.print(" ("); out.print(d.elapsed); out.println("ms)");
} */
Include the timing component in the global template
<cq:include path="timing" resourceType="foundation/components/timing"/>
Open the page in the browser and view the source
Search for “Timing chart URL:” in the source and copy the chart URL.

Open the chart URL in the browser.
This will show the loading time of each component on the page.

Tune the individual component that is taking more time.

Integration with third-party services:

If you are integrating third-party services via HttpClient then make sure the MultiThreadedconnectionManager is implemented to improve performance.

private MultiThreadedHttpConnectionManager connectionManager;
public HttpClient getConfiguredHttpClient() throws TRCommerceException{ HttpClient http = new HttpClient(connectionManager);
http.getParams().setParameter(HttpConnectionParams.CONNECTION_TIMEOUT, PropertiesUtil.toInteger(TRCommerceConstants.getPropertyValueObj(TRCommerceConstants.C ONNECTION_TIMEOUT),TRCommerceConstants.DEFAULT_CONNECTION_TIMEOUT));
http.getParams().setParameter(HttpConnectionParams.SO_TIMEOUT, PropertiesUtil.toInteger(TRCommerceConstants.getPropertyValueObj(TRCommerceConstants.S OCKET_TIMEOUT),TRCommerceConstants.DEFAULT_SOCKET_TIMEOUT));
http.getParams().setConnectionManagerTimeout(CONNECTION_MANAGER_TIMEOUT);
return http; }

Static
{
HttpConnectionManagerParams params = new HttpConnectionManagerParams();
params.setDefaultMaxConnectionsPerHost("100"); params.setMaxTotalConnections("100");
connectionManager = new MultiThreadedHttpConnectionManager(); connectionManager.setParams(params);
}

Enabling compression:
Go to ConfigMgr (http://localhost:4502/system/console/configMgr) Scroll down and click on "Day CQ HTML Library Manager"
Click on the Minify checkbox
Enable Gzip

Log levels:
Make sure the log levels for all the loggers are set as Error to reduce the I/O overhead to improve performance.
Go to configMgr (http://localhost:4503/ system/console/configMgr
Search for “Apache Sling Logging Configuration” and modify the Log level to Error.

Search for “Apache Sling Logging Logger Configuration” logger factory and modify the Log Level for all the logger configurations to Error.

Https Communication:
To improve the performance, we can enable http protocol for internal communication – http communication will not have the SSL overhead so the performance will be improved.

References:
https://docs.adobe.com/docs/en/cq/5-6-1/deploying/performance.html https://helpx.adobe.com/experience-manager/kb/performancetuningtips.html