Debugging Slowness in your AEM instance : Part 2

In this post let’s take a look at some other tools which you can use to find out troubled areas of your application. One such tool is JConsole that comes with JDK. You can use JConsole to find out if your application is having memory leaks, thread deadlock etc. This tool can be used for any java application including AEM.

To run JConsole simply type jconsole in your command line (you should have JDK installed in your system and PATH variable pointing to bin).

Once JConsole comes up you will see a screen like this


Select your AEM process and click on connect (if you are running AEM locally)

In case AEM is running in a different JVM provide the hostname and port number where hostName is the name of the system running the application and portNum is the port number you specified when you enabled the JMX agent when you started the JVM

For more details on remote monitoring see this link

You can use load testing tool (like Apache Jmeter) to generate enough load on your instance .

The JConsole interface is composed of six tabs:
Summary tab: displays summary information on the JVM and monitored values.
Memory tab: displays information on memory use.
Threads tab: displays information on thread use.
Classes tab: displays information on class loading
MBeans tab: displays information on MBeans
VM tab: displays information on the JVM

Once connected you should see below screen. It will give you snapshot of your heap memory usage, running threads, number of class loaded and CPU usages.

Monitor your application over a period of time to get a picture of what is going on in your application


Detecting Memory leak: look at memory graph carefully and observe the trend line, if your application does not have a memory leak it would look something like this.


This kind of graph shows that memory is being reclaimed at intervals (when your GC runs), you can even click on Perform GC button to run garbage collector.

If your application is having memory leak you would see a graph like below, which shows that your memory usages are increasing over time , you should observe this graph for sufficient time interval so your GC run enough number of times.


Once you have found that there are memory leaks in your application, you can use tools like Eclipse MAT (Memory analyzer tool http://www.eclipse.org/mat/ ) to locate memory leaks in your code.

Some other stats that are available to you using JConsole are number of active thread, number of peak threads, blocked and waiting threads, you can also check if there is any deadlock by clicking in detect deadlock.

Threads
Live threads: Current number of live daemon threads plus non-daemon threads
Peak: Highest number of live threads since JVM started.
Daemon threads: Current number of live daemon threads
Total started: Total number of threads started since JVM started (including daemon, non-daemon, and terminated).


You can also see number of class loaded in your application and overall VM summey, in the VM Summary Tab
MAT (Memory analyzer tool)

The Eclipse Memory Analyser Tooling (MAT) is a set of plug-ins for the Eclipse IDE which provides tools to analyze heap dumps from Java application and to identify memory problems in the application. This helps the developer to find memory leaks and high memory consumption issues.

You will need a heap dump file to use MAT. A heap dump is a snapshot of the complete Java object graph on a Java application at a certain point in time. It is stored in a binary format called HPROF.

It includes all objects, fields, primitive types and object references. You can use below jmap command to get heap dump of your running AEM instance.
jmap [ option ] pid

where pid is the process id of your running java process (AEM)

for taking dump use option value as
-dump:[live,]format=b,file=<filename>

Dumps the Java heap in hprof binary format to filename. The live suboption is optional. If specified, only the live objects in the heap are dumped.

It is possible to instruct the JVM to create automatically a heap dump in case that it runs out of memory, i.e. in case of a OutOfMemoryError error. To instruct the JVM to create a heap dump in such a situation, start your Java application with the -XX:+HeapDumpOnOutOfMemoryError option.

To open the heap dump in MAT click on file , open heapdump


Once you open the heap dump file you should see a screen which will provide you option to generate leak suspect report.


The leak suspect report will give you possible memory leaks in your application .


Other than this you have various other reports generated by MAT that will give you enough idea about what objects are occupying the largest heap, you can drill down these reports to refine your findings and come to conclusion, some screenshots below






For more details on reports generated by MAT and how to analyze them please see this link

VisualVM:
VisualVM is a visual tool that integrates some of the commandline JDK tools and gives you powerful profiling capabilities. The commandline tools that it bundles up include jstat, JConsole, jstack, jmap and jinfo, which you get with your standard JDK distribution

VisualVM will allow you to generate and analyze heap data, track down memory leaks, monitor the garbage collector and perform memory and CPU profiling, as well as being able to browse and operate on MBeans. While VisualVM runs on JDK6 it can monitor applications created since JDK1.4.

To start visualVM just type jvisualvm on command line (JDK1.6 or up should be installed and be in your PATH). Once it gets started select AEM Process if running locally or use remote option if AEM is running on remote machine and provide host and port



One it connects to running java process you should start seeing various status related to heap memory, running threads, cpu usage and number of classes loaded. You can even generate heap dump and thread dump from this tool and do the analysis as suggested earlier.


VisualVM allows you to:
Monitor application performance and memory consumption
Monitor application threads
Profile application performance and analyze memory allocation
Take thread or heap dumps
Analyze core dumps
Save snapshots to analyze the application offline.

Other than the tools listed above there are many tools which you can use to debug slowness or performance issues in AEM some are listed below

Apache Bench: Identify memory leaks, selectively analyze response time.
JMeter: Load and functional tests.
JProfiler: In-depth CPU and memory profiling.

YourKit. CPU and memory profiling tool Used when analyzing slow requests during development.

Also use eclipse debug mode to step thru your code by running your AEM instance in debug mode, use breakpoint, code watch to see what your code is doing.

This was about tools that you can use to find performance bottlenecks from backend perspective In next article we will see how to debug performance issues from front end perspective and what tool are available