AEM Tune your JVM For Better Performance

Environment: AEM 6.3, JRE 1.8

• If you are getting out-of-memory errors, either add memory or make that there isn't a memory leak.
• If there is a memory leak, use -XX:-HeapDumpOnOutOfMemoryError to get a heap dump and analyze it to find the root cause.
• You can use any heap dump analyzers, such as a sap memory analyzer, yourkit profiler, or eclipse memory analyzer.
• Turn on garbage collection logs to understand the behavior of your application's memory usage. Make sure that you look at the available JVM options from your JVM vendor. 

For Java sun, it's here: http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp