How to Rotate Logs in CQ / AEM

Use Case:
Rotate all logs in CQ
Logs are taking a lot of space in file system

Solution:
Till CQ5.4, there are different kind of logs in CQ. Please refer http://www.wemblog.com/2012/01/how-to-change-different-log-locations.html for that, here is process to rotate all logs,

CQ Logs:
crx-quickstart/logs/error.log :
You can rotate error log using sling:OsgiConfig under /libs/sling/config/org.apache.sling.commons.log.LogManager. Please override this under /apps/sling/config/org.apache.sling.commons.log.LogManager. You can additionally have environment specific configuration as config.author or config.publish

crx-quickstart/logs/request.log and crx-quickstart/logs/access.log:
http://dev.day.com/content/kb/home/cq5/CQ5SystemAdministration/HowToRotateRequestAndAccessLog.html

CRX Log: crx-quickstart/logs/crx/error.log

You can rotate CRX log through configuration in /crx-quickstart/server/runtime/0/_crx/WEB-INF/log4j.xml by changing configuration of

<appender name="error" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="crx-quickstart/logs/crx/error.log"/>
<param name="maxFileSize" value="10MB"/>
<param name="maxBackupIndex" value="20"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{dd.MM.yyyy HH:mm:ss} *%-5p* %c{1}: %m (%F, line %L)%n"/>
</layout>
</appender>

/crx-quickstart/logs/{stdout,stderr}.log

These logs are not controlled by sling or log4j. You have to use apache log rotate utilities to do this. You might have to do following to rotate these logs (By default they are not rotated)

After
export CQ_LOGDIR
CQ_LOG="${CQ_LOG:-"$CQ_LOGDIR/startup.log"}"

------- ADD ------

CRX_LOG_STDOUT="${CQ_LOG:-"$CQ_LOGDIR/../../logs/stdout.log"}"
export CRX_LOG_STDOUT

CRX_LOG_STDERR="${CQ_LOG:-"$CQ_LOGDIR/../../logs/stderr.log"}"
export CRX_LOG_STDERR

And change your rotate option as,

exec $jvmExe 2>&1 | /usr/sbin/rotatelogs "$CQ_LOG.%Y%m%d" 86400
| /usr/sbin/rotatelogs "$CRX_LOG_STDOUT.%Y%m%d" 86400 | /usr/sbin/rotatelogs "$CRX_LOG_STDERR.%Y%m%d" 86400

That mean rotate both log after 86400 second (1 day).
Tar Journal:

How to rotate TarJournal in Shared Nothing Clustering
Use Case: Your tar Journal is growing and consuming a lot of space.

Solution: If you are using shared clustered Please refer http://dev.day.com/content/kb/home/Crx/Troubleshooting/JournalTooMuchDiskSpace.html

For shared-nothing clustering, you need following configuration (If you want rotation after every 24 Hour) in repository.xml

<Journal class="com.day.crx.persistence.tar.TarJournal">
<param name="maxFileSize" value="104857600" />
<param name="maximumAge" value="PT24H" />
</Journal>

Please note that Age specified as the duration in ISO 8601 or plain format. Journal files that are older than the configured age are automatically deleted. The default is "P1M", which means files older than one month are deleted.

More detail can be found here http://en.wikipedia.org/wiki/ISO_8601

Important Note: If you do the above configuration then make sure that another instance in the cluster is not down for more than the specified time in "maximumAge", Otherwise cluster will get Out Of sync because of rotation of tarJournal.


Server Logs
crx-quickstart/server/logs/{access,startup}.log

Server logs can not be rotated at sling level. You need to rotate these logs at OS level using logrotate utility similar to http://httpd.apache.org/docs/2.0/programs/rotatelogs.html

For this in serverctl script you can do following (Adjust log file location accordingly),

After
export CQ_LOGDIR
CQ_LOG="${CQ_LOG:-"$CQ_LOGDIR/startup.log"}"

------- ADD ------

CQ_LOG_ACCESS="${CQ_LOG:-"$CQ_LOGDIR/access.log"}"
export CQ_LOG_ACCESS

And change your rotate option as,

exec $jvmExe 2>&1 | /usr/sbin/rotatelogs "$CQ_LOG.%Y%m%d" 86400
| /usr/sbin/rotatelogs "$CQ_LOG_ACCESS.%Y%m%d" 86400

That mean rotate both log after 86400 second (1 day).

If you do not have logrotate utility, OOTB CQ support rotation based on size (Not on date or number), For that go to /crx-quickstart/server/etc/server.xml and add following line

<log-file>
<log-file-name>logs/access.log</log-file-name>
<log-max-file-size>20MB</log-max-file-size>
</log-file>

<log-file>
<log-file-name>../logs/server.log</log-file-name>
<log-max-file-size>20MB</log-max-file-size>
</log-file>

Launchpad Logs

crx-quickstart/launchpad/logs/error.log

You can configure launchpad error log using felix configuration or creating a custom configuration in crxde may be under /apps/sling/config. Please see http://sling.apache.org/site/logging.html for more information. You can also control sling logs rotation though /crx-quickstart/launchad/sling.properties

Look for, following properties
org.apache.sling.commons.log.file.number=5
org.apache.sling.commons.log.file.size='.'yyyy-MM-dd
org.apache.sling.commons.log.file=${sling.home}/logs/error.log



Dispatcher log:
For dispatcher log you have to use http://httpd.apache.org/docs/2.0/programs/rotatelogs.html (Similar to server log). Something like

DispatcherLog '|/usr/local/apache/bin/rotatelogs <Path-to-your-log>/logs/%Y-%m-%d-dispatcher.log 86400'

http://dev.day.com/content/kb/home/Dispatcher/faq-s/SetupDispatcherLogRotation.html

CQ5.5
Good thing about CQ5.5 is there is not different locations to control log rotation. You can control all log rotations through sling configuration http://sling.apache.org/site/logging.html. One thing to note here is, unlike CQ5.4 ../logs/<your-log> will point to logs folder under CQ root and not under /crx-quickstart/logs. If you want your logs to go under /crx-quickstart/logs you have to use log location as logs/<your-log> in sling configuration.

It is always recommended to modify your log configuration under /apps/<Path>, That way you can port it to different environment.

Some more reference
http://dev.day.com/content/kb/home/cq5/CQ5SystemAdministration/ReconfigureLogFilePaths.html

http://dev.day.com/content/docs/en/cq/current/deploying/configure_logging.html

Note: For OS level log rotation, You can use any log rotation you want and give path to CQ log files.

You can try this option as well, If log rotate utility option is not working after config through serverctl.

$NOHUP $jvmExe | /usr/sbin/rotatelogs "$CQ_LOG.%Y%m%d" 86400>> /dev/null 2>&1