Configure Data Store and Node Store in AEM 6

In this article we are going to learn how to configure Data Store and Node Store in AEM 6. We all know how to install AEM, but it’s really important to know which type of configuration is best for which type of scenario. What are the different ways to configure data store and node store in AEM 6.1.

Major difference between CQ5.x and AEM6.x:
AEM6.x implements OAK repository whereas older CQ5 uses CRX2.
AEM6.x uses Microkernel , CQ5.x uses Persistence manager.
Custom re-index is possible in AEM 6.x depends upon the queries . (Will cover more on this in later article)
Slightly is introduced in AEM6.x whereas CQ5 uses JSP.
Prerequisite:
AEM 6.1 jar with valid License file.
Decide which Node and Datastore is required for your project.
Configure different Data Store in AEM 6:

There are basically two type of data store available in AEM 6 Amazon S3 bucket and File Datastore.

Amazon S3 Bucket Data Store :
This type of storage requires account with Amazon. We need this type to store more data’s in an external S3 bucket.

Config file name :
org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore.config

Basic config:
accessKey=<provided by Amazon>
secretKey=<provided by Amazon>
s3Bucket=<provided by Amazon>
s3Region=<provided by Amazon>
s3EndPoint=<provided by Amazon>
connectionTimeout=120000
socketTimeout=120000
maxConnections=40
maxErrorRetry=10
writeThreads=20
cacheSize=<size in bytes>
concurrentUploadsThreads=10
asyncUploadLimit=100
cachePurgeTrigFactor=0.95d
path=~/datastore

File Datastore:

This method is required to store all binary data in same local file system .

Config file name :
org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore.config

Basic config:
path=~/datastore
minRecordLength=<values in bytes>

Configure Node Store types in AEM 6:

Document Node store:
This type needs to configure MONGODB. Mongo DB setup is configured for HA of instances.

Config file name :
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService.config

Basic config:
mongouri=mongodb://<hostname>:<port>
db=<db name>
customBlobStore=false
Feel free to drop a comment, if you face any issue while implementing MongoDB

Segment node store:
This method uses to store Metadata , properties in TARMK implementation. By default AEM uses segment store.

Same article contains offline compaction details below.

Config file name :
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.config

Basic config:
customBlobStore=true

NOTE: By default , segment store folder creates under /repository folder. If still needs to change the path , we can use repository.home in config file.

Steps to install AEM 6:
Move Jar and license file to appropriate folder.
Rename jar file as “cq6-p<portnumber>.jar” # specify port number 4 or 5 digit
Just Unzip the jar file

Java –jar cq6-p<portnumber>.jar –unpack

Sample output:
[pradeep@host ~]$ <strong>java -jar cq6-author-p4507.jar -unpack</strong>

Loading quickstart properties: default

Loading quickstart properties: instance

Setting properties from filename '/home/pradeep/cq6-author-p4507.jar'

Option '-quickstart.server.port' set to '4507' from filename cq6-author-p4507.jar

Verbose mode - stdout/err not redirected to files, and stdin not closedResourceProvider paths=[/gui, /gui/default]

quickstart.build

quickstart.properties not found, initial install

UpgradeUtil.handleInstallAndUpgrade has mode INSTALL

Saving build number in quickstart.properties

Upgrade: no files to restore from pre-upgrade backup

31 files extracted from jar file

Running chmod +x /home/pradeep/crx-quickstart/bin/start

Running chmod +x /home/pradeep/crx-quickstart/bin/stop

Running chmod +x /home/pradeep/crx-quickstart/bin/status

Running chmod +x /home/pradeep/crx-quickstart/bin/quickstart

Not starting the Quickstart server as the -unpack option is set

Quickstart files unpacked, server startup scripts can be found under /home/pradeep/crx-quickstart

Once extracted properly, check crx-quickstart folder present in the same path
Inside Crx-quickstart folder , create a folder named as “install” folder.
As per above configuration , create config file and configuration.
Once done, start the instance.
Check all bundles are inactive state and review the error log.
Issues while configuring S3 bucket:

After all config made as per above, we faced issues related to Amazon config and local cache path not created properly.

After lots of verification on config file, we finally identified as the config made under install folder is not override on our OSGI console.

Open config similar to your setup and verify the configuration override properly.



Sample S3 config :


Sample Segment store config:


Offline compaction:
Few may face issues with disk space issue on Segment store folder .

To reduce space , AEM has provided with compaction tool. This post explains about offline compaction techniques.

Steps to perform offline compaction:
Download and install latest oak-run . Please visit below URL to check the updates
https://repository.apache.org/content/repositories/releases/org/apache/jackrabbit/oak-run/
Stop the AEM instance.
Backup the instance .
Check the size before running command
Run the below command
java -jar oak-run-x.x.xx.jar checkpoints <segmentstore path>
java -jar oak-run-x.x.xx.jar checkpoints <segmentstore path> rm-unreferenced
java -jar oak-run-x.x.xx.jar compact <segmentstore path>
Start the instance.
Check the segment store size.

Explanation:
First command will identify the older checkpoints.
Second command, checks for unreferenced checkpoints and remove them
Compact the segment store.

NOTE: Once you are familiar with these steps , please implement in script .

Feel free to drop a comment or write to us on mongo DB setup and configuration related issues.