Site cache not getting updated

Issue

When activating content in your AEM site, the changes show on the publish instances, but not on the actual website.

Cause

There can be many causes for thidepending on the architecture it varies.

The causes can be:

• If you are using a CDN
  • Incorrect TTL settings
  • Wrong or missing cache-related headers (Cache-Control, Pragma Expires)
• The dispatcher or webserver is failing or misconfigured

Resolution

The first step in debugging such an issue is to isolate the problem. We already know that the AEM publish instances are getting updated with the activations. So the issue must reside in the upstream web stack including CDN and Web Server / Dispatcher cache.

I. Check the Web Servers / Dispatchers to see if the content is outdated

If the content on the dispatchers is not outdated, then the problem is with either the CDN TTL configurations or the headers being sent by dispatcher and AEM.  Follow section "II. CDN Issues" below for guidance.
If the dispatchers have outdated content, then something is either misconfigured or malfunctioning on one of those dispatchers.  Follow section "IV. Dispatcher Cache Issues" for guidance.

II. CDN Issues

If content on the CDN is outdated, then you might try the following:

Log in to the CDN management interface and check the TTL configurations. If TTLs are set in the CDN configuration, then make sure they are not set too high.
Access the content directly via the dispatcher and check for existence of cache-related HTTP headers such as Cache-Control: max-age, Expires, and Pragma. For example http://dispatcher-host1/content/geometrixx/en.html
Test bypassing the dispatcher by adding a querystring in the URL, for example http://dispatcher-host1/content/geometrixx/en.html?bypasscache=1. Check the cache-related headers for this response too.  The headers can differ between the ones returned from a cached response. Note that after a dispatcher cache flush, the first HTTP response would be the same as when bypassing dispatcher. Due to this, some CDN edge servers would be receiving the bypassed response and others would be getting the cached response from dispatcher.

Note:

If cache-related headers are omitted from the HTTP response, then some CDNs such as Amazon Cloudfront will cache the response indefinitely (assuming no TTLs are set). 

III. Flush Agent blocked or not configured

If you have determined the issue is not at the CDN level, then the next step is to check the AEM flush agent configurations.

Make sure that you have configured flush agents and that they are enabled 

Check the flush agent queues if the cache flush jobs are getting processed or stuck in queue.

1. Flush agent is in blocked state
If the dispatcher flush agent page shows blocked state then there might be a network connectivity issue between AEM and the dispatcher server(s).  Make sure that all the AEM instances with flush agents configured can reach the dispatcher servers. Modify firewall rules if required to allow connectivity between the servers.
2. Flush agent is in active state with many jobs queued
If the agent shows active state but there are many jobs then there could be:
a. An error during flushing
b. The web server is responding extremely slow to flush requests
c. Or a problem with AEM's job processing
3. Flush agent is in active state with no jobs queued
If the agent shows active state but there are no jobs queued then the webserver is returning a 200 response for the flush requests but not actually performing flushing. 

IV. Dispatcher Cache Issues

TEST A MANUAL DISPATCHER FLUSH REQUEST USING CURL

Test out a dispatcher flush request using cURL. If you get anything other than a 200 response with response body "OK", then the dispatcher flush is not being processed by the dispatcher module. Follow the steps below to debug it. 

1 2 3 4 5

curl -v -H "CQ-Action: Activate" \      -H "CQ-Handle: /content/bar" \      -H "Content-Length: 0" \      -H "Content-Type: application/octet-stream" \      http://dispatcher-server-hostname:port/dispatcher/invalidate.cache

Check the dispatcher.any configuration

Check the allowedClients

Check the /allowedClients configuration of your dispatcher farms configured in the dispatcher.any configurations. Make sure that the IP address of all AEM instances expected to issue flush requests are listed. 

Check the Webserver configuration

Microsoft IIS:

A. Check "Request Filtering"

Log in to the IIS server via RDP as an administrator
Open the Server Manager => Roles => IIS Manager
In the IIS Manager, browse down to your AEM Dispatcher enabled site
Open Request Filtering
Make sure that the filtering configurations do not block GET and POST requests to /dispatcher/invalidate.cache

See the Microsoft documentation for further information on how to access and modify these configurations.

B. Check IIS Rewrite Rules

Log in to the IIS server via RDP as an administrator
Open the Server Manager => Roles => IIS Manager
In the IIS Manager, browse down to your AEM Dispatcher enabled site
Open URL Rewrite
Review the rules to determine if any could block or break GET or POST requests to /dispatcher/invalidate.cache

See the Microsoft documentation for further information on how to access and modify these configurations.

Apache Web Server:

A. Check the RewriteRule configurations

Review and RewriteRule directives in your httpd configurations to validate if any could block or break GET or POST requests to /dispatcher/invalidate.cache

Source: https://helpx.adobe.com/experience-manager/kb/site-cache-not-getting-updated---aem--dispatcher.html