Configuring selective cache invalidation with AEM Dispatcher flush agents

AEM Dispatcher improves response times by serving cached files instead of sending every request to the publish tier. That performance gain depends on keeping the cache trustworthy. When an editor updates a page, publishes a campaign, or replaces an asset, the relevant cached representation must be removed quickly without forcing the entire site to regenerate.

Dispatcher flush agents provide that control. With carefully scoped paths, invalidation rules, and deployment checks, an AEM team can clear only the content affected by a change. This approach is particularly valuable for large Australian sites serving several brands, state-based campaigns, language variations, or high-traffic commerce journeys across cities such as Sydney, Melbourne, Brisbane and Perth.

How dispatcher invalidation works

A flush agent is an AEM replication agent configured to send an invalidation request from the author or publish environment to the Dispatcher endpoint. The request tells Dispatcher that a resource, such as /content/example/en/products/item.html, is no longer current. Dispatcher then removes the matching cached file, allowing the next request to retrieve a fresh response from AEM Publish.

The standard mechanism relies on .stat files. When Dispatcher receives an invalidation request, it updates a stat file in the relevant cache directory. Cached files are considered valid when their timestamp is newer than the applicable stat file. The statfileslevel setting determines how far up the directory tree Dispatcher places these markers. A low value can invalidate broad sections of a site, while a higher value supports more precise cache clearing.

This timestamp model is efficient, but it requires consistent server time and predictable cache paths. Synchronise clocks with a reliable time service and verify that the web server account can create and update stat files. A small time drift between an AEM host and a Dispatcher host can make a selective flush appear unreliable during a release.

Choosing the right invalidation scope

Selective cache invalidation begins with a content dependency map. A page update may affect the page itself, a navigation component, a search result, a sitemap, an experience fragment, or a JSON endpoint consumed by a mobile application. Clearing only the edited HTML file is fast, yet it can leave related responses stale.

Dispatcher flush agent settings such as trigger.activated, trigger.modified, paths, ignoreUrlParams, and invalidation rules should reflect those dependencies. A flush agent can be restricted to a site branch or configured to target specific URL patterns. Use path-based rules for predictable structures, while avoiding broad patterns such as an entire /content tree unless a controlled full-site purge is genuinely required.

The statfileslevel value deserves particular attention. For a site organised by country, state and campaign, a higher level may let a change under /content/brand/au/nsw invalidate only the relevant branch. However, an excessively high value can produce many stat files and make troubleshooting difficult. Test the setting with representative authoring actions, including page activation, asset replacement, rollout and deletion.

Avoid sending private or user-specific responses to a shared cache. Personalised account pages, consent states and pages containing information governed by the Australian Privacy Act 1988 should generally use appropriate cache controls or bypass caching. A fast stale response is a defect; exposing another visitor’s data is a serious security incident.

Building a reliable flush agent

In AEM, open the relevant replication agent configuration and select the Dispatcher flush agent type. The configuration normally includes a title, enabled state, transport URI, authentication details and trigger settings. The transport endpoint must point to the Dispatcher flush servlet or a protected web-server route, rather than an unrestricted public URL.

Secure the transport connection with HTTPS and restrict access by network controls, authentication and web-server rules. Many Australian organisations operate separate author, publish and Dispatcher tiers in cloud regions such as Sydney. Keep the flush route private where possible, and permit only the AEM hosts that need to issue invalidation requests. Record which service account owns the endpoint and rotate its credentials through the organisation’s normal security process.

Use ignoreDefault when custom invalidation rules should replace AEM’s default behaviour, and use invalidate rules to define the paths that should be purged. The exact syntax varies with the Dispatcher and AEM version, so validate the deployed configuration against the supported documentation. A rule that looks correct in a local environment may behave differently when URL extensions, selectors, suffixes or rewrite rules are introduced.

A practical pattern is to separate agents by responsibility: one for ordinary authoring activation, another for a static asset domain, and a tightly controlled agent for broad maintenance purges. This makes logs easier to interpret and reduces the chance that a routine editor action clears an entire customer-facing site.

Connecting invalidation to publishing automation

A flush should occur after the publish operation succeeds, not before. If a cache entry is removed while replication is still in progress, the first visitor can receive an old response, a partially deployed response or a temporary error. Configure the activation sequence so that content reaches every required publish instance before the invalidation request is sent.

Approval-driven teams can use AEM workflows for publishing to place review, activation and cache clearing in a controlled process. A workflow step can identify the changed paths, activate them, and invoke the appropriate flush agent only after the required approval has been recorded. This is useful for regulated industries and for state-based campaigns that must launch at a precise local time.

Continuous delivery introduces another layer. A code deployment may change Sling Models, HTL templates, client libraries or servlet output even when no page has been edited. In those cases, content-only invalidation is insufficient. Build the deployment process to distinguish application changes from content changes, then apply a planned cache purge when rendering logic changes.

A pipeline based on Jenkins build automation can run Dispatcher configuration checks, package deployment, smoke tests and a controlled invalidation request. The pipeline should fail if the flush endpoint is unavailable, if the response does not contain the expected status, or if a post-deployment request still returns an old asset version.

Testing, monitoring and Australian operations

Test selective flushing with a repeatable matrix. Request a page to populate the cache, activate a child page, check that the child is refreshed, and confirm that an unrelated branch remains cached. Repeat the test for HTML, client libraries, images, content fragments, selectors and query strings. Inspect Dispatcher logs, AEM replication logs, web-server access logs and the cache directory rather than relying on browser refresh behaviour.

Use headers and response timing to verify the result. A browser may retain its own cached copy, while a CDN in front of Dispatcher may have a separate time-to-live. If a site uses a content delivery network for visitors in Perth, Adelaide and regional areas, coordinate the origin purge with the Dispatcher flush. Otherwise, Dispatcher can be correct while the edge still serves an older response.

Schedule deployments around audience behaviour. Australian users commonly browse on mobile devices during public transport commutes, lunch breaks and evening shopping, so a brief cache inconsistency can affect a large burst of requests. Teams supporting Sydney and Melbourne may also need to account for Australian Eastern Standard Time and daylight saving differences when planning releases, while Queensland and Western Australia follow different local clock practices.

Keep compliance and commercial rules in the operational checklist. Australian Consumer Law can make accurate pricing and availability important, especially for retail pages cached across a national site. Review cache invalidation whenever prices, promotions, shipping estimates or consent notices change. Monitor flush failure rates, activation latency, cache-hit ratio and origin traffic, and retain enough logging to trace which user or deployment initiated a purge.

A well-designed Dispatcher strategy is selective by default and broad only by deliberate choice. Define dependencies, protect the flush endpoint, test statfile behaviour and connect invalidation to successful publishing and deployment events. Then document the rules for developers, authors and operations staff so that every release clears the right content at the right time. Start with one representative site branch, measure the outcome, and expand the pattern across the AEM estate once the evidence supports it.