AEM HttpCache and Dispatcher: TTL and invalidation layers

A published AEM request can be answered by an edge cache, a Dispatcher cache, AEM's HttpCache, or live rendering. When every layer appears healthy yet colleagues see different campaign content, the issue is usually architectural: expiry and invalidation are being applied in different places.

Understanding AEM's HttpCache and Dispatcher means tracing those decisions separately. TTL controls freshness through time; invalidation removes selected entries because content has changed. A long TTL can preserve stale output after a bad purge, while an overly broad invalidation can send an avoidable wave of requests back to AEM.

CIRCUIT brought Java developers, AEM architects, front-end engineers and systems specialists together in Chicago for two days of workshops and technical sessions. Its 2015 and 2016 recordings preserve valuable context, but cache behaviour still has to be mapped onto each organisation's topology: dispatcher rules, servlet headers, CDN policies and replication agents can each change what a customer receives.

For Australian teams, the operating context adds sharp edges. The market mixes corporate sites, ecommerce stores and campaign landing pages, with audiences spread across Sydney, Melbourne, Brisbane and Perth. Peaks cluster around Click Frenzy, EOFY sales and Christmas, while regional offices often combine NBN with fixed wireless, 4G or 5G. Cache design must absorb those bursts without serving one market yesterday's offer.

Trace the request through the cache stack

The simplest request path starts at an optional CDN or load balancer, passes through Dispatcher, and reaches AEM Publish. If Dispatcher misses, AEM's HttpCache may satisfy the request before Sling resolution or servlet rendering occurs. The eligible response can then be stored independently at both the HttpCache and Dispatcher layers.

These layers cache at different boundaries. HttpCache stores rendered AEM responses within the publish-side application, while Dispatcher stores proxied responses at the publish edge and applies rules for URLs, extensions, selectors, headers and invalidation. Query parameters, cookies, host names and negotiated content can alter the cache key unless the configuration deliberately ignores them or the response declares the relevant variation.

The session programme from CIRCUIT provides useful background for this model because sessions placed dispatcher behaviour beside broader AEM design decisions. Treat the resulting diagrams as a request flow: inspect the edge policy, then the Dispatcher rule, then the AEM response, rather than treating “the cache” as one box.

Let TTL and response headers decide freshness

TTL is an age limit, and its source must be identified. A servlet may return Cache-Control: max-age=300, an edge service may publish its own TTL, and a Dispatcher rule or HttpCache policy may determine how that value is honoured. The effective limit results from policy precedence, not simply the number written in a servlet. The age normally begins when an entry is stored; a cache hit does not reset it unless the response is revalidated or replaced.

Freshness also depends on request variation. Responses marked private or no-store should not enter a shared cache, while Vary, Cookie and query-parameter handling determine whether two requests may share an entry. Caching personalised account pages under a common key risks crossing customer boundaries, which carries particular weight under the Australian Privacy Act 1988 and the Australian Privacy Principles.

A three o'clock arvo campaign refresh in Sydney can race an expiry in Melbourne and a remote branch in Darwin. Long, uniform TTLs smooth those peaks, but the trade-off is stale content. Short TTLs improve freshness yet increase origin load and can expose dispatchers to cache stampedes after a busy sales period. Because daylight-saving clocks differ between Australian states, compare expiry timestamps in UTC rather than relying on a local wall-clock label.

Use invalidation as a targeted control signal

An update normally reaches AEM Publish through replication. A dispatcher flush agent can then send an invalidation request to the cache, while a targeted invalidation handler can address selected paths. TTL remains the fallback for entries nobody explicitly invalidates. A purge and a selective invalidation should therefore be treated as different operations, with different blast radius and audit requirements.

Propagation order matters. If an outer tier is flushed while an inner tier can still return old bytes, the outer tier may immediately repopulate with stale content. Confirm the authoritative origin version, coordinate invalidation across the CDN, Dispatcher and any AEM-side cache, and avoid live testing until each tier has acknowledged the event. Path rules must also account for selectors, JSON responses, query parameters and dependent assets.

Australian teams often say they need to “chase the cache” or get a “clean hit”, but the diagnosis depends on the layer that answered. The XtextCon community provides a useful adjacent example of treating configuration as reviewable text: version dispatcher rules, cache headers and invalidation paths together, then compare their observed behaviour in staging.

Compare environments with a repeatable checklist

The same URL can behave differently in local development, integration, staging and production. Hostnames, authentication, proxy layers and Dispatcher configuration all affect cache keys. This content comparison across environments shows why visual parity between websites is insufficient: request headers and cache policy must also be compared.

Use one repeatable request and inspect the response at every hop. Record Age when present, along with Cache-Control, Expires, ETag, Vary and any tier-specific status header. At Dispatcher, check:

  • Which Dispatcher rule matched the request
  • The effective TTL and header precedence
  • Cache-key treatment of parameters, selectors and headers
  • Whether the invalidation event reached this tier

At AEM Publish, establish whether the response is eligible for HttpCache before changing Dispatcher. Check the following details:

  • The freshness header returned by the servlet
  • Whether authentication changed the response
  • Whether the update also changed a dependency
  • The HttpCache configuration applied to that path

Change one policy at a time, canary it, and compare origin, Dispatcher and edge hits. A production flush is a pressure release, not a diagnostic method.

Make cache behaviour a controlled release

Cache governance should cover publishing as well as infrastructure. Record ownership for Dispatcher rules, cache headers and flush procedures; set expiry windows for campaign, evergreen and personalised content separately; and require an audit trail for invalidation. In regulated Australian organisations, also confirm tenancy region, data-residency obligations and access logging before enabling any shared cache around customer data.

Measure hit ratio, origin latency, purge duration and stale-response reports as release signals. A cache change that lowers origin load but increases complaints about pricing or availability has moved cost rather than improved the service. The useful target is predictable freshness with controlled origin pressure.

Register your team through CIRCUIT registration, then bring a real stale-content scenario to the session. Walk it through the request path and leave with a tested TTL and invalidation runbook for your publish tier.