AEM's replication queue: monitoring and resolving blocked agents
In Adobe Experience Manager (AEM), the replication queue is the hand-off point between content creation and delivery. When an author instance changes content, a replication agent packages that change and sends it to a publish instance; dispatcher agents then invalidate or flush cached content. A blocked agent cannot complete that hand-off, so pages remain stale or missing while the underlying content already exists in the author repository.
For Australian teams, the operating context matters. Estates often span Sydney, Melbourne, Brisbane and Perth, with some workloads on premises and others in cloud services. A release started at 9 a.m. in Sydney can meet a Perth on-call engineer during the Australian morning, while state public holidays and 24x7 support rotas can interrupt a repair. Queue monitoring therefore needs clear ownership, local escalation times and evidence that survives handovers.
How the replication path works
An AEM replication agent operates between an author instance and its configured destination. The agent watches for changed content, creates replication jobs and transfers them to publish. From there, dispatcher flush and invalidation agents propagate the change into cached delivery layers. Each stage has its own status, logs and failure modes, so an apparently healthy publish instance can still serve outdated content when the dispatcher stage is blocked.
In commonly deployed AEM 6.x estates, replication work is managed through Sling job queues, with job state persisted in the Oak repository. The replication console shows whether an agent is active, paused, queued or failing, but the headline status alone rarely identifies the cause. Administrators should also examine individual jobs, validation messages, timestamps and the point at which processing stopped.
A queue forms whenever work arrives faster than the agent can complete it, even if the destination is healthy. A short burst after a bulk migration can be harmless; a steadily growing queue with a rising oldest-item age is not. Throttling may be temporary, while a permission, certificate or mapping error can affect every delivery. Triage starts by deciding whether the agent is slow, stalled or repeatedly failing.
Common causes of blocked agents
Connectivity and authentication account for many incidents. An endpoint may be unreachable through a firewall, VPN or proxy, while expired credentials, an invalid certificate or a changed service account can prevent a connection from being established. DNS resolution may work from a developer workstation but fail from the AEM runtime. Test from the same network boundary and security context used by the agent.
Before changing content, check agent configuration, destination credentials and the source permissions required by the transform. A component update can also fail when an agent cannot resolve a protected node or an incompatible version; component version access controls are therefore relevant when the source looks valid in the author UI.
Australian retail, banking and government estates often combine corporate data centres with hosted services, so an AEM instance can sit behind different proxies, VPN paths and identity providers. A package may leave the author successfully and fail at the next network boundary. Test the actual route used by the agent, including TLS termination and service-account access, rather than assuming a browser test is equivalent.
Building a practical monitoring routine
Queue health needs several signals rather than a single counter. Track current depth, the age of the oldest job, successful and failed job rates, time since the last confirmed delivery and agent status. Record these values before and after releases so normal peaks can be distinguished from abnormal growth. A synthetic test that publishes a disposable marker to a controlled destination can reveal a broken route before customers encounter it.
Alerts should identify the environment, agent, queue and failure category. Teams can connect queue alarms to deployment notifications in Slack, provided each message includes the age of the oldest item and the last successful delivery. Static queue-size thresholds can be noisy, so combine them with stalled-job duration and recent error fingerprints.
Make routine checks part of the working day rather than an emergency procedure. A short review during the morning handover can compare overnight jobs with release activity, while a weekly review can identify slow destinations, recurring validation failures and agents that repeatedly fall behind. Australian teams that use chat-based release coordination can attach queue evidence to the same conversation used for deployment approval, reducing context switching during incident response.
Tracing failures through the delivery chain
Begin with a failed job’s timestamp and reference identifier. Correlate that identifier across the replication agent, Sling job queue, validation and serialisation logs, publish-side logs and dispatcher activity. The earliest error usually provides the strongest clue; later entries often describe a consequence. Preserve the original error before restarting an agent or replaying content, because restart activity can obscure the original stack trace.
Separate author-side failures from destination-side failures. If jobs remain queued in the author, inspect transformation, credentials and connectivity. If they complete but the destination lacks the content, compare repository paths and versions on both instances. If publish contains the correct version but visitors see stale pages, move the investigation to dispatcher flushes, cache rules and web server reachability.
Operational patterns shared by the DevPulseCon community can help teams distinguish AEM failures from upstream deployment errors. Time-zone-aware timestamps are particularly valuable in Australia: Brisbane does not observe daylight saving, while Sydney, Melbourne and Perth follow different offsets across the year. Diagnostic logs can also contain content paths, user identifiers and URLs that reveal personal information. Under the Privacy Act 1988 and Australian Privacy Principles, access and retention should follow organisational policy; a queue blockage is not automatically a data breach, but privacy owners should be involved when affected content includes personal information.
Restoring service without losing content
Use a controlled sequence. Confirm the failure category, correct the underlying configuration or connectivity issue, and allow the queue to drain before introducing additional traffic. Avoid clearing a queue indiscriminately: jobs may represent different content versions, and removing them without a reliable source can leave publish and dispatcher states inconsistent.
Where one malformed item repeatedly blocks the queue, preserve its job details and reproduce it in a non-production environment. Repair or remove the offending source through an approved procedure, then replay only the affected content and verify both the publish repository and the delivered page. Resume normal traffic gradually while watching queue depth and error rate, because an immediate return to peak load can expose the original bottleneck again.
A compact triage matrix separates the most common delivery paths:
| Observed pattern | Strongest clue | Verification | Corrective action |
|---|---|---|---|
| Queue grows with no successes | Network, endpoint or credentials | Test the agent route and inspect connection errors | Restore connectivity or rotate valid credentials |
| Only selected jobs fail | Transformation or permissions | Review validation messages and source access | Correct mappings, paths or component access |
| One old job blocks later work | Repeated validation or serialisation failure | Isolate and reproduce the job | Repair the source and replay it under runbook control |
| Publish is current but pages are stale | Dispatcher or cache layer | Check flush status and cache headers | Repair the flush route and confirm invalidation |
| Failures follow a release | Package or version mismatch | Compare source and destination versions | Deploy a compatible version and revalidate |
Turn queue health into an operational standard
Start with one author-to-publish route that carries business-critical content. Assign a primary owner and a backup, document expected publish and cache timings, and create alerts for failed jobs and stalled oldest-item age. Use the CIRCUIT session recordings to compare queue, integration and architecture approaches with teams that have tackled similar delivery problems.
Run the first audit today: choose the highest-impact agent, record its status, baseline and destination, and rehearse one failed delivery from alert to verified publish. A tested route, shared by the Australian operations team and the release manager, turns a blocked agent from a late-night mystery into a visible, recoverable service event.