AEM and MongoDB Deciding When External Persistence Makes Sense

Adobe Experience Manager persists content inside a JCR repository backed by Jackrabbit Oak. By default Oak writes node data to file-based segment stores, often called Tar storage, while binaries sit on local disk or a shared filesystem. Teams scaling beyond a single author instance often consider switching the document store to MongoDB, which keeps the same JCR model but distributes nodes across a cluster.

Australian architects have spent the past decade modernising platforms for banking, retail, mining, and federal agencies. The Privacy Act and the Notifiable Data Breaches scheme push organisations towards clearer records of where customer data sits. AEM deployments holding marketing content and asset metadata benefit from architecture choices that make persistence layers visible and portable.

The MongoDB document model maps reasonably well to the JCR tree because both think in nested key-value structures. Replica sets give an author cluster the ability to survive a node loss, which matters when a hosting provider in an Australian region goes through maintenance. The cost of running that cluster is non-trivial, so the choice has to be justified by the workload.

The real question is not whether external persistence is possible, but whether it is necessary. The sections below look at where MongoDB-backed storage pays off, where the default backend is enough, and what Australian practitioners have learned along the way.

How AEM Persists Content Under the Hood

Oak organises content as a tree of nodes with named properties and child references. In a standalone installation the NodeStore and Tar segment files sit on local disk, and the persistence layer is invisible to most developers. Each segment file contains a packed snapshot of part of the tree, and Oak rolls them forward through a journal that records every change.

When a second author joins the cluster, Oak needs a shared store. The simplest path is a network filesystem such as NFS, but that introduces a single point of failure. Switching the NodeStore to a MongoDB DocumentNodeStore lets multiple authors write to the same backing collection through MongoDB replication, keeping JCR semantics intact while removing the shared-disk dependency.

Binary storage follows the same logic. Small binaries stay inside the node store, while larger ones offload to a binary store backed by S3, Azure Blob, or a local filesystem. Asset-heavy sites often combine MongoDB with an external binary store, since the bulk of disk usage rarely sits in the node documents.

Scenarios That Justify External Persistence

Author clusters spanning more than two nodes are the most common reason Australian teams reach for MongoDB. Once that line is crossed, the operational complexity of a shared filesystem starts to outweigh the complexity of running a replica set. The same applies to multi-region setups where an authoring team in Sydney and a publishing farm in Singapore both need to attach to the same source of truth.

The Big Four banks run internal platforms that mirror these patterns, and AEM deployments in that space often sit on a MongoDB cluster for author resilience. Mining companies operating across Western Australia and Queensland benefit from a setup where the authoring tier can fail over without taking down content approvals, even when the connection back to headquarters is patchy.

Public sector work adds another constraint. The Notifiable Data Breaches scheme under the Privacy Act 1988 means any incident involving customer data has to be reported and contained quickly. A replicated document store makes it easier to demonstrate where content lives, how it is backed up, and how quickly a compromised node can be replaced.

When the Default Tar Backend Is Enough

Not every site needs a MongoDB cluster. A single author serving a regional tourism board, a small university faculty, or a boutique retail brand rarely produces enough write volume to strain Tar segment files. The default backend is also easier to back up, since the segment store is just a folder that can be snapshotted and shipped offsite.

Teams running AEM in a managed cloud often inherit a Tar configuration from the vendor and only see a reason to change when they outgrow the included capacity. That is a healthy place to be, because switching the NodeStore is not a reversible migration. Existing segment stores cannot simply be copied into a new collection; the cluster has to be bootstrapped from scratch and reindexed.

For many Australian SMBs the calculus is simple. Tar storage on a well-provisioned VM, with backups going to an object store in the same region, covers the business need and keeps the runbook short. Adding MongoDB only makes sense once the site crosses thresholds that justify a dedicated platform engineer.

Data Residency and Privacy Considerations

Australian legislation shapes how and where AEM data can live. The Privacy Act applies to organisations with an annual turnover above AUD 3 million, which catches most enterprises that would deploy AEM. Hosting the MongoDB cluster in AWS Sydney or Azure Australia East keeps content under domestic jurisdiction and simplifies Notifiable Data Breaches records.

Data sovereignty comes up with state government work. Several state agencies require that the entire content stack stays inside Australian borders. That rules out the cheapest international Atlas tiers and pushes teams towards local clusters or Atlas multi-region setups with a primary in Sydney. The Australian Signals Directorate's Essential Eight also feeds into hardening expectations around backup immutability and credential segregation.

Smaller operators sometimes discover these requirements late. A regional council that grew into a shared authoring setup can find itself needing both a regional MongoDB cluster and a separate binary store to satisfy an audit. Building that visibility into the architecture early avoids a rushed migration under compliance pressure.

Performance and Replication Trade-offs

MongoDB replication is asynchronous by default. A write acknowledged by the primary may not yet be visible on a secondary, which means an author failover can lose the last few hundred milliseconds of edits. Most marketing workflows can tolerate that, but teams integrating AEM with transactional systems should think carefully about what gets written through Oak.

Sharding adds another layer of operational complexity. Most AEM deployments do not need to shard the document store, because the working set for the authoring tier is smaller than it looks. The bigger lever is usually the binary store, since high-resolution images and videos dominate storage growth on retail and media sites. Teams watching Oak metrics closely often find the slow path is the asset pipeline, not the NodeStore.

Replica set health needs to live in the same observability stack as AEM. Lag spikes, election events, and disk pressure on the primary are all signals that should page the same on-call rotation. The goal is to treat MongoDB as part of AEM rather than as a side database someone else looks after.

Integration With Microservices and APIs

External persistence often goes hand in hand with a microservice backend. When an AEM instance hands content over to a downstream service, that service typically reads through the same MongoDB collection or through an event stream fed from Oak commits. Pairing AEM with a Spring Boot service for backend integration has become a recurring conference topic, and a session recording covers the microservice backend integration approach in detail.

Choosing the right implementation partner matters when the architecture touches both the content tier and the platform tier. The ICF Olson write-up is a useful starting point for teams weighing trade-offs between a tightly coupled AEM setup and a more decoupled backend topology.

A clean separation helps when a team wants to expose AEM content through a headless API. Reads can be served from a read-optimised replica or a denormalised projection in a separate collection, while writes still flow through Oak. That pattern shows up in Australian retail, where a Sydney team feeds one catalogue into a website, an app, and a kiosk.

Operational Readiness Checklist

Before flipping the switch, an Australian team should walk through a short list. Confirm the cluster is sized for the working set, not the total content footprint. Verify backups use a tool that understands MongoDB consistency, like a snapshot on a quiesced secondary or a managed backup service that respects the oplog.

Runbooks need to cover the boring but important cases. What happens if the primary fails during a publish cycle? How is a stale secondary brought back into the set without triggering a re-sync that lasts hours? Who has the credentials to break glass into the cluster during a Sev1? These answers should sit next to the AEM runbooks, not a wiki nobody reads.

Rehearse the disaster recovery story. Spin up a fresh author instance against a restored MongoDB snapshot, point it at a binary store copy, and time how long it takes to be back in business. Australian teams that run this drill quarterly, often using an ICF Olson runbook template, tend to discover weak points in their backup chain well before a real incident.

Session recordings from the 2015 and 2016 CIRCUIT conferences in Chicago covered AEM architecture, integrations, and content storage in depth. Original presentations, speaker notes, and pricing history remain available for teams wanting to revisit the conversation or hear how the community has applied these ideas since. Watch the recordings and pair them with load tests against a MongoDB-backed cluster to shape your decision.