Building Custom AEM Indexes for Faster JCR Queries

In Adobe Experience Manager, every page render, every component lookup, and every author search eventually hits the Java Content Repository underneath. When those queries are slow, the entire authoring experience slows down with them, and Australian content teams running multi-brand sites for retailers or federal agencies feel the pain first. The good news is that the Oak repository gives developers a deterministic path forward through custom indexes that can turn multi-second waits into sub-200-millisecond responses.

Most AEM performance incidents in production trace back to one of three patterns: traversal queries against deep node hierarchies, wildcard searches across properties that lack an index, or full-text searches executed against the wrong index type. Each is solvable through a declarative index definition under /oak:index, and the same configuration file runs identically in Sydney, Melbourne, and Brisbane data centres.

Why JCR Queries Become Slow

Oak evaluates every XPath, JCR-SQL2, and QueryBuilder query through a cost-based planner that chooses between available indexes. When the planner cannot find a suitable index, it falls back to a traversal that walks the repository tree, counting nodes as it goes. On a healthy site with a few thousand pages this is barely noticeable, but on a long-running enterprise site that has accumulated years of content, the cost grows linearly with node count.

The most common offender in Australian AEM installs is a poorly constrained author-side search. A marketing team at a Melbourne-based bank might run a QueryBuilder query that filters by cq:template and sorts by jcr:lastModified, expecting sub-second results. If the only matching index is a property index on cq:template without the sort property included, Oak has to gather every matching node, then perform an in-memory sort before returning the first ten. Multiply that by dozens of editors during a campaign launch and the author instance buckles.

Full-text search trips up many teams because property indexes do not analyse strings. A query for contains(jcr:content/jcr:title, 'woolworths') against a property index forces a full table scan. Lucene indexes analyse content only when the property is declared as analyzed in the index definition, a detail engineers frequently miss.

Choosing Between Oak Index Types

Oak ships with four index families, each suited to a different query shape. Picking the right one early avoids the kind of rework that wastes weeks of engineering time. The table below summarises the practical trade-offs most teams encounter on Australian enterprise deployments.

Index Type	Best Use Case	Tokenisation	Sort Support	Re-index Cost
Property Index	Exact match on a single property	None	Limited to indexed property	Low
Lucene Property Index	Full-text search across multiple properties	Full (analyzed)	Strong, with payload stored	Medium
Lucene Full-Text Index	Mixed structural and free-text queries	Full with n-gram, synonyms, stemming	Strong	High
Solr Index	Distributed search across large repositories	External, configurable	Strong via Solr cores	Very high

Property indexes are the cheapest to maintain and the easiest to reason about. They shine when the application issues a small set of queries against well-known properties, such as looking up a customer by customerId or finding content by cq:tags. They fall apart the moment a query asks for anything other than equality, because Oak cannot range-scan, analyse, or sort efficiently on a flat property index.

Lucene indexes are the workhorse of modern AEM installations. The Lucene Property Index variant covers most author-side search needs without dragging in the heavier machinery of n-gram tokenisation and faceted search. The Lucene Full-Text Index adds stemming, synonyms, and the ability to combine structural constraints with free-text queries in a single plan. Solr indexes come into play when the repository grows past what a single Lucene instance can handle, which is rare in Australian contexts but worth knowing about for global rollouts.

Reading an Oak Index Definition

An index in Oak is a node of type oak:QueryIndexDefinition stored under /oak:index, carrying a type property, the index name, and subnodes that declare which properties to include, whether they are tokenised, and what kind of query result they should produce. Developers in the Sydney AEM community often describe this file as the single most cost-effective piece of configuration in an entire AEM project, because one well-built index can eliminate a category of support tickets.

A minimal Lucene index that supports both structural and full-text search on an cq:Page looks like this in CND notation:

- /oak:index/contentPages (oak:QueryIndexDefinition)
  - type = "lucene"
  - compatVersion = 2
  - async = "async"
  - indexRules (nt:unstructured)
    - jcr:primaryType (String) = "nt:unstructured"
      - cq:Page (nt:unstructured)
        - properties (nt:unstructured)
          - jcr:content/jcr:title (String) = "analyzed"
          - cq:template (String) = "notanalyzed"
          - jcr:lastModified (Date) = "ordered"

Three settings deserve attention. async = "async" tells Oak to keep the index current through background observation rather than blocking writes, which matters on author instances where editors expect immediate saves. Setting compatVersion = 2 enables the modern Lucene 6 backend and unlocks features like fuzzy matching and span queries. Declaring each property as analyzed, notanalyzed, or ordered shapes how Oak treats the value at query time.

Designing the Index Around Real Queries

Before writing any definition file, capture the actual queries your application issues. Enable the Oak query log through the Felix console (org.apache.jackrabbit.oak.plugins.index.search.LuceneIndexProvider at DEBUG) and let it run for a full business day across peak authoring hours. A Perth-based retailer will see a different query mix on weekday afternoons than on weekend campaign windows, so the sample needs to cover both.

Group the captured queries by the path they traverse, the properties they filter on, and the columns they sort by. An index only helps if every property it covers appears as a constraint in the query, so this exercise often reveals queries that benefit from a small code change in the QueryBuilder predicates. Once you have three or four distinct query shapes, you have the foundation for an index that serves real workload rather than an imagined one.

Watch out for queries that use ISDESCENDANTNODE combined with multiple LIKE clauses. These are notoriously difficult to optimise and sometimes justify splitting into two queries and merging results in application code. Engineers in Brisbane working on large university sites have found that a hybrid approach, one Lucene index for the broad filter and a secondary property index for the exact constraint, often outperforms any single index.

Validating an Index Before Rollout

Index misconfiguration is one of the top causes of AEM outages in production, so treat every new index as a candidate for the same rigour as a schema migration. The Oak Query Engine exposes an explain endpoint that returns the chosen plan, including the index used and the estimated cost. Run it before and after adding the index definition to confirm the planner actually picks your new index for the queries you care about.

Performance testing belongs in a representative environment, not a developer laptop. Mirror the production repository size as closely as possible by replaying a recent package install from the live author instance. Measure both cold queries (right after repository restart) and warm queries (after the index has been populated), because Lucene cold-start times can mask issues that only appear after a node restart. Australian teams operating under ACSC guidance often pair this with a load test that simulates concurrent editor traffic, since author contention is a known amplifier of query latency.

Finally, test re-index behaviour explicitly. A reindex flag on the index definition forces Oak to rebuild the entire index from scratch, which can take hours on a multi-million-node repository. Plan a maintenance window, communicate it to the business stakeholders, and verify the index status through the JMX bean IndexStats before declaring the rollout complete. Teams that want an outside perspective on their index strategy often engage the consultants at ICF Olson for an independent audit before going live.

Operating Custom Indexes at Scale

Once an index is live, it needs the same care as any other production artefact. Oak writes index statistics to the repository itself, and a small dashboard that surfaces the IndexStats MBean for the development team in your Sydney office is worth the half-day investment. Watch for indexes that drift in size without corresponding content growth, which usually indicates stale entries from deleted nodes, and for indexes whose update queue grows faster than it drains.

For organisations running AEM across multiple geographies, including Australian publishers serving both domestic and New Zealand audiences, the index strategy needs to account for content replication patterns. Indexes on the author instance differ from those on publish, and a publish-farm index that handles traffic from both Sydney and Auckland data centres may need higher memory allocation than a single-region instance. For organisations layering real-time content analytics on top of AEM search, the integration patterns described in AEM with Apache Flink cover how streamed query results feed into live dashboards without compromising index health.

Watch the recordings from CIRCUIT 2015 and 2016 for deep-dive sessions on Oak internals, then bring the speaker team into your next architecture review.