Solr Query Tuning and Troubleshooting in SAP Commerce

"Why does searching for camera show tripods first?" is the most common search complaint on any commerce project, and it is answerable with complete precision, because Solr will explain every scoring decision it makes if you know how to ask. This guide covers the anatomy of the query SAP Commerce sends, the configuration that shapes it, and the troubleshooting paths for the recurring failure patterns. It merges and updates two CX Works pieces (understanding Solr queries, and troubleshooting Solr).

Before the Query: What Happened to Your Search Terms#

Search terms are transformed before matching. Field types in schema.xml define analyzer chains (a tokenizer plus filters), and the same chain applies at index time and query time. A representative chain:

<fieldType name="text_spell_en" class="solr.TextField" positionIncrementGap="100">
  <analyzer>
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>
    <filter class="solr.PatternReplaceFilterFactory" pattern="(['’])" replacement=" "/>
    <filter class="solr.EnglishMinimalStemFilterFactory"/>
    <filter class="de.hybris.platform.solrfacetsearch.ySolr.synonyms.HybrisSynonymFilterFactory"
            ignoreCase="true" synonyms="en" coreName="${solr.core.name}"/>
    <filter class="de.hybris.platform.solrfacetsearch.ySolr.stopwords.HybrisStopWordsFilterFactory"
            ignoreCase="true" lang="en" coreName="${solr.core.name}"/>
    <filter class="solr.StopFilterFactory" words="lang/stopwords_en.txt" ignoreCase="true"/>
    <filter class="solr.TrimFilterFactory"/>
    <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
  </analyzer>
</fieldType>

Note the two Hybris filter factories: they are how the synonyms and stop words that business users maintain in Backoffice enter the analysis chain without anyone editing schema files. Business users edit, the handler updates the core, search behaviour changes on the live site.

When search behaves strangely at the term level ("why does BLACK power drills match blank?"), the Solr admin console's Analysis screen (per core, left navigation) shows the token stream after each filter for any input, for both the index and query side. It answers term-transformation questions in seconds that log reading answers in hours.

Anatomy of the Query Commerce Sends#

A single-term storefront search for "camera" produces (extracted from the Solr log, trimmed):

q=(ean_string:(camera^200.0 OR camera*^100.0)) OR
  (code_string:(camera^180.0 OR camera*^90.0)) OR
  (name_text_en:(camera^100.0 OR camera*^50.0 OR camera~^25.0)) OR
  (manufacturerName_text:(camera^80.0 OR camera*^40.0 OR camera~^20.0)) OR
  (keywords_text_en:(camera^40.0 OR camera*^20.0 OR camera~^10.0)) OR
  (categoryName_text_en_mv:(camera^20.0 OR camera*^10.0 OR camera~^5.0))
&fq=(((catalogId:"electronicsProductCatalog") AND (catalogVersion:Online)))
&sort=score desc,inStockFlag_boolean desc,score desc
&rows=20&start=0
&defType=edismax
&facet=true&facet.field={!ex=fk5}price_usd_string&facet.field=brand_string_mv&...
&spellcheck=true&spellcheck.q=camera

The parameters that matter:

• q is the scored query. Every searched field appears with up to three sub-queries per term: exact (camera^200.0), wildcard-prefix (camera*^100.0), and fuzzy (camera~^25.0, Levenshtein matching that catches "camra" and "kamera"). The ^ values are boosts, and the exact:wildcard:fuzzy ratio per field is what makes an exact EAN match outrank a fuzzy category name match.
• fq filters before scoring, like a WHERE clause, and hits Solr's filterCache. Catalog version, category, and facet selections belong here, never in q: they are binary conditions, not relevance signals, and caching them is most of your query throughput.
• rows/start: page size and offset.
• Phrase proximity is also available: "camera battery"~5^100.0 scores documents where both words appear within five positions. If multi-word relevance matters to your business (it usually does), a custom FreeTextQueryBuilder adding proximity clauses is one of the highest-value search customizations available.

Where the Query Comes From, and How to Change It#

The field list and boosts live in the commerceSearchTextPopulator bean (commerceservices), as a list of FreeTextQueryBuilder entries. Two distinctions to keep straight:

• Field boost (here, in Spring) weights where a match occurs. Set by developers, deployed with releases.
• Term boost / boost rules (Backoffice, Commerce Search configuration) weight what is matched or push specific products up or down. Set by business users at runtime. When result order surprises you, check both layers; runtime boost rules are invisible in the Spring config and regularly forgotten.

Adding a searchable field is a merged list extension:

<alias name="acmeCommerceSearchTextPopulator" alias="commerceSearchTextPopulator"/>
<bean id="acmeCommerceSearchTextPopulator" parent="defaultCommerceSearchTextPopulator">
  <property name="freeTextQueryBuilders">
    <list merge="true">
      <bean class="de.hybris.platform.commerceservices.search.solrfacetsearch.querybuilder.impl.DefaultFreeTextQueryBuilder">
        <property name="propertyName" value="description"/>
        <property name="boost" value="30"/>
      </bean>
    </list>
  </property>
</bean>

Changing an existing field's boost is uglier: the list must be redeclared wholesale (merge="false") with every builder repeated and your changed value in place. Version that bean carefully; it is now the authoritative relevance definition for the site.

Query builder choice is also per field: DefaultFreeTextQueryBuilder emits exact plus wildcard plus fuzzy; NonFuzzyFreeTextQueryBuilder emits exact only, which is correct for identifier fields like EAN and code where fuzzy matching produces nonsense; and a custom FreeTextQueryBuilder implementation is the escape hatch for phrase proximity, per-word-count logic, or anything else.

The one discipline that keeps tuning sane: maintain a baseline query set. Ten to thirty searches with agreed expected top results, run after every boost or schema change. Relevance tuning is a whack-a-mole game (improving "camera" can degrade "camera bag"), and the baseline set is what tells you a fix broke something three queries away. There is no perfect query; there is a documented compromise your business signed off on.

The Debugging Loop: Extract, Replay, Explain#

Every relevance investigation follows the same three steps.

1. Extract the real query. Either from the Solr request log, or from the platform by enabling the search debug logger:

log4j2.logger.solrfacetsearch.name=de.hybris.platform.solrfacetsearch.search.impl
log4j2.logger.solrfacetsearch.level=DEBUG
log4j2.logger.solrfacetsearch.appenderRef.stdout.ref=STDOUT

(or register the SolrQueryDebuggingListener through Backoffice for targeted capture).

2. Replay it in the Solr admin console against the live core, with modifications that make output readable:

Recent Solr versions may require a default field (df=code_string) when replaying raw queries; add it if the console complains.

3. Read the explain output. With debugQuery=true, Solr itemizes every document's score: which field matched, which clause (exact, wildcard, fuzzy), the boost applied, the term frequency contribution. The tripod outranking the camera stops being a mystery and becomes "the tripod's keywords_text_en contains camera three times and the boost ratio lets that beat a fuzzy name match". Then you know which lever to pull: field boost, keyword data quality, or query builder choice.

Troubleshooting Patterns#

The recurring production issues, with their diagnosis paths.

Same search, different results on refresh#

Queries round-robin across replicas; inconsistent results mean the cores disagree. Check replication on the Solr admin's Replication screen: on the primary, replication enable must be true (direct-mode full indexing disables it during the run; it must come back) with replicateAfter: commit, startup; on replicas, compare Version/Gen against the primary and check the last-iteration timestamp and Next Run. Lagging generation that never converges is your problem.

"No live SolrServers available to handle this request"#

Connection failure: wrong endpoint URLs (Backoffice, Facet Search Config, Solr server configuration), a downed node, or network policy. After environment restores, remember the endpoint config lives in the database: a staging environment restored from production is now configured with production's Solr addresses, which is both a functional bug and, if the network allows it, a way to destroy the production index from staging.

404 core not found ("Expected mime type application/octet-stream but got text/html ... HTTP ERROR 404")#

Commerce reached Solr, but the core does not exist, typically after an environment rebuild or collection deletion. Run the full indexing cronjob; core creation happens during full indexing.

Storefront 404 on search results#

Frequently a category deleted (or recoded) while still referenced by indexed documents; rendering fails looking up the localized category name, and the log says so directly:

ERROR [solrIndexThreadLogger] Category with code '578' not found!
  (Active session catalogversions: electronicsProductCatalog.Online)

Reindex after category removals, and treat category deletions as index-invalidating events in your release process.

Data is stale#

Work through the indexing chain in order: did the update cronjob run and succeed (Backoffice, Facet Search Config, CRON JOBS tab: full-<name>-cronJob, update-<name>-cronJob, last result SUCCESS, schedule sane)? Does the indexer query actually select the changed items (run it in HAC, substituting ?lastIndexTime with a literal timestamp, as the cronjob's Run As user, because a restricted user silently sees fewer products)? If indexing is clean, check replication. A quick sanity check: the indexer query's row count should match the core's Num Docs in the Solr console when no customization filters between them.

A facet is configured but invisible#

Four causes, in observed frequency order: the IndexedProperty is missing facet=true/visible flags; the value provider is wrong for the data (classification attributes need CommerceClassificationPropertyValueProvider); a Backoffice boost rule or search profile interferes; or, the subtle one, the facet value covers 100 percent of the current result set, in which case ConfigurableFacetSearchResultFacetsPopulator removes it as useless (a facet everyone matches filters nothing). Replay the query with facet.mincount=0 to see what Solr actually returned before commerce post-processing.

Results in the wrong order#

debugQuery=true scoring explanation first, Backoffice boost rules second. If sorting by anything other than relevance behaves oddly, check the sort parameter in the extracted query; the sort chain (score desc, inStockFlag_boolean desc, ...) sometimes encodes surprises someone configured years ago.

Backoffice search results missing fields#

By design: Backoffice search needs only PKs (with restrictFieldsInResponse=true for performance), and the shipped schema stores only pk, catalogVersion, code, autosuggest, and spellcheck fields (stored="true"), leaving the rest indexed-but-not-stored. Indexed-not-stored fields are fully searchable; they just do not appear in responses. If you genuinely need a field returned, flip its schema declaration to stored="true" (dynamic-field rules like <dynamicField name="*_int" ... stored="true"/> work) and restart Solr, accepting the index-size cost.

Which core am I even looking at?#

Two-phase indexing keeps two cores per config, suffixed _flip and _flop, one serving queries while the other rebuilds. The active one is recorded in Backoffice: the SolrIndex items under the Facet Search Config, where the Active flag marks the query core. Diagnosing "wrong data" against the inactive core is a rite of passage; do it once, then always check the flag first.

Full indexing takes too long#

In order of typical yield: fix the indexer query (especially the update variant's delta detection), stop indexing languages the storefront does not serve, prune unused IndexedProperties, drop the Staged catalog version from the storefront index (index Online; Backoffice preview needs are served by the separate Backoffice indexed type), and move volatile fields to partial updates. Value provider performance is the other half of this story, covered in the indexing guide.

The Meta-Skill#

Every issue above yields to the same method: find out what commerce actually sent (logs), reproduce it against Solr directly (admin console), and make Solr explain itself (debugQuery, Analysis screen, Replication screen). Teams that internalize the loop stop filing "search is weird" tickets and start filing "categoryName boost beats name boost for two-word queries, proposal attached" tickets, and that is the difference between a search you apologize for and a search you tune.