A/B Test
The A/B Test Module allows you to test several different configurations of your FACT-Finder in parallel over a defined period of time, in order to find out which of them is the most effective.
For the use of A/B tests in Suggest, there is a new parameter in the Suggest API: sid. This parameter contains the SessionId that is used to select an A/B test. Without this parameter, the individual requests cannot be clearly assigned to the A or B variant.
Resource Usage and Number of Tests
Regular A/B tests require only one additional copy of the configuration on memory per test, and for Ranking A/B tests only one additional list of product IDs with their respective ranking score.
In principle, you can create as many A/B tests as you want. But the more tests you create, the longer each individual test needs to make a reliable statement.
The following modules/features are A/B testable:
- Thesaurus
- Stopwords
- Preprocessor Entries
- Trash Can Entries
- Campaigns
- Search Algorithm Settings
- Result listing Settings
- Personalization
except for (de)activating the module
- Semantic Enhancer
- Facets & ASN: Filters
- Single Word Search
- Loop54
- Ranking Rules, but with the following exceptions:
- Ranking Rules are only A/B testable for the scopes 'Search' and 'Navigation', not for the scope 'Suggest'.
- The ranking for the suggest module will always use the Ranking Rules from the base variant and never apply changes from the B variant.
The following modules/features are not A/B testable:
- ASO
- Field settings
- Recommendations
- GEO
- CSP
- Suggest
- Suggest-Ranking
- Import and Database Settings