Our client, an e-commerce company in the cosmetics segment, was faced with the challenge of improving the conversion rate on its product pages across multiple channels. Despite numerous A/B tests, the results were often not meaningful enough to make informed decisions. The unstructured collection and analysis of data as well as challenges such as statistical misinterpretations and split errors were hindering the optimization of the product pages.

To solve this problem, we used advanced AI solutions to automate the A/B testing and analysis and push the content to be tested to multiple marketplaces. First, we exported and cleansed the data from the backend of the application to create a solid database. Customer data, product data and purchase data were precisely analysed and harmonized. In the next step, we implemented AI-supported algorithms to analyze the test results. These algorithms enabled automated detection of better performing variants and identified different segments within the test data. By using machine learning, we were able to make informed, data-driven hypotheses that went beyond mere personal opinions.

AI also helped us to implement and monitor the tests efficiently. Through a real-time data pipeline, we were able to continuously integrate new user data and dynamically adapt the recommendation engine. The focus here was on the correct implementation of the A and B variants and on minimizing statistical misinterpretations and splitting errors.

By using AI, we were able to significantly increase the statistical power of the A/B tests. The positive change in the conversion rate averaged +5.88% across multiple products, which was a significant improvement. The AI-supported analysis enabled a detailed view of user interactions and led to more precise results. At the same time, the process set up enables the testing of additional product categories.