The challenge
Our client implemented product markdowns to clear inventory and encourage customers to purchase products that were slow-selling or going out of season. However, they faced the challenge of inefficiency in manual A/B testing of markdowns, resulting in potential calculated losses that could be optimized.
The retailer aimed to automate their pricing determination process for markdowns but encountered skepticism regarding the relevance of automated procedures. The client sought proof that automated procedures are the future of the company. This was the moment they reached out to VirtusLab.
They established distinct prices for stores using both merchandisers, conducting manual markdowns, and machine learning models to finally compare the outcomes and identify the most effective strategy. The idea was to prove the pricing model’s efficiency by conducting A/B tests for various pricing strategies in stores.
The solution
VirtusLab developed a framework to streamline and fully automate A/B tests. By providing the Automated Testing and Orchestration (ATO) with a list of products and models to compare, the A/B tests were run entirely autonomously.
This included training the models, optimizing the prices, selecting stores, uploading different prices to stores, and retrieving the test results from an API endpoint once the tests were complete.
We successfully integrated various components, such as different APIs for store pricing, ML pipelines for model training and obtaining optimized results, and a store selection service.
The workflow of our framework
- Users defines A/B test: choose which models to compare, define the products, define the percentage of stores for each group of the A/B test.
- ATO automatically selects the stores and determines the prices for the different groups of the A/B test.
- ATO uploads prices to stores.
- Users can retrieve the result of the A/B test.
