Implementation of demand forecasting using machine learning methods in retail. A/B test results

Automating the ordering process is a crucial need for grocery retailers with a large number of SKUs, which can run into the thousands, and for country-level chains, which can have tens of thousands. Optimal order formation for suppliers is not only a labour-intensive task but also directly impacts key retail indicators, such as inventory turnover rate and out-of-stock events, which can result in lost profits.

Improving Order Accuracy

As we noted in our previous articles, the following formula underlies the auto-ordering system:

Planned order = Sales forecast + Safety stock — Beginning inventory

Thus, in order to increase the accuracy of the auto-ordering system, it is necessary to work with all components of the formula.

Beginning inventory is a known value, but the absence of regulations for write-offs or regular inventories can distort it. For example, mismatches can create negative balances for some SKUs, or conversely, ‘phantom’ availability in stock. Accurate inventory management is a mandatory requirement for building an auto-ordering system.

Ensuring accurate forecasting presents its greatest challenges in the sales forecast and forming of the safety stock.

The safety stock provides a buffer to circumvent possible shortages due to forecast errors and takes into account shelf-stocking rules. The better the quality of the forecast, the lower the error rate and, accordingly, the size of the safety stock, which releases liquidity frozen in the safety stocks and increases inventory turnover.

To strike a balance between having sufficient stock and avoiding stock-outs, the use of machine learning algorithms to increase sales planning accuracy can be helpful.

To test this claim, we conducted an A/B test on the example of stores belonging to a retail chain, the results of which we present in this article.

Conditions for conducting A/B testing

A machine learning (ML) based demand forecasting solution was implemented by a retail chain in late September 2019. To evaluate the effectiveness of this solution, the third quarter of 2020 and the corresponding period of the previous year were chosen for comparison, representing periods before and after the implementation.

The A/B testing was conducted for 84 stores that operated both before and after the implementation of ML forecasting. The assortment of goods sold during both periods was selected for testing.

To analyse the test results, the chain stores were divided into 3 categories:

• Premium supermarkets (with an average assortment of about 17,000 SKUs)
• Supermarkets with an assortment of 8,000 to 15,000 SKUs
• Supermarkets with an assortment of 3,000 to 6,000 SKUs

To calculate the lost profit (product’s retail price minus its cost price), the number of out-of-stock days and the average sales rate of each product in each store during the selected period were considered. To smooth the influence of consumer price growth when estimating lost profit, the relative indicator was used, i.e., the share of lost profit in the store’s sales for the period.

Test results for Premium class stores

The main feature of Premium class stores is the high level of service and almost complete elimination of out of stock products by overstocking and maintaining high safety stocks, which reduced the share of lost profits on average by 15%.

However, due to increased forecasting accuracy and reduced «surplus» orders, turnover coefficient was improved by 1.34 times.

Average number of SKUs: 17,312

Table 1: Results of A/B testing. Format: Premium Supermarkets

Test results for stores with 8,000-15,000 SKUs

The key KPI for supermarkets is inventory turnover. In trying to maintain it at the target level, shortages and empty shelves may occur.

Thanks to accurate forecasting, the indicator was kept within the target range, and the share of lost profits in sales was reduced by 2.5 times.

The average number of SKUs is 11,837

Table 2: A/B test results. Format: supermarkets with 8,000-15,000 SKUs.

Test results for corner-shop stores with 3,000-6,000 SKUs

Similarly to large supermarkets, the key KPI for corner-shop supermarkets is turnover ratio.

Due to accurate forecasting, the share of lost profits in sales for stores in this class was reduced by 2.25 times.

Average number of items: 5,002 SKUs

Table 3: A/B test results. Format: corner-shops with 3,000-6,000 SKUs.

General results of demand forecasting implementation using machine learning methods

Improvement in forecasting accuracy (up to 95% in a category) has led to a 10% increase in turnover across the network and a 2-fold decrease in lost profits across the network.

In addition, the following benefits of implementing such solutions should be noted:

• 5 times reduction in the purchasing department’s workload

Automating forecasting reduces the labour intensity of business processes, eliminating the need for manual forecast adjustments, including during promotional periods. Prior to the project launch, 15 employees were involved in this work, whereas after the project launch, only 6 are required.

• 20% reduction in losses and markdowns

Reducing the number of write-offs and product markdowns, unplanned sales, and the number of overlapping promotions in a single product category.

• Fast deployment of the solution

It takes 1.5-2 months to develop and test the model.

• Algorithm adaptability

In situations with unstable demand, only 1 week is needed for adaptation.

Furthermore, the Datanomics Demand Forecast service has high performance and fault tolerance. With Microsoft Azure cloud is used as a computing resource, the service generates around 56.8 million forecasts for the customer daily at an average speed of 710,000 forecasts per minute.