Forecast Accuracy Measurement: MAD, MAPE and WMAPE
Welcome
to the Forecast Solutions website
Training Courses

Forecasting in Excel


Improvement Plans

Forecasting Service

Software
Pricing Models

Weather Effects

Seasonality

Demand Forecast Accuracy Measurement

Measurement of demand forecast accuracy is important for the calculation of safety stock, to ensure that the forecasting system is under control and to investigate existing or potential problems in the supply chain. A range of sales forecast accuracy reports can be used including simple summaries, exception reports, alert lists and average accuracy reports with measures such as MAPE and WMAPE.

The calculation of forecast accuracy starts with the calculation of forecast error. Here are a few of the common measures:

  • E (error e.g. actual - forecast)

  • AE (absolute error)

  • MAE (mean absolute error) or MAD (mean absolute deviation)

  • PE (percentage error)

  • APE (absolute percentage error)

  • MAPE (mean absolute percentage error)

  • WMAPE (weighted mean absolute percentage error)

  • RMSE (root mean squared error)

  • sMAPE (symmetric MAPE)

  • MASE (mean absolute scaled error)

Many companies feel that it is better to dwell on the positive and report accuracy rather than error. Forecast accuracy measures are calculated by deducting the % error from 100% (although this does create a mathematical issue when the error is greater than 100%).

Forecast accuracy health check

Forecast Solutions can carry out a forecast accuracy health check. It comprises a review of methods and procedures together with an analysis of the levels of forecasting accuracy that are being achieved in the current sales forecasting system. Forecasts can be checked for bias. In a full forecast accuracy analysis, a forecast simulation can be set up using powerful sales forecasting software in order to compare the forecast accuracy thus achieved with that from the existing process. The health check can be an important component in the early stages of a sales forecasting improvement program

WMAPE (weighted mean absolute percentage error)

Weighted Mean Absolute Percentage Error (WMAPE) reports are particularly useful and are becoming very popular. They are easily calculated and give a concise forecast accuracy measurement that can be used to summarise performance at any detailed level (e.g. SKU) across any summary level (e.g. total business). WMAPE is frequently used in corporate KPIs.

The easiest and most robust way to calculate WMAPE is simply to add together the absolute errors at the detailed level, then calculate the total of the errors as a percentage of total sales. If a measure of accuracy is preferred over a measure of error, this is easily calculated as 100 - WMAPE.

Safety Stock

When stock replenishment for fast moving products is planned in line with the short-term forecast, safety stock is calculated using a formula that includes the variability of the forecast error as measured by standard deviation. Safety stock can then be expressed as a quantity or in terms of days cover, the latter being preferable for seasonal products. So achievement of the lowest possible forecast error is vital in striving for the minimum safety stock.

Some reasons for measuring forecast accuracy

Investigation or anticipation of problems

A problem may have occurred in the supply chain and scrutiny of forecast accuracy may be a helpful step in analysing and dealing with it. It is often useful to screen for exceptional instances of high forecast error, in order to anticipate and prevent problems from occurring. Simple measures can usually be used for these purposes, using routine or ad hoc reports to describe forecast accuracy for single entities in single time periods.

Monitoring forecast accuracy against targets or KPIs

Companies need to monitor the ongoing forecast accuracy that is being achieved in order to ensure the forecasting process is under control and meeting any KPIs that have been adopted. Ideally one would wish to see continual improvement taking place over time. For complex businesses it is not practical to report forecast error for every individual product or listing in every individual time period, therefore summary measure are needed to allow concise reporting.

Choosing between alternative forecast models

A common approach when selecting a forecasting model is to simulate forecast error over a range of historical time periods, then choose the one that would have resulted in the lowest errors. True, performance in the past is not necessarily a sure guide to performance in the future, but it is nevertheless a good starting point. A number of forecast error measures can be used for this purpose, of which WMAPE is a good example.

The most common family of methods for forecasting is time series forecasting, where analysis of historical data is carried out in order to calculate seasonal indices and quantify any persistent trends in sales volume.  A number of different forecasting models are then avaolable to project sales into the future, including moving average, curve fitting and exponential smoothing models.  Another approach is causal modelling , where statistical techniques are used to quantify the effect on sales or market share of potential causal factors such as pricing, weather or economic indices.

Click here for information on price modelling.

Click here for details about weather sensitivity analysis.

Home Page     |   Forecasting Training    |    Consulting Services    |    Links    |    Site Map