## Key Points on MAPE:

**Mean Absolute Percent Error (MAPE) is a useful measure of forecast accuracy and should be used appropriately.**- Because of its limitations, one should use it in conjunction with other metrics.
- While a point value of the metric is good, the focus should be on the trend line to ensure that the metric is improving over time.

Businesses often use forecast to project what they are going to sell. This allows them to prepare themselves for the future sales in terms of raw material, labor, and other requirements they might have. When done right, this allows a business to keep the customer happy while keeping the costs in check.

One of the key questions in the forecasting process has to do with the measuring of the forecast accuracy. There is a very long list of metrics that different businesses use to measure this forecast accuracy. Let’s explore the nuances of one of them.

Mean Absolute Percent Error (MAPE) is a very commonly used metric for forecast accuracy. The MAPE formula consists of two parts: M and APE. The formula for APE is:

The M stands for mean (or average) and is simply the average of the calculated APE numbers across different periods. It is derived by dividing the APE by the number of periods considered. Let’s look at an example below:

Since MAPE is a measure of error, high numbers are bad and low numbers are good. For reporting purposes, some companies will translate this to accuracy numbers by subtracting the MAPE from 100. You can think of that as the mean absolute percent accuracy (MAPA; however this is not an industry recognized acronym).

100 – MAPE = MAPA

MAPE in its ‘textbook’ version is the most popular accuracy measure for these reasons:

- It’s fairly easy to explain. Its popularity probably feeds back into this.
- It does not depend on scale and can apply easily to both high and low volume products.

However, there are reasons why this error measure has its detractors:

- If MAPE is calculated at a high level (think product family, or business level or across different periods) the pluses and minuses cancel each other out to (often) paint a very rosy picture.
- For this reason, consider Weighted MAPE (WMAPE) when reporting the forecast error to management as they only look at the forecast error at a very high level.
- This example is obvious in the first table. When calculated at the aggregated level, we get an APE of 4% whereas taking the average calculates a MAPE of 26%.

- MAPE does not provide a good way to differentiate the important from not so important. For example, what if the error is 90% on two products; one averages 1 million units per month and the other 10 units per month. Both get the same error score of 10%, but obviously one is way more important than the other.
- For this reason, consider using Mean Absolute Deviation (MAD) alongside MAPE, or consider weighted MAPE (more on these in a separate post in the future).
- MAPE is asymmetric and reports higher errors if the forecast is more than the actual and lower errors when the forecast is less than the actual.
- As the author (Armstrong, 1985, p. 348) says: “This can be explained by looking at the extremes: a forecast of 0 can never be off by more than 100%, but there is no limit to the errors on the high side.”
- In business terms, a high forecast has the potential to give unlimited percentage error when the observations (actuals) drop unexpectedly. This is more common because of plant shutdowns etc. than sudden huge increases.
- A discerning forecaster might well minimize their MAPE by purposely forecasting low. This will probably encourage pre-existing ‘sandbagging’ behavior which is reinforced in organizations via wrong bonus/reward structure to encourage “beating the forecast.”
- To look at this from yet another angle, see example below: Customer 1 buys an average of 90 units per month; customer 2 buys an average of 100 units per month. See table below. The same absolute error (10) produces an error of 11.1% in one case and 10% in another. However, for the same product, a miss of 10 units is equally important in both cases.

- Some companies have a tendency to over forecast which can very often be attributed to overconfidence bias. If this is the case, dividing by actuals (a smaller number in this example) results in higher error rather than dividing by forecast. This is one reason why these organizations have adapted a different version of MAPE where the denominator is the forecast.
- This, however, is also biased and encourages putting in higher numbers as forecast.

- What is the percent error when the actuals are 0 or a small number (< 1)? The error percentage calculated is very high and skews the results. (This problem does not go away when you change the denominator to the forecast, it just shows up elsewhere). Most practitioners deal with it by using a cap (say 9999%) on the error, or ignoring the ‘outliers.’

I hope this is useful info on the MAPE as a forecast accuracy metric. Since MAPE is so popular, it has many variations which I have captured in this post titled the family tree of MAPE. I am interested in your thoughts and comments.

Like this blog? Follow us on LinkedIn or Twitter, and we will send you notifications on all future blogs.

SamuelJuly 15, 2015 at 11:11 pmHi Sujit,

I am doing Supply Chain with S.A.P.I.C.S, I just realized that M.A.P.E is no different from MAD (Mean Absolute Deviation) it differs in the manner in which it’s applied depending on the organization.

Sujit SinghJuly 16, 2015 at 3:50 pmHi Samuel,

Thanks for the comment(s). MAD is only the magnitude of the error and MAPE is the relative significance (Percentage) of the error. So, they are different, at least at the definition level. But once you understand how to interpret, one might be enough.

Sujit

SamuelJuly 21, 2015 at 9:16 amThank you Sujit, so informative.

Stefan de KokJuly 23, 2015 at 6:55 amHi Sujit,

even though the MAPE is indeed asymmetrical the example you use in the table does not illustrate this. The reason the MAPE is different between customers 1 and 2 is because the actual demand is different. A fair comparison would have been if actual demand were 100 units in both cases but forecasts were 90 and 110 respectively.

For the example you give it is indeed correct that 10 units error for demand of 90 is slightly worse than 10 units error for demand of 100. Consider an extreme example: if typical demand is 10 units then an error of 10 units is highly significant. Whereas if typical demand is 1,000,000 units then 10 units error is insignificant. The MAPE does proper justice to this. The asymmetry is purely due to MAPE being bounded below and unbounded above.

Personally I am one of the detractors of the MAPE, but not for its asymmetry. Rather because it is utterly useless for slow moving items: even a single period of zero demand will cause the MAPE to be undefined. Consider that even fast moving consumer goods companies these days typically have over 90% of SKU-locations in the long tail (i.e. more periods with zero demand than positive demand). The MAPE is thus only useful for at most 10% of their portfolio. Of course you can measure it instead at aggregate levels, but as you correctly state the MAPE paints a very rosy picture when you do this. Great for sweeping issues under the rug, not for a true representation of the error.

My $0.02,

Stefan

mohammed salem awadhSeptember 30, 2015 at 4:28 pmDears

I developed accuracy forecasting matrix, concerning two measuring factors

1- coefficient of correlation

2- signal tracking

please take a look to my webs

http://www.slideshare.net/forecasting

mohammed salem

JhoomDecember 2, 2015 at 2:02 pmGood points about MAPE.

I think the most important problem is that, as was noticed above, MAPE does not represent accuracy under symmetric loss.

I wanted to suggest some recent papers that discover additional effects that render MAPE quite difficult to interpret. These papers also show that the most indicative measure would be geometric mean of Relative MAEs or geometric mean of MAD/Mean ratios:

http://www.researchgate.net/publication/282136084_Measuring_Forecasting_Accuracy_Problems_and_Recommendations_(by_the_Example_of_SKU-Level_Judgmental_Adjustments)

http://www.researchgate.net/publication/284947381_Forecast_Error_Measures_Critical_Review_and_Practical_Recommendations

Sujit SinghDecember 15, 2015 at 8:48 pmThanks Jhoom. Very good papers.