Precision v. Accuracy: How about using a range?

The concept of precision verses accuracy was first brought home to me back in the depths of time while I doing Physics "A" Level in high school and calculating measurement errors in experiments. From this I learned that no matter how precise one's measurements were, there an inherent error in the tool which was used to measure the results. Taking into account that lack of precision provided more accurate results, which is what we sought.

When I got involved in finance during my time in business school, the measurement tool was currency, and so precision was deemed to be the smallest currency unit, i.e. a cent. This precision was encouraged by the wonderful computing power of Lotus 1-2-3 and then Excel which allowed you to do forecasts that were precise to the cent going out years based on a few assumptions. While the precision was there, accuracy was foregone as we built huge models on a few simplistic assumptions.

However, upon leaving business school and entering the real world my illusions were quickly dashed. Having been at work less than a week, I presented my boss with a 10 year forecasting model which showed profit and loss statements, balance sheets, cash flows statements and of course a discounted cash flow valuation. He looked at it and asked how confident I was in my analysis. As I defended it, he looked at me and asked in a sort of Baynesian way, "Would you bet your bonus on company making the revenue numbers you have for this year?" (we were a quarter of the way into the year). 

"No!" was my quick reply. 

"In that case, how can you defend ten years of numbers which all build off that initial number?" he asked. It hit me and that has lived with me ever since that we quickly get sucked into the precision of our models, but really the numbers are so inaccurate.

During my career, I have see precise projections for sales, projects, costs being bandied around in companies by everyone, including the CFO. Yet few, if any, would bet a modest amount, let alone their bonuses, on those numbers being the final results. Thus we end up with the corporate situation where everyone knows the numbers are wrong but cannot admit it.

However, when working with these same companies and trying to get them to accept the concept of ranges in forecasts and probability distributions, I am regarded as though I had just stated that the earth was flat and supported by elephants. One CFO of a public company looked at me with total distane and asked "Why would anyone ever need something like that?".

When I responded that I was seeking more accuracy, albeit with less precision, in financial projections it was obvious that had I reverted to speaking Fanagalo. Another CFO used precise numbers (down to the dollar when their revenue was in excess of $20 million, and none of their forecasts were ever correct but he could not accept that forecasting to the million was acceptable. I heard that their last results were a huge surprise given the forecasts; however given the forecasting process I was not surprised at all.

This trade off of precision in return for greater accuracy continually is rejected by many of the financial professionals that I meet. I have wondered if the cause could be the focus on precision encouraged by the accounting profession that make it hard for these professionals to let it go.

So going back to my boss' logic - if you would not bet on a single number being accurate, would you be prepared to bet on a range? For example, the forecast may say that next years' revenue will be $10 million, but when pushed no one will bet their bonus on it. So would you be willing to be that it will be between $9 and $11 million or $8 and $12 million? When you reach a range that you would be willing to bet on, then surely that is the range to use in the forecasts.

Using ranges like this and applying techniques like Monte Carlo simulations results in far great accuracy in forecasts and an understanding of the risk profile of the results, i.e. there is a 50% probability that revenue will between $9 million and $10 million or that profits will between $0 and $1 million. Furthermore one can see what items are key to the final results what is not. Thus company's can focus on those key items rather than getting distracted by the noise.

It is best to remember Nils Bohr's quote "Prediction is very difficult, especially if it's about the future." Therefore, to make the predictions more useful, it is best to forego precision in return for greater accuracy.

Copyright 2013 Marc A. Borrelli

What are you measuring and why?

In my discussions with many CEOs and CFOs it is always interesting to see what is measured and tracked. However, many times they are not measuring the right things. Here are five suggestions of what and howto measure.

1. Measure your mission / differentiation goals. 

In reading the goals or mission statements of many companies, often you will see "we provide superior service to our clients". This is a very worthwhile goal or aim; however, once I get involved with a company I notice that there is no metric of customer service and satisfaction that is tied into key metrics and that drive performance bonuses. Furthermore, those that are measured are often measured in isolation and are not tracked in blind tests against those the of competitors or potential customers that didn't choose the company's products or services. Whatever is claimed to be the company's differentiation and vision should be measured and tracked to ensure that is what is being delivered.

2. Make sure you measure your strategy expectations

If embarking on a new strategic initiative ensure that you know what your expectations from the strategy are and then develop the appropriate metrics to measure its performance. Again many companies embark on new strategies and the metric tracked is often the financial contribution. Tracking financial contribution is also a worthwhile aim; however, it is the result and cannot always tell you what is driving those results. I like to think of it as measuring Return on Equity, rather than doing it through the Du Pont analysis. While both give you the same result, the latter is much more informative as it breaks down the ROE return into different performance metrics allowing you to understand what is driving the ROE as well as do better industry comparisons. In addition, this analysis should be broken down more to look at the key drivers of revenue and expenses. As I have said on prior blogs, usually its a handful of items drive 90% of the returns so these are the one that need to be focused on, not all the noise around them.

3. Measure cash

Cash is king and all CFOs know this. However, I find that many CEOs and CFOs in small to midsized companies are not always sure of what revenue does to cash. A simple measure that I like to look at is what I call Cash generated for $1 of Revenue. The formula for this is:

Cash Generated/Revenue = Net Margin - (Net Working Capital / Revenue) + [(Depreciation + Amortization - Capital Expenditure) / Revenue]

A company that I was involved with was proud of their net margin but was always cash strapped. Applying the above formula I got:

C/R = 9.3% - 24.5% + 1.5% = -13.5%

Thus for every additional dollar of revenue, the company needed $0.135 of additional cash to fund the cash shortfall. Thus the company's strategy of aggressively growing sales could drive them into bankruptcy if they were unable to raise capital to fund this growth. Many companies have found success to be lethal and this was another example.

As I learned many years ago from a wise adviser - fix the pipe before forcing more product through it. The key in the above was to focus on the company's Net Working Capital and reduce it so that the company was generating cash for every dollar of additional revenue. Once that had been accomplished they could drive sales aggressively.

4. Tie compensation to variables that employees can influence and will continue to influence

During my career I have worked with a number of large and small companies and can say that often a large piece of the incentive compensation of many departments has been tied to things that they have no control over. Examples of this are:

a. Net contribution of the department

In firm  bonuses were tied to net contribution of the department to the corporate profits. While this makes sense at a high level, the issue faced was that 50% of the department's costs were allocated by corporate HQ over which the department had no control. Doing some research it was determined that if the department could operated on a true stand alone basis with no allocated costs, it would save 10% of its costs. Thus it was proposed setting up the division separately where they would have full control over their costs, but this was rejected on the basis that if they did that then the additional costs would have to born by other departments.

Tie your employees bonuses to their direct costs which they control and measure those! If the HQ is allocating central costs then senior management is in charge of those costs and they should be rewarded on control of those costs. To often bloated central costs are divided up between departments and no looks at them carefully because they are not measured in a meaningful way.

b. Tied to operational results which excluded the departments activities

In another firm I have experience with the M&A department's compensation was tied to corporate profitability; however, the M&A department is really a cost center and there is little it can do for the overall costs of the company. Better metrics would be to tie it to the value generated by the transactions it executed, the time and effectiveness of integration planning, minimizing the costs of third party professional in doing transactions etc. These are items it could control and employees would be more incentivized to meet their objectives. So the key is to develop the appropriate metrics to drive the results that are sought.

c. Sales compensation tied to a 10 year future income stream that is not guaranteed.

Finally, I have come across situations where the sales team is incentivized by the potential revenue they generate for the company. Again this would seem to be the right motivation. However, the salesmen were being paid the present value of a ten year contract and in no example did I ever see the projections for income do anything but go upwards in a linear fashion (unless it was a hockey stick!). However, looking back over the company's history this had never once been the case in a ten year contract - Yes Virginia business cycles are real. Finally, once the salesmen signed a client they had no incentive to ensure the client stayed with the company for the full 10 years or generated the potential income on which they had been compensated. When I discussed this with the finance department they were all aware of the issues but said that if they changed the compensation all the salesmen would leave. They were helpless. A worrying signal about the corporate culture!

5. General Statistics

This may be an odd heading but I see time and time again people using statistical tools without the full appreciation of them so here are two simple reminders. For example I see people doing lots of regression analysis in Excel to project some future cost or revenue item, but when I ask if the date sample is normally distributed I get blank stares - you cannot do regression analysis if its not!

a. Need sufficient data to draw conclusions

It is amazing how many projections or statistical inferences are developed off a few data samples. Just as a reminder if you have 100 samples the error bounds on the results are 10%, to get the error bounds down to 3% the sample size needs to be 1,000. Therefore, it is always worrying when a few samples, i.e. less than 50 have been used to generate some estimate and the estimate is given with definite precision from an Excel model. There are no ranges in the model, and it all falls back into the precision vs accuracy issue.

b. Correlation does not imply causality

This seemly obvious rule is one that it would appear no television forecast has ever been made aware of. Listening to statements like "the team that is leading at the end of the 1st quarter always wins" makes me want to cry. But I see the same logic applied in corporate modeling and measurements where the equivalent of "the coin has been tails the last four throws so it will be again" is expressed with confidence. There is little question of what drives what or why the results are correlated. Remember you derive a formula to match any set of data points but it may be meaningless.

c. Measure the important stuff

As I mentioned above and as represented in the Pareto Principle, 20% of the variables drive 80% of the results. Thus focus on those items that drive the results and spend time and effort to measure and understand them correctly. The rest are not as important and can easily estimated as they will not affect the ultimate outcome significantly.