67% of GRC Statistics Are Made Up Just for Blogs

A French knight, an Italian math guy and a pair of dice ride into a casino. . .well, maybe that’s not exactly how the story of probability begins, but if I’m going to get you to read something other than the picture caption, I’ve got to grab your attention.

What are the odds your GRC metrics will hold up under scrutiny?

What are the odds your GRC metrics will hold up under scrutiny?

No matter where you work, you’ve likely sat through a meeting that included a statement like, “What gets measured gets done.” This statement is inevitably followed by some new thing to measure that will no doubt allow your team to “exceed stakeholder expectations” or some other amorphous goal – you’ll probably even get to hear the tale of “Company X” and how when they did the same thing it “revolutionized their business.”

It’s easy to fall in love with numeric measurements; they cut through the buzz words and provide something concrete. We put a lot of trust in numbers. However, are our numbers built on assumptions or facts?

This same question led to a new way to view math: probability. If you’re a GRC professional, or any professional who deals with numbers and conclusions, take a moment and add some history to your day. The following is a true account, but it reads just like an Aesop fable – the hero battles a problem and in the process illustrates a great truth.

Doubling Down on Good Intentions
During the 17th Century, a French nobleman named the Chevalier de Méré was taken with games of chance, particularly those involving dice. A favorite game involved betting others that he could roll a six within 4 rolls of a single die. Seeing that there were 6 possible outcomes on a single roll, he believed it was more likely than not a six would come up over the course of 4 rolls – seems like a 67% chance of success, right? (1-in-6 odds on each roll, 4 rolls would equal a 4-in-6 shot – thus, 67%) If you were putting money down, you’d probably side with de Méré too. More often than not, de Méré walked away a winner.

Over time, his bar chums got wind that betting against de Méré was a bad investment and they stopped playing dice with the guy. This prompted de Méré to up the ante; now he bet that he could roll double sixes over the course of 24 rolls of 2 dice. Using his previous math, de Méré thought he was just re-packaging the single dice game. Since there were 36 possible combinations for a roll of 2 dice (6×6) his chance of rolling double sixes on any given roll would be 1/36. Therefore, he thought rolling 24 times would give him favorable odds of 24-in-36 – again, a 67% chance of success. However, de Méré found himself losing more than winning in this iteration of the game.

Don’t Bet the Horse on Bad Math
Why did the fractions fail the Frenchmen for the second game? Beyond that, was he EVER figuring the odds correctly? To get to the bottom of this, de Méré sought help from the famous Blaise Pascal – yes, the triangle guy. Through a series of letters exchanged between Pascal and Pierre de Fermat, Pascal exposed the problem with de Méré’s logic.

It turns out de Méré wasn’t even getting the odds of the single dice game correct, let alone the 2 dice game. While he correctly figured out the chance of getting a six in one dice roll was 1-in-6, he incorrectly assumed how that probability would compound over multiple throws. So what’s the right way to figure this out?

To accurately find the odds in a single-dice game, Pascal first determined the total number of outcomes for 4 rolls of a 6-sided die:

  • Total outcomes: 1296 (6x6x6x6)

Next, he found the outcomes that would be a loss (rolling something other than a six):

  • Losing outcomes: 625 (5x5x5x5)

Using these values, he came up with the winning outcomes:

  • Winning outcomes: 671 (first result minus the second result)

This gave de Méré a 51.8% probability of winning his first dice game. While de Méré was able to profit from this game, he wasn’t winning by as large of a margin as he thought! Using this as a model, when you investigate the second dice game, it explains why de Méré was unable to win with the same frequency:

  • Total outcomes: 22,452,257,707,354,600,000,000,000,000,000,000,000 (36x36x. . .) That’s 22 unedecillion if you’re curious; the same unit used to measure the number of possible IP addresses in IPv6 format.
  • Losing outcomes: 11,419,131,242,070,600,000,000,000,000,000,000,000
  • Winning outcomes: 11,033,126,465,284,000,000,000,000,000,000,000,000

Parsing through these unimaginable numbers (don’t ask me how two 17th century guys figured that one out), you can see that it’s more likely you’re going to lose the 2 dice game – the win probability for rolling double sixes across 24 attempts is only 49.1%. From this inquiry, de Méré was able to fine tune his game and Pascal laid the foundation for a whole new field of math. So the next time you’re in Vegas, you can show off by telling the origin story of professional gambling, i.e. probability. 

Now What? Extracting GRC Wisdom from a Long-Winded Blog Post
How is this useful? I think this serves as a warning to all of us whose job involves interpreting data. In our story, de Méré’s original assumption (that adding fractions creates true odds) broke down when he expanded the scale of his game. Had he understood why he was truly winning the first game, he could have saved himself some money and built the second game around a different set of criteria.

How often have you encountered the following scenarios?

  • Stumbling across a metric that appears to correlate exactly with a specific outcome, but failing to truly investigate the patterns. (This is what our friend de Méré did.)
  • Starting with the conclusion and then reviewing the numbers until you find data that supports your assumption. (This is a favorite during election season.)
  • Tracking behaviors that, while interesting, don’t affect whether your objectives will be achieved. (Does it matter that “Joe” completes his assessment two days sooner than “Bob” if both meet the deadline and thoroughly and accurately respond to the questions?)

I encourage you to take time today, or at least this week, and review your top five metrics. Are they proven? Will they hold up if you expand your scale? Are you just making up numbers for the sake of having numbers to report? We are a numbers-driven society, but the numbers can tell a variety of stories. Before you generate and share your next eye chart or report, take a moment and ensure that the story you are sharing is one worth listening to.

–Jonathan Kitchin, OrangePoint

References

This entry was posted in Risk Management and tagged . Bookmark the permalink.

2 Responses to 67% of GRC Statistics Are Made Up Just for Blogs

  1. BJ says:

    I believe it is closer to 83% per Goliath’s Mr. Stinson

  2. site says:

    Hello to every single one, it’s in fact a pleasant for me to go to see this web site
    site, it includes precious Information. I am actually
    site glad to read this web site posts which includes
    lots of useful data, thanks for providing these kinds of site information.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s