Table of Contents

Preface to the Third Edition xiii Acknowledgments xix About the Author xxi Part I The Measurement Solution Exists 1 Chapter 1 The Challenge of Intangibles 3 The Alleged Intangibles 4 Yes, I Mean Anything 5 The Proposal: It's about Decisions 7 A "Power Tools" Approach to Measurement 10 A Guide to the Rest of the Book 11 Chapter 2 An Intuitive Measurement Habit: Eratosthenes, Enrico, and Emily 15 How an Ancient Greek Measured the Size of Earth 16 Estimating: Be Like Fermi 17 Experiments: Not Just for Adults 20 Notes on What to Learn from Eratosthenes, Enrico, and Emily 25 Notes 27 Chapter 3 The Illusion of Intangibles: Why Immeasurables Aren't 29 The Concept of Measurement 30 The Object of Measurement 37 The Methods of Measurement 40 Economic Objections to Measurement 48 The Broader Objection to the Usefulness of "Statistics" 52 Ethical Objections to Measurement 55 Reversing Old Assumptions 58 Notes 65 Part II Before You Measure 69 Chapter 4 Clarifying the Measurement Problem 71 Toward a Universal Approach to Measurement 73 The Unexpected Challenge of Defining a Decision 74 If You Understand It, You Can Model It 80 Getting the Language Right: What "Uncertainty" and "Risk" Really Mean 83 An Example of a Clarified Decision 84 Notes 90 Chapter 5 Calibrated Estimates: How Much Do You Know Now? 93 Calibration Exercise 95 Calibration Trick: Bet Money (or Even Just Pretend To) 101 Further Improvements on Calibration 104 Conceptual Obstacles to Calibration 106 The Effects of Calibration Training 111 Notes 118 Chapter 6 Quantifying Risk through Modeling 123 How Not to Quantify Risk 123 Real Risk Analysis: The Monte Carlo 125 An Example of the Monte Carlo Method and Risk 127 Tools and Other Resources for Monte Carlo Simulations 136 The Risk Paradox and the Need for Better Risk Analysis 140 Notes 143 Chapter 7 Quantifying the Value of Information 145 The Chance of Being Wrong and the Cost of Being Wrong: Expected Opportunity Loss 146 The Value of Information for Ranges 149 Beyond Yes/No: Decisions on a Continuum 156 The Imperfect World: The Value of Partial Uncertainty Reduction 159 The Epiphany Equation: How the Value of Information Changes Everything 166 Summarizing Uncertainty, Risk, and Information Value: The Pre-Measurements 171 Notes 172 Part III Measurement Methods 173 Chapter 8 The Transition: From What to Measure to How to Measure 175 Tools of Observation: Introduction to the Instrument of Measurement 177 Decomposition 180 Secondary Research: Assuming You Weren't the First to Measure It 184 The Basic Methods of Observation: If One Doesn't Work, Try the Next 186 Measure Just Enough 188 Consider the Error 189 Choose and Design the Instrument 194 Notes 196 Chapter 9 Sampling Reality: How Observing Some Things Tells Us about All Things 197 Building an Intuition for Random Sampling: The Jelly Bean Example 199 A Little about Little Samples: A Beer Brewer's Approach 200 Are Small Samples Really "Statistically Significant"? 204 When Outliers Matter Most 208 The Easiest Sample Statistics Ever 210 A Biased Sample of Sampling Methods 214 Experiment 226 Seeing Relationships in the Data: An Introduction to Regression Modeling 235 Notes 243 Chapter 10 Bayes: Adding to What You Know Now 247 The Basics and Bayes 248 Using Your Natural Bayesian Instinct 257 Heterogeneous Benchmarking: A "Brand Damage" Application 263 Bayesian Inversion for Ranges: An Overview 267 The Lessons of Bayes 276 Notes 282 Part IV Beyond the Basics 285 Chapter 11 Preference and Attitudes: The Softer Side of Measurement 287 Observing Opinions, Values, and the Pursuit of Happiness 287 A Willingness to Pay: Measuring Value via Trade-Offs 292 Putting It All on the Line: Quantifying Risk Tolerance 296 Quantifying Subjective Trade-Offs: Dealing with Multiple Conflicting Preferences 299 Keeping the Big Picture in Mind: Profit Maximization versus Purely Subjective Trade-Offs 302 Notes 304 Chapter 12 The Ultimate Measurement Instrument: Human Judges 307 Homo Absurdus: The Weird Reasons behind Our Decisions 308 Getting Organized: A Performance Evaluation Example 313 Surprisingly Simple Linear Models 315 How to Standardize Any Evaluation: Rasch Models 316 Removing Human Inconsistency: The Lens Model 320 Panacea or Placebo?: Questionable Methods of Measurement 325 Comparing the Methods 333 Example: A Scientist Measures the Performance of a Decision Model 335 Notes 336 Chapter 13 New Measurement Instruments for Management 339 The Twenty-First-Century Tracker: Keeping Tabs with Technology 339 Measuring the World: The Internet as an Instrument 342 Prediction Markets: A Dynamic Aggregation of Opinions 346 Notes 353 Chapter 14 A Universal Measurement Method: Applied Information Economics 357 Bringing the Pieces Together 358 Case: The Value of the System That Monitors Your Drinking Water 362 Case: Forecasting Fuel for the Marine Corps 367 Case: Measuring the Value of ACORD Standards 373 Ideas for Getting Started: A Few Final Examples 378 Summarizing the Philosophy 384 Notes 385 Appendix Calibration Tests (and Their Answers) 387 Index 397

About the Author

DOUGLAS W. HUBBARD is the inventor of Applied InformationEconomics (AIE), a measurement methodology that has earned himcritical praise from The Gartner Group, Giga Information Group, andForrester Research. He is an internationally recognized expert inthe field of decision analysis and challenging measurements and isa popular speaker at numerous conferences. He has written articlesfor Information Week, CIO Enterprise, and DBMSMagazine. He is the author of The Failure of RiskManagement: Why It s Broken and How to Fix It andPulse: The New Science of Harnessing Internet Buzz to TrackThreats and Opportunities.