Table of Contents

Preface xi

Acknowledgments xiv

Introduction xv

1 First Principles 1

1.1 Random Experiment, Sample Space, Event 1

1.2 What Is a Probability? 3

1.3 Probability Function 4

1.4 Properties of Probabilities 7

1.5 Equally Likely Outcomes 10

1.6 Counting I 12

1.7 Problem-Solving Strategies: Complements, Inclusion-Exclusion 14

1.8 Random Variables 18

1.9 A Closer Look at Random Variables 21

1.10 A First Look at Simulation 22

1.11 Summary 26

Exercises 27

2 Conditional Probability 34

2.1 Conditional Probability 34

2.2 New Information Changes the Sample Space 39

2.3 Finding P(A and B) 40

2.4 Conditioning and the Law of Total Probability 49

2.5 Bayes Formula and Inverting a Conditional Probability 57

2.6 Summary 61

Exercises 62

3 Independence and Independent Trials 68

3.1 Independence and Dependence 68

3.2 Independent Random Variables 76

3.3 Bernoulli Sequences 77

3.4 Counting II 79

3.5 Binomial Distribution 88

3.6 Stirling's Approximation 95

3.7 Poisson Distribution 96

3.8 Product Spaces 105

3.9 Summary 107

Exercises 109

4 Random Variables 117

4.1 Expectation 118

4.2 Functions of Random Variables 121

4.3 Joint Distributions 125

4.4 Independent Random Variables 130

4.5 Linearity of Expectation 135

4.6 Variance and Standard Deviation 140

4.7 Covariance and Correlation 149

4.8 Conditional Distribution 156

4.9 Properties of Covariance and Correlation 162

4.10 Expectation of a Function of a Random Variable 164

4.11 Summary 165

Exercises 168

5 A Bounty of Discrete Distributions 176

5.1 Geometric Distribution 176

5.2 Negative Binomial-Up from the Geometric 184

5.3 Hypergeometric-Sampling Without Replacement 189

5.4 From Binomial to Multinomial 194

5.5 Benford's Law 201

5.6 Summary 203

Exercises 205

6 Continuous Probability 211

6.1 Probability Density Function 213

6.2 Cumulative Distribution Function 216

6.3 Uniform Distribution 220

6.4 Expectation and Variance 222

6.5 Exponential Distribution 224

6.6 Functions of Random Variables I 229

6.7 Joint Distributions 235

6.8 Independence 243

6.9 Covariance, Correlation 249

6.10 Functions of Random Variables II 251

6.11 Geometric Probability 256

6.12 Summary 262

Exercises 265

7 Continuous Distributions 273

7.1 Normal Distribution 273

7.2 Gamma Distribution 290

7.3 Poisson Process 296

7.4 Beta Distribution 304

7.5 Pareto Distribution, Power Laws, and the 80-20 Rule 308

7.6 Summary 312

Exercises 315

8 Conditional Distribution, Expectation, and Variance 322

8.1 Conditional Distributions 322

8.2 Discrete and Continuous: Mixing it up 328

8.3 Conditional Expectation 332

8.4 Computing Probabilities by Conditioning 342

8.5 Conditional Variance 346

8.6 Summary 352

Exercises 353

9 Limits 359

9.1 Weak Law of Large Numbers 361

9.2 Strong Law of Large Numbers 367

9.3 Monte Carlo Integration 372

9.4 Central Limit Theorem 376

9.5 Moment-Generating Functions 385

9.6 Summary 391

Exercises 392

10 Additional Topics 399

10.1 Bivariate Normal Distribution 399

10.2 Transformations of Two Random Variables 407

10.3 Method of Moments 411

10.4 Random Walk on Graphs 413

10.5 Random Walks on Weighted Graphs and Markov Chains 421

10.6 From Markov Chain to Markov Chain Monte Carlo 429

10.7 Summary 440

Exercises 442

Appendix A Getting Started with R 447

Appendix B Probability Distributions in R 458

Appendix C Summary of Probability Distributions 459

Appendix D Reminders from Algebra and Calculus 462

Appendix E More Problems for Practice 464

Solutions to Exercises 469

References 487

Index 491

About the Author

ROBERT P. DOBROW, PhD, is Professor of Mathematics at Carleton College. He has taught probability for over fifteen years and has authored numerous papers in probability theory, Markov chains, and statistics.