Table of Contents

Preface to 2nd Edition Preface to 1st Edition Acknowledgements Introduction to Meta-Analysis Integration Research Findings Across Studies General problem and an example Problems with statistical significance tests Is statistical power the solution? Confidence intervals Meta-analysis Role of meta-analysis in the behavioral and social sciences Role of meta-analysis in theory development Increasing use of meta-analysis Meta-analysis in industrial-organizational psychology Wider impact of meta-analysis on psychology Impact of meta-analysis outside psychology Meta-analysis and social policy Meta-analysis and theories of data Conclusions Study Artifacts and Their Impact on Study Outcomes Study Artifacts Sampling error, statistical power, and the interpretation of research literatures When and how to cumulate Undercorrection for artifacts in the corrected standard deviation Coding study characteristics and capitalization on sampling error in moderator analysis A look ahead in the book Meta-Analysis of Correlations Meta-Analysis of Correlations Corrected Individually for Artifacts Introduction and Overview Bare bones meta-analysis: Correcting for sampling error only Artifacts other than sampling error Multiple simultaneous artifacts Meta-analysis of individually corrected correlations A worked example: Indirect range restriction Summary of meta-analysis correcting each correlation individually Exercise 1: Bare bones meta-analysis Exercise 2: Meta-analysis correcting each correlation individually Meta-Analysis of Correlations Using Artifact Distributions Full artifact distribution meta-analysis Accuracy of corrections for artifacts Mixed meta-analysis: Partial artifact information in individual studies Summary of artifact distribution of meta-analysis of correlations Exercise: Artifact distribution meta-analysis Technical Questions in Meta-Analysis of Correlations r versus : Which should be used? r vs. regression slopes and intercepts in meta-analysis Technical factors that cause overestimation of Fixed and random models in meta-analysis Credibility vs. confidence intervals in meta-analysis Computing confidence intervals in meta-analysis Range Restriction in meta-analysis: New technical analysis Criticisms of meta-analysis procedures for correlations Meta-Analysis of Experimental Effects and Other Dichotomous Comparisons Treatment Effects: Experimental Artifacts and Their Impact Quantification of the treatment effect: The d statistic and the point-biserial correlation Sampling error in d values: Illustrations Error of measurement in the dependent variable Error of measurement in the treatment variable Variation across studies in treatment strength Range variation on the dependent variable Dichotomization of the dependent variable Imperfect construct validity in the dependent variable Imperfect construct validity in the treatment variable Bias in the effect size (d statistic) Recording, computational, and transcriptional errors Multiple artifacts and corrections Meta-Analysis Methods for d Values Effect size indices: d and r An Alternative to d: Glass' d Sampling error in the d statistic Cumulation and correction of the variance for sampling error Analysis of moderator variables Correcting d values for measurement error in the dependent variable Measurement error in the independent variable in experiments Other artifacts and their effects Correcting for multiple artifacts Summary of meta-analysis of d values Exercise: Meta-Analysis of d-Values Technical Questions in Meta-Analysis of d Values Alternative experimental designs Within-subjects experimental designs Meta-analysis and the within-subjects design Statistical power in the two designs Threats to internal and external validity Bias in observed d values Use of multiple regression in moderation analysis of d values General Issues in Meta-Analysis General Technical Issues in Meta-analysis Fixed effects versus random effects models in meta-analysis Second order sampling error: General principles Detecting moderators not hypothesized a priori Second order meta-analysis Large N studies and meta-analysis Second order sampling error: Technical treatment The detection of moderator variables: Summary Hierarchical analysis of moderator variables Exercise: Second order meta-analysis Cumulation of Findings Within Studies Fully replicated designs Conceptual replications Conceptual replications and confirmatory factor analysis Conceptual replications: A alternative approach Analysis of subgroups Summary Methods of Integrating Findings Across Studies and Related Software The traditional narrative procedure The traditional voting method Cumulation of p-values across studies Statistically correct vote counting procedures Meta-analysis of research studies Unresolved problems in meta-analysis Summary of methods of integrating studies Computer programs for meta-analysis Locating, Evaluating, and Coding Studies Conducting a thorough literature search What to do about studies with "methodological weaknesses" Coding studies in meta-analysis What to include in the meta-analysis report Information needed in reports of primary studies Appendix: Coding sheet for validity studies Availability and Source Bias in Meta-Analysis Some evidence on bias Effects of methodological quality on mean effect sizes from different sources Multiple hypotheses and other considerations in availability bias Methods for detecting availability bias Methods for correcting for availability bias Summary of Psychometric Meta-Analysis Meta-analysis methods and theories of data What is the ultimate purpose of meta-analysis? Appendix: Windows Based Meta-Analysis Software Package References Author Index Subject Index About the Authors

About the Author

John E. (Jack) Hunter (1939--2002) was a professor in the Department of Psychology at Michigan State University. He received his Ph.D. in quantitative psychology from the University of Illinois. Jack coauthored four books and authored or coauthored over 200 articles and book chapters on a wide variety of methodological topics, including confirmatory and exploratory factor analysis, measurement theory and methods, statistics, and research methods. He also published numerous research articles on such substantive topics as intelligence, attitude change, the relationship between attitudes and behavior, validity generalization, differential validity/selection fairness, and selection utility. Much of his research on attitudes was in the field of communications, and the American Communications Association named a research award in his honor. Professor Hunter received the Distinguished Scientific Award for Contributions to Applied Psychology from the American Psychological Association (APA) (jointly with Frank Schmidt) and the Distinguished Scientific Contributions Award from the Society for Industrial/Organizational Psychology (SIOP) (also jointly with Frank Schmidt). He was a Fellow of APA, APS, and SIOP, and was a past president of the Midwestern Society for Multivariate Experimental Psychology. For the story of Jack's life, see Schmidt (2003). Frank L. Schmidt is the Gary F. Fethke Leadership Professor Emeritus in the Department of Management and Organization in the Tippie College of Business at the University of Iowa. He received his Ph.D. in industrial/organizational psychology from Purdue University and has been on the faculties of Michigan State and George Washington Universities. He has authored or coauthored seven books and nearly 200 articles and book chapters on measurement, statistics, research methods, individual differences, and personnel selection. He headed a research program in the U.S. Office of Personnel Management in Washington, D.C., for 11 years, during which time he published numerous research studies in personnel psychology, primarily with John Hunter. Their research on the generalizability of employment selection method validities led to the development of the meta-analysis methods presented in this book. Professor Schmidt has received the Distinguished Scientific Award for Contributions to Applied Psychology from the American Psychological Association (APA) (jointly with John Hunter) and the Distinguished Scientific Contributions Award from the Society for Industrial/Organizational Psychology (SIOP) (also jointly with John Hunter). He has also received the Ingram Olkin Award and the Frederick Mosteller Award, both for contributions to meta-analysis methodology; the Scientific Award for Applications of Psychology from the Association for Psychological Science (APS); the Gold Medal Lifetime Achievement award from the APA Foundation; the Distinguished Career Award for Contributions to Human Resources, and the Distinguished Career Achievement Award for Contributions to Research Methods, both from the Academy of Management. He is a Fellow of the APA, the Association for Psychological Science, and SIOP, and is past president of Division 5 (Measurement, Statistics, & Evaluation) of the APA.

Reviews

"Clearly written and compellingly argued, this book explains the procedures and benefits of correcting for measurement error and range restriction and details the methodological developments in meta-analysis over the last decade. No one should consider conducting a meta-analysis without first reading this book. It is essential reading for all scientists." -- Michael A. McDaniel
"A book that will certainly appeal not only to the students, but will also be a great reference source for the technically sophisticated professional. The breadth and depth of the coverage, not to mention the novelty and clarity of writing, makes this book a classic in the field. It covers (and at times introduces) many novel issues that will be in the forefront for some years to come-as such a must read for all meta-analysts." -- Vish C. Viswesvaran, Ph.D.