Chapter 1: Introduction to Educational Assessment
I. Introduction
II. The Language of Assessment
A. Tests, Measurement, & Assessment
B. Types of Tests
C. Types of Scores Interpretations
III. Assumptions of Educational Assessment
A. Psychological and educational constructs exist.
B. Psychological and educational constructs can be measured.
C. While we can measure constructs, our measurement is not perfect.
D. There are different ways to measure any given construct.
E. All assessment procedures have strengths and limitations.
F. Multiple sources of information should be part of the assessment process.
G. Performance on tests can be generalized to non-test behaviors.
H. Assessment can provide information that helps educators make better educational decisions.
I. Assessments can be conducted in a fair manner.
J. Testing and assessment can benefit our educational institutions and society as a whole.
IV. Participants in the Assessment Process
A. People who develop tests.
B. People who use tests.
C. People who take tests.
D. Other people involved in the assessment process.
V. Common Application of Educational Assessments
A. Student Evaluation
B. Instructional Decisions
C. Selection, Placement, and Classification Decisions
D. Policy Decisions
E. Counseling and Guidance Decisions
VI. What Teachers Need to Know About Assessment
A. Teachers should be proficient in selecting professionally developed assessment
procedures that are appropriate for making instructional decisions.
B. Teachers should be proficient in developing assessment procedures that are
appropriate for making instructional decisions.
C. Teachers should be proficient in administering, scoring, and interpreting
professionally developed and teacher-made assessment procedures.
D. Teachers should be proficient in using assessment results when making
educational decisions.
E. Teachers should be proficient in developing valid grading procedures that
incorporate assessment information.
F. Teachers should be proficient in communicating assessment results.
G. Teachers should be proficient in recognizing unethical, illegal, and other
inappropriate uses of assessment procedures or information.
VII. Educational Assessment in the 21st Century
A. Computerized Adaptive Testing (CAT) and Other Technological Advances.
B. Authentic Assessments
C. Educational Accountability and High-Stakes Assessment
D. Trends in the Assessment of Students with Disabilities
VIII. Summary
Tables
A. Table 1.1: Major Categories of Tests
B. Table 1.2: Norm- and Criterion-Referenced Scores
C. Table 1.3: Assumptions of Educational Assessment
D. Table 1.4: Common Applications of Educational Assessments
E. Table 1.5: Teacher Competencies in Educational Assessment
Special Interest Topics
A. Special Interest Topic 1.1: Cognitive Diagnostic Assessment — Another Step Toward Unifying Assessment and Instruction
B. Special Interest Topic 1.2: Technology and Assessment in the Schools
C. Special Interest Topic 1.3: Princeton Review's Rankings of High-Stakes Testing Programs
D. Special Interest Topic 1.4: The “Nation's Report Card”
E. Special Interest Topic 1.5: What Does the 21st Century Hold for the Assessment Profession?
Chapter 2: The Basic Mathematics of Measurement
I. The Role of Mathematics in Assessment
II. Scales of Measurement
A. What is Measurement?
B. Nominal Scales
C. Ordinal Scales
D. Interval Scales
E. Ratio Scales
III. The Description of Test Scores
A. Distributions
B. Measures of Central Tendency
C. Measures of Variability
IV. Correlation Coefficients
A. Scatterplots
B. Correlation and Prediction
C. Types of Correlation Coefficients
D. Correlation and Causality
V. Summary
Tables
Table 2.1: Common Nominal, Ordinal, Interval, & Ratio Scales
Table 2.2: Distribution of Scores for 20 Students
Table 2.3: Ungrouped Frequency Distribution
Table 2.4: Group Frequency Distribution
Table 2.5: Calculating the Standard Deviation and Variance
Table 2.6: Calculating a Pearson Correlation Coefficient
Figures
Figure 2.1: Graph of the Homework Scores
Figure 2.2: Hypothetical Distribution of Large Standardization
Sample
Figure 2.3: Negatively Skewed Distribution
Figure 2.4: Positively Skewed Distribution
Figure 2.5: Bimodal Distribution
Figure 2.6: Relationship between Mean, Median, and Mode in Normal and Skewed Distributions
Figure 2.7: Three Distributions with Different Degrees of Variability
Figure 2.8: Scatterplots of Different Correlation Coefficients
Special Interest Topics
Special Interest Topic 2.1: Population Parameters and Sample Statistics
Special Interest Topic 2.2: A Public Outrage: Physicians Overcharge Their Patients
Special Interest Topic 2.3: Is the Variance Always Larger Than the Standard Deviation?
Special Interest Topic 2.4: Caution: Drawing Conclusions of Causality
Chapter 3: The Meaning of Test Scores
I. Introduction
II. Norm-Referenced & Criterion-Referenced Score Interpretations
A. Norm-Referenced Interpretations
B. Criterion-Referenced Interpretations
III. Norm-Referenced, Criterion-Referenced, or Both?
IV. Qualitative Description of Scores
V. Summary
Tables
Table 3.1: Transforming Raw Scores to Standard Scores
Table 3.2: Relationship of Different Standard Score Formats
Table 3.3: Converting Standard Scores From One Format to Another
Table 3.4: Characteristics of Norm-Referenced and Criterion-Referenced Scores
Figures
Figure 3.1: Illustration of the Normal Distribution
Figure 3.2: Normal Distribution with Mean, Standard Deviation, & Percentages.
Figure 3.3: Normal Distribution Illustrating the Relationship among Standard Scores.
Special Interest Topics Special Interest Topic 3.1: The “Flynn Effect.” Special Interest Topic 3.2: Whence the Normal Curve?
Special Interest Topic 3.3: Why do IQ Tests use a Mean of 100 and Standard Deviation of 15?
Special Interest Topic 3.4: The History of Stanine Scores
Special Interest Topic 3.5: Every Child on Grade Level?
Chapter 4: Reliability for Teachers
I. Introduction
II. Errors of Measurement
A. Sources of Measurement Error
III. Methods of Estimating Reliability
A. Test-Retest Reliability
B. Alternate Form Reliability
C. Internal Consistency Reliability
D. Inter-Rater Reliability
E. Reliability of Composite Scores
F. Selecting a Reliability Coefficient
G. Evaluating Reliability Coefficients
H. How to Improve Reliability
I. Special Problems in Estimating Reliability
IV. The Standard Error of Measurement
A. Evaluating the Standard Error of Measurement
V. Reliability: Practical Strategies for Teachers
VI. Summary
Tables
Table 4.1: Major Types of Reliability
Table 4.2: Half-Test Coefficients and Corresponding Full-Test Coefficients Corrected
with the Spearman-Brown Formula
Table 4.3: Calculation of KR 20
Table 4.4: Calculation of Coefficient Alpha
Table 4.5: Calculating Inter-Rater Agreement
Table 4.6: Source of Error Variance Associated with Major Types of Reliability
Table 4.7: Reliability Expected When Increasing the Numbers of Items
Table 4.8: Standard Errors of Measurement for Values of Reliability and Standard Deviations
Table 4.9: Reliability Estimates for Tests with a Mean of 80%
Figures
Figure 4.1: Partitioning the Variance
Special Interest Topics
Special Interest Topic 4.1: Generalizability Theory
Special Interest Topic 4.2: Consistency of Classification with Mastery Tests
Special Interest Topic 4.3: A Quick Way To Estimate Reliability for Classroom Exams
Chapter 5: Validity for Teachers
I. Introduction
A. Threats to Validity
B. Reliability & Validity
II. "Types of Validity" versus "Types of Validity Evidence"
III. Types of Validity Evidence
A. Evidence Based on Test Content
B. Evidence Based on Relations to Other Variables
C. Evidence Based on Internal Structure
D. Evidence Based on Response Processes
E. Evidence Based on Consequences of Testing
F. Integrating Evidence of Validity
IV. Validity: Practical Strategies for Teachers
V. Chapter Summary
Tables
Table 5.1: Tracing Historical Trends in the Concept of Validity
Table 5.2: Sources of Validity Evidence
Figures
Figure 5.1: Illustration of Item Relevance
Figure 5.2: Illustration of Content Coverage
Figure 5.3: Predictive and Concurrent Studies
Figure 5.4: Graph of a Regression Line
Special Interest Topic
Special Interest Topic 5.1: Regression, Prediction, and Your First Algebra Class
Chapter 6: Item Analysis for Teachers
I. Introduction
II. Item Difficulty Index (or Item Difficulty Level)
A. Special Assessment Situations and Item Difficulty
III. Item Discrimination
A. Item Discrimination on Mastery Tests
B. Difficulty and Discrimination on Speed Tests
IV. Distracter Analysis
A. How Distracters Influence Item Difficulty and Discrimination
V. Item Analysis: Practical Strategies for Teachers
VI. Using Item Analysis to Improve Items
VII. Item Analysis and Performance Assessments
VIII. Qualitative Item Analysis
IX. Using Item Analysis to Improve Classroom Instruction
X. Summary
Tables
Table 6.1: Optimal p Values for Items with Varying Numbers of Choices
Table 6.2: Guidelines for Evaluating D Values
Table 6.3: Maximum D Values at Different Difficulty Levels
Table 6.4: Two Examples of Test Scoring and Item Analysis Programs
Special Interest Topics
Special Interest Topic 6.1: Item Difficulty Indexes and Power Tests
Special Interest Topic 6.2: Item Analysis for Constructed Response Items
Special Interest Topic 6.3: Developing a Test Bank
Chapter 7: The Initial Steps in Developing a Classroom Test:
Deciding What to Test and How to Test It
I. Introduction
II. Characteristics of Educational Objectives
III. Taxonomy of Educational Objectives
A. Cognitive Domain
B. Affective Domain
C. Psychomotor Domain
IV. Behavioral versus Nonbehavioral Educational Objectives
V. Writing Educational Objectives
VI. Developing a Table of Specifications
VII. Implementing the Table of Specifications and Developing an Assessment
A. Norm-Referenced versus Criterion-Referenced Assessment
B. Selecting which types of items to use
C. Putting the Assessment Together
VIII. Preparing your Students and Administering the Assessment.
IX. Summary
Tables
Table 7.1: Bloom's Taxonomy of Educational Objectives
Table 7.2: Krathwohl's Taxonomy of Affective Objectives
Table 7.3: Harrow's Taxonomy of Psychomotor Objectives
Table 7.4: Learning Objectives for Chapter 2: The Basic Math of Measurement
Table 7.5: Table of Specifications for Test on Chapter 2: Based on Content Areas
Table 7.6: Table of Specifications for Test on Chapter 2: Content Areas with Percentages
Table 7.7: Strengths and Weaknesses of Selected-Response Items
Table 7.8: Strengths and Weaknesses of Constructed-Response Items
Table 7.9: Practical Suggestions for Assembling an Assessment
Special Interest Topics
A. Special Interest Topic 7.1: Suggestions for Reducing Test Anxiety.
B. Special Interest Topic 7.2: Strategies for Preventing Cheating
Chapter 8: The Development and Use of Selected-Response Items
I. Introduction
II. Multiple-choice Items
A. Guidelines for Developing Multiple-choice Items
B. Strengths and Weaknesses of Multiple-choice Items
III. True-False Items
A. Guidelines for Developing Multiple-choice Items
B. Strengths and Weaknesses of Multiple-choice Items
IV. Matching Items
A. Guidelines for Developing Matching Items
B. Strengths and Weaknesses of Matching Items
V. Summary
Tables
Table 8.1: Checklist for the Development of Multiple-choice Items
Table 8.2: Strengths and Weaknesses of Multiple-choice Items
Table 8.3: Checklist for the Development of True-False Items
Table 8.4: Strengths and Weaknesses of True-False Items
Table 8.5: Checklist for the Development of Matching Items
Table 8.6: Strengths and Weaknesses of Matching Items
Special Interest Topics
Special Interest Topic 8.1: Do Multiple-choice Items Penalize Creative Students?
Special Interest Topic 8.2: Correction for Guessing
Special Interest Topic 8.3: What research says about "Changing your answer?"
Chapter 9: The Development and Use of Constructed-Response Items
I. Introduction
II. Oral Testing: The Oral Essay as a Precursor of Constructed-Response Items
III. Essay Items
A. Purposes of Essay Items
B. Essay Items at Different Levels of Complexity
C. Restricted-Response versus Extended-Response Essays
D. Guidelines for Developing Essay Items
E. Strengths and Weaknesses of Essay Items
F. Guidelines for Scoring Essay Items
III. Short-Answer Items
A. Guidelines for Developing Short-Answer Items
B. Strengths and Weaknesses of Short-Answer Items
IV. A Final Note: Constructed-Response versus Selected-Response Items
V. Summary
Tables
Table 9.1: Purposes of Essay Testing
Table 9.2: Guidelines for the Development of Essay Items
Table 9.3: Strengths and Weaknesses of Essay Items
Table 9.4: Holistic Scoring Rubric
Table 9.5: Analytic Scoring Rubric
Table 9.6: Guidelines for Scoring Essay Items
Table 9.7: Guidelines for the Development of Short-Answer Items
Table 9.8: Strengths and Weaknesses of Short-Answer Items
Special Interest Topics
Special Interest Topic 9.1: Computer Scoring of Essay Items
Chapter 10: Performance Assessments & Portfolios
I. Introduction - What Are Performance Assessments?
II. Guidelines for Developing Effective Performance Assessments
A. Selecting Appropriate Performance Tasks
B. Developing Instructions
C. Developing Procedures for Scoring Responses
D. Implementing Procedures to Minimize Errors in Rating
III. Strengths & Weaknesses of Performance Assessments
IV. Portfolios
V. Guidelines for Developing Portfolio Assessments
VI. Strengths & Weaknesses of Portfolios
VII. Summary
List of Tables
Table 10.1: Guidelines for Selecting Performance Tasks.
Table 10.2: Guidelines for Developing Instructions for Performance Assessments.
Table 10.3: Example of a Rating Scale using Verbal Descriptions
Table 10.4: Example of a Numerical Rating Scale
Table 10.5: Example of a Graphic Rating Scale
Table 10.6: Example of a Descriptive Graphic Rating Scale
Table 10.7: Example of a Checklist Used with Preschool Children
Table 10.8: Guidelines for Developing and Implementing Scoring Procedures
Table 10.9: Strengths & Weaknesses of Performance Assessments
Table 10.10: Guidelines for Developing Portfolio Assessments
Table 10.11: Strengths and Weaknesses of Portfolios Assessments
Special Interest Topics
Special Interest Topic 10.1: Example of a Performance Assessment in Mathematics
Special Interest Topic 10.2: Reliability Issues in Performance Assessments
Special Interest Topic 10.3: Performance Assessments in High-Stakes Testing.
Chapter 11: Assigning Grades on the Basis of Classroom Assessments
This text employs a pragmatic approach to the study of educational tests and measurement so that teachers will understand essential psychometric concepts and be able to apply them in the classroom. The principles that guide the development of this text are: (1) What essential knowledge and skills do classroom teachers need to conduct student assessments in a professional manner? (2) What does the research on educational assessment tell us? This focus has resulted in a uniquely approachable and technically accurate presentation of the material. While providing a slightly more technical presentation of measurement and assessment than more basic texts, this text is both approachable and comprehensive. The text includes a gentle introduction to the basic mathematics of measurement, and expands traditional coverage to include a thorough discussion of performance and portfolio assessments, a complete presentation of assessment accommodations for students with disabilities, and a practical discussion of professional best practices in educational measurement.
|
Ask a Question About this Product More... |
|
