- - - - - - - - - - - - - - - - - - - - - -
 Divisions
  Overview
  Academic Policy & Innovation
  Accountability, Assessment, and Data Systems
  Career and College Readiness
  Communications, Partnerships, and Grants
  Curriculum, Assessment & Accountability
     Overview
     Assessment & Accountability
        Overview
        2001 - 2002 Student Publications
        2002 - 2003 Staff Publications
        2002-2003 HSA Technical Report
        2002-2003 Student Publications
        2003 - 2004 Staff Publications
        2003 - 2004 Student Publications
        2003-2004 Alternate Maryland School Assessment
        2004 - 2005 Staff Publications
        2001 - 2002 Staff Publications
        1999 - 2000 Student Publications
        2000 - 2001 Staff Publications
        2000 - 2001 Student Publications
        Staff and Student Publications
        1998 - 1999 Student Publications
        1999 - 2000 Staff Publications
     Title II, Part B: Mathematics and Science Partnerships
     Maryland Teacher Professional Development
     Physical Education
     MD Professional Development Planning Guide
     Comprehensive Health Education
     Reginald F. Lewis Museum of Maryland
     World Languages
     World Languages Elementary STEM Curriculum Modules
  Educator Effectiveness
  Early Childhood Development
  Finance
  Information Technology
  Library Services
  Office of the State Superintendent
  Rehabilitation Services
  Special Education and Early Intervention
  Student, Family, and School Support
Divisions
Divisions > Curriculum, Assessment & Accountability > Assessment & Accountability
2003-2004 Alternate Maryland School Assessment

Alternate Maryland School Assessment
Mathematics and Reading
Grades 3, 5, 8 and 10

2003-2004 Technical Report



TABLE OF CONTENTS

PURPOSE

 

1.0 HISTORICAL OVERVIEW
     
  1.1 OVERVIEW OF THE ALTERNATE ASSESSMENT
  Background
  Purpose of the Assessment
  Participation in the ALT-MSA
  Organizations and Groups Involved
     
Maryland State Department of Education (MSDE)
Pearson Educational Measurement
Inclusive Large-Scale Standards and Assessment Group (ILSSA)
Advisory Committee
       
1.2 TEST DESIGN AND BLUEPRINT
    Review Standards
    Test Design
    Test Blueprint
       
2.0 PORTFOLIO ASSESSMENT CONSTRUCTION & ADMINISTRATION
       
  2.1 TIMELINE
       
  2.2 CONTRIBUTORS
    Local Accountability Coordinator
    Principal
    School Testing Coordinator
    Test Examiners
    Instructional Assistants (Under the supervision of the test examiners)
    Student
    Parents/Guardians
    ALT-MSA Facilitator
       
  2.3 ALT-MSA DEVELOPMENT AND ADMINISTRATION
    ALT-MSA Portfolio Planning and Development
    Acceptable Evidence of Mastery
    Eligible Test Examiners
       
  2.4 PORTFOLIO ORGANIZATION
     
3.0 SCORING AND REPORTING
       
  3.1 SCORING
    Recruitment of Scorers and Scoring Supervisors
    Rangefinding
    Training
      The Training Process
      Introduction
      Anchor Portfolio Set and Scoring Guide
      Practice Portfolio Sets
      Qualifying Portfolio Sets
      Training of Scoring Supervisors
    Distribution of Portfolios to Scoring Teams
    Scoring Procedure
    Quality Control
      Backreading
      Validity Sets
      Reports Generated to Monitor Scorers and Scoring Accuracy and Control Scorer Drift
      Security at the Scoring Site
     
  3.2 STANDARD SETTING
     
  3.3 REPORTS
    Description and Interpretation of Scores
      Mastery Objective Score
      Mastery Percentage Score
      Evidence and Indicators of Positive Practice
    Reports
      Accountability Reports
      Non-Accountability Reports
       
4.0 RELIABILITY AND VALIDITY
     
  4.1 RELIABILITY
    Reader Agreement
     
  4.2 VALIDITY
    Intrinsic Rational Validity Evidence
    Content- and Curricular-Related Validity Evidence
    Face Validity
    Consequential Validity Evidence
    Criterion-Related Validity Evidence
       
REFERENCES
 
APPENDICES
 
  Appendix A: Tables
     
  Table 1: Participation by Grade, Gender, Ethnicity and SES
  Table 2:  Scorer Qualification Results
  Table 3:  Summary of Performance on Validity Sets
  Table 4: Percentage of Mastery Objectives Scored "Mastered" by Reading Content Standard
  Table 5: Percentage of Mastery Objectives Scored "Mastered" by Mathematics Content Standard
  Table 6: Reading Mastery Percentages for All Students Tested
  Table 7: Mathematics Mastery Percentages for All Students Tested
  Table 8:  Average Reading and Mathematics Mastery Percentage Scores
  Table 9: Reading Proficiency level Frequencies
  Table 10: Reading Proficiency level Frequencies by Free/Reduced Lunch Designation
  Table 11: Reading Proficiency Level Frequencies by Ethnicity
  Table 12: Mathematics Proficiency Level Frequencies
  Table 13: Mathematics Proficiency Level Frequencies by Free/Reduced Lunch Designation
  Table 14: Mathematics Proficiency Level Frequencies by Ethnicity
  Table 15: Percentage of Portfolios Showing Evidence of Positive Practice
  Table 16: Percent Perfect Reader Agreement by Reading Content Standard
  Table 17: Percent Perfect Reader Agreement by Mathematics Content Standard
  Table 18: Percent Perfect Reader Agreement Over All Students
  Table 19: Correlation between Positive Practice Scores and Reading/Mathematics Mastery Scores
     
  Appendix B: ALT-MSA Timeline for 2003-2004
  Appendix C: ALT-MSA Mastery Objective Review Form
  Appendix D: Samples of Required Forms
  Appendix E: Performance Scoring Center Staff Roles and Responsibilities
  Appendix F: Rangefinding Portfolio Selection Criteria
  Appendix G: 2003-2004 Scoring Process Document
  Appendix H: Steps to Monitor Scoring Accuracy and Remedy Drift
  Appendix I: Sample Score Reports
 
A complete copy of the Alternate Maryland School Assessment 2003-2004 Technical Manual is also available.

Contact Information
Leslie Wilson, Assistant State Superintendent
Division of Accountability and Assessment
Maryland State Department of Education
200 West Baltimore Street
Baltimore, MD 21201
Maryland State Department of Education
200 West Baltimore Street
Baltimore, MD 21201
MSDE Privacy Statement Disclaimer  | Copyright © 2003 MSDE