On the Roles of External Knowledge Representations in Assessment Design. CSE Report 722

People use external knowledge representations (EKRs) to identify, depict, transform, store, share, and archive information. Learning how to work with EKRs is central to becoming proficient in virtually every discipline. As such, EKRs play central roles in curriculum, instruction, and assessment. Five key roles of EKRs in educational assessment are described: (1) An assessment is itself an EKR, which makes explicit the knowledge that is valued, ways it is used, and standards of good work; (2) Analysis of any domain in which learning is to be assessed must include the identification and analysis of the EKRs in that domain; (3) Assessment tasks can be structured around the knowledge, relationships, and uses of domain EKRs; (4) "Design EKRs" can be created to organize knowledge about a domain in forms that support the design of assessment; and (5) EKRs from the discipline of assessment design can guide and structure the domain analyses noted in (2), task construction (3), and the creation and use of design EKRs noted in (4). The third and fourth roles are discussed and illustrated in greater detail, through the perspective of an "evidence-centered" assessment design framework that reflects the fifth role. Connections with automated task construction and scoring are highlighted. Ideas are illustrated with two examples: "generate examples" tasks and simulation-based tasks for assessing computer network design and troubleshooting skills. (Contains 16 figures and 2 tables.)