Measurement

Direct Measures
- Consistent with the action verb and the learning context stated in the student learning outcome
- Prioritize embedded measures. Embedded measures are already integrated into the regular curricular process.
Choosing the Right Measure
- Valid: Are you measuring the outcome?
- Reliable: Are the results consistent?
- Actionable: Do the results clearly tell you what students can or cannot do?
- Triangulation: Are there multiple lines of evidence for the same SLO?
- Meaningful and engaging: Are the faculty engaged? Do students care?
- Sustainable: Can the process be managed effectively within the program context?
Indirect Measures
Direct vs. Indirect
Direct
Student behaiors or products that demonstrate their mastery of SLO
- Exam/Quiz
- Paper/Presentation
- Project/Portfolio
- Recital/Exhibition
- Peer evaluation
Indirect
Reported perceptions about student mastery of SLO
- Self-reflection essay
- Self-report survey
- Interview
- Focus group
- Report by alumni, employer, etc.
Direct evidence helps tell us “what,” and indirect evidence helps tells us “why.”
A note about Indirect Measures
Survey
- Flexible question focus and format
- Quick and easy to administer
- Cost effective
- Quality of question?
- Appropriate sampling and response rate?
- Survey fatigue
- Social desirability bias
Focus Group
- “Richer” in-depth information
- Tailored, immediate follow-up for further elaboration
- Participants react to each other’s ideas
- Quality of protocol?
- Facilitator’s skills?
- Appropriate sampling?
- Time and resources for data collection and analysis?
Interview
- “Richer” in-depth information
- Tailored, immediate follow-up for further elaboration
- Privacy and personal attention
- Quality of protocol?
- Facilitator’s skills?
- Appropriate sampling?
- Time and resources for data collection and analysis?
Evaluating Student Evidence: Rubrics
- Evaluating Student Evidence: Rubrics
- A four-point scale is highly recommended. This will drive the respondent to choose between positive or negative.
- Use an existing rubric, such as AAC&U Value Rubrics as base, then tweak to fit the needs of the given program.
- Each cell of a rubric assumes mastery of the previous cell, as the cells describe higher and higher levels of performance.
- Zero is an option if the work sample does not meet level one performance on the given line of the rubric.