Measurement

Close-up of a tape measure tape.

Direct Measures

  • Consistent with the action verb and the learning context stated in the student learning outcome
  • Prioritize embedded measures. Embedded measures are already integrated into the regular curricular process.

Choosing the Right Measure

  • Valid: Are you measuring the outcome?
  • Reliable: Are the results consistent?
  • Actionable: Do the results clearly tell you what students can or cannot do?
  • Triangulation: Are there multiple lines of evidence for the same SLO?
  • Meaningful and engaging: Are the faculty engaged? Do students care?
  • Sustainable: Can the process be managed effectively within the program context?

Indirect Measures

Direct vs. Indirect

Direct

Student behaiors or products that demonstrate their mastery of SLO

  • Exam/Quiz
  • Paper/Presentation
  • Project/Portfolio
  • Recital/Exhibition
  • Peer evaluation

Indirect

Reported perceptions about student mastery of SLO

  • Self-reflection essay
  • Self-report survey
  • Interview
  • Focus group
  • Report by alumni, employer, etc.

Direct evidence helps tell us “what,” and indirect evidence helps tells us “why.”

A note about Indirect Measures

Survey

  • Flexible question focus and format
  • Quick and easy to administer
  • Cost effective
  • Quality of question?
  • Appropriate sampling and response rate?
  • Survey fatigue
  • Social desirability bias

Focus Group

  • “Richer” in-depth information
  • Tailored, immediate follow-up for further elaboration
  • Participants react to each other’s ideas
  • Quality of protocol?
  • Facilitator’s skills?
  • Appropriate sampling?
  • Time and resources for data collection and analysis?

Interview

  • “Richer” in-depth information
  • Tailored, immediate follow-up for further elaboration
  • Privacy and personal attention
  • Quality of protocol?
  • Facilitator’s skills?
  • Appropriate sampling?
  • Time and resources for data collection and analysis?

Evaluating Student Evidence: Rubrics

  • Evaluating Student Evidence: Rubrics
    • A four-point scale is highly recommended. This will drive the respondent to choose between positive or negative.
    • Use an existing rubric, such as AAC&U Value Rubrics as base, then tweak to fit the needs of the given program.
    • Each cell of a rubric assumes mastery of the previous cell, as the cells describe higher and higher levels of performance.
    • Zero is an option if the work sample does not meet level one performance on the given line of the rubric.