Questions for consideration
- Instructor’s Perspective: What are some of the benefits of developing, implementing, and scoring performance assessments?
- Instructor’s Perspective: What are some of the challenges an instructor or instructional designer might face when creating a performance assessment for an online environment?
- Learner’s Perspective: What are some of the benefits of performance assessments?
- Learner’s perspective: What are some of the challenges learners might face when participating in performance assessments in online environments?
- What are some strategies you have learned for overcoming some of the challenges of developing, administering, completing and scoring authentic online assessments?
I have to admit, this week’s discussion concept (online performance assessments) initially threw me for a loop because of the upcoming application assignment. Allow me to explain.
First, performance assessments as discussed in our text meet the following criteria:
- “Specific behaviors or outcome of behaviors are to be observed
- The behaviors represent performance objectives or goals of the course
- It is possible to judge the appropriateness of learners’ actions or at least to identify whether one possible response is more appropriate than some alternative response
- The behavior or outcome cannot be directly measured using a paper-and-pencil test” (Oosterhof, Conrad, & Ely, 2008, p. 144).
Everything so far makes complete sense.
Our text provides ample examples of advantages that are beneficial to both instructor and learner.
- “Performance assessments can measure skills that written tests cannot . . .this is the most significant advantage of performance assessments.
- Performance assessments influence what is taught and learned by expanding what is assessed.
- Performance assessments facilitate the assessment of a learner’s process, rather than the product resulting from the process.
- Performance assessments tend to provide better insights into a learner’s knowledge.
- Performance assessments can better tolerate settings with low test security” (Oosterhof et al., 2008, p. 145-146).
Again, I agree completely with these statements. I have long been a proponent of performance assessments as better able to determine true learning, certainly better than black and white multiple-choice assessments. Performance assessments typically involved ongoing projects that result in an ultimate project. Along the way, the learner demonstrates continuous growth and development. In many ways, online performance assessmments are a natural fit because the learner’s are already familiar with explaining reasoning and thought processes, participating in ongoing projects, and participating in authentic activities. There are many disciplines that are taught online that easily lend themselves to performance assessment type activities. For instance, our instructional design course is a great example. Throughout the program each learner has had invaluable opportunities to practice their skills through course projects, application assignments, as well as the discussion forums. In my mind, this program has evidenced the best of the written assessment and performance assessment worlds.
Disadvantages / Limitations
This is where my reasoning became a little befuddled.
Once again our text provided ample characterizations of the many issues that are problematic with online performance assessments.
The first caveat or concern, “when activity is used as a performance assessment, considerations such as validity and generalizability of performance become critical considerations” (Oosterhof et al., 2008, p. 142). Alas, but this was just one of the significant considerations. “Often the only option when not face to face is to indirectly observe the process, such as through student journals. Online learners do not benefit as much from formative assessments since, compared with summative assessments, a formative assessment relies very heavily on observing the process students use when completing the task” (Oosterhof et al., 2008, p. 145)
Additional issues include:
- Performance assessments are less efficient . . . They generally require more time to produce, administer and particularly score. Students require more time to complete performance assessments than written tests.
- Scoring of performance assessments is subjective… susceptible to rater bias.
- Performance assessments have problems with generalizability” (Oosterhof et al., 2008, p. 147).
Most importantly, “many complex skills are presently difficult and sometimes impossible to assess online” (Oosterhof et al., 2008, p. 143). “Present technology limits the assessment of motor skills online, this being a major factor that determines which content areas can and cannot be taught through distance learning” (Oosterhof et al., 2008, p. 145).
It was at this point I became particularly vexed because our assignment for the week clearly requires performance assessments of what appear to me to be motor skill in nature (i.e, execute a three point turn, parallel park, illustrate proper use of turn signals, etc.). I was concerned on two fronts. First, I honestly do believe the performance assessments for these skills would best be served face-to-face. Second, if there was some way of creating a simulation style environment within which an avatar could demonstrate the abilities there would be issues in (1) my ability to create such an intricate assessment, and more importantly, (2) whether or not the assessment would remain valid and generalizable outside of the simulation context. I was immeasurably relieved when Dr. Van Erp assured me that the intent was to promote creativity. Plus, I am not surprised that Walden would engineer an assessment that would require out of the box thinking and compromise. There may be many such situations in our instructional and/or instructional design futures.
Therefore, when faced with situations where the process is to be evaluated, “indirect methods for observing the process usually must be employed. A common indirect method is for learners to subject a journal that describes the process in writing” (Oosterhof et al., 2008, p. 144). Another method, which we learned earlier in the course, includes video taping of the task being performed. The only concern here is that the instructor must be comfortable with the possibility that the learner could videotape themselves several times performing the task and submit the best representation. This, to my mind, is acceptable. Again, I am a believer in ultimate mastery of the concept, not immediate perfection. (Unfortunately, I tend to hold myself to a higher standard, which is why these issues have bothered me so).
I must admit I am looking forward to my cohorts’ suggestions on how to alleviate some of these issues, especially in light of our upcoming assignment.
Oosterhof, A., Conrad, R. M., & Ely, D. P. (2008). Assessing learners online. Upper Saddle River, NJ: Pearson.