What are the opportunities today’s technologies provide for assessment in the online environment?
The advances in technology over the last five years alone provide instructors with an entirely different landscape of assessment opportunities within the online environment. For instance, social and/or (synchronous/asynchronous) communication technologies through mobile devices, texting, video and audio chat, blogs, and podcasts, in addition to collaborative technologies such as wikis have all afforded instructors new and varied options for e-assessments. Specifically, “technology offers new measures for assessing learning that will yield rich sources of data and expand the ways in which educators understand both learning mastery, and teaching effectiveness. The use of information technologies and e-learning to augment the assessment process may include: pre and post testing, diagnostic analysis, student tracking, rubric use, the support and delivery of authentic assessment through project based learning, artifact collection, and data aggregation and analysis” (Buzetto-More & Alade, 2006, p. 251). In addition, advancing technologies in the area of virtual realities have opened the door to performance assessments in an online environment sans previously experienced constraints of poor validity, reliability and generalizability, particularly due to the inherent immersive nature of the construct (Clarke-Midura & Dede, 2010). Further, “innovative e-assessment formats have been developed, including drag-and drop, hotspot, matrix or extended matching questions, voice responses, as well as the use of certainty-based marking to capture the students confidence in their level of understanding (Crisp, 2010, p. 1-2). Ryan Watkins expresses the new online possibilities succinctly as “technology is giving us more options: more options in the classroom, more options online, more options to assess performance in the actual performance environment” (Laureate Education, Inc. [Laureate], n.d., p. 1). With all of the opportunities available, why does it seem like education and/or instructors are dragging their proverbial feet when it comes to adopting these new innovative methodologies?
Given the affordances of modern technologies and the multiple purposes of assessment, why haven’t more educators, trainers, or instructors (or their institutions) adopted new media and adapted old frameworks?
Change is difficult under the best of circumstances. Within education, change is especially slow-paced due to the innumerable number of stakeholder constraints. All things being equal, The Diffusion of Innovations theory posited by Rogers aid significantly in identification of “factors affecting the adoption of an e-assessment system” (McCann, 2010, p. 799) providing plausible explanations as to the relatively slow adoption of e-assessments in education. Specifically, according to Rogers (2003):
“Five attributes explain 49–87% of the variance in the rate of adoption of innovations: relative advantage, compatibility, trialability, observability and complexity. Relative advantage is the degree to which an innovation is viewed as better than the idea it supersedes. The greater the perceived advantage, the faster is the rate of adoption. Incentives and mandates to adopt increase the relative advantage of an innovation. Compatibility is the degree to which the innovation is seen ‘as consistent with the existing values, past experiences and needs of potential adopters’. The rate of adoption increases as compatibility increases. Complexity is the degree to which an innovation is viewed as difficult to understand and use. Adoption increases as complexity decreases. Trialability is the degree to which an innovation can be experimented with on a limited basis. An innovation that can be tried creates less uncertainty to the potential adopter and so speeds up the adoption rate. Observability is the degree to which results of an innovation can be seen by potential adopters. Adoption increases if adopters can see the results of the innovation” (as cited in McCann, 2010, p. 799).
In applying this theory to the arena of education it becomes all too clear why the public has not seen an overwhelming adoption of new technology in this area. For instance, to date, many school districts struggle with basic technological advances, much less expanding usage to include assessment. I was considered a maverick utilizing my own projector for PowerPoint, and investing my time and energy researching new and innovative lesson plans utilizing technology. In fact, the last school district I worked in did not allow students to use cell phones on campus, much less in the classroom. Further, most classes were lucky to have 1-3 computers at most. The cultural environment was wholly incompatible to utilizing technology. In addition, the instructors often are so busy with meetings, instruction, and preparation for instruction they do not have the time to “figure out” new technologies that seem very complicated from a novice’s perspective. These same instructors likely have little time to participate in any “ trial and error” scenarios because every single second counts. One mistake could yield many sleepless nights. With that much pressure it is no wonder why they decide to stick with the safe route of traditional testing. In this environment, it is also difficult to observe any other instructor being successful because someone has to be the first.
In the online learning environment, the trends are more progressive in utilizing various e-assessments. According to Boettcher (Laureate Education, Inc. [Laureate], n.d.) there is a positive trend toward forward learning assessment (i.e., formative assessment). In addition, she described various types of performance assessment used in online learning environments such as automated quizzing, discussion postings, peer reviews, involvement in projects and collaborative work project (Laureate, n.d.). However, these few assessments are still not taking complete advantage of the entire online learning environment and technologies have to offer in terms of e-assessments.
What are the challenges of developing and implementing online assessments versus traditional assessments?
Traditionally, paper-pencil assessments have garnered more support because of problems associated with validity, reliability and accountability when utilizing performance based assessments. “When it comes to testing in accountability setting, multiple-choice tests have been the favored choice because they have satisfied psychometric criteria, are more cost effective, and are easier to scale. Movements for more authentic or performance-based assessments that are better aligned with how students learn rarely get enough traction against their multiple-choice counterparts. For example, performance-based measures using physical objects in real-world settings were shown to be not as psychometrically reliable or practical as item-based tests, are expensive, and are burdened with task-dependency” (Clarke-Midura & Dede, 2010, p. 311). In addition, “assessment systems must take into account issues of interface, accessibility, security, usability, the information to be collected, hardware and software technology, and information storage and processing” (Buzetto-More & Alade, 2006, p. 259).
E-assessment has increased in usage; however, as with most innovations, e-assessment’s utilization has barely scratched the surface of the possible technological capabilities due in large part to the issues (political, monetary and stakeholder) discussed herein. When e-assessment is used regularly, it is most often for simplistic operations such as multiple choice testing assessing the “acquisition of declarative knowledge, or knowing ‘what’” (Crisp, 2010, p. 1) rather than to assess higher-order thinking.
At this point, it is simply a matter of time until education catches up with technology. As demand increases, education will have to comply.
AL-Smadi, M., Guetl, C., & Kappe, F. (2010, March). Peer assessment system for modern learning settings: Towards a flexible e-assessment system. iJET, 5(Special Issue 2: “MIPRO 2009”), 5-11. doi:10.3991/ijet.v5s2.1229
Buzetto-More, N., & Alade, A. J. (2006). Best practices in e-assessment. Journal of Information Technology Education, 5, 251-269. Retrieved from EBSCOhost
Clarke-Midura, J., & Dede, C. (2010). Assessment, technology, and change. Journal of Research on Technology in Education, 42(3), 309-328. Retrieved from EBSCOhost
Colburn, A. (2009). An assessment primer. Science Teacher, 76(4), 10. Retrieved from EBSCOhost
Crisp, G. (2010, October). Interactive e-assessment – practical approaches to constructing more sophisticated online tasks. Journal of Learning Design, 3(3), 1-10. Retrieved from EBSCOhost
Laureate Education, Inc. (Producer). (n.d.). Assessment: Historical perspectives and future trends [Video]. Available from https://class.waldenu.edu/webapps/portal/frameset.jsp?tab_tab_group_id=_2_1&url=%2Fwebapps%2Fblackboard%2Fexecute%2Flauncher%3Ftype%3DCourse%26id%3D_551855_1%26url%3D.
Laureate Education, Inc. (Producer). (n.d.). Technology and assessment [Video]. Available from http://mym.cdn.laureate-media.com/2dett4d/Walden/EIDT/6511/01/mm/tech_assess/index.html.
McCann, A. (2010, December). Factors affecting the adoption of an e-assessment system. Assessment & Evaluation in Higher Education, 35(7), 799-818. doi:10.1080/02602930902981139
Oosterhof, A., Conrad, R. M., & Ely, D. P. (2008). Assessing learners online. Upper Saddle River, NJ: Pearson.