Evaluating Distance Learning Experiences



Example 2: A foreign language course that is intended to promote learners’ spoken and written fluency of the language.


I admit I selected this example because of the three it had the best possibility of success in a strictly distance learning environment. The other two examples could be successful in a blended/hybrid distance learning environment. They would both require some face-to-face time, if for different reasons. The midwifery example has a learners’ outcome of being able to home-deliver babies. No matter how many interactive videos, discussions and/or even simulations are included in the program, the learners definitely need to have some practical hands-on experience with delivery procedures, women in labor, and newborns. Similarly, the interpersonal communication example has a learners’ outcome of being able to deliver speeches in front of a LIVE audience. Again, no matter how many interactive videos, video conferencing, discussions and/or simulations are included in the program, the learners need to have some real life LIVE experience delivering a speech in front of a LIVE audience because it is a completely different environment.


On the other hand, learning a foreign language (Swahili), improving spoken and written fluency in Swahili is a definite possibility in a successful distance learning environment, as long as it is appropriately designed. For instance, the discussions could be asynchronous with the learners required to respond in Swahili to a variety of philosophical prompts designed to encourage interaction and further conversation. Another option would be providing relevant texts in Swahili or vice versa for translation. Application and/or formative assessments determining learners’ progress could include weekly recorded podcasts uploaded to the instructor and/or the class, who could view them and then offer written and/or oral feedback in Swahili. This course would be a lot of fun to do in a distance learning environment because the discussion topics and podcast topics could incorporate many of the possible scenarios a fluent speaker could find themselves, political topics, current topics, or philosophical topics. For the final summative assessment, the learners could turn in a culmination of their podcasts as one long dialogue on a certain topic of their choice, portfolio-style. I would be very challenged by a course like this, but I would also enjoy it.


Determining the effectiveness of the course brings up another set of issues and many questions. For example, who are the target learners? Are the learners feeding from Swahili I to Swahili II or some other part of an academic series? Are the learners taking the course for job related or to travel abroad? Is the course voluntary or required for some educational degree/certified program? Despite these being relevant questions to answer, one of the best methods to determine if the course is effective, at least from the learners’ standpoint, would be by doing a comparison (pre-test/post-test) of the learners’ abilities. The pretest would also act as a diagnostic tool for the instructor to determine at what level to gear the class, upwards, downwards, or right in the middle. This particular aspect of the course could vary considerably depending on the target learning audience and/or purpose of the course.


The Fortune and Keith’s AEIOU evaluation process (Simonson, Smaldino, Albright, & Zvacek, 2009) would actually be very helpful in evaluating the effectiveness of courses because it covers so many different aspects that may not have been considered previously. It is especially useful for a course such as the aforementioned.


Component 1: Accountability (A): “Did the project planners do what they said they were going to do?” (Simonson et al., 2009, p. 352). This aspect of the evaluation is one of the easiest to determine, as it is typically administratively managed. Questions Such as how any students enrolled in the Swahili course? How many sets of materials were ordered/distributed? How many class discussions were there? How many students participated? How many application assignments where there? How many students turned them in? How many students completed the course?


Component 2: Effectiveness (E): “How well done was the project?” (Simonson et al., 2009, p. 352). This aspect of the evaluation attempts to determine effectiveness using more traditional criteria such as standardized measures, grades, surveys, and attitude questionnaires. This is probably where my pre-test/post-test data would be useful. Survey questions need to include attitude of the learners, as well as the feelings of the learners regarding their progress and/or satisfaction with the course outcomes. This could also include how the learners’ felt about the materials, teacher and distance learning environment.


Component 3: Impact (I): “Did the project, course, or program make a difference?” (Simonson et al., 2009, p. 353). According to the text this aspect speaks to the impact on the educational institution as a result of the course, not so much as to the learner. Accordingly, questions such as did the students sign up for another Swahili course? Or did the students sign up for other distance courses in other areas of study? Personally, I would also expect the impact on the learner would be important in this category. Perhaps, it is meant for the previous section. But, if data are going to be collected longitudinally to see improvements and/or increases in different aspects of the learners’ behaviors, would it not be important to also attempt to verify if the learners are, in fact, utilizing their newly acquired skill? For instance, did my Swahili students actually go on a trip and how did it go? Did they find themselves out of their depth or were they adequately prepared? If this was a beginner’s course and the students went on to a more advanced course, how did they do at the next level? Did they feel appropriately prepared? What would they have changed?


Component 4: Organizational Content (O): “What structures, policies, or events in the organization or environment helped or hindered the project in accomplishing its goals?” (Simonson et al., 2009, p. 354). This aspect of the evaluation is perhaps at least as important as the others, especially in a distance learning environment. As we have already learned, it is imperative that the structure of the course be as complete as possible PRIOR to the beginning of the course. The text suggests questions such as “what factors made it difficult to implement the project or to successfully complete the course? What contributed most to the success or failure of the project or the students in the course? What should be done differently to improve things and make the course more effective?” (Simonson et al., 2009, p. 354). Personally, I cannot imagine attempting to evaluate the effectiveness of any learning environment without posing these questions. For example, I may discover, upon asking these questions, that my students needed more guidance is how to create a podcast. If a significant number of the students had no idea what a podcast was, this could create a significant problem towards completion of the course. Also, it may have been that the course was perceived as too long, too short, too difficult, or too easy. The materials could have been perceived as too dry, boring, or difficult to understand. All very valuable information to build on in the future.


Component 5: Unanticipated Consequences (U): “What changes or consequences of importance happened as a result of the project that was not expected?” (Simonson et al., 2009, p. 354). Pardon my expression, but Shedaisy! If I had a dime for every time something unforeseen occurred, I would be one rich teacher. Sometimes the unanticipated consequence was absolutely amazing, as the light goes on in a student’s eyes for the first time. Sometimes, the unexpected is not so great, such as when the students lit a firecracker in the room, rather than participate in the day’s lesson. Sometimes it is great such as when a group finds their niche and put together an amazing project for the class through teamwork. In one of my classes a group of students put together an entire video for a project using animations and voice overs. It was terrific. On the other hand, I have also had students misuse an encouraging motivator against another student, creating harsh feelings or situations. This could also easily occur in a distance learning environment. For instance, perhaps someone in my Swahili discussions accidentally insults another student by using the wrong vocabulary. It would be imperative that I be able to diffuse the situation quickly, reassuring both the person who made the mistake and the person who was on the other end of it.




For the most part, all of the aspects of the AEIOU evaluation system have been important in education for a long time. I can remember reflecting on many of these questions and others as I taught my classes. It makes complete sense to apply them to distance learning as well. Further, I wish I had been taught this system structure before teaching. It would have saved me a great deal of “chasing my tail.” I work much better with structure and organization. This would have been an invaluable tool in my face-to-face teaching. It WILL definitely be an invaluable tool in my instructional design career and/or future.





Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. (2009). Teaching and learning at a distance: Foundations of distance education (4th Ed.). Boston, MA: Pearson Education.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s