Background

The growth of online learning has led to online classes being built recklessly, in the sense that those building the courses have not had training in online instructional design or the processes for evaluation or the build. In this article, Sims, Dobbs, and Hand (2002) identified areas in the process of instructional design process where evaluation should take place, rather than relying on the traditional way of evaluation that is done at the end of a course. The authors stressed the importance of evaluation to monitor the balance of the relationship between the course content, computer-learner environment, and learning outcomes (Sims et al., 2002).

The authors emphasized planning and design being focused on those unfamiliar with an online learning environment (Sims et al., 2002). Additionally, the foci should also include the areas that are often not covered in the traditional evaluation setting: criteria surrounding the environment and resources (Sims et al., 2002). All materials should be evaluated for effective and efficient scaffolding for the learner. In doing so, this allows for the more traditional formative and summative evaluations to be beneficial and thorough at the conclusion of the online learning period.

Key Points

• Proactive evaluation was designed specifically to extend existing evaluations and for scaffolding purposes and for these to be integrated during the course production phase.
• In using proactive evaluation, even those with less experience than others should have a successful course design.
• Most important segments of instructional design process are the planning and design segments.
• Underlying principles with proactive evaluation lend themselves towards online learning environments, further strengthening the development of a strong online course regardless of teacher training in e-learning.
• The principles of proactive evaluation are as follows (Sims et al., 2002):
  • Strategic intent—Understanding the purpose
  • Content—The use of technology-based resources for course materials
  • Learning design—Learner-centered environments created by online materials
  • Interface design—Visual impact and flow of the learning environment
  • Interactivity—Positive, successful communication (multifaceted)
  • Assessment—Should match online environment and not mirror a traditional classroom assessment
  • Student support—Making resources available to students
  • Utility of content—Review if content can be used in different environments (think potential global site blockages, accessibility, etc)
  • Outcomes—Measuring how successful the teacher has been with the above

Discussion

Each online learning environment contains major areas of emphasis for design and context and have specific areas that affect the design of the environment. Each of these areas should be taken into consideration with evaluation when building and maintaining the online course. This is done to make sure that the environment remains efficient and effective in translating materials to the independent online learner so that maximum learning happens in the online environment.

Understanding the content and learning design and the impact they will have on the environment that is being built is critical to success. However, Sims and colleagues (2002) stressed that this is not the sole focus and that the teachers must be aware of all layers of online learning development.

Table 1(b): This table showed what Sims and colleagues (2002) believed to be the major parts to online content (p. 139). For each area, potential issues were provided for those developing the online environment and how the developer (in most cases, the instructor) can overcome these issues to build a course that is as successful as possible. These issues were presented heavily as a transition from traditional classrooms to an online environment. For example, Sims and colleagues (2002) presented the component of accessibility (p. 139). Here, the issues were presented by bringing up accessibility in a traditional sense (disability adaptability) but also ensuring that the content developer is using the correct vernacular and terminology throughout the course that reflects the level of the learner (i.e. speaking about HTML or CSS coding terms to a group of first-time, young learners would be inappropriate and inaccessible in most cases).

Table 2(a): This table showed different areas that can impact that process of online learning design. First and foremost, pedagogy must be selected, where Sims and colleagues (2002) emphasized selection between an instructivist or constuctivist strategies. The decision of the pedagogical alignment will then impact the last two areas on the table: learning outcomes and resources. Learning outcomes are a direct connection between the pedagogical strategy and outcome to help drive the design of the learning environment: Will it be more problem-solving? Will the instructor need to engage motivation-based methods? The last area focuses on resources. These were based on how they will impact the areas of learning and overall learning outcomes.

Table 2(b): This table mirrored the layout of Table 1(b), however, this table focused on the major components of online learning design. Sims et al. (2002) provided seven different components to online learning that they considered to be critical to the design process. For each component, ways to overcome issues with the component as well as designing for that specific component were provided. For example, one component that was included was level of learning. Sims et al. (2002) posed a question, asking how government standards could impact the design surrounding this component. Programs that require standardized testing or state-mandated achievement levels or learning outcomes must all be taken into consideration with online environment development.

Questions

1. What, if any, areas do you think could be overlooked during a proactive evaluation of an online course compared to a traditional end-of-the-course evaluation?
2. Share an experience of a good online course you have taken. What made it a positive experience and made it stand out in your mind? Have you had any negative experiences? Describe what made them bad online environments and how you could improve them with proactive evaluation if you were redesigning the course.
3. Refer to Table 1(a) on page 138. Using a current course or a course you are planning on designing, provide examples of online content for each of the five content types.
4. Which section of the proactive evaluation process would be most challenging for you if you were the instructional designer? Why? How would you face and overcome the challenge?

Additional Resources

• Biskupic, I.O., Lackovic, S., & Jurina, K. (2015). Successful and proactive e-learning environment fostered by teachers’ motivation in technology use. Procedia—Social and Behavioral Sciences, 174, 3656–3662. https://doi.org/10.1016/j.sbspro.2015.01.1086.
• Garrison, D.R. & Anderson, T. (2003). E-learning in the 21st century. New York, NY: Routledge
• Magno, C. (2013). Proactive evaluation [SlideShare slides]. Retrieved from https://www.slideshare.net/crlmgn/proactive-evaluation.