abstract:
we are now delivering thousands of online courses to millions of students around the world. we have progressed beyond the early explorations into "how to teach online," to the more complex challenge of "how well?" this paper summarizes the results of a nationwide empirical study performed in the united states, in an effort to systematically identify the factors that determine the quality of e-learning courses. our investigations sought how to design quality into web-enhanced courses, how to ensure delivery of a quality support system for online learners, and how to assess the quality of online courses. this is the first in a series of reports on the outcomes of this investigation, and how to use what we learned about how to ensure and assess quality in web-based learning. this first report focuses on instructional design of quality courses.
introduction
after reviewing hundreds of studies on education, lion gardiner concluded:
"[w]hen we subject the quality of our collective work as educators to the same close examination we demand in our disciplines, we find a substantial body of evidence that clearly demonstrates a crisis of educational quality in our nation's colleges and universities. … we need to begin immediately to assess, evaluate, and improve the quality of our work."
gardiner further explains that, by combining best-practice instructional design methods with internet technologies, it is possible (but not necessarily true) that much greater higher-order learning and higher-value educational outcomes can be achieved.
one contextual challenge course developers face is that quality is not solely an intrinsic feature of a course; but is expressible through the viewpoints, values, and needs of the course consumer, whether he is a student, an instructor, a department chair, an academic vice president or financial officer, an institutional planner, or a corporate human resources director. there is no one checklist by which we can design or evaluate quality in a course. we must make explicit the viewpoint from which the course is being assessed. in the second section of this paper, we will summarize the criteria for quality native to these different consumer viewpoints and thoroughly analyze course quality as viewed from the perspective of the student consumer.
simply put, it is very difficult to design and judge course quality. we know this from the proliferation of thousands of mediocre online courses. from the educational research literature, we are often given guiding principles that are too general to be instructive at the level of classroom practice and course design. making class "happen" in a web-based environment is so new, and different, that neither broad principles nor narrowly prescriptive practices are helpful when we sit at a computer and try to reinvent our teaching in this new environment.
after all, if an instructor who undertakes to design a web-based class is told "follow the principle of giving frequent feedback in the form of formative assessments," he will say "you haven't helped me a bit; i already apply that principle to my classroom-based class. if, on the other hand, you tell him "require students to e-mail you a synopsis of each week's reading," you haven't helped him either because that course process may neither apply to course nor complement his course design.
if the concept of designing and evaluating a course by narrow prescriptive formulae is misguided, what's an instructor to do? in our investigation, we found many categories of good experience-proven practical ideas that seemed to sort themselves into three basic levels of guidance for quality online pedagogy: principles of learning science; practices in instructional artistry; and applications of online systems.
the instructor's best strategy is to arm himself with a few sturdy principles, to be guided by a translation of these principles into practices effective in a web-based environment and to think creatively all the while using specific application methods as a point of reference. at some level, the instructor must do precisely what each student must do and teach himself by drawing inferences and extending from the known to the unknown. the quality of the inferences and extensions, however, can be positively influenced by the solidity of the principles and the quality of the illustrations used as guides.
principles, practices, and specific applications sort themselves out along a scale of specificity:
§ a principle is applicable to nearly any formal learning situation, independent of the delivery medium. that is because these "principles of learning science" derive primarily from characteristics of learners and the functions of human brain-learning, and not from the specific course nor the medium of course delivery.
§ a practice is manifests the "art of instruction" within and suited to a specific mode of delivery (e.g., class based, online, mixed mode). practices in the art of instruction reflect a merging of learning science principles with the artistry of instruction.
§ an application (of learning science principles and instructional artistry) is specific to a particular course, within a specific delivery infrastructure.
principles, practices, and applications can also be indexed by degree of prescriptiveness.
at the foundation of the pyramid, there are fewer principles. we can more readily enumerate major principles of learning science that broadly address the foundation upon which most all learning is based. they are less specifically prescriptive. practices are more specific, and they apply to a particular delivery medium. but, they are still so abstract as to almost demand experienced expertise in order to utilize them in any concrete way. on the low end of the spectrum, methods are immediately applicable. but they are so narrowly specific that they are innumerable. infeasible to simply present a complete enumeration of all possible methods that might be highly beneficial to a specific learning activity.
the abundance of specific methods (learning activities, "assignable units," projects) by which you may assure the success of a given practice and the application of a given principle will bewilder you if you seek a simple formula to answer the question "what do i do to create and assess quality in e-learning?"
approach
this paper reports findings developed as part of a nationwide study of quality in web-based and web-intensive learning. the study is ongoing, and addresses the instructional design of quality pedagogy, quality assurance in course delivery, and assessment of quality in existing courses.
this study does not involve basic scientific research, but we have relied heavily on the scientific and educational research literature. research topics have encompassed neurophysiological and neuropsychological factors underlying human memory and learning processes. secondly, the study has also investigated basic and applied research on instructional design models. third, we have incorporated much practical knowledge of the current state of the art in instructional delivery systems. from these three foundations of investigation, the overall study is based on "the science of learning, the art of instruction, and the application of systems."
this investigation was begun informally in 1994, and continued as such until a formal national investigation was launched in november 1999. the plan of investigation has involved a very labor-intensive process of targeted information collection. (we wanted to avoid the common pitfalls in overly homogenized findings from a monolithic survey of unknown sources.) besides translating basic research into learning science, our approach has been to carefully identify experts in the field who bring both learned expertise and practical application experiences. therefore, the process of simply identifying these experts has been a key element of the study. we have lurked on national listserv's to identify experts, and to pick up new ideas. we have followed up with many of them, in conducting interviews. we have organized round table discussions and panel sessions at the leading national conferences and then followed up with emails with those participants. we have examined online faculty guides from leading institutions, and online course-sites on instructional design, and followed up with their authors where that seemed helpful. not surprisingly, one of the most informative sources of ideas and exemplars (both positive and weak) has been many online course sites themselves. in all, the study has to date involved several hundred pages of research literature, hundreds of listserv postings and follow up emails, over one hundred actual online course sites, hundreds of conference attendees, and dozens of interviews with individual experts. this investigation is ongoing, and further findings are in development for release in subsequent papers.
summary of solution:
learning science yields learning principles, and these principles stand without need for reference to mode of delivery. these principles derive from features of the human brain and characteristics of human learning. they do not derive from the features and characteristics of a given course.
further, in designing any course we must keep in mind that knowledge is not transmitted; it is constructed. learning happens in the brain of the student, and not necessarily "on the web" or in class at 10:32 a.m. or via interactive video.
in our findings, we list and define nine learning principles before translating each one into practices effective in a web-based environment. we then describe an application method for each principle and practice, to illustrate how all of these principles and practices of course quality can be incorporated into the instructional design.
exogenous (uncontrollable) factors that affect "apparent quality"
course or academic-program "quality" is more than just "how good" they are. it is equally a matter of "fit." how well does the academic program fit the students' plans? how well do the course competencies match the job-performance needs of the students who paid the tuition? how well does the pace and learning style fit the particular students, who might even be accessing the web from hotel dialup connections? in the broad sense, course quality depends upon who is judging the quality and exogenous factors that may be wholly outside the domain of pedagogical design.
the "apparent-quality" of a course is a combination of both course-intrinsic design and delivery issues and external factors such as the following:
· "curricular cohesion"
(i.e., the extent of symbiosis among and between course-content structures and also among the various instructional designs)
· academic advising
(i.e., the extent that students are able to successfully knit their courses, co-curricular activities, and life experiences into desired educational achievements)
· class size
· system features (e.g., portal, lms, administrative systems)
· learner characteristics
o motivation
o pre-requisite skills and tools
o intelligence and learning style
o life style
· instructor
o training
o instructional style
o tools
o trade-offs
this paper is focused on the internal course-design elements, rather than these external factors.
findings
#1: knowledge is constructed.
this principle is about the neurophysiological view of human brain learning. this view holds that a) all new knowledge is constructed on a foundation of prior knowledge, and b) this new knowledge, once inter-linked and referenced to the prior knowledge, forms a foundation of new prior knowledge.
the solidifying of this new knowledge in reference to prior knowledge can be compared to the way that discrete ice crystals grow and join to form solid frost on a window. the ability to recall discrete bits of new knowledge is a lower level form of learning. higher order learning involves constructing links between and among bits of new knowledge. furthermore, the neurophysiology of learning suggests that learning is enhanced when the learner must exert deliberate effort to construct the links between and among bits of new knowledge.
bloom's taxonomy of educational objectives lists six progressively "higher" levels of learning (i.e., "critical thinking") that a student may achieve: recall, comprehension, application, analysis, synthesis, evaluation. most educators and employers place a much higher value on the three higher order levels of learning. the "level of learning" that a student achieves on this scale of 1-6 is strongly linked to the extent of his active involvement in the learning process (i.e., not just passive listening).
good instructional design does not "over feed" the student by providing all the "answers." the best teaching leaves some things untaught. it prompts the student to exert creative effort to acquire new knowledge by constructing these cognitive links. poor instructional design, in this instance, demands no constructive effort and serves up knowledge as something to be memorized.
practice: it is important not to employ your site as a "document dump" and to flood the student with information. good practice follows the socratic ideal of prompting students to rediscover knowledge by constructing it for themselves. just a socrates refrained from creating documents of this teachings in order to force each new generation of thinkers to examine issues for themselves, so should an instructor allow the student to exert himself.
the internet is perfectly suited to practices that prompt a student to construct data or knowledge from a vast welter of data. the internet has no defined structure and, more demanding yet to the student, it provides a plethora of competing and often biased partial structures. the internet is an ideal medium for problem-based learning. problem-based learning activities ask a student to 1) refine a statement of a problem, 2) contrive or develop a sense of the structure of knowledge and reasoning relevant to the problem, and 3) to go find the information needed to solve the problem.
application notes: the development of a webliography is an excellent means of prompting students to construct their own knowledge. the instructor assigns a topic and leaves the student to "sweat" the process of defining the topic, gathering information, sorting and evaluating the information in order to impose a structure onto it, and producing a final product. in the case of a webliography, the product can have the structure of an outline that reflects the student's learning process. as a final note, the instructor should resist the temptation to permanently incorporate a student's work in "resources" section of the site. doing this may serve up a greater array of web resources for subsequent students, but it robs them of the stretch and struggle of constructing knowledge for themselves.
a problem-based research paper is an excellent means of leveraging the web environment to get students to construct knowledge in a manner that crosses and spans disciplines. the idea is to set them up to construct linkages between and among competency areas that academic programs tend to isolate. a learning activity based on the subjects of microeconomics and ethics and demanding a position paper on profit and endangered species or agriculture and immigration, for instance, will pitch the student into the information chaos of the web where he will have to construct meaning and the complex linkages we associate with higher order learning.
