Executive Summary of Results

Based on the notion of a technology imperative, more technology in the asynchronous lecture condition should have resulted in improved course and instructor satisfaction. Differences in student achievement and perceptions between the asynchronous and face-to-face lecture conditions indicate that students receiving lectures asynchronously performed poorer on tests and had negative affective reactions to the course and instructor after controlling for relative expected grades. These results are consistent with ineffective communication caused by communication oversimplification among students in the asynchronous lecture condition.

Need for the Project

According to the U.S. Department of Education, AIBI is one the fastest growing modes of distance education being used on campuses across the country (NCES, 1999). And yet, it has received little attention as a pedagogical tool, particularly in the important area of the efficacy of the instructional medium.

Use of Technology

Students in the face-to-face lecture condition came to class as usual. The students in the asynchronous lecture condition could access lecture material from any computer equipped with the Windows © operating system and a CD-ROM drive. The asynchronous group in each section received a CD-ROM computer disk that contained all of the course lectures in the form of narrated PowerPoint© slide shows. Readers are invited to browse lectures online by visiting the textbook website for Krietner & Kinicki (2000) using Microsoft Internet Explorer © at: http://www.mhhe.com/business/management/kreitner5e/student/olc/ch01lecture.mhtml.

Instructional Design Plan

Describe how the use of technology used supported your teaching approach:

• What student learning outcomes were expected?

Measures
Webster & Hackley (1997) proposed the examination of a broad range of effective student evaluations. This study looked at such factors as course usefulness (5 items), instructor satisfaction (7 items), course satisfaction (6 items), courseware satisfaction (4 items), perceived stress (4 items), relative expected grade & absolute expected grade (Greenwald & Gillimore, 1997), (see Appendix A). Perceptual, psychological measures and demographic data were gathered the first day of class, and perceptual data was also gathered before each of three exams. Survey data was gathered in a totally anonymous fashion. Marsh & Roche, 1997, indicated that student evaluations should reflect the multidimensional nature of teaching to be useful. The approach being taken here is to use factor analysis together with reliability analysis to select items that were both prescriptive and reliable (for a treatment of these issues as applied to test theory see: McDonald, 1999). All of the items in a similar content area were submitted to factor analysis thus assuring their connection to the larger overall construct of instructor satisfaction. The process was repeated for each of the concepts that will be addressed in this investigation.

Student Performance, Course and Instructor Perceptions

• AIBI v F-T-F: Students’ grades and perceptions in an AIBI class were compared to those in a traditional F-T-F course. When lectures and instruction were provided asynchronously over the Internet rather than face-to-face, grades, student satisfaction with the instructor, the course, and perceived usefulness of the course were lower. The differences being statistically significant. (In the AIBI condition, first-exposure lectures were streamed asynchronously and instruction was provided by means of Course-mail, chat, and threaded discussion groups.)
• AIBI v Hybrid: Students’ grades and perceptions in AIBI classes were compared to those in hybrid classes. Grades were lower in the AIBI condition, however students’ were equally satisfied with the instructor after controlling for relative expected grades and attitude toward technology. (In this hybrid version, lectures were provided on-demand in an asynchronous fashion. There were no pre-tests. Although students were strongly encouraged to watch lectures, there was not a mechanism built-in to the course that would ensure students doing so. In this hybrid version, classroom time was used to engage students in interactive exercises.)

Students and Oncourse:

• Students agreed or strongly agreed that they were satisfied with their experiences using courseware.
• The amount of courseware use was not related to student learning whether instruction took place face-to-face or over the Internet.
• Results generalized across a broad range of learner characteristics; the notable exception being students’ attitude toward the technology used in the course. With respect to students’ beginning attitude toward courseware, more positive attitudes were associated with greater use of and satisfaction with courseware.
• Use of courseware was greater among students being instructed over the Internet than those being instructed face-to-face.
• Student satisfaction with courseware generalized across both types of instruction, students receiving instruction over the Internet were no more satisfied with their experience using courseware than students instructed face-to-face.
• Courseware may have an important impact on student course satisfaction; greater satisfaction with courseware was associated with greater satisfaction with the course.
• A limitation of the study is worth noting. Lacking a control group in which courseware was not used, it cannot be stated that students taking a course employing courseware would be more satisfied than those without it.
• Students began as novices, yet student confidence with using courseware dramatically improved over the term of a semester.
• The increase in student confidence with using courseware was just as great among students receiving face-to-face instruction as for those receiving instruction over the Internet.

• How did your teaching plan encourage active learning, provide feedback, address
  respective learning styles, and build collaboration?
• What resources were used? (campus technology support services, etc.)

Each semester of the 2000 – 2001 academic year, the author taught two sections of an undergraduate organizational behavior class. Student volunteers from each section were assigned to receive lectures either asynchronously or face-to-face. Apart from the way students received lecture material (face-to-face or asynchronously), students in both sections were treated in as an identical fashion as possible. Lectures in both cases were based on the same set of scripts. Assignments and tests for both groups were identical. All students came to class five times for the same exams and presentations. A survey was administered the first day of class and immediately following the completion of each exam. All students participated in three group projects, one individual Internet project with a paper and presentation, and three in-class essay exams. All exams were graded using criterion-based rubrics in a fashion whereby the instructor did not know the identity of the student. Students were given information on their progress as they completed each event by using the built-in online grade book that is part of the courseware to report grades.

Students in the face-to-face lecture condition came to class as usual. The students in the asynchronous lecture condition could access lecture material from any computer equipped with the Windows © operating system and a CD-ROM drive. The asynchronous group in each section received a CD-ROM computer disk that contained all of the course lectures in the form of narrated PowerPoint© slide shows.

Lectures
Scripts were prepared for each of 25 lectures based on the learning objectives that begin each chapter of the textbook. In the face-to-face lecture condition, the instructor used a computer connected to a projector, to project PowerPoint © presentations in-class while lecturing to students. In-class interactions between the instructor and students were not controlled for and were considered to be part of the nature of the face-to-face lecture condition. Lecture scripts were tightly integrated with materials in the textbook so that at appropriate times references were made to textbook pages containing material, exercises, and cases. The slides presented to students face-to-face or asynchronously were identical. The instructor prepared for class by referring to scripts that were brought to class to be used as reference material during lectures. In the asynchronous lecture condition, the instructor used the scripts and the sound recorder software that accompanies Microsoft’s Windows © operating system to create sound files that were embedded in the PowerPoint© presentations for later asynchronous playback. These sound files were attached to each slide in the lectures presented in the course. The material was compressed for easy streaming over the Internet or dissemination using a single CD-ROM for each student.

• Would your project be easily understood and executed by other faculty?

Implications of Courseware for Other Instructors
Results that should generalize include:

• The instructor found courseware easy to learn and use.
• The instructor found courseware created administrative savings by reducing the amount of paper handed out to students.
• The instructor found the convenience and flexibility of the Internet in the form of courseware reduced the administrative burden associated with teaching. For example, making announcements, distributing instructional materials, and handing back assignments may be better handled over the Internet using courseware rather than using face-to-face in class.
• The instructor found courseware increased student and instructor interactions by providing both with different avenues of communication.
• The instructor found that students did a better job of researching their papers because of the wide range of powerful online tools available 24X7.

Potential to Impact Student Learning

Clearly define how your project improved student learning - include specific examples of how your project:

• Fostered depth of learning
• Supported efforts to improve retention
• Fostered the learning of large numbers of students
• Has broader applications that will affect learning K-12 or community environments?

See attached journal submissions which fully answer these questions.

Assessment Plan

Briefly explain the effectiveness of your assessment plan:

• What measure of assessment was used?
• How did your approach obtain measures of performance?
• Did you feel your measurement was accurate and effective?

See attached journal submissions which fully answer these questions.

Plan for Colleague Development

Describe your role and activities as a mentor:

• What mentoring activities were implemented to assist colleagues?
• How did you help to develop colleague's instructional technology skills?
• How do you think your contributions to the Knowledge Base will help colleagues planning similar projects?
• Is your project approach potentially applicable to other contexts or disciplines?
• What tips would you offer?

I remain committed to sharing what I have learned with my colleagues at IUS and elsewhere. Have done brown bag lunches here on campus. Presented at the Indiana Academy of Sciences conference and Educause conference (both in October 2001). I have made two submissions to Academy of Management Online Journal and Decision Sciences Online Journal. Will be attending and presenting at the Eastern Academy of Management Conference.

For many of my colleagues Internet-based instruction is seen as threat. Carsten & Worsfold (2000) are concerned about the intellectual cost of the online classroom such as the elimination of interpersonal relationships in the online classroom. The current high level of economic investments into AIBI means that it probably is short sighted, if not wrong, to assume an unassailable superiority of traditional face-to-face instruction. Right now, traditional face-to-face instruction is a process that has been refined over may hundreds of years. It enjoys the benefits of that development built around high levels of social interaction among all the participants. As technology evolves, the traditional classroom will have to evolve as well.

Some interesting sites to visit are:

http://www.educause.edu/nlii/

http://www.educause.edu

Final Comments on Project Results

• Do you think you met the goals of your project?
• What, if anything,would you do differently?
• Did you have any unexpected outcomes? (positive or negative)
• Would you recommend this teaching approach to colleagues?
• What other comments or suggestions would you offer to complete this report?

Yes, I have met the goals of my project. I am increasingly teaching using the Internet. I have developed an online course. I have learned to stream my lectures over the Internet. I have learned to use online testing. I am experimenting with ways of using the Internet to improve teaching and learning. Along the way, I have completed several articles and papers. Most important, I realize I have just begun to tap into the value of the Internet as an educational tool.

The Internet does offer unparalleled convenience and flexibility. Internet instruction versus face-to-face instruction should not be looked as a tradeoff between utilitarian concerns and teaching effectiveness. The Internet can be used to actually increase interactions in a face-to-face course by providing the instructor and students with different communication channels not otherwise available. In a hybrid course, an instructor may use the flexibility of the Internet to its best advantage. The Internet’s greatest contribution to education may be as a tool for experimentation and innovation to improve teaching and learning.

Return to Final Reports page

AT&T Fellows Home Page

Last updated: 18 May, 2007

Comments and Questions | (317) 278-4833
Copyright © 1999-2003, The Trustees of Indiana University