Executive Summary of Results

Need for the Project

The purpose of this project is to improve learning, retention, and persistence in Computer Science courses by recasting lecture content into streaming media, removing the time and place linkage between lecture and classroom, and refocusing class sessions to incorporate Just-in-Time Teaching and active learning modules.

Use of Technology

• How did the project take advantage of IU’s learning environment?
  
  The Just-in-Time Teaching component of the project leveraged roster and gradebook capabilities of the Original Oncourse.  The two impacted courses, CSCI 230 Computing I and CSCI 240 Computing II, were administered through Original Oncourse prior to the initiation of this project.  Both classes continued to use Original Oncourse during the Spring 2006 (control) and Fall 2006 (experimental) semesters.  The Warm Up Center software integrated with the Oncourse roster export and gradebook import features to create user accounts at the beginning of the semester for warm up exercises, and to load participation percentages at the end of the semester.
  
• Did the project lead to collaboration that will help IU improve or expand classroom   and online learning environment?
  
  This project produced materials in collaboration with Andy Harris of the Computer Science Streaming Media Lab (http://www.cs.iupui.edu/facilities/index.html).  This work expanded the use of Just-in-Time Teaching pioneered at IUPUI (http://www.jitt.org).  The creation of streaming media enabled the offering of CSCI 230 at the Columbus campus of IUPUI.
  
  Experience with producing streaming media lectures has produced process and procedures that can be easily used by a colleague port a course to the active learning classroom and streaming lectures.
  
  Several active learning modules involve collaboration between students.  One example is an activity involving writing algorithms.  A collection of fifteen to twenty problems is distributed to the class.  Groups form of size three or four to work on an algorithm together.  After about 15 minutes, work stops and each group presents the problem, inputs and outputs, assumptions, and algorithm.  The class then critiques the work in an open and friendly tone set by the instructor. 

• How did the project make use of instructional technology and design?

Several instructional technologies were leveraged for this project.  CamStudio video recording technology enabled instructors to produce streaming media remotely.  The Computer Science departmental streaming media server supported remote access to course lectures (http://wally.cs.iupui.edu). 

The courses use instructional design based on Felder-Silverman Model of Learning Styles, Cognitivism and Constructionism, and Moore’s Model of Interactions.  Traditional lecture focuses on the cognitivism that Moore describes as learner-content interaction.  The streaming media delivery does not change the cognitive interaction, but frees time in the classroom for Moore’s other modes of interaction: learner-instructor and learner-learner.

Instructional Design Plan

• What student learning outcomes were expected?
  
  Improved exam scores, lower DWF rates, and improved evaluation global scores are specific outcomes expected. 
  
  Prospective outcomes include improved retention, persistence, and progression for undergraduate Computer Science students.
  
• How did your teaching plan encourage active learning, provide feedback, address respective learning styles, and build collaboration? 
    

• Active Learners
• More movement and group activities in the classroom through the introduction of active learning modules.
• Can stop streaming when attention span is exhausted.
• Can stop streaming and implement code as they go along.
• Can view media in small groups in which members take turns explaining the material to each other?
• Reflective Learners
• Can stop streaming periodically to review material and build connections.
• Can stop stream and write short summaries to improve retention.
• Sensory Learners
• Improve learning by relating material to the real world.  This is accomplished by the “Computer Science in the Real World” component of classroom time.  More interactive classroom discussions are enabled by the removal of lecture time from the classroom.
• Stop streaming and work out specific examples on their laptop in order to make abstract concepts more concrete.
• Intuitive Learners
• Module design measure that students can skip through material from topic to topic concentrating on intuitive abstract concepts.  While watching the video, intuitive connections can be made, and then the video can be viewed again seeing how the examples sit into the conceptual framework.
• Visual Learners
• Visual learners particularly like the video format that includes diagrams, pictures, and other visual demonstrations.
• Students can stop and watch the video as often as necessary to grasp concepts.
• Visual learns also benefit by stopping the video, reading the textbook, and resuming the video.
• Verbal Learners
• Verbal learners benefit by hearing the lectures.  The video can be stopped and have words repeated for any concepts that are missed.  This level of repeating material is not possible in the classroom.
• Verbal learners may especially benefit from listing and discussing lectures within a small group.  This level of interaction is not possible with a Gateway course lecture.
• Sequential Learners
• Sequential learners will like to follow the lectures step-by-step from the beginning to the end.  They will be able to stop and review the video when they feel they have fallen off the logic path.
• Sequential learning will also like being able to repeat the lecture in its entirety in preparation for examinations.
• Global Learners
• Global learners can use the modular design and fast-forward through examples to get a general feel for the material, and then go back and fill in the details from the examples.  This ability to reorganize the lecture is not possible in a classroom setting.

• What resources were used? (campus technology support services, etc.)
  
• Streaming Media Lab
• Andy Harris, Distributed Education Mentor
• Scott Orr, Departmental Systems Administrator
• Kurtis Rush, digital media engineer
• Jonathan Beams, Tanmay Dharmadhikari, Deepthi Jonnala, Kirthi Krishna, Active Learning Module designers
• Michael McCarty, Warm Up Center developer
  

Potential to Impact Student Learning

• Fostered depth of learning
  
  Extending the depth of learning is a premier benefit of this approach due to the increased time in the classroom.  The following examples demonstrate improved learning the leverages the defined classroom experience.
  
  1.  Protracted examples: Students are introduced to the Turing Machine model of computation via a series of increasingly complex examples culminating in a group project.  The development of the topic takes an entire lecture.
  2.  Constructive examples: Students solve a series of problems demonstrating addition, multiplication, subtraction and division using binary representations of positive integers, negative integers, and floating point numbers.  Students demonstrate their proficiency by constructing solutions to given problems.
  3. Simulations: Students participation in a kinetic simulation of the fetch-execute cycle and polling vs interrupt event handlers.
  4. Students connect coursework to real life example through Computer Science in the Real World presentations.   For the majority of the semester, two students present a 5-10 PowerPoint presentation of any computer science-related topic of interest.  Examples include the use of radio frequency tags to track Alzheimer patients to keep them from accidentally wandering out of the facility, the $100 PC targeted for developing countries, and computer technology on the battlefield.
  

• Supported efforts to improve retention
• Learning Styles:  The restructured course supports learning styles better than lecture-format courses.
• Positive self-esteem: Previous experience with Distributed Education courses identifies that students feel more comfortable with the streaming media that is presented in an up-close fireside chat manner, as opposed to listening to lectures in stadium seating.   Students have also shared that they often feel intimidated by other students that seem to pick up with material with little effort.  They contrast this with the feeling of watching the video in their own home and having family members make positive remarks about their performance in the course.
• Archived Content:  Lecture contents as well as all ancillary material are archived for easy reference.
• Modular Curriculum Design: Topics are decomposed into reusable instruction modules.  Reusable modules and be mixed with custom modules to create complete courses.  Segments can be replaced as they grow obsolete.
• Multi-modal Delivery: Learning is improved by combining video, audio, feedback and active learning together.
• An unexpected contributor to improved retention is the ability for student to move between on-campus and distributed education sections of the course instead of withdrawing.  This factor alone accounted for a 3% reduction in the DWF rate.

• Fostered the learning of large numbers of students
  
  The active learning techniques involved with this redesign can be applied to a large number of students provided that sufficient board space is available in the classroom.  At IUPUI, only a few classrooms (e.g. IT-167) offer white boards across both the front and sides of the classroom.  There are no issues with the scalability of the active learning modules themselves, only enough space for everyone to participate.
  
  The distributed education section of the class is limited only by capacity to grade the assignment.  The streaming media server and JiTT Warm Up Center can support hundreds of students per section.  This project supports prior experience that suggests that opening a distributed education section of a course increases enrollment.  There is a market for distributed education sections of courses that attracts students that would never take an on-campus section.
  
• Has broader applications that will affect learning K-12 or community environments
  
  With a streaming media backbone, the active learning classroom is directly transferrable to the K-12 environment.  Students can use streaming media for lectures and the high school teacher can execute active learning modules in the classroom.  For this reason, each of the active learning modules is written up in a teachers guide.
  
  One high school student has already taken part in CSCI 230 and CSCI 240 through the SPAN program.
  
  There is some discussion of porting CSCI 230 and CSCI 240 to Java to that a high school offering can be made to support Advanced Placement A and A/B courses.

Assessment Plan

• What measure of assessment was used? How did your approach obtain measures of performance?
  
  All tests and projects were kept constant between the traditional Spring 2006 and active learning Fall 2006 sections of the course.
  
   Standard IUPUI course evaluations shall be used to compare student satisfaction of the two sections.
  
• Did you feel your measurement was accurate and effective?
  
  While the assessment measurements are accurate, several factors limited their usefulness in reaching definitive conclusions.
  
  1.  The number of students is too small forcing the 90% confidence intervals for the average scores of each section to overlap.
  2. I may have introduced some “research bias” because I unintentionally improved teaching in the course because I knew it was being evaluated.
  3. Not enough sections of the course have been taught to control for the class size.  It is a well known fact that smaller classes improve learning.  Some of the active learning sections had smaller enrollment than the baseline traditional section.
  4. While results are consistent with conclusions, this research does not reach a scientifically significant conclusion.
  
What were the student learning outcomes?

Three positive outcomes are listed below.

1. Improved DWF Rate: 33% - 27% and 27% - 8%.
2. Improved Learning: Exams 83% - 86%
3. Positive student feedback


Plan for Colleague Development

• What mentoring activities were implemented to assist colleagues?
  
  An overview of this project was presented to Computer Science faculty. 
  
  Streaming media record and production improvements were communicated to Andy Harris.
  
• How did you help to develop colleague's instructional technology skills?
  
  A demonstration and training session in the use of CamStudio was provided to Jeff Allen.
  
• How do you think your contributions to the Knowledge Base will help colleagues planning similar projects? 
  
  This project plan for this source serves as a reference point for similar projects.  A copy of the project plan is available upon request.
  
• Is your project approach potentially applicable to other contexts or disciplines?
  
  Active learning, JiTT, and streaming media techniques in this project are particularly appropriate for courses with a heavy lecture component.  The lecture component is naturally transferrable to streaming media and frees up a lots of time in the classroom.
  
  Courses that already use alternate teaching methods such as Demonstration/Performance, Discussion, Case Study, or Simulation methods (http://www.utexas.edu/academic/cte/sourcebook/teachingmethods.pdf) are harder or impossible to displace to streaming media.  These courses already redefine the classroom lecture experience.
  
• What tips would you offer?
1.  Create streaming media components on a one semester lag behind a live section.  Be disciplines to record the streaming media immediately after teaching the same material live.  Be sure to revise the materials based on student feedback prior to record, even if it means taking an extra day to revise materials.
2. Reference only topic names in introductions.  Avoid all references to a particular course number or title in the streaming media.  (There are lots of references to the course on the web site; you don’t need it in the recording.)  This allows you to reorganize materials between courses in the curriculum without retaping.
3. Try not to make direct references to prior lecture.  Refer to topics but don’t reference topics numbers.  This allows you to reorder topics without retaping the lectures.
4. Plan on taping all topics twice.  Start taping one and do not stop for errors.  Tape a second time stopping and starting over for every error on the introductory slides.  After passing through the introduction, continue through the end of the lecture without stopping.  You will notice that the second recording is much better and coherent than the first.
5. Do not even attempt to record fast-moving video during your lecture.  At least with CamStudio is will come out choppy.
6. Plan to offer CD and DVD media so students can watch the lectures without an internet connection.  If you offer media, remember that all streaming media recording must be complete at least a month before the beginning of the semester.  (This does not support Just-in-Time-Recording.)
7. Active Learning Modules require significant board space.  Find an acceptable room and reserve it early.  The following pictures show my favorite room, IT-167.









Final Comments on Project Results

• Do you think you met the goals of your project?
  
  This project met is execution goals:  90 streaming media segments were recorded, 30 active learning modules were created, 24 warm-up exercises were development, the Warm Up Center was deployed, and two courses were delivered using the new format.
  
  This project met its assessment goals:  Projects and Exam were held constant and compared between sections, a background survey and learning style survey were delivered.
  
  This project did not meet collaboration goals:  Processes and procedures for developing and Active Learning courses using JiTT were still under development.  The methodology was not mature enough to share with colleagues.  Sharing can begin from this point forward.
  
• What, if anything, would you do differently?
  
• Consider using Oncourse CL for the JiTT warm up exercises.  The Warm Up Center was independently developed to support the JiTT framework for other universities not using Oncourse.  Consideration should be made to porting warm up exercises to Oncourse using guest support.
• Place greater emphasis on room selection.  It is difficult to teach Active Learning Modules without available board space.
• Conduct more research in instructional design.  It is difficult to generate active learning modules with varying modes of interaction.  Creating good active learning modules is a research project on its own.
• Supplies have a strong impact on students.  A large number of index cards and markers are necessary.  Buy only black markers, not colored sets.  You need a number of markers equal to the number of students in order to support groups of two students and replacing the marker at least once a semester.  (Markers can be shared across multiple sections.)
• There is not accountability to ensure that lectures are actually watched before taking the warm up exercise or coming to class.  Students’ level of participation and benefit from the active learning modules is greatly diminished if they do not watch the material prior to class.
• JiTT feedback is not well designed for distributed education if it is tied to a particular date.  Distributed Education students tend to flex their schedules more than traditional students.  Much of the work occurs on weekends and the warm up exercises and feedback must take this into account.
  
• Did you have any unexpected outcomes? (positive or negative)
  
  There were two unexpected outcomes from the project.  First, the streaming media that was used to free up time to redesign the lecture allowed a distributed education section of the course to be offered.  Extra enrollment occurred because the distributed market only partially overlaps the on-campus market.  Second, additional improvement in the DWF rate was experienced because student can transfer between on-campus and distributed education sections.  Some students experience life event difficulties and can no longer attend class.  Those students are able to transfer into the distributed education section to complete the course.  Some students find that more assistance is needed than the distributed education section can offer, so they transfer into the active learning on-campus section.  Students that transferred both direction expressed satisfaction and appreciation for the ability to transfer.
  
  Note that care must be taken in the calculation of DWF rates so that cross section transfers do not adversely effect the DWF rate of the sections losing the students.
  
• Would you recommend this teaching approach to colleagues?
  
  This teaching approach is appropriate for courses that contain a significant lecture component.  Most courses in this format can improve student learning by introducing more active learning.  This approach is also appropriate if there is a desire to offer the course in a distributed education mode by delivering lecture content in streaming media format.  It should be noted that other modules of distributed education are available without streaming lecture, but significantly more development time is necessary to make such an offering.  It is a natural progression to go from lecturing in the class to lecturing outside the classroom.  Significant creativity is required to create active learning modules in an instructionally sounds ways and varying modules of learning.
  
• What other comments or suggestions would you offer to complete this report?
  
  Some departments debate the need for active learning in upper division undergraduate courses.
  
  There is an opportunity to build customized JiTT support within Oncourse.  Such an offering should be able to set warm up availability dates by student, by section, or for multiple sections of a course.  Calendar reminders could prompt students to take the warm ups.  Some tracking could be introduced to verify that students watch streaming media prior to taking the warm up.

Return to Final Reports page

---

AT&T Fellows Home Page

Last updated: 31 July, 2007

Comments and Questions | (317) 278-4833
Copyright © 1999-2003, The Trustees of Indiana University