Executive Summary of Results

Multiple animations (in both QuickTime and Flash format) were developed to improve learning and increase understanding of cardiovascular embryology. Data analysis showed that viewing the animations aided understanding of complex embryology events, as determined by comparing pretest scores to posttest scores of assessment participants. Participants stated they found the animations easier to understand and use than traditional teaching tools. Further, Web-based animations were accessed worldwide, indicating the extensive learning potential for this technology.

Need for the Project

Gross Human Anatomy (A550-A551) is a challenging course for 1st year medical students. It is challenging for the student to understand and conceptualize the three-dimensional relationships of the human body. For the gross anatomy part of the course, this conceptualization is aided by learning tools in the lab, such as the use of cross-sectional images of the body, 3-D models of certain organs, and the dissection of human cadavers.

In addition to learning gross anatomy, we also devote a substantial amount of time examining the embryology (growth and development) of all organ systems. Knowledge of embryology is essential for health care professionals, as many pathologies and anomalies can be traced to errors in organ development. Students find embryology particularly difficult to understand for a variety of reasons:

1. Embryology explores dynamic processes as opposed to static structures. In order to understand embryology, I must describe a set of complex steps that occur over a period of time. Due to this 4th dimension of time, these dynamic processes are almost always very complex and difficult to conceptualize.
2. By their nature, embryology texts use static, 2-dimensional images to illustrate the dynamic developmental processes. It is difficult for a student to look at 2-dimensional image A and image B, and understand the development that took place for the changes to occur between image A and image B.
3. Currently, there are few embryology animations on the market that can clearly and accurately demonstrate these dynamic processes. In May 2001 I attended the 2nd Embryology, Imaging and Education Conference and the common theme among these educators and embryology researchers was that dynamic computerized embryology animations were desperately needed.

Thus, in order to increase student comprehension of this subject, I began collaborating with Jim Hull in Graphics Services to create a series of computerized embryology animations. Our collaboration began in summer 2000. We have focused on cardiovascular (heart & blood vessel) development, as this aspect of embryology tends to be the most challenging for our students. I received a Graphics Services grant (offered by Indiana University) to pay for Jim’s services for the 2000-2001 and 2001-2002 school years. Each animation demonstrates a specific dynamic event that occurs during heart development. The animations may be found at the following URL: http://www.indiana.edu/~anat550/cvanim.

The funding from my SBC grant allowed me to hire a graduate student (Ruben “JR” Montoya) to develop additional cardiovascular embryology animations for me. JR Montoya is knowledgeable about heart anatomy, and he is experienced with Macromedia Shockwave Flash animations. To date, he has prepared multiple Flash animations that illustrate specific cardiovascular embryology events.

In addition, JR’s master’s thesis involves examining the assessment data from these animations and determining to what extent these animations aided the learning of difficult embryology concepts, how individuals felt about their ease of use and effectiveness, etc. JR and I are working closely together to examine this data and incorporate the feedback received to improve these and future embryology animations.

Use of Technology

As mentioned above, most teaching of embryology is via didactic lecture and static, 2-dimensional pictures. My project focused on developing dynamic computerized animations that showed the dynamic growth processes that occur in the developing human.

There currently are 8 cardiovascular embryology animations on the web (two of these animations have been combined into one large animation). There are two more animations in the final stages of development. Seven of these 10 animations are QuickTime movies, whereas the other 3 are interactive Flash animations. The QuickTime movies are relatively short (running time: 1 minute – 8 minutes, depending upon the animation). The Flash animations contain a series of “steps” that the viewer can work with at his or her own pace.

Few anatomists have explored the use of QuickTime and/or Flash animations to examine cardiovascular embryology events in detail. Since the animations are Web-based, anyone in the world with a Web browser can use them.

The students in my Human Gross Anatomy class (Anatomy A550-551) were encouraged to view the animations outside of class, before lectures on Heart Embryology commenced. Examination of the assessment data collected indicated that virtually every student reviewed these animations and participated in the assessments for each animation. During my lecture, I showed the animations as I was explaining specific, difficult concepts in heart embryology.

Whenever anyone views one of the animations, a JavaScript pop-up window appears asking them if they would like to participate in a formal assessment of the animation. If they click “ok”, they are taken to an online consent form where they again click a yes/no radio button for consent, and put their email address down for identification. When they click the submit button, they then are taken to an online multiple-choice pretest that asks them a series of questions about heart embryology. When they click the “submit” button, their answers are stored in a delimited text file on the website. When they submit their answers, they can view the animation multiple times. After they have finished viewing the animation, they are taken to a posttest that also contains multiple choice questions. Posttest scores are stored in a different comma delimited file on the website. Once they have finished the posttest, they are taken to a general assessment survey and asked several questions about their perceived learning from the animation.

If individuals do not wish to participate in the formal assessment, they can click on the “cancel” button in the JavaScript pop-up window. They then are taken to a pretest (for their own learning: their answers are NOT recorded) – and then they can view the animation.

Instructional Design Plan

Describe how the use of technology used supported your teaching approach:

• What student learning outcomes were expected?
• How did your teaching plan encourage active learning, provide feedback, address respective learning styles, and build collaboration?
• What resources were used? (campus technology support services, etc.)
• Would your project be easily understood and executed by other faculty?

In my SBC (formerly Ameritech) Fellows Midterm report, I mentioned the following learning outcomes expected included the following:

1. Medical students and other assessment participants will find the animations a more effective, easier, efficient learning tool than 2-dimensional images and lecture without animations.
2. For each animation, a comparison of pretest and posttest scores should show that posttest scores are higher (indicating that learning resulted from viewing the animation).
3. Fall 2002 A550 medical students should score equally well or better on the same cardiovascular embryology questions compared to an earlie4 May, 2007here no animations were used.
4. Fall 2002 A550 medical students who take the formal assessments multiple times should show an increase in pretest scores following each viewing of the animation, indicating that long-term retention of the material was enhanced by viewing the animation.
5. Anatomy A550 lectures on embryology of the heart were perceived to be more understandable and clear when the animations were used in lecture.

We still are in the process of collecting and analyzing Web assessment data for these animations. However, the data we have examined thus far indicate the following learning outcomes were achieved:

1. Based on feedback from my medical students, unsolicited emails from assessment participants, and Web-based assessment survey comments, I have found that these individuals found these animations effective, easy to use and very efficient learning tools. Many reported that the animations were more effective learning tools than 2-dimensional images and lecture without animations. A sampling of comments listed on the Web-based assessment survey include the following:
   • “A great learning tool for the development of the heart. I wish my school provided such a great tool.”
   • “Animation is the best way to understand changing relationships.”
   • “Very helpful, I could not have pictured this process without help from the animation.”
     
     
2. For each animation, a comparison of pretest and posttest scores show that the posttest scores are much higher (indicating that the animation aided learning of these embryologic events). On average, posttest scores we 40-60% higher than the pretest scores. (Please see the graph on the following page of comparison of pretest scores and posttest scores, with data collected as of 3/1/03 – note that we still are in the process of collecting more data.)
   
   Chart above illustrates pretest and posttest score comparisons for 6 of the 7 animations on the Web site.
   
   
3. The Fall 2002 class scored just as well or better on most, but not all, embryology exam questions as a previous year (Fall 1998) that did not use the animations. There were one or two questions on the exam where the Fall 2002 class did not do as well as Fall 1998. The Fall 2002 class has 3 individuals (out of 30) who were struggling with the entire class, so it may be that the Fall 2002 numbers are not related to the animations, but rather to the fact that these 3 students were struggling. I plan to collect additional data next year (Fall 03) and ask this same question about whether the animations help improve exam performance.
   
   
4. Despite my request that the medical students take the assessments multiple times, only a small portion of the class actually did so. Thus, we currently do not have enough data to determine if those who take the formal assessments multiple times show an increase in pretest scores following each viewing of the animation, indicating that long-term retention of the material was enhanced by viewing the animation. Again, data collection is ongoing and both medical students and other participants will be encouraged to view the animations multiple times, so we can collect enough data to answer this question.
   
   
5. As mentioned above, my students commented that the lectures were easier to understand and clearer when they had an opportunity to view the animation in lecture and receive didactic instruction as well.

Staff from Teaching and Learning Technologies Lab (TLTL) were extremely helpful with many aspects of this project. Staff members helped us modify JavaScript and IU Transform forms to develop the Web-based assessments. They developed an efficient tracking system and data collection system on our Web site for the assessment data. They also helped redesign the “splash” page.

This project can be easily understood and executed by other faculty. The multimedia programs used (QuickTime and Flash) can be manipulated fairly easily, and the Web forms used can be copied and modified for another technology project. Several anatomy instructors at IUPUI, the country of Turkey, and the country of Australia have used my animations in their classrooms to illustrate these processes. These instructors have told me they found the animations easy to use and their students seemed to enjoy them.

The assessment survey for each animation provided us with immediate feedback about each animation, and allowed us to make changes if necessary. For example, several respondents told us they had trouble reading the text in the Flash animations (the text was too small), so we enlarged the font. Some navigation tools weren’t clearly understood, so we tried to modify these tools so they would be easier to use. Several individuals told us they would like to see sound added to some of the animations – I have not done this yet, but I am thinking of doing this in the future.

Potential to Impact Student Learning

Clearly define how your project improved student learning - include specific examples of how your project:

• Fostered depth of learning
• Supported efforts to improve retention
• Fostered the learning of large numbers of students
• Has broader applications that will affect learning K-12 or community environments?

The Web-based animations fostered learning for students throughout the world. In addition to students, faculty and health care practitioners in the state of Indiana, we have had participants from other US schools and sites around the world. Analysis of IP web addresses indicate these animations have been accessed by multiple individuals from the following countries:

I have had several emails from students, instructors and physicians alike who thanked us for the animations and said the animations have helped them learn the difficult material.

As mentioned above, we are continuing to collect data so as to increase our sample sizes for each animation. Detailed analysis of our assessment-data-to-date has been reported above. Comments on assessment surveys have been positive and individuals believe the animations helped them understand the cardiovascular embryology concepts better than traditional teaching tools.

Assessment Plan

Briefly explain the effectiveness of your assessment plan:

• What measure of assessment was used?
• How did your approach obtain measures of performance?
• Did you feel your measurement was accurate and effective?

Most of our assessments were Web-based and included the following for each animation:

1. An online multiple-choice pretest is taken before one can view the animation.
2. After one views the animation, the person takes an online multiple-choice posttest (that contains the same questions as the pretest).
3. A general assessment survey for each animation follows each posttest, asking the individual about their perceived learning from the animation, and suggestions to improve the animation, from what geographic region did they visit the site from, basic demographic information, etc.

Pretest and posttest scores from each individual (identified only by their email address) are compared to determine if posttest scores show improvement (which would indicate that the animation assisted with learning). If an individual views the animation and takes a pretest more than once, the later pretests are examined to determine if retention of that learning occurred. Comments from the assessment survey are examined to determine individuals’ perceptions about their learning based on using the animation.

Whenever anyone views one of the animations, a JavaScript pop-up window appears asking them if they would like to participate in a formal assessment of the animation. If they click “ok”, they are taken to an online consent form where they again click a yes/no radio button for consent, and put their email address down for identification. When they click the submit button, they then are taken to an online multiple-choice pretest that asks them a series of questions about heart embryology. When they click the “submit” button, their answers are stored in a comma delimited file on the Web site . When they submit their answers, they can view the animation multiple times. After they have finished viewing the animation, they are taken to a posttest that also contains multiple choice questions. Posttest scores are stored in a different comma delimited file on the Web site. Once they have finished the posttest, they are taken to a general assessment survey and asked several questions about their perceived learning from the animation.

In addition, Web site traffic was analyzed to determine how many people visit the site but chose not to participate in the formal assessments. We have been surprised at how many individuals have accessed the site, and their geographic locations. Please see the above section for a listing of geographic locations, based on IP addresses.

An additional assessment was to compare Anatomy A550 test scores between the “experimental group” (Anatomy A550 Fall 2002 – used the animations) and a control group (Anatomy A550 Fall 1998 – no animations were developed at this time). As mentioned above, the data analysis yielded conflicting results, which may be due to small sample sizes (each class had 30 students in it) and the fact that a small number of Fall 2002 students who were struggling with the entire class may have skewed the results for the analysis of the animations alone. I plan to collect further data in Fall 2003 to increase the sample and try to “flesh out” the information from the data.

In general, I feel our assessment measures are as accurate and effective as they can be, given certain limitations. We have discovered that although the Web site had over 10,000 people access the animations (as based on Web data over a 6 month period), a grand average of 11.4% of those agreed to participate in assessing the animations. In addition, of that 11.4%, there was a large number of individuals who did not complete all pretest/posttest assessments (so we could not use their data). We are continuing to collect data and I plan on soliciting further volunteers for the assessments, so we may increase our sample numbers.

Finally, the general assessment survey is somewhat subjective, but will allow us to gauge individuals’ perceptions about their own learning and how comfortable they were with using this Web-based medium.

Plan for Colleague Development

Describe your role and activities as a mentor:

• What mentoring activities were implemented to assist colleagues?
• How did you help to develop colleague's instructional technology skills?
• How do you think your contributions to the Knowledge Base will help colleagues planning similar projects?
• Is your project approach potentially applicable to other contexts or disciplines?
• What tips would you offer?

I believe my project approach is potentially applicable to multiple contexts and disciplines. The main assessments used are online pretests, posttests, assessment surveys, and in-class exams. These basic assessments could be implemented with any short animation or Flash exercise.

I already presented the modules to IU faculty as part of the Teaching and Learning Technologies Lab Faculty Project Showcase in May 2001. In addition, JR and I have presented preliminary findings or showcased our animations at the following national meetings:

1. April 2002: IHETS IPSE Share the Learning Space conference (Bloomington, IN)
2. April 2002: FASEB (Experimental Biology) meetings (New Orleans, LA)
3. June 2002: Slice of Life Conference for Medical Educators (Toronto, Canada)
4. April 2003: FASEB (Experimental Biology) meetings (San Diego, CA)

I already have worked with several colleagues (both part of the IU School of Medicine and in other countries) about how they can utilize my animations in their classrooms.

I would give the following tips to others:

1. It is vital to collaborate with others. I would have been lost had I not had Jim Hull, JR Montoya and the TLTL staff to help me.
2. Consult with TLTL or your representative educational technology office for advice. TLTL had many valuable suggestions for me and provided guidance when I felt lost.
3. Make sure you get your project approved by the Human Subjects (IRB) committee early, if you plan on presenting your findings. In addition, recognize that online consent forms (like the ones I used) may need to be approved first by the committee.
4. Think big, but start small. I was amazed how each animation and its assessments took longer to develop than expected.

Final Comments on Project Results

• Do you think you met the goals of your project?
• What, if anything,would you do differently?
• Did you have any unexpected outcomes? (positive or negative)
• Would you recommend this teaching approach to colleagues?
• What other comments or suggestions would you offer to complete this report?

Based on our data collection to date, I believe I have met almost all of the goals of my project. I still feel that we need to collect more data for a thoroughly “sound” educational study, so we plan on collecting additional assessment data through next year. I would recommend utilizing Web-based animations in one’s teaching - *especially* if one is teaching dynamic developmental processes. In general, I don’t think I would have done anything differently – other than perhaps utilize Flash animations more than the QuickTime animations. I am in the process of doing so for future animations.

In terms of unexpected outcomes, I continue to be amazed at the worldwide accessibility of the Web. I never would have guessed that so many people from so many different countries (for example – United Arab Emirates?) would want to utilize these animations. I am THRILLED that people from around the world find my Web site useful. As a result, I have applied for additional funding from other venues to fund embryology animations of other body systems (like head and neck embryology, digestive system, reproductive system, etc).

I initially developed this project out of frustration that there were no embryology animations on the market that fit my needs. I am amazed now at the fact that this idea has “snowballed” into an expansive project that is used by individuals from around the world!

I want to THANK the SBC Fellows Program for the awarding of this grant and believing in this project. THANK YOU!!

Return to Final Reports page

AT&T Fellows Home Page

Last updated: 18 May, 2007

Comments and Questions | (317) 278-4833
Copyright © 1999-2003, The Trustees of Indiana University