AT&T Fellows Final Reports

2006

Name
Rachel Applegate
Title
Assistant Professor
Department
School of Library and Information Science
Campus
Indianapolis
Project Title
Outcomes Evaluation of IUPUI University Library Information Commons
Project Goal
To explore the effect on student learning outcomes of the innovative physical/technological facility, the Information Commons; to create a collaborative work group of faculty and librarians researching student information and technology use.
Type of Technology Used in the Project

Information Commons, University Library:  group-oriented workstations, relevant services.  OnCourse CL Workspace and electronic portfolio.

Executive Summary of Results

Data from student surveys and ‘scoring’ of student projects (papers and presentations) on information and technology use showed the utilization of the University Library’s resources, facilities, and services—particularly the Information Commons--to achieve learning outcomes.  There was a significant positive correlation between undergraduate students’ use of the Information Commons space and services and their demonstrated competency in information use and technology use as seen in classroom projects.  Both undergraduate and graduate students used the newly-designed technological workspaces for group and individual projects and 40% of undergraduates used librarian information services. 

          An OnCourse workspace was a useful platform for the cooperative efforts of a team of six librarians and faculty members.  The format allowed file-sharing, administrative support by a graduate assistant, and group communications.  The electronic portfolio function was explored.  It is not yet adaptable to serve as a structure for anonymous outcomes measurement.

Need for the Project

Research on effective pedagogy needs to consider not only in-classroom experiences but incorporate a holistic assessment of the student’s entire learning environment.  Within universities there is a growing awareness of the value of out-of-classroom resources, facilities, and services for student learning.  OnCourse is one manifestation of a ‘virtual environment’ designed to enhance learning, and the ePortfolio of a ‘virtual environment’ in which to facilitate learning assessment and teaching improvement. 
         Appropriate physical environments have a role, as well.  The University Library at IUPUI has consistently been at the forefront of both virtual and physical support for learning activities, with cutting-edge wired and wireless installations, heavy promotion of electronic information, and design and implementation of innovative searching (metasearch) and content (digital library, institutional repository) resources. 
         The UL staff recently conducted a needs assessment which revealed a growing need for student group work spaces.  Many programs and courses, such as in Business, Informatics, and Social Work, involve their students in complex group assignments.  At Indianapolis, the Library is an essential partner in making these assignments feasible learning experiences, as the student population is almost entirely commuter rather than residential, and there are limited technologically-equipped group work areas on campus. 

            Faculty in the project continued their usual teaching patterns and practices.  This research concentrated on student actions and choices.  It evaluated the extent to which the provided physical environment (the Information Commons) served student learning needs.

Use of Technology

There were two learning environments examined in this study:  the University Library’s Information Commons, and OnCourse, in its group work modules and the electronic portfolio portfolio function. 
         The University Library’s Information Commons was designed to provide effective group-oriented, technologically capable study facilities, information resources, and computer and reference assistance.  This expanded the learning environment outside of the classroom, and enhanced social interaction.  An average of 33% of time that students spent on their projects was spent at the library. 
         The OnCourse module helped librarians and faculty members coordinate their efforts—always challenging on a big campus with busy people—in this collaborative evaluation effort. 

            I had extensive discussions with Jay Fern regarding the use of the electronic portfolio to support assessment of student work.  At the present time it was not adaptable to the sort of anonymous document handling that this project involved.  Anonymity (of students) can be a useful feature of program-level outcomes evaluation and is often a feature of library patron relations.  Here it was also required by the IRB approved design.

Instructional Design Plan

A presentation to Library faculty in January 2007 highlighted for them the demonstrated benefits of the Information Commons for student learning outcomes.  This will feature in their in-class presentations to students on making the best use of the library.  Two librarians will involve classes in Spring 2007 in further research building upon this pilot examination. 

            I had assistance from Jay Fern (UITS, OnCourse lead developer), Anastasia Morrone (Center for Teaching and Learning) and Sharon Kahn (Center for Research and Teaching).

Potential to Impact Student Learning

It is vital for students to have effective resources to help their learning experiences.  When professors determine that group projects are an effective learning mechanism, then environments must be available where that can happen.  When professors ask students to go beyond textbooks into independent information-gathering, it is important to provide the information tools and professional advice that makes that successful.

            This Information Commons evaluation project explored a pilot means of testing whether those necessary conditions are present.  It showed that students found the physical facilities and professional services to be valuable in completing course projects.  Students who asked for assistance had higher-quality use of information resources.

Assessment Plan

This project itself was designed as a trial of student project scoring as a measurement tool.  It used survey data matched with expert scoring of student projects.  Classroom and library faculty scored projects with four-point scales for information and technology use.  The results show a remarkable consistency, speaking to the usefulness of simple expert judgments.  In exactly half of all cases all judges scored the projects the same; in only 17% of cases were the scores more than 1 point apart. 


Point Discrepancies

Info

Tech

Scores

Scores

0

16

11

1

10

14

2

1

2

            On the other hand, students had some difficulty interpreting the formatting of the surveys, where they were asked to provide percentages of time spent in each of several physical locations, and a sub-percentage for library use.  In nearly half of all cases, the cumulative percentages did not add up to 100.  This is valuable information for revising the survey instrument for the future.

Plan for Colleague Development

This project was an extremely useful first step for librarians and classroom faculty to explore environment effects on student learning outcomes, and functioned very well as a component of a project to explore the comparative value of a variety of library assessment tools.  Its particular approach represents the aspirational gold standard for academic librarians:  to see an effect of their resources, facilities, and services.  . 

            Based on this research, I will be delivering a paper at a national conference on the topic, and I have developed a network of nine Indiana academic libraries who are proceeding with further studies in 2006-2007 (and beyond).  All of the libraries are contributing data on usage and will employ site-specific surveys plus (with faculty cooperation) student project scoring.  This is a valuable contribution to library scholarship, moving beyond input/output calculations to focus on academic achievement. 

Final Comments on Project Results

I am grateful to have had the support and endorsement of this grant.  I am not sure I needed all of the money—the librarian collaborators were eager to donate their time!  Some problems were that two of the faculty collaborators had difficulty scheduling their time and ended up not contributing.  Some students had difficulty filling out the surveys in a mathematically accurate way.  The electronic portfolio is not designed to support anonymous reviewing, although such reviewing could be a very useful part of program review and other overview assessments. 

            It is wonderful and challenging to try to discern a direct “library effect” on learning outcomes.  It would be unfortunate to imagine that this kind of very intense examination is always possible and that libraries cannot demonstrate their worth in a multitude of indirect ways as well.

Return to Final Reports page


AT&T Fellows Home Page

Last updated: 18 May, 2007

Comments and Questions | (317) 278-4833
Copyright © 1999-2003, The Trustees of Indiana University