Project Season and Curriculum IL Assessment

Some academic librarians are on 10-month contracts, which is fabulous and, I hope, restorative!

For the rest of us, summer is often “project season,” as the pace of academic life slows and we finally get a chance to catch our breath. At least that’s the theory! (Summers used to feel slow and long, but I’ve noticed the pace of May-July shifting. I have meetings scheduled every day this week, which would have been unheard of in June even five years ago.)

One of the biggest summer projects which we undertake at my library is our “information fluency assessment project.” (We use the term “fluency” because we believe it denotes a higher level of ease and facility than “literacy” does.) With this project, we attempt to rate the level of information fluency skills demonstrated by our first-year students and by students who have received several years of a Seton Hill education.

This project was approved by the university’s IRB and begun in 2017, and we have continued it every summer since then. A librarian contacts all instructors of the first-year writing course and asks them to send us their students’ final research papers. We also contact the instructors of all 300- and 400-level courses which seem likely to contain a significant research component, and we ask them to send us their students’ biggest research projects. 

Our circulation assistant anonymizes the papers, redacts any potentially identifying information, and pulls a random sample of 60 papers from the first-year writing courses. She then does the same for papers collected from the upper-division courses. She keeps a coded record of which papers came from which course and which academic school, but this information is not shared with the librarians until after all of the papers are read and scored. 

The sample papers are divided into three batches, and each of the three librarians is assigned to read and rate the papers in two of these three groups (each batch contains a mix of first-year and upper-level papers, with no indication which is which). We developed a rubric which assesses resource pertinence; source integration; source validity; source currency; use of primary sources; the engagement with different viewpoints; and whether or not the sources seem to have been accessed legally and ethically. That last category is an all-or-nothing rating and was added after a few students listed bootlegged films in their bibliographies!

This gives us a snapshot of where our first-year students’ information fluency skills are at the end of the first-year writing course. It also gives us a similar sense of where our juniors’ and seniors’ skills have or have not developed. And, while we are careful to not directly compare different academic schools to one another, it is helpful for liaison librarians to see the specific skill areas in which their liaison schools are strong or weak. Faculty have been very interested in the results of the project, and especially in how their own academic schools perform in the evaluation. We have several instructors who choose to not have their classes participate, but on the whole the buy-in has been good. 

Our results indicate that, in the course of their Seton Hill education, our students significantly improve their ability to integrate sources into their own discussion of a topic; make better use of primary sources; and use fewer outdated sources. We are working with our institutional researcher to further analyze the results and to think of more ways that we can use this information to improve the learning and development of our students. So far it has helped us as librarians to see where we need to be improving our instruction, creating more “point of need” learning resources, or spending more time talking about particular aspects of information fluency.