Our Library Assessment in Student Learning Lab is undertaking ambitious work this year. We designed, conducted, and coded a student survey in Fall 2019, which had over 500 responses. In Winter, 2020, we designed, conducted and are mid-way through coding 22 individual student interviews. Both of these research efforts provided our Fellows with significant learning opportunities around coding. For most of our Fellows, creating a coding book and applying codes in Dedoose are entirely new research skills they have learned.
In looking at our Fall 2019 student survey, much of the development of our coding book was directed by the librarians since they had more experience than the students. LIbrarians began a draft codebook, then students tried to apply codes to the survey questions taking note on what additional codes may be needed and which were not necessary. We revised the codebook several times to arrive at clear wording and consistent application of the codes to the survey responses. Our second foray into coding was very different.
In Winter 2020, we conducted individual student interviews which included a role playing activity. We needed to develop a codebook in order to code transcripts from all 22 interviews. Coding would be done for some open ended questions as well as a role playing activity and finally a student reflection section at the end of the interviews. We decided to take a different approach to developing a codebook. Each of our 4 research fellows read 2 transcripts. The coded for big concepts, and not each word or phrase mentioned by students. Each fellow then developed a list of codes based on their individual work. Then, fellows met in pairs with the Lab Director, Soo Young Rieh, to review transcripts together and improve code application and inter-coder reliability. We then met as an entire lab and combined all coding terms into one giant list. We removed duplicates and consolidated very similar concepts. However, we still left in a very wide variety of terms, some which may not end up being in the final version however, we wanted to be as inclusive as possible at this early stage. For a second time, each fellow was asked to code two new transcripts using our newly amalgamated codebook to see again which codes worked, what may still be needed and which no longer make sense. The lab will meet as a whole group again in a week to further refine this codebook based on the recommendations of our fellows. Then our fellows will code all 22 interviews using the final codebook.
This process has been challenging and rewarding for two very different reasons. The first is that none of our fellows had developed codebooks before. For each of these research activities, the survey then the interviews, the fellows gain two very valuable and different experiences. The first was learning an approach of anticipating what codes may be needed and developing a codebook first before reviewing the actual survey data. Then adding/removing codes based on data that was unanticipated. The second method, reviewing the interview transcript data and letting this data guide development of a codebook, was a very different approach. Both approaches are valuable and allow our fellows to see options in coding and data analysis. I believe that it reinforced the notion that there is no single way to conduct such research and gave them very valuable experiences.
Several fellows shared with us that the deep, rich experiences they have had in our research lab have been focal points of some of their cover letters for job applications. I think this speaks volumes for our approach of giving graduate students real world experiences in a research lab environment to foster engaged learning at the graduate level.
Doreen Bradley obtained a B.A. in French Literature, Linguistics, and Translation from the University of Toronto and a Master of Information and Library Studies (M.I.L.S) from the University of Michigan. She has over 17 years experience as a health sciences librarian, and for the past 11 years has focused on information literacy particularly for undergraduate students. She is the Director of Learning Programs and Initiatives unit in the U-M Library.