• Introduction
• Philosophy
• Competency A
• Competency B
• Competency C
• Competency D
• Competency E
• Competency F
• Competency G
• Competency H
• Competency I
• Competency J
• Competency K
• Competency L
• Competency M
• Competency N
• Conclusion

Introduction

Socrates opines that "the unexamined life is not worth living". Centuries later, this stance is still open for debate, but it's certainly true that the unexamined library may not be offering its patrons the best programs and services possible. The administrators of the unexamined library may also not be able to justify programs and services when budgets come under review. Thus, it's important for information professionals to be able to turn a critical eye towards their own organizations, to be familiar with methods of evaluation in a wide variety of contexts, and to be able to plan and implement an effective evaluation.

Evaluation is an iterative process: the collection of information about a program or service may prompt changes, which themselves need to be evaluated to determine their effectiveness. Evaluation is a process that occurs in a continually changing environment: depending on what is being evaluated, it may be necessary to revisit the evaluation process at regular intervals. Evaluation can be effectively carried out with varying levels of formality: both rigorously designed data collection processes and informal opinion polls of the next ten patrons that walk through the door can provide valuable information and fresh perspectives. And, evaluation can mean the collection of qualitative or quantitative data: scores on pre-tests and post-tests can shed light on the effectiveness of instruction efforts, while focus group or individual interviews can provide useful insights into behaviors, opinions, and emotions surrounding a particular library program.

Equally important to methods of evaluation are the development of criteria against which a program or service is to be evaluated. Our profession has developed a number of standards and guidelines that can help us to evaluate how particular programs and services measure up. These standards and guidelines can form a framework of criteria to use in the evaluation of a service. One set of guidelines that I have often referred to in both my academic and professional work has been the Reference and User Services Association's Guidelines for Behavioral Performance of Reference and Information Service Providers (2004). These guidelines are so widely used, and applicable to such a variety of information-seeking environments, that they act as a sort of universal professional benchmark against which reference services can be measured. Of course, some evaluative needs may be more peculiar to a particular information organization, in which case it may be necessary to develop meaningful evaluative criteria that are tailored to the particular situation. In these cases, it's important to clearly define the goals of the evaluation, the particular dimensions that will be examined (are we interested in patron attitudes towards a service, or patron use of a service?), the data collection techniques that will be used, and the resources and staff time needed to carry out the evaluation.

My coursework has provided me with ample opportunity to evaluate programs and services, and to develop data collection methodologies to achieve evaluation goals. Since I have emphasized reference and information services in the selection of my classes, I've devoted a lot of thought to the evaluation of reference service. I have also developed a survey tool to evaluate high school student attitudes towards school library services. With the evidence presented below I demonstrate my competency in the area of evaluating programs and services against either external or internally developed standards.

Evidence

Reference services are increasingly provided via a range of different media, from chat, to email, to good old-fashioned face-to-face, and even via avatars populating virtual immersive worlds such as Second Life. Although each mode of delivery imposes its own restrictions on the reference interaction, standards of good reference service can still be applied remarkably consistently across formats. In each of the works presented here, I have evaluated reference interactions occurring in different environments according to RUSA Guidelines (2004). Two of these assignments ("secret shopper" evaluations of face-to-face and chat reference interactions) were completed in fulfillment of the requirements for Reference and Information Services (LIBR 210). One assignment, the evaluation of a classmate's responses to reference questions submitted to the Internet Public Library's Ask an IPL Librarian service (IPL, 2008), was completed for Advanced Information Resources and Services (LIBR 228). For this assignment, I also evaluated the rubric used to frame responses to reference questions. I have included another assignment from LIBR 228 in which I evaluated 1) reference services on offer in libraries in Second Life and 2) a simple staged reference interaction that I had with one of my classmates. This assignment was later used as the basis for a column co-authored with two of my classmates, which was published in Reference and User Services Quarterly (Gerardin, Yamamoto & Gordon, 2008): see competency I for a description of and link to the column. I submit these works as evidence of my ability to apply the RUSA guidelines (2004) to the evaluation of reference services delivered via a variety of different modes.

Useful evaluation of the services and programs of an information organization may be carried out on an informal basis if the results of the evaluation will only be used internally and if time and funding for collecting such information is at a premium. If the evaluation results will be used to justify programs to outside funding agencies, or if the collectors of the information intend to publish the results of their efforts in the library and information science literature, it's often desirable to design a rigorous data collection protocol. In my Research Methods in Library and Information Sciences class (LIBR 285), I worked with two other students to develop a protocol to collect survey data from hypothetical high school students, in order to determine attitudes towards existing programs and needs for new programs. The project entailed the development of the survey instrument, a protocol for collecting and analyzing survey data, and an analysis of the survey design process. I submit this work as evidence of my ability to work with a team to create a sound data collection methodology for program evaluation.

Conclusion

I have demonstrated with the evidence presented above that I have achieved competency in assessing a service (reference service) in the context of multiple delivery modes, as well as in developing and assessing tools and data collection methods that can be used to evaluate patron attitudes towards programs. I have already applied my understanding of the methodology underlying effective evaluation in my current position, assessing the effectiveness of an online information literacy tutorial. I believe that the understanding of research design that I've acquired during my time at SLIS, as well as my familiarity with some of the guidelines and standards used as the basis of evaluation in our profession, will allow me to effectively develop evaluation strategies in a variety of information organizations, in response to a wide range of needs.

References

Gerardin, J., Yamamoto, M., and Gordon, K. (2008). Fresh perspectives on reference work in Second Life. (FOR YOUR ENRICHMENT). Reference and User Services Quarterly. 47(4), 324-330.

Internet Public Library. (2008). Ask an IPL Librarian. Retrieved on February 15, 2009 from http://www.ipl.org/div/askus/

Reference and User Services Association. (2004). Guidelines for behavioral performance of reference and information service providers. Retrieved on 03/15/2009 from http://www.ala.org/ala/mgrps/divs/rusa/resources/guidelines/guidelinesbehavioral.cfm