We examined 45 transcripts of online reference chat interactions where reference librarians from other institutions provided service to patrons who entered the service through NU's web page.

Of 45 transactions (these are mostly librarians from other colleges answering questions, not NU librarians):

• 15 were fully answered
• 6 were partially answered
• 7 were unanswered
• 7 patrons dropped off after the dialog started but before it was complete

Of 45 transactions:

• 14 reference interviews of some sort were conducted
• 14 reference interviews were not conducted when they should have been
• 17 reference interviews were not needed (usually cases where the person needed citation help and explained what they needed pretty well)

Observations:

• Counting disconnects, the chances of a patron getting a complete answer to his or her question is 33%.

• The librarians seem to be trying to help patrons, though there was a tendency to answer questions based upon what web pages could be pushed, rather than the questions themselves.

• Many of the reference interviews that were counted as being done weren’t necessarily done well. For example, librarians asked "yes or no" questions or focused on format: "do you need articles or books?"

• Patrons who asked research questions or even hard factual questions tended not to get their questions fully answered. Many of the fully-answered questions were to citation requests.

• Librarians tend to push web pages to patrons and aren't really using their own collections very much (with a couple of good exceptions). When it comes to citation help, librarians also often push pages rather than address the specific question (with a couple of good exceptions).

• There were only a handful of "value added" transactions where the librarian brought something extra to the interaction.

• Many of the questions that came through Niagara's site did not appear to be from Niagara students.

Overall, including the disconnects, it doesn’t seem that the virtual reference service is likely to result in a very pleasant user experience.

The technology doesn't seem suited to research questions. The service works better for specific requests, but only if a student happens to catch a Niagara librarian on duty. For example, a librarian from another institution would not be able to tell a Niagara student what our passwords are or provide a referral to a local librarian by name.

Students usually would have been better off calling the Library's reference desk, so starting in the Fall of 2005, we are promoting this service as an alternative.

We are not giving up on online reference service, but we will explore a home-grown alternative or wait until the quality of cooperative virtual reference service improves.