Comparing User Research Experience and
Outcomes in Real & Virtual Environments.
by Tula Giannini, Ph.D., M.L.S., M.M.,
Associate Professor, Pratt-SILS

Venues and technology of research at the
crossroads of real and cyber space.
Library Research Seminar II 
University of Maryland, November 2001


Most studies focus on end-user interactions
with information retrieval systems.

This study looks at online searching in real and virtual environments from a macro perspective by exploring the wider parameters of the research landscape through the perspective of three variables:

1. Research environment (place)

2. Research experience (user)

3. Research outcomes (search results)



Users are no longer tied to physical collections and have
choices of where to carry out research especially as
online fulltext sources increase dramatically.


Comparing online searching in 3 venues:
Home = virtual
user searches solo in personal environment)
Library = real collaborative, sharing and community environment, Integrated = combined VE & RE
The quality of user experience is greatly influenced by RE and SO and ultimately determines which RE users choose and thus, directly impacts the future of libraries.


Evaluating the information users retrieve for its quality and usefulness.

Choices users make as to where to carry out research are based on perceptions about environment, experience and outcomes, rather than any particular search system, database service, etc.

If online searching at home alone is perceived by users to be more convenient and comfortable than in the library, and further, if in the future, access to online sources off site are comparable to the library's, will users recognize advantages to library research that will counterbalance these enticements of home?

Importantly, user perceptions of the advantages and disadvantages of each environment are pivotal to the choices that they make for online searching.

This study attempts to measures the impact of user perceptions of the research landscape, and to understand what trade-offs users are willing to make in the context of what they perceive as advantages and disadvantages of the real and the virtual library environment.


5Web search engine (80%) and specific database (60%) were students' top choices for where to begin online research.
1. Of the following options, which one do you generally
find most useful for beginning your research?
Ranked most useful (1) to least useful (6).
Rankings 1 2 3 4 5 6
Web Search Engine 12 5 3
        % ranked 1-3


         % ranked 4-6

Web Directory 1 4 5 5 3 7
         % ranked 1-3


         % ranked 4-6

Web Portal 4 5 4 2 4 6
         % ranked 1-3


         % ranked 4-6 

Specific Database 4 7 4 4 3 3
         % ranked 1-3


         % ranked 4-6

Database Service 3 5 5 4 7 0
         % ranked 1-3


         % ranked 4-6

Library Catalog 1 2 6 5 4 7
         % ranked 1-3


         % ranked 4-6

6Significantly, 68% preferred work or home to the library
for online searching.

2. Ranking preferences of search environment.
Responses show a slight preference for the work environment.
  Environments ranked as first preference with percent.
     Work    10 = 40%
     Library   8 = 32%
     Home     7 = 28%
3. What do you see as the most significant limitation of the library environment for online searching?  Identify two or three.
 Summary of responses- 
focus on convenience and research atmosphere
  • Limiting computer time and waiting to gain access
  • Need for more computers
  • Lack of privacy
  • Not adequate help with searching; staff not helpful
  • Access to online databases, programs and services not integrated
  • Slow Internet connections
  • Hardware and software problems
  • Crowded
  • Noisy
  • No printing and or downloading
  • Pay to print
  • Not adequate fulltext online

  • Library home page not user friendly

    This response underscores contradictions in student thinking -
    Although claiming to prefer online searching at home or work, they
    recognize the added value the library brings to the research process.

    4. Is online searching enhanced in the library environment?
        Responses: Yes - 23 (92%); No - 2 (8%)    next

Students appreciated the richness of online library resources, most of which were not available from home, as well as having free access to them.  They preferred the library environment for research.  Happy to be away from the distractions of home and work, they valued the professional help and expertise of librarians as an important if not essential aspect of research, and distinct advantage of online searching in the library.

9 Students give their reasons as to why libraries enhance 
online searching.
Sources and Services Environment Library Staff
  • Many fulltext sources available (xxxxxxx) 
  • Sources are close by
  • There are no fees for online databases
  • Learn about a new resource
  • The in-library online resources are far richer than off site (xx).
  • Many databases are not fulltext, so that collections support databases and vice versa.
  • Printers and copiers are available
  • Books available for multitasking next
  • One works better in a library
  • Work without interruptions
  • Better atmosphere for study - distraction free, sources at hand.
  • Library staff can guide you towards the proper database
  • Librarians are there to assist users (xxxxxxx)
  • Reference service 
  • Friendly librarians
  • Librarian help is readily available
  • No fee services

5. Based on your experience in research libraries, what recommendations do you have for improving online research services?
  • More help with searching by knowledgeable librarians xxxx
  • More knowledgeable staff in library's databases 
  • More computers and computer time xxxx
  • More user instruction xxx
  • Provide information on print sources that are cited in databases xx
  • More information for patrons about databases to assist patrons in database selection
  • More staff to answer patron search questions
  • Live online help (not e-mail)
  • More Web based services
  • Upgrade to highest level
  • Offer a library portal with all online sources available off site
  • Make search interface more user friendly 
  • Develop common interfaces for searching.
  • Integrate online resources to avoid moving from one terminal to another
  • Better fulltext coverage for remote access  xxx
  • More off site access to databases
  • More collection development of online sources
  • More work space 
  • Faster Internet connection xx
  • Separate area for search stations
  • Have computers for tasks such as email, news, etc., and others for true research.
  • Research area should be not be near crowded and noisy areas.
  • More research instruction and more explanation of databases
  • More technical support  next

Student answers show that they recognize the
library's important social aspects and human values and
the experience of  a landscape totally devoted to research.

6. Comparing real and virtual environments,
mark each of the following items with an "R" or "V"
to indicate which environment you feel best supports it.
Listed from highest number of "Rs",
ending with the highest number of "Vs" -
V's were in majority for only the last 6 concepts.

 Concept                                Real     Virtual   Uncertain
 Memorable experience             24          0         1?
 Social relationships                   23          2
 Human interaction                    23          2
 Research assistance                  21          4
 Positive feedback                     20           3        2?
 Study                                         19          6
 Concentration                           19          6
 Understanding info. retrieved   19          5         1?
 Goal attainment                        16           8        1?
 Worthwhile experience            16           7        1?
 User satisfaction                       14           9        2?
 Knowledge development          13         11        1?
 Progress in research                 10         13         2?
 Information seeking behavior  10         15
 Information exchange                8         15         1?
 Information acquisition              4         21
 Time savings                               3        22
 Convenience                               1        24 

7. What factors influence your choice of where to do research when choosing between online searching at home or at the library? 
Environmental factors Resource factors Logistics and Research Strategies Technology issues
  • The physical environment of the library
  • At home, kids, telephone, etc. stop me from working; I am much more productive in the library.
  • The home environment has interruptions; in the library you are free of interruptions and can work.
  • Peace and quiet - there are too many distractions at home.  I am more focused in a library setting.
  • Library subscribes to special databases xxxxxx
  • The materials available in the library xxxxxx
  • I will do research in a library if I know that they have resources that I need such as an extensive periodicals collection - but, if I can find it online in fulltext, I prefer doing research at home.
  • The quality of the library's physical collections.
  • Time factors xxxxxx
  • Type of research
  • Availability of printing
  • I write at home and do research at libraries.
  • Preliminary search in the online environment at home will give indications of what resources are lacking in the virtual environment, leading to targeted search in the real environment.
  • The library has broader access to databases and information; at home I can manage my own time, conditions and feel comfortable.
  • I work at a library and prefer doing research there during the day to working at home in the evening.
  • Cost factors - if I do not subscribe to a service, I will use the library
  • If  I am not sure of what I need the library is valuable for assistance.
  • For extensive research I use the library.
  • If I know that I can find the answer in the VE than I will not got to the library. next
  • Are good computers available?
  • Use library for fast Internet connection
  • No online access at home.


8. When doing research in the library, what is the most likely reason
for you to seek help with online searching?
Rank 1-7, where 1 is most likely.

Most students ranked the given list of reasons for seeking help as follows:

 1. Learn what databases are available
 2. Help with search strategy
 3. Database selection
 4. Search protocols and techniques
 5. Understand how the online sources relate to the collections
 6. Help with finding cited articles
 7. Help with evaluation of search results
Other, specify - 3 students noted help with printers.

9. When you have sought help in a library with online searching, 
have you received the help you needed?
If yes, what was it?  If no, why not?
Yes 15
No 10
  • Search strategy and terms
  • Learn about library's databases; search strategy - xxxxxxxx 
  • Librarian offered suggestions that I had not thought about. 
  • Librarians knowledgeable and could show how databases worked.
  • Librarians pointed to other useful sources.
  • For medical research the librarian was extremely helpful.
  • Only help with printing, but not searching.
  • I only ask for help when I can see that  librarians show a willingness to give it.   next
  • Librarians impatient and intimidating
  • Seek help only for tech. problems
  • Have adequate search skills xx
  • Use customer hotlines of paid services
  • Need librarians with better research skills.
  • Not enough staff; questions about searching are not welcome.
  • I was asked to consult the computer system for information.

10. When you are doing online research at home,
       what do you do if you need assistance?
  •    Use database online help xxxxxxxxxxx
  •    Try things until I get it to work xxxxx
  •    Call a librarian or colleague xxxx
  •    Seek help later at a library or work xxxx
  •    Use e-mail xx 
  •    Call customer service for subscription databases
  •    Use database print materials           next

11. Do you use library services from home?  If yes, which ones?
Yes - 21 No -  3
NYPL electronic resources xxxxxxxxxx 
NYPL OPAC xxxxxxxxx
Library catalogs xxxxxxxxx
Electronic databases xxxxx
Books: borrowing, hold, renewal xxxx
Library of Congress OPAC xxx
Brooklyn Public Library electronic resources
Pratt Library's electronic resources



1812. Looking ahead 5 years, how do you envision online research servics in libraries will develop to meet the needs of users?
What will these developments be?
Resources and Access
Facilities & Services
  • More online fulltext  xxxxxxxxx
  • Search interfaces that look more like the Web xx
  • Better home pages to support searching
  • Books will remain in print
  • More resources through remote access
  • Online services more tailored to at-home users as more homes get high speed access.
  • Develop live online help for home user.
  • Library Web portals that consolidate sources and services         next
  • More and better trained staff xxxx
  • Library as educational center xxxxx
  • More user instruction for online searching xx
  • Librarians that will teach users proactively xxx
  • More collection development of online resources xxx
  • Better human interaction
  • More hours of opening xx
  • More computers xx
  • Better access to more databases in all libraries, not just research libraries
  • 24 hour reference desk 
  • Need for increased funding for maintenance and upgrade of computers and systems xx
  • Library café 
  • Libraries as they exist today will not be able to meet users research needs xx
  • Better internet connection
1913. Based on your experience with online searching, what do you consider the most significant challenges that end-users face.  Rate the following from 1 (most significant) to 12.  Comment of your top 2 choices.
1. Having access to subscription databases 
2. The cost of fulltext services
3. Evaluating search results
4. Finding information that responds to research need.
5. Taking too much time finding useful information
6. Selecting the most appropriate sources for a topic.
7. Understanding and interpreting results
8. Feeling confident in information found
9. Using information selected to make decisions
10. Using free information wisely 
11. Keeping within legal and ethical boundaries of information use.
12. Access to high speed Internet connections
Student comments: 
*Subscriptions and cost relate to a divide between who those who have
  access and those who don't.
*It can be very time consuming to identify a needed source if you do not
  have access to subscription databases. 
*Most research libraries restrict who can use their databases both on site 
  and off. next

20Another questionnaire of 40 Pratt students reinforces the findings of the
above study. Students were asked about which environment they preferred
for online searching - home or library.
          Home     (24) 60%
          Library   (16) 40%
Students were asked to list some of the qualities of the environment
they preferred.  Responses in order of ranking.


  • Atmosphere conducive to study
  • Access to many online databases and resources
  • Reference librarians
  • Fast Internet access 
  • Books
  • Print materials
  • Study space
  • Quantity & quality of materials
  • More choices, real and virtual
  • Free printing 
  • No time limits for online searching
  • Comfort
  • Convenience
  • No travel time
  • Privacy
  • Quiet
  • not intimidating
  • Refreshments

Conflicting responses show students' mixed emotions.
Even though 60% of students said they preferred the home search environment, when they were asked, "Do you think that where you carry out your research and online searching makes a difference to your search outcomes?" 67.5% said that the library produced better results.

This contradiction reveals user conflict and difficulty assigning meaning to what they perceive as the advantages and disadvantages of the real and virtual.

How users resolve this conflict of values will greatly impact libraries.

Users are deciding what trade-offs they are willing to make. Is it worth giving up better resources for more convenience and comfort?

Clearly, libraries must take a new path with a new vision of service that provides the best of both worlds.  Libraries will need to connect users in both real and virtual, allowing them to customize service to meet a variety of research needs and to fine tune their research strategies.
The study reveals reveal that:

  • Users select different sources in the real and virtual
  • Selection process and criteria used for each were different.
  • The virtual was guided by convenience, time andavailability of online fulltext.
  • In the real, information was explored in greater depth;
  • more formats and documents were consulted.
  • Real library time was more concentrated and focused with attention given to the research strategies.
  • Home was characterized more as online searching.
  • The library was characterized by the process of research.
The research environment as flowing between the real & virtual.

Students identified the following factors as being important to them.
Factors identified by the most number of students:
Best of both worlds.
Online sources enhance use of print collections.
Virtual allows you to perform preliminary work prior to entering
the library so as to maximize time there.
Ability to use the Internet to search library's Web site to determine
resource needs.
Ability to move from online to print.
Users can evaluate library sources before visiting.
Able to focus research at home before getting to library.



Interaction and communication in cyberspace offers useful lessons
about the research process and tools that can be applied to real space.

One such tool for facilitating library research is computer mediated communication using the Internet and Web as a means to show information and to provide a common information workspace of shared content to which both librarian and user can react and interact.

A "collaboration support system" that models the reference process online designed by Shigeo Sugimoto and presented at the 1995 Digital Libraries Conference, Austin, Texas, makes clear the need for comparable collaboration and information sharing in the real.

Similarly, library Web pages, online databases and catalogs need to move from source lists and fill in the blanks search forms, to information displays
that show and communicate collection content and services.

  Control Tool: manages session between a user and a librarian.
  TV-phone: displays real time video pictures of session
  Shared Virtual Display (SVD): multiple windows for input/output operations
  White Board: participants can input character text and draw images
  Image Tool:  input images using a scanner shown on both workstations


Seeds of  Change: Social Effects of Political Crisis
Point to a Greater Role for Library Community.

Students were asked how the current political crisis might effect libraries and whether "people will seek to build library communities to meet new needs."

Out of 34 responses, 29 (85%) agreed with this notion,
4 were not sure, noting budget cuts, and one disagreed.

These results show that users will likely seek greater connection and
participation in community.

How can the library world meet this new challenge.



Shifting Gears: from community library to library community, from buildings that house collections
to building communities that use them.

Today, we have new tools to achieve a new kind of library community and library community networks through advanced information and communication technologies especially the Internet and World Wide Web.

Shifting our notion that a library is essentially a service that meets users needs, to one in which users become community members that participate in the library, and where service flows in both directions, from user to library and from library to user, could have far reaching effects.

In this type of environment, users will not have to choose between real and virtual, but rather will have an array of options in response to a broad spectrum of research and information needs.


28Building library community through online services that link and integrate the real and the virtual. 
Search skills gained from user instruction can help users participate in online community
 The library should work to extend its institutional culture to the online environment so that the real and virtual become parts of a whole.
A new information architecture is needed to reconfigure reference service transforming it to research services where librarians work with scholars using online searching. 
The user/library relationship should flow between the real and the virtual connecting a user's solo endeavour in private space to a collaborative one in pubic space. 

29Will researchers retreat to the suburbs of cyberspace
    or join library communities?

Online services link library communities and users to libraries.
The Internet and Web provide the platform for community
communication, collaboration and sharing.
Library communities can link to form library networks.
Virtual library services can be mirrored in the real:
e-mail, chat, collaborative research, online instruction,
interactive tutorials, digital media -
The Web can serve to communicate collection content,
research methods and strategies, and to connect with library
users at home to reinforce sense of community membership. next

Local library communities can link with state, regional and national communities to form library community networks for researchers, scholars, students and faculty. Such networks would do much to erase the digital resource divide between large research institutions and smaller colleges and universities so that students could have equal access to resources. Members of one library community could use facilities of another.  A library community network would give the library and library profession a greater voice and presence, and also offer members a library commons or forum to exchange information and address issues.

Libraries have already begun to transform themselves in ways that support community building through new approaches to IA in the both the real and virtual.
Collaborative work places
Coffee bar
Electronic research centers
Instructional classrooms
Computer labs
24/7 service areas
meeting and conference rooms and auditoriums
green architecture
quiet areas for independent research, good lighting
ergonomics, etc.
social interaction in secure environment
librarian/user interaction and collaboration
Online services to off cite users keep them connected to library community.
Library Web pages that include:
community news
library discussion groups
communicate collection content in ways that stimulate research
and interest in the library's collections
show users how collections might support topics for research.

Hi tech architecture links old and new to build community and benefit users.
A library atmosphere conducive to study and research was identified by students as a high priority.
Research libraries have begun to transform their buildings to meet the challenges of  technology and of
bridging the gap between the paper and digital worlds.
Library renovation and reconstruction designed to integrate real and virtual extends service to users off site and in the library through online searching.
Three examples show how libraries are being transformed: a new building for the British Library, in Paris, a new building for the Archives Nationales linking to the existing 17th century ones, and in New York, NYPL adds a new interior building to the Research Libraries at 42nd street for teaching and access. next
NYPL's South Court Pavilion, a new 5-story training center within the South Court courtyard at the Humanities and Social Sciences Library maintains the original Carrère and Hastings building. Coupling old and new, it symbolizes the dynamic relationship between them, and utilizes technology to access both research collections and online resources bringing new focus to teaching a range of technology and research skills using the hi-tech workstations and smart classrooms.  next

34Archives Nationales - Paris, France

The British Library, London.


M.D. Anderson Library (University of Houston):


"Proposed entry to the expanded and renovated Anderson Library.
The new entry will be partially covered, providing an outdoor seating area and protection from inclement weather. A 24 hour Student Café will be easily accessible inside the front entrance."

"Increase student study spaces to over 3,000. 
The Electronic Publications Center (EPC) is one of the largest and most advanced electronic library environments in the United States.
Students are often seen waiting in line for a computer terminal to access
one of the more than 100 databases available through the EPC.
Usage continues to double each year, yet the library lacks the space
or infrastructure to expand access to electronic information resources.
The number of EPC workstations available to students will be tripled
when the new wing is completed."
Usable space: increased from 357,000 to 498,000 sq. ft.
Book capacity: increased from 1.6 million volumes to 2.8 million volumes
Student study spaces: increased from 800 to 3,100.
Electronic library systems: expanded and upgraded.
Electronic information workstations: increased from 190 to 535.
Group study rooms: increased from 12 to 40.
Technology-assisted classrooms: three new rooms and an auditorium added.
24-hour study lounge: additional study space.
Special Collections and Archives Suite: expanded and renovated.  next 

38The University of Houston's  library expansion of the and renovation makes clear its move toward building library community, and the process of integrating the real and virtual. Its vision illustrates current trends in reinventing the library and redefining its role and mission in the university community in teaching, education, communication, and access to online information, placing online searching at the heart of the process - the point at which users
connect to collections,
to each other,
to the community
and to the world