Proposal submitted to IMLS in February 2005
Institutional Repositories: Ensuring Continued Access to Learning Objects
ASSESSMENT OF NEED
Colleges and universities are examining ways to capture and reuse the intellectual output of teaching and research activities such as data files from research projects, instrumentation, courseware, e-prints, and learning objects. One approach has been to create institutional repositories-"digital collections that capture and preserve the intellectual output of universities" (Crow, 2002a). Institutional repositories have also been defined as "a set of services that a university offers to the members of its community for the management and dissemination of digital materials created by the institution and its community members" (Lynch, 2003). This definition allows for a great deal of variety in institutional repositories. Institutional repositories operate under different administrative models, utilize a variety of technologies (e.g., DSpace, EPrints, FEDORA), and contain diverse content (Genoni, 2004). For example, the Harvard University Digital Repository Policy Guide notes, "Content deposited into the Repository is limited to library-like materials from any Harvard organizational entity. 'Library-like' materials are those that: support research, have persistent value, and are expected to be on deposit indefinitely." This guide goes on to state specific technical requirements for deposit. Other repositories, such as MIT's, have focused primarily on faculty papers and technical reports.
The primary focus of this proposal is investigate the development of institutional repositories in colleges and universities to identify models and best practices in the administration, technical infrastructure, and access to repository collections. Lynch (2003, p.) identifies effectiveness for institutional repositories in these terms:
An effective institutional repository of necessity represents a collaboration among librarians, information technologists, archives and records mangers, faculty, and university administrators and policymakers. At any given point in time, an institutional repository will be supported by a set of information technologies, but a key part of the services that comprise an institutional repository is the management of technological changes, and the migration of digital content from one set of technologies to the next as part of the organizational commitment to providing repository services (p.2).
While Lynch refers broadly to services, end users and continued access are not featured prominently in his definition of effectiveness. Crow (2002b) offers practical advice and guidance in the development of institutional repositories. In research literature, Nixon (2003) compared DSpace and Eprints at the University of Glasgow's institutional repository with respect to administrative functionalities. His study suggests that DSpace and Eprints have common features and work as complementary products that enable a university to collect different types of scholarly information. Kim (2005) examined end-users of DSpace and Eprints and finds problems with both systems' interfaces. She identified aspects and features of these systems that require improvement and suggests additional research on search options, results displays, and the means of linking to digital documents. Several other case studies have also detailed specific elements of success (Ashworth, Mackie, & Nixon, 2004; Barton & Walker, 2003; Beier & Velden, 2004; Rogers, 2003). Absent from these case studies are generalizable analyses of the necessary organizational, technical, and social elements in an institutional repository, whether each of these is equally important in institutional repository development, or whether some, such as intra-institutional coalitions, can successfully be accomplished later in the process. Finally, and most importantly, there is also little attention paid to the users of learning objects in these repositories. The investigators of the proposed research project believe that effectiveness cannot be evaluated without examining users and access issues.
Although an increasing number of institutions of higher education are planning for institutional repositories, repositories are in still in their infancy compared to other online information systems and services such as online public access catalogs, online cataloging systems, computer-based registration systems, online database systems, library gateways, etc. The time is right to undertake a research project to identify specific factors contributing to the success of institutional repositories and effective ways of accessing and using repositories. Project outcomes will assist administrators, staff, and executive officers at institutions of higher education in their planning and decision-making processes.
The proposed project addresses the effectiveness of institutional repositories, it accounts for the perspectives of both users and administrative staff. The following six key research questions will guide the investigators' research efforts (the Activities [see DESIGN PLAN] in which data will be collected to answer each question are also cited):
1. What makes an institutional repository successful, especially in terms of its organizational placement, administration, contributions processing, systems for resource discovery, content, use, and users? (Activities 1, 2, & 3)
2. Does level of expertise for metadata assignment play an important role in an institutional repository's success story? (Activity 3)
3. What automatic tools would make repository staff more efficient in assigning metadata and streamlining the assignment process? If we introduce needed tools, would they increase staff efficiency and streamline assignment? (Activity 3)
4. What do members of the learning community know about institutional-repository content, systems, and services? (Activity 4)
5. With regard to users of institutional repositories, what are their starting points and when do they turn to an institutional repository in the course of their search for information? (Activities 4 & 5)
6. Do people want to search institutional repositories at the same time they search more traditional resources, i.e., books, journal articles, newspaper articles, web sites, etc.? (Activities 4 & 5)
NATIONAL IMPACT AND INTENDED RESULTS
The development of institutional repositories is a very new enterprise for so many colleges and universities nationwide. If implemented well, institutional repositories could revolutionalize scholarly publication in learning communities, opening up access to research much earlier in the discovery process and reaching entirely new audiences (Lim, 1996; Prosser, 2003). A glimpse of the possibilities is evident in the physics community (O'Connell, 2002; Pinfield, 2001). In light of the potential impact in scholarly publication and communication, a baseline study of institutional repositories is timely. The proposed research will have an impact on extant repositories as well as on those in the decision-making and planning stages. This project's investigators intend to examine diverse variables and develop a series of outcomes that identify critical success factors at each stage of institutional repository development that pertain to the effective management, infrastructure, and access to these collections. Many of this project's outcomes will be applicable to institutional repositories in other environments (e.g., discipline-specific repositories and open-access repositories such as MERLOT). Finally, this project's outcomes will increase understanding of the development of institutional repositories for practitioners and researchers and provide guidelines and tools for practitioners to apply to the repository initiatives at their institutions. (See the EVALUATION for a list of outcomes.)
The proposed project requires five activities over 3 years (October 2005 to September 2008) that will involve surveying institutional repositories, conducting follow-up phone interviews, conducting cases studies, surveying current and prospective repository users, and studying users searching digital-repository resources.
ACTIVITY 1: Survey institutional Repositories (9 Months, Oct. 2005 to Jun. 2006)
From a literature survey and web-searches, the project team will identify institutional repositories. We will also query our contacts at these repositories for additional suggestions. Although our objective is to be as inclusive as possible, we may not be able to include all institutional repositories in North America due to limited time and resources. If we limited the survey to a purposive sample, we would sample repositories that have different intended audiences, objectives, object types, etc. We would also call on our advisory group for guidance and rely on comparable studies to fill in details (Cole at al. 2004). Our objectives in Activity 1 are to identify the wide range of practices, policies, and operations in institutional repositories and to begin to characterize the successful cases.
The subtasks required to accomplish Activity 1 are: (1) identifying repositories, (2) drafting, pretesting and refining the web-based questionnaire, (3) collecting data including encouraging and reminding repository staff to complete web-based questionnaires, and (4) analyzing data for fruitful areas of inquiry and in search of successful repositories for follow-up phone interviews.
ACTIVITY 2: Conduct Follow-Up Interviews (7 Months, Jun. 2006 to Dec. 2006)
Our analysis of survey data will enable us to begin characterizing successful institutional repositories. The team's next step is to conduct follow-up phone interviews with the staff at about two dozen seemingly successful repositories. Our objectives in Activity 2 will be to learn more about successful, unique, and fruitful practices, policies, and operations at institutional repositories and focus on the more successful ones for model building.
The subtasks required to accomplish Activity 2 are: (1) drafting phone interview protocols, (2) conducting interviews, (3) analyzing data in search of variables influencing success factors, model policies, and fruitful areas of inquiry for follow-up site visits.
ACTIVITY 3: Conduct Case Studies at 5 Model Institutional Repositories (8 Months, Dec. 2006 to Jul. 2007)
A Michigan project team member will visit up to five model institutional repositories and conduct personal interviews with a wide range of classes of people associated with the repository: (a) current and potential contributors: research and teaching faculty, research scientists, graduate research assistants, lecturers, donors; (b) repository caretakers: librarians, archivists, historians, curators, system administrators, volunteers; (c) institutional officers and administrators: library directors, museum administrators, vice presidents for research; and (d) prospective users: students, faculty, scholars, research scientists, general public, web users. This activity's objective is to determine how model institutional repositories operate, serve users, and maintain themselves. Activity 3 can be subdivided into several subtasks: (1) identifying repository liaisons to host the visit of the project team staff member and arrange interviews and activities in advance, (2) making one-week visits to each repository, (3) establishing rapport, interviewing, probing, observing, collecting documentation, etc., at visited sites, (4) getting answers to follow-up questions via phone calls, email messages, and Internet chat, (5) comparing metadata assignment by different classes of people at different levels of professional expertise and experience in metadata assignment, and (6) assessing the usefulness of OCLC-supplied vocabulary tools.
ACTIVITY 4: Survey Current and Prospective Repository Users (9 Months, Aug. 2007 to Apr. 2008)
Institutional repository resources are different from the resources that result from users' searches of online databases. Here are a few important differences-repository resources span a wide array of academic disciplines, they may require customized software to use, and they are rarely self-contained resources that can stand on their own without reference to other resources for use and interpretation. Our objectives for Activity 4 will be to learn firsthand from repository users-first-time, one-time and intermittent users, repeat users, and nonusers-about their use of, expectations, needs for institutional repository system functionality and repository content, and suggestions for improvement. Also, we will find out why people do not use institutional repositories and how and why nonusers would become institutional-repository users. Activity 4 subtasks are: (1) drafting, pretesting and refining web-based questionnaires for nonusers, first-time or intermittent users, and repeat users, (2) collecting data, and (3) analyzing data.
ACTIVITY 5: Investigate How People Search, Retrieve, and Use Institutional-Repository Resources (9 Months, Jan. 2008 to Sept. 2008)
Two methods will be enlisted to conduct user studies: (1) analysis of transaction logs and (2) experimental search tests. Analysis of logged data and search-task data will enable the investigators to determine how people query institutional-repository systems, what they expect to find and whether they find it, what they do with the information they find, how easy or difficult it is to find what they wanted, what improvements they would make to the repository's functionality, whether they return in the future, what prompts them to return, and whether they tell their friends and colleagues about the repository.
An estimate of nine months will be given to accomplishing Activity 5's four subtasks: (1) drafting, pretesting and refining interviewer-administered data collection instruments, (2) collecting transaction log data, (3) conducting experiments with prospective repository users and follow-up contacts, and (4)analyzing search-task and transaction-log data.
This section enumerates four major project outcomes and the contributions of the project's five activities to these outcomes.
Outcome #1. Case Studies of institutional Repositories that Illustrate Key Elements Contributing to Their Success.
Project investigators will purposefully vary case studies and focus on different types of digital collections. These studies will be designed to highlight different approaches to administration, collaborations, and technical issues such as platform and metadata assignment. Case studies will also feature sections on intended and projected user communities. Although the results of Activity 3 directly address case studies, Activities 1 and 2 are foundational and lead up to Activity 3.
By the time project staff initiate Activity 3, we will have winnowed our list of repositories down to a handful of successful ones. A project team member will travel to five separate repositories, observe repository staff in action, for example, staff soliciting contributions, processing contributions, assigning metadata, and handling inquiries. We will encourage repository staff to train our interviewer-observer to process contributions and assign metadata. We will enlist untrained volunteers and students from our School of Information at various class levels to assign metadata so we can compare their efforts. The project team will also collect organizational charts, systems documentation, training manuals, budgets, annual reports, web sites, and other documentation for later in-depth study. We will want to round out our knowledge of repository administration, items processing, systems, use and users, preservation plans, models for sustainability, future needs, etc., determine the levels of training and expertise needed for repository operations, and determine what repository operations and tasks are most costly and require high levels of training and expertise to accomplish them. Our modus operandi will be to probe, observe, and follow fruitful areas of inquiry instead of sticking to a rigid interview schedule. Because interviews will yield qualitative data, project investigators will enlist QSR's n6 to analyze collected data.
Together with repository staff, we will learn about repository staff needs for vocabulary tools that will speed, streamline, or simplify metadata assignment and convey our findings to OCLC's Office of Research staff. OCLC Research staff will make its existing vocabulary tools available for use by repository staff. Repository staff will use these tools and report back to project staff with their self-assessments of overall viability, ease of use, efficiency, and suggested improvements.
Outcome #2: Specification of Variables that Influence Success Factors in institutional Repositories
A written report will document the key elements involved in planning, development, implementation, and operational stages of successful institutional repositories. Although the results of Activities 1, 2, and 3 will figure into this report, findings from the user studies in Activities 4 and 5 may address the variables that influence success factors in institutional repositories.
In Activity 1, repository officers, administrators, and staff will be encouraged to complete web-based questionnaires that will collect substantive information about the repository: (1) its administration, e.g., the impetus for its establishment, current governance and significant changes from previous governance, mission, institutional champions, current caretakers and significant changes from previous caretakers, institutional and non-institutional partners, contributions policy, contributors, soliciting contributions, training contributors, contributions volume, staffing, staff training, repository and non-repository job responsibilities, (2) items processing, e.g., metadata assignment, selection of a metadata standard(s), ongoing responsibility for metadata creation, level of expertise needed for and training for metadata assignment, maintaining metadata standards, automatic processing, backlog, disposition of unprocessed item(s), (3) tools for resource discovery: functionality for retrieval, display, and use of repository items, tools development and enhancement, interoperability, (4) use and users: current and prospective users, promoting repository use, links from non-repository sources such as search engines and directories, reporting metadata to search engines and directories, training users in system use, restrictions on users, volume of use, (5) preservation plans, (6) models for sustainability, and (7) future needs.
Phone interviews in Activity 2 will enable project investigators to fill in details about repository administration, items processing, tools and systems, use and users, preservation plans, sustainability models, future needs, etc. The open-ended nature of telephone calls will enable repository staff and project team members to carry on a conversation, explore potentially fruitful areas of inquiry instead of sticking to an interview schedule, probe, and pursue threads and themes that cannot be foreseen in advance. Also team members can make inquiries on subjects that may be too sensitive to cover in online questionnaires, e.g., financial models, accuracy of contributor-generated metadata, partnerships, governance, future needs, etc.
Activity 3 site visits will complete our understanding of the operations of successful institutional repositories. Although our interviewer-observer will not be able to ask all questions and fill all gaps during a one-week on-site visit, she will become familiar to staff and build trust and confidence so that repository staff will be forthcoming about sharing information through follow-up contacts.
Outcome #3: Evaluation of institutional Repositories Based on User Studies
Project investigators will study members of the learning community and how they conduct research using institutional-repository content. Specifically, we will determine how people learn about digital-repository content, systems, and services, where they begin the information-seeking process, at what point in their research they turn to institutional repositories, and how they benefit from repository content. Data from Activities 4 and 5 will figure prominently in the fulfillment of this outcome.
In Activity 4, Michigan project staff will develop web-administered questionnaires and link them to the institutional repository web site, a likely location for recruiting repository users, and to the parent institution's site, a likely location for recruiting nonusers. Searches of the institutional repository could also trigger an online questionnaire so that we can collect data about users' actual ongoing searches to completed questionnaires. Our intent is to put these questionnaires at institutions participating in Activity 3 case studies; however, we could fall back on institutions participating in Activity 2 phone interviews in the event that a case-study site is not able to host web-administered questionnaires. Analyses of questionnaire data will enable us learn firsthand from repository first-time, one-time and intermittent users, repeat users, and nonusers about their use of, expectations, needs for institutional repository system functionality and repository content, and suggestions for improvement. Also, we will find out why people do not use institutional repositories and how and why nonusers would become institutional-repository users.
Finally in Activity 5, project team members will monitor and observe actual use and users of institutional repositories. Case-study sites will log queries and other relevant transaction data, e.g., number of retrieved items, number of retrieved items inspected, and the varieties of customized programs that users launch to use retrieved items so that team members can analyze these logs to determine whether information retrieval in institutional repositories is comparable to retrieval in traditional online databases. Through surveys, interviews, and the case studies the team will have identified core constituencies for the institutional repositories. The team will recruit members of these constituencies at the University of Michigan learning community with real information needs, and ask them to conduct two searches in the institutional repository: (1) one search for information that addresses a topic of their own interest and (2) a second search for information on a search task that an interviewer assigns to them. About three months after the interview, team members will ask these same users to respond to a web-based questionnaire that asks them whether they returned to the institutional repository or a comparable web-based repository to find additional information on the same or a different topic, whether they found useful information, and how they used what they found. Users who participate in the study will be paid $25 for the initial in-person contact and $15 for subsequent online contacts. Because much transaction log data and search interviews will yield quantitative data, project investigators will enlist SPSS to analyze collected data.
Outcome #4: Instruments and Protocols for Re-Use by Other Investigators and Repository Staff
Data-collection instruments and protocols will be aimed at the core constituencies of the learning community-students, faculty as teachers, faculty as scholar-researchers. Instruments and protocols will be tested and validated during the course of the grant and be ready for staff colleges and universities to use after this project's data-collecting tasks are done. All five project Activities will require instruments and protocols to collect information from repository staff (web-based questionnaires in Activity 1, phone interviews in Activity 2, interview schedules in Activity 3) and repository users (web-based questionnaires in Activity 4; interviewer-administered questionnaires in Activity 5).
The School of Information's administrative staff will be responsible for accounting, funds disbursement, and policy compliance. Their experience with these matters is extensive due to the School's high standing (7th) amongst all University of Michigan units in the acquisition of grants and contracts.
Co-principal investigators Rieh, Yakel, and Markey are colleagues in the School of Information at the University of Michigan. Each will apply her own high-level expertise in information use and users in various contexts, web (Rieh), archives and records management (Yakel), and public and academic libraries (Markey) to the project.
While Rieh will be responsible for the overall conduct of the project, both co-investigators will take leading roles in this project's major activities as follows:
Activity 2 Follow-up Phone Interviews Rieh & Markey
Activity 3 Case Studies of Selected Repositories Rieh & Yakel
Activity 4 Survey Repository Users & Nonusers Rieh & Yakel
Activity 5 Retrieval Tests with Users Rieh & Markey
The project team has designated Ji-hyun Kim as Project Manager; Kim is currently an SI doctoral student who has research experience with institutional repositories. The project team of three investigators, project manager, and hourly student workers will hold weekly project meetings to discuss project business.
The project team will enlist the expertise of an advisory group to assist them in the identification of repositories, review drafts of the web-based questionnaire and analyses of questionnaire data, suggest repositories for follow-up interviews, and brainstorm the factors that make a successful repository. Advisory group members have experience in system building, tool building, research, and consulting for institutional repository systems. Members are Kate Nevins, Director, SOLINET; Michael Seadle, Assistant Director for Systems & Digital Services, Michigan State University Libraries; Diane Vizine-Goetz, Consulting Research Scientist, OCLC; and Marcia Zeng, Professor, Kent State University. The ATTACHMENTS section contains email messages that confirm their acceptance of our invitation to serve on the advisory group.
OCLC Research staff will make its existing vocabulary tools available for use by repository staff. Project staff will place these tools in repository staff's hands for use and self-assessments of tool usefulness. Because Diane Vizine-Goetz of OCLC is serving on our advisory group, she will learn about repository needs for vocabulary tools at the earliest phases of the project and be able to mobilize her staff to respond accordingly to their needs with useful tools. The ATTACHMENTS section contains an email message from Lorcan Dempsey, Vice President for Research, OCLC, that confirms OCLC's contribution in this regard.
A large proportion of the budget will support personnel (principal investigator Rieh, co-investigators Markey and Yakel, and manager Kim). We also request one temporary research assistant to help with all aspects of the project, including data input and data analysis. The cost for site visits, payment to subjects, software, and transcription services are important items in the budget. As stipulated in IMLS application guidelines, travel funds to attend the outcomes-based planning and evaluation (OBE) training and other IMLS-designated meetings are requested. The BUDGET JUSTIFICATION gives detail regarding the allocation of personnel time and other expenses.
The School of Information is committed to the proposed project and recognizes its importance in the form of cost sharing. Cost sharing will support the salary and benefits of the project personnel and partial tuition for a doctoral student: (1) Rieh in 15% of Academic Year salary for three years; (2) Markey in 15% of Academic Year salary for three years; (3) Yakel in 15% of Academic Year salary for three years; (4) Kim by covering the out of state cost differential for her tuition which equal slightly more than half the cost of tuition. Cost sharing amounts to one-third of the project's total budget. Details are found in the BUDGET JUSTIFICATION.
Project investigators have a combined 55 years of research and practical experience studying users and uses of information systems and services. Rieh conducts research on information-seeking behavior and web searching and evaluates information retrieval systems used in industry and academe. As a Human Factors Research Engineer at Excite@Home, she researched people's web searching behavior in their home environments and evaluated the interface design of the Excite search engine. She has participated in the TREC (Text Retrieval Conference) projects and used observations and interviews in her research. Her dissertation (Eugene Garfield-ALISE Doctoral Dissertation Award, 2002), dealt with people's judgments of information quality and authority on the web, analyzed search logs, think-aloud verbal protocols, and post-search interviews collected in the laboratory settings. She can apply her award-winning research experiences and knowledge directly to this project in the design of data-collection instruments and analysis of data that address use and access issues.
Markey has 25 years of research experience in end users, subject access, search strategies, and library-user education. Managing research projects sponsored by the Council on Library Resources (CLR), OCLC, Department of Education, and IMLS, she has gained considerable experience with the methods used in the proposed project and has conducted numerous retrieval tests with objectives that are similar to those in this project. Markey is the author of four books, two dozen research reports, and six dozen journal articles. She has won prestigious awards from the American Library Association: Kilgour Award for Research in Library and Information Technology (1998) and Shera Award for Distinguished Published Research (1999).
Elizabeth Yakel has 20 years of professional and research experience in the areas of archival access systems and the evaluation of use and user services related to primary sources. Yakel is currently involved in two complementary research projects. The first, "Assessing Access and Accessibility of Interfaces for Primary Sources," is funded by the National Historical Publications and Records Commission (NHPRC) and examines the searching strategies of researchers seeking primary sources online. The second, "Developing Archival Use and User Metrics," funded by the Mellon Foundation, is aimed at standardizing the information archives collect on use and users to improve their operations. Her experience with access to primary sources, representational issues, and design of archival access systems makes her well-suited for this project.
Ji-hyun Kim is an SI doctoral student who is involved in research projects on the usability of access systems for archival information. She is presently participating in a research funded by the NHPRC on the usability of archival-access tools and has tested various tools in institutional repositories and identified problems with their interfaces. Kim can directly employ her research experience and knowledge of institutional repositories to the proposed project.
To make this project successful, it is critical for the Michigan team to assure that the project is fully exposed to administrators, officers, and staff of existing institutional repositories. Wide recognition of this project will encourage active participation of many institutional repositories throughout the project. A project web site will be created immediately so that the project proposal, interim reports, data-collection instruments, and other relevant resources are made available. Preliminary findings will be given in semi-annual interim reports so the participating institutions and interested individuals can monitor project progress. The team will make every effort to give talks at professional conferences and publish in scholarly publications. The primary professional affiliations of each member of the Michigan team are not the same (Rieh = ASIST, Markey = ALA, Yakel = SAA), thus we are well positioned to present papers at different professional and scholarly venues. The team will work diligently to publish in both scholarly journal articles and practitioner-oriented magazines that project findings reach the people who are most likely to effect changes in the development, design, and maintenance of institutional repositories. We will also publish in peer-reviewed electronic journals to take advantages of fast turn-around on publication.
This project will address the problem of institutional repositories from the perspectives of repository administrative staff and repository users. Project investigators will accomplish this by developing models and providing assessment tools that can be used by administrative staff in these repositories. For those working in institutions that already have developed and maintained institutional repositories, the results of this project can offer directions for refining their future plans especially for designing activities that can better meet the needs and use patterns of their user communities. For those who are in the initial stages of developing institutional repositories, we anticipate that the project will serve as guidelines for creating more efficient management plans based on this project's empirical evidence of successful repositories. For both, this project will contribute to better understanding of the nature of managing institutional repositories from both perspectives of preservation and use. It will also serve as a baseline study that can be used over time to measure the development, impact, and viability of institutional repositories in later studies.
Unlike well-established information institutions such as libraries, archives, and museums, institutional repositories have not developed a sense of community because of their short history. Therefore, to date, sharing successful cases and tools is based on ad-hoc direct contacts among repositories. As the first project of surveying current status of institutional repositories, we believe that this project will have significant impact in the field by identifying the types of institutional repositories and successful cases in terms of repository administration, content, infrastructure, use and users. Once the project is making progress, it will be a useful step toward establishing communication channels for various stakeholders who are involved in developing and maintaining digital collections.
Ashworth, S., Mackie, M., & Nixon, W. J. (2004). The DAEDALUS Project, Developing Institutional Repositories at Glasgow University: The Story So Far. Library Review, 53(5), 259-264. http://eprints.gla.ac.uk/archive/00000408/
Barton, M. R. & Walker, J. H. (2003). Building a Business Plan for DSpace: MIT Libraries' Digital Institutional Repository. Journal of Digital Information 4 (2).
Beier, G. & Velden, T. (2004). The eDoc-Server Project: Building an Institutional Repository for the Max Planck Society. High Energy Physics Libraries Webzine, 9,
Cole, T. et al. (2004). IMLS Collection Registry & Metadata Repository.
Crow, R. (2002a). The Case for Institutional Repositories: A SPARC Position Paper.
Crow, R. (2002b). SPARC Institutional Repository Checklist & Resource Guide,
Lynch, C. A. (February 2003). Institutional Repositories: Essential Infrastructure for Scholarship in the Digital Age, ARL Newsletter no. 226, 1-7.
O'Connell, H. B. (March 2002). Physicists Thriving with Paperless Publishing. High Energy Physics Libraries Webzine.
Pinfield, S. (December 2001). How Do Physicists Use an E-Print Archive? Implications for Institutional E-Print Services. D-Lib Magazine, 7.