In the consumer sphere, search has become practically magical. Our personal devices-phones, computers, smart speakers-respond to queries and questions with remarkable precision and context. Google- whether accessed through a web browser on a computer, tablet, or phone, or through a Google Home assistant-has consistently set an enormously high bar for search, relying on its massive content indexes, ranking algorithms, AI, and other content and technology components. While Google dominates the search engine market, Microsoft's Bing likewise delivers phenomenal results. These commercial services benefit from massive engineering resources, global infrastructure, and state-of-the-art user experience expertise.
Despite the perceived capabilities of the consumer search engines, they are far from perfect as tools for seeking objective information. While they work well for practical tasks and topics, commercial search engines are subject to manipulation and bias. They generally do not reveal the details of what sources they index and how they rank search results. In this era of fake news, any search engine faces ever more difficult challenges in delivering reliable results. It is also important to keep in mind that these services operate primarily as advertising platforms and are optimized to maximize revenue. They deliver search results in ways that satisfy users and perpetuate their services, but do not necessarily operate transparently.
Libraries provide search and discovery services with a much different set of requirements and usually less than magical interfaces and search capabilities. These services must respond to queries objectively, document what resources they address, and reveal the methods used for indexing, retrieval, and ranking. The expectations for privacy are diametrically opposed to commercial services: Those provided by the library must be designed to protect the privacy of the search session and to carefully guard the personal details of the patron-in contrast to those within the adbased ecosystem that depend on extracting any available personal and contextual data for the benefit of targeted ad placements.
Despite these overarching differences, library discovery services must provide interfaces that will be appreciated by patrons accustomed to consumer search services. Library search interfaces that are difficult to use or that perform too slowly will cause frustration and may drive patrons to the consumer search services. Despite these challenging requirements and the limited resources available in the library development community (including both commercial companies and open source projects), there has been great progress among the discovery products and services that libraries offer to their patrons.
The Demise of the Academic Library Catalog
Currently, the traditional online catalog is increasingly giving way to discovery tools, especially in the academic library sphere. An online catalog comes as a component of an ILS and gives patrons the ability to search and make requests for the collection materials the library owns. These catalogs, based on the MARC records managed within an ILS, can offer sophisticated features for searching and browsing the library collection, although some patrons may find their interfaces complex and difficult to use.
The concept of an online catalog does not apply to the library services platforms, such as Ex Libris Alma, OCLC's WorldShare Management Services, and the open source FOLIO project. These products do not include a patron-facing interface limited to the traditional library inventory; instead, they rely entirely on a discovery service that also provides access to a broader realm of relevant content. The vendor or developer may bundle its library service platform with its own discovery service, but there is some flexibility in how these two types of products are implemented. Despite the possibilities for mix-andmatch deployments, the vast majority of libraries opt for a bundled package from a single vendor.
Index-Based Search Dominates Academic Libraries
Library collections almost always extend beyond the materials managed within their ILSs. An ILS manages materials such as books and journal subscriptions but does not describe more granular information resources (such as individual articles, book chapters, or research reports). The limitations of online catalogs led to the emergence of index-based discovery services about a decade ago. These products rely on massive indexes that represent almost all the body of scholarly and professional materials of interest to libraries. As these products have developed, the gaps of missing content have narrowed, although there continue to be points of differentiation in coverage among the major products. Index-based discovery services have been widely implemented in academic and research libraries. Major products include Primo and Summon from Ex Libris, WorldCat Discovery Service from OCLC, and EBSCO Discovery Service. Although exact statistics are not available, I estimate that these services have been implemented by 80% to 90% of academic libraries in the U.S.
Index-based discovery services have had a decade to mature, but they still face new challenges. The body of scholarly content expands rapidly, fueling the need to continually develop agreements with publishers. Expectations for access to current materials drive the need for ever faster velocity in the ingestion of currently published resources. Changes in the scholarly publishing industry also have ramifications.
There is a seismic shift toward OA taking place in the scholarly communications arena, and transformational publish-and-read agreements between publishers and library groups have major implications for discovery services. The genre of index-based discovery services, to a large extent, is based on a business model of subscriptions. The central index encompasses all material, with each institution's implementation configured to provide access to the materials covered by their subscriptions. Libraries select their portfolios of subscription packages from a comprehensive knowledgebase of the global potential offerings so that patron searches can be scoped to the materials available to them. This scope can also be expanded to non-subscribed materials, with the expectation that these items can be requested through interlibrary loan or document delivery services.
OA materials require different treatment. They should be made available to all searchers regardless of whether they are covered within the library's body of subscriptions. This functionality depends on definitive indicators for OA materials. For pure OA journals, these indicators might be set automatically. But for hybrid journals, indicators must be set individually. NISO has published the Access and License Indicators recommended practice, which describes the metadata elements publishers can use as they provide content to discovery services (niso.org/publications/rp-222015-ali). Universal implementation of these indicators will be essential to the discovery services ecosystem as OA gains stronger momentum.
In their next phase of development, discovery services for academic libraries have ongoing opportunities for improvement in many areas. Their interfaces must continue to improve in step with the current user experience and become more easily used by mobile devices. The ways in which related records are grouped and the ranking of search results will also evolve to become more intuitive and understandable. Preferred conventions for web and mobile interfaces evolve continually, and it is essential for library products to stay current.
Another important area of interest relates to finding available copies of articles residing behind publisher paywalls not covered by the library's subscriptions. Researchers will often place copies of their papers on institutional or disciplinary repositories even when the authoritative version is published in a subscription-based journal. Services such as Unpaywall, Kopernio, and Lean Library facilitate gaining access to OA copies of articles when available. These services can be deployed via browser plugins and are increasingly directly integrated within library-provided discovery services.
Despite constant improvement of library-provided discovery services, patrons mostly use other means to find resources in support of their studies and research. Google and Google Scholar tend to be much more popular starting points for academic research than the library's website or discovery service. The familiarity and convenience of these general search tools seem to matter more than the high-quality results and more sophisticated capabilities seen in the discovery services from the library. Ongoing improvements in these products are not likely to substantially change the use patterns.
Even though library discovery services may not always play the leading role in the way that patrons gain access to research materials, I do not anticipate any diminishment in their implementations. Even when patrons begin research elsewhere, they may also need more powerful tools in later stages of their research. Libraries, with only a few notable exceptions, seem unwilling to divest discovery entirely to the consumer sphere. Librarians value tools under their control that can be configured to conform to institutional privacy policies and that draw their results exclusively from vetted resources. Libraries make enormous investments in their collections and appreciate any technologies that are able to maximize their use.