Discovery services have come a long way in improving the ways that libraries provide patrons with access to their collections. This genre of software has become pretty well established since its introduction almost a decade ago. Libraries are increasingly abandoning the ILS online catalog modules or relegating them to specialized use. Discovery services take a broader view of library collections, attempting not just to address the materials managed directly in the library's ILS, but also to find ways to incorporate the content available to users by virtue of subscriptions to electronic resources and to specialized collections of digitized documents, images, sound recordings, and video.
These new waves of discovery products have also modernized the interface, incorporating features well established on web destinations outside the library space, such as relevancy ranked results spanning heterogeneous content formats, with faceted navigation to help users hone in on specific items of interest. The interfaces now routinely deliver a more attractive visual experience, beyond cosmetic presentation, and also include helpful functionality such as virtual shelf browsing.
Pros and Cons
This realm of discovery services is not universally accepted. Many librarians appreciate what these new products are trying to achieve but also regret some of their weaknesses and the loss of some capabilities on which they depend. I continue to hear voices of dissent from not just librarians, but also from groups of library users with more complex requirements. In talks that I give about discovery services, I often mention that while users may love the new search tool offered by their libraries based on one of these new discovery services, many librarians and specialized users may even see them as a step backward. Content providers also express some reservations. They are concerned about a loss of control in the way that library users experience their resources and that their resources may suffer variations in usage levels, which in turn will have an impact on whether or not libraries will maintain their subscriptions. As these products advance to the next generation, it will be important to introduce features that will either directly address these concerns or ameliorate them in other ways.
For now, most libraries deal with the positive features of discovery services and their shortcomings by offering the discovery service as the first-line approach for most searchers and making their older online catalogs available to library personnel and users with specialized needs. This strategy can only work as a short-term or medium-term stopgap since these online catalogs will eventually find their demise. Very few of the online catalog modules of ILS products see continued development, and most have not been updated for many years. All new development is currently directed toward discovery services or new generation discovery interfaces. The new slate of library services platforms (including OCLC WorldShare Management Services; Ex Libris Alma; Sierra Services Platform from Innovative Interfaces, Inc.; and Kuali OLE) will not include the traditional concept of the online catalog but will rely on one of the discovery services for patron access.
While I think that discovery systems have made great improvements over the online catalogs that came before them, we will need to have an ambitious vision of how they will be further developed in the future. Before online catalogs disappear entirely from the scene, it's essential for these discovery services to continue to evolve in ways that will better meet the needs not only of casual users, but also of specialized researchers and especially the needs of librarians in all their various roles, such as information literacy, collection development, and reference support. I also see that the next generation of discovery interfaces must break past the bounds of discovery and content delivery to seamlessly embody a more robust representation of the services libraries offer.
Toward Comprehensive Coverage
Thanks to the recent movement toward web-scale index-based search, the scope of search has expanded dramatically. Library users are presented with a single search box from which they expect to be able to find content from anything available to them. Unfortunately, none of the discovery services can really live up to these expectations, especially for libraries with large numbers of electronic subscriptions.
One of the most important areas of development will be to close the gaps on content not yet available in discovery systems. The problem isn't technical; it's a matter of business decisions and strategies. Some publishers and aggregators prefer to see their content made available through interfaces under their control rather than through a thirdparty discovery service where they have no particular way to influence the ranking or positioning of their content in search results. I personally believe that the time will soon come when any given publisher's content will be disadvantaged if it's not indexed by the major library discovery services. In the end, the advantages, or not, of discovery services to publishers will be seen in the usage statistics from the libraries that make use of these services. If discovery services live up to their promise, publishers should see increased use of the content they license to libraries. In my ideal dream world, discovery services offer significant advantages to publishers as well as to libraries and their users.
But to the extent that important channels of content remain outside of the scope of discovery services, library users lose out. Either they remain unaware of items of interest that are available to them that were not in their results or they work hard to assess what's covered or not covered in the discovery service and then access the remainders separately. Until discovery services achieve near comprehensiveness of coverage, they will not necessarily fill the needs of serious researchers. It's not until discovery services truly provide access to a comprehensive representation of all the library's collection components that they can achieve their true potential.
The comprehensiveness of the coverage of a discovery index needs to be easily verified. Libraries should be able to know exactly how well their collections are represented and the extent of indexing available, such as full text or citation. This transparency is essential to effectively evaluate the coverage of the competing discovery services relative to any given library's collection. Identifying the best practices for achieving this kind of transparency is one of the goals of the Open Discovery Initiative, a workgroup recently launched by the National Information Standards Organization (NISO).
Ever Improving Relevancy
As these discovery services incorporate larger volumes of content into their indexes, the methods for ordering search results become an incredibly important factor in their effectiveness. It's essential for the discovery service to present results so that the most important items rise to the top. The discovery services available today have already made a great deal of progress in the way they order search results. I see the need to continue to improve the relevancy algorithms so that even the most sophisticated searchers will have confidence that they will be presented with the best representation of the resources available from the library for their research.
Taking relevancy to the next level in sophistication will have to go far beyond the ways that keywords match the query. Search systems will have to rely on an expanding set of factors or signals to do justice to the nuances of importance represented in the realm of library content resources. Calculating relevancy for library discovery services is an incredibly difficult proposition.
On the web, relevancy is based on the full content of webpages and such factors as inbound and outbound links and the frequency on which links are clicked by users. For library discovery services, in many cases the indexes have access to only thin metadata about an item, such as a book or an article, providing only a bare minimum of data for the search and relevancy process. In an ideal world, discovery services would have access to the full text of all the materials that comprise the library's collection. But the most important improvements to relevancy in a library context will involve incorporating other factors that measure the importance of any given result candidate relative to the user's query. The web search engines have taken an increasingly personalized approach to search, altering the relevancy rankings according to qualities inferred by physical location, previous searches, and click- throughs. Library search will also benefit from a more personalized approach. Information such as the user's academic department and previous search history can be great clues regarding the kinds of materials that would be considered most relevant in the user's search results.
Seamless and Unified Interface Across All Service Offerings
I worry that libraries are still in the mode of offering a menu of different applications that are only loosely aggregated together on their websites. That isn't the way that the rest of the web works. Any major web destination consists of a thoroughly unified interface entirely free from any feeling that the users are moving from one underlying application to another as they perform different tasks. Even the most complex sites, such as Facebook have been able to blend an incredible number of features and services into an increasingly complex, but inherently intuitive, interface. Even though we read massive grumblings every time that Facebook makes a change in its interface, things always settle down as users acclimate to new ways of doing things. At least part of the success of these interfaces lies in their consistency.
The discovery interface tends to be one of many different components of a fragmented library website. As visitors select different options on the site, they tend to be handed off from one application or product to another. Contextual information related to the user's visit, such as identity or login status, session history, and the like, may or may not survive the handoff. Each of the applications or website components might follow different navigational and interface conventions. I've found very few library websites with a unified user experience across all the different service offerings. Some of the different components that typically exist within a library website, each with its own user interface, might include the traditional online catalog, a discovery interface, interlibrary loan or document delivery request system, subject-specific guides to library collections, link resolution and recommender systems, ebook lending services, staff directories, informational pages about the libraries, digital collections, news and events, and probably many others. While I think that libraries have become quite used to this fractured approach, our users would benefit from the radical integration typical of other web destinations.
The solution to the problem of the fragmented interface isn't simple. To accomplish an integration of the library's web presence effectively, there needs to be a single presentation layer that owns the user experience across all the different service options, even when each option is based on different applications or products. Unfortunately, each of these products comes with its own user interface, which isn't easily separated from its underlying functionality.
I would suggest that one way to achieve a more unified library presence would be through separation of backend functionality from user interface. In addition to any delivered user interface, each application would also offer a set of APIs that would allow the library to take advantage of its functionality through its own user interface layer. Librarians who want to take more control of their users' experiences could work with the APIs to construct their web presence. Such a task would involve a pretty complex web development project and would not necessarily even be possible until all the many different library-oriented products and services open up their capabilities through APIs.
I also see bringing together a wider view of the library's content and services as an opportunity for a future generation of library discovery services. These discovery services have been working toward better integration across an increasingly broad view of content and for an array of delivery options. Extending the functionality of these discovery services to a more comprehensive library portal would be a natural opportunity for future development.
It's a pretty big step for the library to consider any product as a full website replacement. The key to success for any such library portal product would be to give the library complete control over layout, organization, navigation, and presentation and the ability to plug into each of the different genres and brands of back-end infrastructure employed by librarians.
Social Discovery
I also see that the next generation of library discovery will bring in an increasing set of social characteristics, going far beyond the My Account and personalization features available today. Some existing discovery products have taken on more social character than others, especially BiblioCommons, an increasingly popular option for public libraries. But I see this as an especially appealing opportunity for academic libraries as well. A user's social network can provide additional data that could be applied to how results are ranked and used in the identification of resources not necessarily likely to come up in search results. User-to-user interactions could supplement assistance given by librarians in identifying items of interest or answering research questions.
The ability for library users to share and collaborate within the library's web presence seems to me to be an obvious path of development as we look toward the next generation of this genre.
Additional Considerations
As library discovery services evolve, there are some other characteristics that I would like to see them embrace.
Predictability of search results. While discovery environments have become more complex, it's essential that search results be predictable and replicable. One of the complaints that I often hear from discovery services relates to how different results can be returned for the same query. Of course, as rankings include more personalized factors, the ability to replicate results becomes challenging. The ability to see results sans personalization will be helpful in this area.
Collection browsing. Librarians seem to really miss the ability to browse library collections through discovery services, which was offered in most online catalogs. Integrated library systems typically incorporate a system of authority control that includes name and subject headings that can then be used to view orderly lists of results. But such browsing capability can be enormously difficult in a discovery service that includes a majority of content from outside the library's ILS that is not under authority control. Names present the classic problem. In scholarly articles, names tend to be formed with the surname and initials, as opposed to the more complete forms represented in the Library of Congress authority file for names. The ability to bring browsing into the next generation of discovery interfaces that span heterogeneous content resources will depend on the implementation of consistent metadata for names and subject terms, which isn't necessarily an unsolvable problem. Given the scale of the materials involved, manual application of authorized names and headings isn't necessarily an option. Rather, it should be possible to exploit various existing resources, such as OCLC's WorldCat Identities, to layer in the consistent metadata needed to bring browsing into the web-scale discovery genre. While there may not be universal agreement that library users would make extensive use of such a capability, it would make a lot of librarians happy.
Provider neutrality. I believe that discovery services should present search results in the most objective ways possible without bias to one publisher over another. While libraries might want ways to set their own preferences over sourcing options when the same item is available from multiple publishers, the discovery service itself should not necessarily have built-in preferences. Given that some of those offering discovery services are themselves providers of content resources, libraries need to be reassured that performance is tuned in accordance to the library's preferences and not just to best position that vendor's content. There may be circumstances when these two interests are aligned and others where they may be in tension. This is another area where it's important to have transparency in the constitution of the discovery index and the way that it returns results.
Overall, I remain impressed with the progress that discovery services have made as they have begun to mature. We're well past the times when discovery layers might be considered a cutting-edge technology; rather, we're in a phase of routine deployments. The percentage of libraries implementing them is not especially high, but it's climbing rapidly, especially when we count the new-generation interfaces offered with ILS products with similar interface features (though not necessarily associated with a web-scale index).
Yet these discovery products have made more of an incremental difference in the way that libraries provide access to their content and services than what might be considered a transformational change. That's why I think that it's important to keep pushing the envelope on the capabilities of these services, possibly including some of the ones that I've mentioned here. While I'm sure that the providers of some of the current products may suggest that their current offerings are quite transformative relative to the previous generation of convoluted search tools, I'm still hoping for a much more ambitious set of technologies where libraries can excel in ways far beyond present-day capabilities.
