1. The National Digital Heritage Archive Programme
1.1. What is the NDHA Programme?
NDHA stands for National Digital Heritage Archive. The NDHA Programme is working in partnership with Ex Libris Group and Sun Microsystems (Sun) to develop a digital archive and preservation management system.
A successfully completed NDHA Programme will ensure the National Library of New Zealand has the infrastructure and technology it needs to provide ongoing access to and preservation of digital heritage collections under the guardianship of the National Library and the Alexander Turnbull Library.
Digital preservation is a key factor in the development and enhancement of the Ex Libris Group products, which run in 32 national libraries around the world. The goal of this large-scale, trusted digital preservation system is to meet the cultural heritage preservation and management needs of national libraries now and in the future via a commercially available system.
The work will continue in collaboration with an international peer review group (PRG) made up of recognised thought leaders and innovators from library and academic communities. This group has served as an independent resource for the NDHA Programme, consulting on the scope and design of the initiative and its compliance with emerging industry standards. Consultation with the PRG will ensure that the system conforms to general access and archiving trends so that it may be broadly applied.
2. The System
2.1. What exactly will the NDHA system be?
The National Library of New Zealand requires a system that secures the integrity, authenticity and therefore trustworthiness of digital material deposited with the National Library while integrating with other software applications that it uses to deliver digital library services.
The NDHA will use a standards-based, commercial digital archive and preservation software system developed by the National Library’s software supplier Ex Libris Group based on functional requirements developed by the National Library. It will operate on Sun Microsystems hardware in line with the National Library’s enterprise technology strategy. The hardware and software is designed to be scalable over time as the digital collections grow.
The proposed system will be replicable in other organisations wishing to preserve and give access to information, and will serve as an international model for the implementation of digital preservation and ongoing management of digital materials.
2.2. How will the NDHA integrate with the National Library’s other systems?
The NDHA must integrate with other software applications, which the National Library uses to deliver digital library services to users. Concurrent with the NDHA’s development, the Library will develop integration software and tools to integrate the NDHA system with the Library’s collection management systems and access products.
The National Library is also developing improved and new business processes for ingest of digital material from publishers, donors, and the Library’s internal image and sound digitisation programmes and web-harvesting activities. New staff positions will research and manage digital preservation and the handling of digital material collected through legal deposit and unpublished donations.
2.3. What standards will be applied to the system?
Digital preservation is a new field and standards are emerging. The Open Archival Information System (OAIS) reference model, an International Organisation for Standardization (ISO) reference model, is one of the first recognised standards and the development partnership has agreed to apply it.
The OAIS highlights standardisation as the key method to preserving data in the long term, but acknowledges that emulation and migration are also key components. OAIS provides a framework for capturing information packages in a way that they may be stored (migrated or emulated) according to two layers of metadata. In order to keep metadata complexity low, any archive following OAIS is asked to implement a dependency classification system for structural, semantic, and representational metadata.
2.4. What digital materials will the NDHA store and how does the National Library collect them?
Digital materials that make up the digital heritage collections under the National Library’s guardianship come to the National Library via four main sources: legal deposit, web-harvesting, donors, and digitisation programmes.
The National Library of New Zealand Act 2003 extended legal deposit to include electronic publications, and came into force in August 2006. Since then, digital publications have been received by the National Library from publishers via an online submission process (e.g., online publications) or by the receipt of physical items (e.g., CD’s DVD’s) or collected by a Web-harvesting programme.
Unpublished digital heritage material generally comes to the National Library from donors via the same mechanisms. One of the largest ingest streams for digital material is from internal sound, audio-visual, image, and print digitisation programmes.
2.5. What will the commercialised product be called?
Ex Libris Group is currently exploring naming options for the commercial system.
2.6. Why was the Ex Libris Group platform not considered previously?
Ex Libris Group did not respond to the National Library’s RFI and was therefore not considered during the initial procurement process. Prior to that, when the NDHA business case was presented in early 2004, a number of existing products were assessed for their potential as a base for a digital repository and preservation management solution.
DigiTool, launched in October 2001, was one of them. At the time, none of the products was assessed as offering the long-term solution that the National Library required. In the interim, DigiTool, now at version 3.0, has had the benefit of significant commitment of resources to its development and now meets more of the NDHA’s requirements than it did in 2003/2004. Ex Libris Group is committed to investing significant resources to develop the Digital Preservation System, aimed at fulfilling all of the NDHA requirements.
2.7. When will the NDHA be operational?
The NDHA will be implemented in two phases. The NDHA will be operational for ingest, storage and access functions by the end of 2008, at which time it will be made available commercially by Ex Libris Group. The balance of functionality, including support for digital preservation capability, will be in place by the end of 2009.
3. NDHA Programme Partnership Structure
3.1. How did the initial partnership come about?
The passing of the National Library of New Zealand (Te Puna Mātauranga o Aotearoa) Act 2003, coupled with funding approval for the project and the establishment of a defined timeframe for delivering a solution, provided the National Library with a limited window of time to achieve its goal of delivering a reliable archive for the preservation of digital content and to ensure ongoing access to it.
In July 2004, a core project team was assembled, governance and oversight structures established, and the charter for the NDHA Programme developed and approved. Developing the solution internally was not a viable option, nor was pairing with a vendor to deliver a customised solution. Ultimately, the National Library elected to work with a third-party vendor who could devote their resources to the development of a digital access and preservation management system based on the NDHA Programme’s extensive functional requirement groundwork.
As a result of robust procurement processes and building on the strong relationships the National Library already enjoyed with both Endeavor Information Systems (acquired by Ex Libris Group in December 2006) and Sun, the three organisations reached an agreement to partner on the development of this project in August 2006.
3.2. How did the partnership with Ex Libris Group develop?
Francisco Partners (FP) completed acquisition of Endeavor Information Systems Inc. on 22 December 2006 and merged Endeavor with Ex Libris Group. Ex Libris had earlier been acquired by Francisco Partners, a technology-focused private equity fund.
Prior to the merger, Ex Libris and Endeavor products were in direct competition. The merger of the two companies combined product lines and market share making the Ex Libris Group the second largest player in this sector. The combined company now possesses a global reach with significant operations in North and South America, Europe, Middle East, and Asia-Pacific with a suite of library software and service offerings deployed at more than 4,400 institutions.
Endeavor and Ex Libris completed analysis and review of their combined product portfolio in early 2007. The National Library and Ex Libris met prior to the final acquisition announcement to discuss the future of the NDHA Programme. The outcome of that meeting was a commitment by Ex Libris to honour Endeavor’s contract, potentially using Ex Libris' DigiTool product as the base rather than undertaking a complete new development, or to negotiate a settlement.
A gap analysis, mapping the NDHA requirements to DigiTool, began in early January 2007 and was completed in March 2007. Upon completion of the gap analysis, Ex Libris submitted a proposal to the National Library regarding the development of a new Digital Preservation System based in part on DigiTool architecture. This proposal was considered and an agreement made to complete the NDHA development was reached.
3.3. What other relationships do the partners have?
Since the development of the ALEPH integrated library system on the Sun® platform, Ex Libris has become a Sun Higher Education Partner. Over the years, Ex Libris and Sun have worked closely to provide real solutions for the constantly evolving library market. Currently, 80% of Ex Libris customers run on Sun operating systems. 90% of Ex Libris customers running the Voyager Integrated Library System run Sun operating systems.
3.4. How is the international community involved in the partnership?
The formation of the Peer Review Group (PRG) was initiated by the partnership. The PRG consists of recognised thought leaders and innovators from the international library and academic communities, all with institutional expertise in the areas of digital preservation and permanent access. The PRG serves as an independent resource for the NDHA Programme.
In addition to validating the scope and design of the initiative, as well as its compliance with industry standards, the Peer Review Group is charged with ensuring that the NDHA is developed in concert with general access and archiving trends, so that it may be broadly applied at other research organisations in the future. Ultimately, the PRG's mandate is to guide the partnership and the resulting creation of a commercially viable solution.
Members of the Peer Review Group represent such institutions as the British Library, Cornell University Library, the Getty Research Institute, Helsinki University Library, Koninklijke Bibliotheek, National Library of China, Singapore National Library, the University of Glasgow, and Yale University.
4. NDHA Programme Partners
4.1. National Library of New Zealand
The National Library of New Zealand Te Puna Mātauranga o Aotearoa has a vision of New Zealanders connected with information important to all aspects of their lives. The National Library and the Alexander Turnbull Library preserve and provide access to New Zealand’s documentary heritage so that future generations of New Zealanders can explore and enjoy it. The National Library also provides resources to schools to support teaching and learning in New Zealand, and fosters relationships with communities, including Māori, in New Zealand and throughout the world.
4.1.1. Why is this a National Library project?
The passing of the National Library of New Zealand (Te Puna Mātauranga o Aotearoa) Act 2003, in May of that year, required the National Library to collect, through legal deposit, preserve and make accessible New Zealand digital publications, along with the traditional paper publications, in ways that ensure current and future access to documentary heritage under the National Library’s guardianship. This highlighted the need for the National Library to put in place a reliable archive for the preservation of digital content to ensure its ongoing access.
4.1.2. What will the NDHA offer the National Library’s users?
The completed NDHA will ensure the National Library has the capability to collect and preserve in perpetuity New Zealand’s digital heritage in line with agreed collection policies, and preservation and interoperability standards. The National Library will make its digital heritage collections accessible in perpetuity in accordance with relevant legislation and other agreements while respecting the rights of the producers.
In simple terms, the NDHA will be the National Library’s storehouse for digital assets such as Websites, CDs, DVDs, images, and digitised copies of print assets, that make up our digital heritage collections. The NDHA will preserve these items in their original form and ensure that they can still be viewed, listened to, and explored in the future, even if the original technology has become obsolete.
4.1.3. What experience in digital preservation does National Library have?
Since 2000, the National Library has successfully initiated a programme of digital preservation initiatives, including development of a preservation metadata schema and data dictionary, development of software for the automated extraction of preservation metadata from key file formats, and involvement in international activities such as PREMIS: Preservation Metadata Implementation Strategies.
The Library has also implemented an Object Management System to test some elements of the digital preservation process and developed, along with the British Library, a Web curator tool for the harvesting of Web sites under the auspices of the International Internet Preservation Consortium.
These milestones, achieved under the NDHA Programme, are already contributing directly to the achievement of the government’s digital strategies and the National Library’s digital heritage responsibilities. The National Library is also proactively sharing its growing digital preservation knowledge and expertise with the rest of the public sector.
4.2. Ex Libris Group
Ex Libris is a world leader in the development of cutting-edge library automation solutions for academic, national, and research libraries worldwide. Since its establishment over two decades ago, Ex Libris has continually been ahead of the technological wave in providing right-on-time solutions to the evolving library automation market.
Today, Ex Libris boasts a suite of products that provides libraries of every type and size with a complete solution for all material types—from traditional print publications through the gamut of digital and electronic resources and data. Ex Libris Group’s flagship ALEPH 500 and Voyager integrated library solutions are in use at over 3,000 sites worldwide.
Other products from the Ex Libris suite, deployed at more than 1,300 sites, focus on the digital library and offer state-of-the-art, user-centric solutions for managing and providing informed access to electronic resources and digital assets.
A consistent ground-breaker in the delivery of solutions such as SFX, DigiTool, Verde, and Primo, Ex Libris continues its constant analysis of the market and the latest in technological developments to deliver systems aimed at empowering libraries to better serve the ever-changing needs of information patrons and the institutions that manage and preserve it.
4.3. Sun Microsystems
Sun is a leading provider of open network computing solutions to colleges and universities around the world, powering academic, research and high performance computing systems, campus administration, digital libraries and student instruction systems. In addition, Sun is committed to connecting the world's students to the Internet, beginning with primary and secondary schools and extending to all levels of higher education. For information about Sun in Education, visit www.sun.com/edu
Sun Microsystems, Inc’s (NASDAQ: SUNW) singular vision -- "The Network Is The Computer"(TM) -- guides Sun in the development of technologies that power the world's most important markets. Sun's philosophy of sharing innovation and building communities is at the forefront of the next wave of computing: the Participation Age. Sun can be found in more than 100 countries at www.sun.com
4.3.1. What is Sun contributing to the NDHA Programme?
Sun will provide architecture expertise, hardware and where relevant, software. Sun developed early reference architecture for the NDHA that provided a comprehensive overview of the necessary infrastructure for running and maintaining a digital archiving and preservation system.
4.3.2. What is the significance of the Sun Centre of Excellence Digital Futures in Libraries agreement?
Agreed in August 2004, the National Library became a Sun Microsystems Centre of Excellence for Digital Futures in Libraries. Ex Libris Group is the third party partner in this relationship.
The National Library already leverages Sun's local consulting expertise, as well as Sun NZ’s partner Gen-i. This association with Sun gives the National Library access to intellectual property, innovation, research and development that reside within Sun Microsystems globally, supplementing the National Library's capacity and drawing it into Sun’s Centre of Excellence community.
The partnership will be contributing to an environment and standards for managing and operating digital repositories that will be applicable to the whole digital preservation community.