At the request of Jerry Specht, three veterans of the Harvard University Library (HUL) Office for Information Systems (OIS) have searched their memories and existing documentation to assemble an account of Harvard's selection, implementation and local development of NOTIS. These are
Missing from this roster is Priscilla Caplan, who also played a major role in NOTIS installation, specification and HOLLIS development, then left Harvard in 1993 for professional opportunities in Chicago and subsequently in Florida where she is less conveniently consulted than the three of us who remain in the Boston area.
The collaborators in this effort have often been surprised at the gaps in their memories and in the readily available documentation. The reader is advised not to rely on this account as definitive. Considerable detail is in the realm of best guess.
We want to make it clear that the success of NOTIS at Harvard is due not just to our efforts but to those of many, many Harvard colleagues, especially our fellow members of the library systems office (OSPR subsequently OIS) who, as a team, worked to install, maintain, enhance, and operate NOTIS-based HOLLIS over its Harvard lifetime of nearly two decades.
This section may seem an awfully long introduction to the central theme of this story, but conversion of pre-existent data from several sources and production systems to fit into a NOTIS framework is a signal feature of Harvard's experience; so here we go.
The Harvard University Library began using automated techniques in the early mid-1960's, quaint as those seem fifty years later. There were small independent projects run by the libraries of the schools of Business and Medicine, but the major focus of development lay in the central assemblage of units known as the College Library, among which Widener was the giant in size and influence. (All in all there were about 90 identifiable units in HUL at that time, many of them tiny. There has been considerable consolidation since.) The College Library Data Processing Department in Widener, founded in 1964, was the home of local automation development. It was all local then.
The first production system in use at Widener was a unit record circulation system developed in-house by Foster Palmer, at that time Head of Widener's Public Services Department. Loans were recorded on IBM punched cards and manipulated by sorting and collating machines. A nice sidelight on this pioneering effort is that it replaced an edge-notched card system implemented by Frederick Kilgour, when he worked in Widener circulation right after graduating from Harvard. Kilgour's innovation was designed to cope with increased loan volume. Palmer's system was developed for exactly the same reason. It, too, became cumbersome as circulation activity continued to increase, although crisis was deferred by progressive shifts of operations from card sorter and collator to an IBM 1401 computer and then, before 1980, to IBM 360 series machines. Because the Palmer system represented dates sequentially in four decimal digits, starting from 1 January 1963, a date later than 17 May 1990 would break it.
The second major 1960's system was conversion of the Widener Library Shelflist to an electronic file originally maintained on magnetic tape, working again from punched card input. This project has been well described in an article by Richard DeGennaro in College and Research Libraries, 1970 v.31(5) p.318-331. Its relevance to the NOTIS story is that the never-finished project produced by the end of its 1970's heyday a file of more than a million brief bibliographic records corresponding to Widener holdings as represented in handwritten shelflist entries.
Finally, the locally developed Computer Assisted Order System (cheekily known by its acronym, CAOS) went into production. The impetus for acquisition automation was the need for current information on the status of a large number of restricted funds. Again, the machinable input was punched cards. Original outputs were paper purchase orders and claims, vouchers for payment of invoices, and a variety of fiscal reports. First implemented for the 1401 in 1967, it was made considerably more powerful when it was reprogrammed for PL/1 in 1970. Sometime thereafter, several voluminous paper reports were superseded by computer output microfiche (COM). The specifics of original CAOS functionality are now obscured by its evolution around 1976 into a still more capable version known as CAPS, the Computer Assisted Processing System. A problem: both CAOS and CAPS used typed search slips as source documents, thereby requiring much of the data to be keyed twice.
Cataloging in OCLC and participation in CONSER soon provoked development of other local systems. These activities generated MARC records that were collected and maintained, in a Bibliographic Master File (BMF) and a CONSER serials file respectively. CONSER, especially, aimed to capture data representing holdings of all HUL units for a Central Serials File. A few library units had ways to produce MARC records on site without recourse to OCLC or, eventually, to RLIN. OSPR accumulated these under the rubric "Local MARC."
In that Bicentennial year and the years immediately following, a number of important things happened to affect Harvard's automation efforts.
There followed a period of transition of leadership in the University Library and in OSPR that precluded major initiatives but allowed opportunity to consider what might and what should be undertaken. It was clear that systems built to handle the needs of library units throughout the university would be needed to fulfill the mission of OSPR. The university's technical environment was advancing to a point where batch processing systems that relied on keypunch machine preparation of input data seemed antiquated, Some operations, notably the duplicate keying of CAPS input, became burdensome. The provision of CAPS information via batch reports lacked the desired timeliness.
It was also a period of technical transition outside the university. Greatly expanded storage capacities and new telecommunication possibilities opened opportunities for new ways of approaching library problems. The first Online Public Access Catalogs debuted. The earliest generation of integrated library system products appeared on the market—turnkey systems, most of them.
At the dawn of the 1980's with Professor Oscar Handlin newly appointed as Director of the University Library and Dale assuming the leadership of OSPR, Harvard was well positioned to consider improving its aging library automation efforts.
That last point begs a digression: Harvard's traditional philosophy of financial organization has long been characterized by the phrase "Every Tub on its Own Bottom," which implies a great deal of independence in the way the College and each graduate school manages its own business. A similar style of particularity pertains within the hierarchy of each school. There are agencies that provide university-wide service but they are relatively few. Operationally, ETOIOB means that if one unit chooses--it is usually a choice—to receive services from another, it typically has to pay for them (One consequence of this is an enormous chart of accounts. The various branches of the College Library alone have access to a few hundred restricted funds; adding those of other faculties brings the total to around a thousand. But the chart of accounts is only one example, practically any sort of functional entity you can name, any table you might need to build, exhibits a similar degree of granularity.)
A small study conducted jointly by Harvard's Office of Information Technology and OSPR indicated the economic feasibility of replacing CAPS with an online system, a prospect unimaginable only a few years earlier. But enlarging the scope of the online acquisitions system to serve the University Library campus-wide would be very costly, not least because of the need to purchase a large number of terminals and build a network for them. The university approached the Pew Memorial Trust for support. The appeal was successful, resulting in the award in 1983 of a grant of .2 million for the purpose.
As early as the end of 1980, an Online Acquisitions Planning Committee had been appointed to examine online acquisitions processing systems in use or development elsewhere and to specify the functions a single such system must have to accommodate the needs of the University Library's many components units.
A concomitant part of the committee's charge was to determine whether Harvard should buy an existing system or design its own. While home-grown design was attractive, rough preliminary estimates suggested that the cost of acquiring an existing system would be about two thirds of that for building one in-house. But which one would best meet Harvard's needs, or seemed susceptible of modification to do so? Three types of external CAPS replacement options were examined:
The committee's investigation identified NOTIS as the best candidate. A contract to purchase the system was signed in 1982. The strengths of NOTIS included the following points.
In the spring of 1983, with the NOTIS software installed on a mainframe at the Harvard Computing Center and the Pew grant in hand, we needed a name for the prospective system. Cilla and Charlie independently hit upon HOLLIS, the Harvard On-Line Library Information System, the acronym appropriately recalling the family name of the library's most generous eighteenth century donors. Cilla's vigorous promulgation of the name sold the idea. The Online Acquisitions Planning Committee was renamed the HOLLIS Implementation Planning Committee. The project assumed a new level of reality as more people beyond the IPC and OSPR became involved in getting ready for implementation. Work that needed to be done can be grouped under a few large headings, but these are not silos. There are many interactions required between areas.
Feedback from the working groups led shortly to modifying the NOTIS that we had received from Northwestern. A memorable early decision was to eliminate the separate copy holdings record in favor of a new repeatable field in bibliographic records. Its numeric tag would be 995. It would commonly be known and displayed as Location (LOC). That field, with its numerous subfields, served us well throughout the life of the system. Also, probably at this time, the NOTIS-3 volume holdings record was replaced by a record in the MARC Holdings format. Further divergence would come later.
It is good to remember that at this point Harvard was preparing to implement only NOTIS acquisitions, cataloging and serials functions; i.e., only technical services staff functions. Circulation and a public access catalog awaited NOTIS-4.
HOLLIS began production on 1 July 1985. (The choice of a fiscal year boundary simplified several operations, notably fund file initialization.) The Harvard Librarian issue of that September reported that:
On Day One, there were 32 administrative units in the HUL system actively participating in HOLLIS, using 132 terminals; other library units may join HOLLIS later…Like the DUC, HOLLIS is intended to bring to the University Library some of the advantages of centralization, while preserving the relative autonomy of individual units that characterizes Harvard's library system. Participation in HOLLIS is voluntary, and decisions on whether to participate and the degree of participation are made by individual units. Generalizing broadly, most of Harvard's larger libraries except the Business School's Baker Library, which already had a computerized acquisition system in place, are now ordering materials through HOLLIS.
The same source recorded initial file sizes. The Bibliographic file contained 412,756 records, of which 73.9% were monographic. "About 30 percent of these were ... in HOLLIS terminology, standard cataloguing records" the balance being "provisional." The Order File contained 101,736 records, of which 40.5 percent were for monographs. The holdings file contained 81,614 records. A modest beginning.
At the opening ceremony, Professor Handlin, who had recently been succeeded as HUL Director by Professor Sidney Verba, characterized HOLLIS as the product of "massive consultation." He was right, but we believe that extensive involvement of library staff in design and implementation decisions was an important part of the ready acceptance of HOLLIS. Change is hard, after all, but especially if it is into the unknown.
After the 1985 HOLLIS launch, we shipped our specs and code for the fund accounting and invoice generation modules off to Northwestern where they became known as "the Harvard enhancements" until their incorporation, in revised form, in NOTIS-4. It would have made little sense to retrofit the canonical NOTIS version into our working system; so fund accounting became one more NOTIS/HOLLIS divergence.
However, NOTIS-4 did contain an updated circulation module and revised OPAC code both of which we wanted to incorporate into HOLLIS.
The primary focus after the initial HOLLIS launch was on the complex task of bringing HOLLIS out of the library's back rooms to serve library users directly with a public access catalog. Achieving that goal would require several sorts of development:
The HOLLIS OPAC became available, cautiously billed as "a test version," on 6 September 1988. Staff members at a few units reported watching students come in and begin to use the new catalog without apparent difficulty. The official OPAC launch took place on 29 September amid pride, joy and relief. As in the 1985 debut, those emotions were shared by a large number of staff members who had played roles in specifying particular parts of the functionality. As least half a dozen small working groups had been formed—and given catchy acronyms by Cilla--to deal with specification particulars ranging from screen layout to duplicate detection and resolution rules. In the Director's annual report for 1988-89, Professor Verba referred to the creation of the catalog as a "gargantuan effort." The microfiche DUC could safely be retired.
The HOLLIS OPAC had several interesting features, some of which were definitely Harvard innovations. Others may have been present in a NOTIS-4 release. At this point we are not clear about their provenance, so we'll simply talk about them without implying a claim of responsibility.
Aside: One the documents examined as we prepared this report is a paper I wrote giving a fairly detailed description of LI790BAL, the central program for processing catalog mode requests, as it existed in HOLLIS just before the public catalog debut Unfortunately, differences from the distributed NOTIS version are not clearly identified. The smallest certain difference was its name, for we had developed a variant naming convention. Reading it now I am impressed by a number of things, first and least by my ability to understand it moderately well twenty-six years later. But more important, it is possible from this distance to admire the program's complexity and observe that, not only did it work, but it was constructed so well that modifying it successfully for our needs was fairly easy. In the heat of battle you don't have time to appreciate things aesthetically.
Meanwhile, while catalog development specification and development were underway, planning for circulation implementation began, too, for the Palmer system in Widener would blow up in late autumn 1989 when the due date for extended loans would fall beyond day 9999. Hence circulation presented an urgent deadline. We added another person, Caren Smith, to the OSPR Development group for the circulation project.
There was a great deal of interest from libraries other than Widener to participate in the new system; thus interest in participating in the planning. As in other applications, local unit autonomy enhanced complexity of the job. Loan and fine policies varied widely across campus. Some variation could be preserved, some required compromise. As the application that has the most direct effect on individual users, circulation raises more sensitive practical issues than others. So, social matters were more complicated than technical ones and building the necessary tables was challenging. Technical issues centered on equipment:
Widener began HOLLIS circulation in July of 1989. Several other libraries followed soon thereafter. Implementation went fairly smoothly. Memorable little software glitches arose from the difference between NOTIS-3 and NOTIS-4 specifications of the fullword Julian forms for date and time variables. The date bug arose in December when it assigned due dates earlier than the charge dates. The time bug was similar but did not surface until a few years later when we began to use the system for hourly loans of reserve materials near midnight. Hard to find, easy to fix.
In the fall of 1987, the HOLLIS network included 175 terminals, serving the acquisitions and cataloging functions. Two years later, after addition of the OPAC and circulation, the network had grown to a total of about 450 terminals, including 145 for the catalog, over 60 for circulation, the remainder attributable to growing participation in the tech services applications.
While the HOLLIS catalog, covering post-1976 cataloging, was able to satisfy a significant fraction of user requirements, the continuing need also to consult the pre-1977 Union Catalog on cards for a complete search was an annoyance both practical and strategic. Planning commenced for the conversion of the big catalog and its integration in HOLLIS. A cogent presentation by the libraries convinced university administration to fund the project, which we called, unimaginatively, RECON.
To effect such a massive conversion in a reasonable length of time, required mobilization on many fronts. The most obvious need would be getting the actual data conversion accomplished. The method chosen was to contract with OCLC to search its database, claim matching records for the appropriate holding codes, and key unfound records. OCLC worked directly from the catalog cards, which were shipped in large batches managed at the Harvard end by a team also responsible for quality assurance checking of OCLC's keying/claiming operation.
But interesting modifications were also needed to OIS operations and NOTIS/HOLLIS software and database management:
RECON also provided an opportunity to address with new automated procedures two other situations resulting from our strategy for conversion to HOLLIS.
The OCLC/Harvard RECON contract ran from September 1992 through December 1996. In that interval OCLC claimed or keyed over 4.54 million records for us and added new holdings to an additional 133 thousand. Looked at another way, the contract achieved a throughput of just under three thousand records per day, every day, throughout the contract period. Local RECON operations and the transactions generated by processes described above added non-negligible amounts of data to the input queues.
The reader of this section has suffered through a lot of detail, but RECON more than any other project shows the challenging kinds of extensions to NOTIS capabilities that Harvard could successfully implement. They testify again to the basic soundness of NOTIS and to our satisfaction in having chosen it.
Entr'acte: A few notes before going on the final chapter of Harvard's NOTIS-era story. HOLLIS generally ran well even at the furious computing pace set by RECON activity. Though the processing was automated as much as we could manage, things surely would not have gone so well had we not had an excellent manager of production operations on staff. Linda Marean managed to remain calm, or at least to retain the appearance of calmness, even during a few events that were pretty tricky and threatened to become emergencies. After Cilla left for the University of Chicago in 1993, Linda and I worked quite closely together. We consulted frequently on operational matters. When Linda announced her intention to retire, I so feared the prospect of being asked to take over her responsibilities, even temporarily, that I signed up for a stress reduction course. Fortunately for all concerned, Linda's successor was hired quickly, avoiding a gap.
One task I assumed after Cilla's departure was that of broadcasting messages to staff users in the event of service disruptions, planned or otherwise. I attempted to explain, as much as possible, the cause and expected duration of the disruption. Frequently it was possible to tell the story with a little humor. I cannot remember any specifics, but even into my retirement I continue to encounter people who remembered those message or who knew me only through them—an odd kind of fame. One message I do recall grew from my frustration with the number of staff users who would call in saying, "I see the word ABEND on my screen. What does that mean?" I posted a long message that began "When the word ABEND appears on your screen it does not imply that the sun is setting over the Rhine. What's going down is your job!" I went on to explain that the numbers or letters after the ABEND word were important and named common ones that were inconvenient but not dire. I did collect transaction dumps until the end of NOTIS /HOLLIS—quite a lot of shelf space was involved—and found time to investigate and eliminate a few frequently occurring ones.
Questions began to arise as early as 1989 at Harvard about the inclusion of electronic resources in our collections and how they could be accommodated in HOLLIS. Birth of the World-Wide Web, which happened that same year, would in a very short time recast those questions into a general consideration of the future of information transfer. Something big was happening. In retrospect the '90s appear as a revolutionary decade.
In November 1993, Dale organized a meeting at Harvard for heads of automation at eight large academic libraries to talk about the status of their “local systems” and what each institution saw as likely evolutionary prospects for its own system. The discussion could be characterized by
At the meeting, Dale stated that "Harvard would prefer to buy its next system but a look at the marketplace is not encouraging." In fact it took nearly four more years before the new generation of vendor products would mature enough to permit a productive comparative evaluation of them to begin. Even then our requirement that a candidate system must have a large installation in operation had to be relaxed.
Meanwhile local needs and expectations that the NOTIS framework could not easily satisfy were multiplying, largely because of excitement about the World Wide Web, the potential of which was being exploited globally with remarkable speed. The proliferation of personal computers, especially when they served as internet nodes, was also affecting the ways research and information seeking would be conducted.
At the same time Harvard, like many other universities, was interested in replacing mainframes with newer technology. As on other campuses, it would be very costly to be the last customer on the big machine. This prospect exerted a gentle push to migrate, but the pull of increased functionality was the stronger motivation for us to advance HOLLIS beyond its NOTIS framework. In summary, major reasons for replacing NOTIS were:
By early in 1997 the Automation Planning Committee had identified nine companies whose products seemed worth at least initial consideration as NOTIS successors. Investigation narrowed the field to the point that a year later Ex Libris (Aleph) and DRA (Taos) were the only remaining serious contenders. Taos was by a small margin the favorite and Harvard signed a letter of intent in 1998 with DRA to purchase Taos on the condition that its installation at UCLA proved successful. Its failure to do so eventually left Aleph, after one more state-of -the-market review, the clear choice. In November 2000 Harvard contracted with Ex Libris to purchase Aleph.
The deliberate pace of the selection process had made it apparent that we would still be operating NOTIS-based HOLLIS when two significant deadlines passed, so the final technical work on that system addressed those issues.
Aleph-based HOLLIS went into production 2 July 2002, with a database of 9.25 million bibliographic records, about 22.5 times as many as were involved in the July 1985 HOLLIS launch. We have not been able to track down reliable numbers for other types of records: orders, holdings, items, funds, invoices, patrons, loans, etc. at the time of migration, but they surely would reflect similarly impressive growth.
"Massive consultation," "gargantuan effort,"--those are phrases chosen by two successive Directors of the Harvard University Library to characterize the building of HOLLIS. The present narrative has focused on the activities of OSPR. On reading a draft of this paper, Cilla commented, "The first thing I remember is that we coordinated battalions of committees who wrote so many specs and did so much reviewing and testing. The iceberg to me is all that committee work under the surface and the coding and development sticking up.” This is a just observation, even if an iceberg metaphor may feel a little injudicious at Harvard. Professors Handlin and Verba were certainly thinking of the depth and breadth of staff participation when they made their remarks, and of the unifying force the creation of HOLLIS exerted upon the University Library as a whole. In 1976 when the decision to build a Central Serials File was made, Karl Nyren waggishly announced the plan in Library Journal under the headline, "Harvard Plans Cooperation with Itself." By 2002 that idea was no longer novel because of the coordinated participation by staff throughout HUL, far too many to enumerate, in the successful creation of HOLLIS from NOTIS.
Similarly the employees at NOTIS, far too many to enumerate, can claim credit for the system's success at Harvard and elsewhere, for they created and supported a good product. One of the items in the questionnaire Jerry Specht sent to former NOTIS clients in connection with the building of a NOTIS historical website solicited recollections of particular NOTIS staff members. Here are a few of ours.
Dale says, "Jane Burke and I were colleagues for well over 20 years, starting when she was at CLSI. In addition to working together on NOTIS, we had long discussions of libraries and technology over dinner a couple times a year for decades. She was aggressive, and a rigorous thinker, and she could be a tough vendor, which one had to admire although at times it certainly got your back up. While she showed good judgment of technical developments, she was not a technologist, and so at times would say/promise things she shouldn't have (but what manager hasn't?). She was smart, perceptive, full of energy and ideas (she never stopped probing for new ideas). After Jim and Velma, it was Jane that made NOTIS a success, and she was certainly one of the great contributors to the development of IT in libraries."
Jim Aagaard and Velma Veneziano of course we admired in the design of the system and in the consistency of the code. Tracey remembers Dale, Cilla, and me being especially impressed by the bibliographic indexing techniques when we first examined them in detail. I was acquainted with Velma before we were NOTIS customers. We both had interests in MARBI at ALA where we would get involved in MARC discussions.
Dale and Cilla mention John Kolman. Dale says, "I believe he did work on the system in the early days (he was one of Jim's students, I believe). Then later he returned as something like VP for development where he was a major player in the evolution of the system. John was not just smart and technically competent, but also wonderfully honest and straightforward. He really was one of the strengths of the company."
In user service matters we remember Jerry Specht and Randy Menakes as helpful and pleasant to deal with. It must be admitted that our memories are fresher of them than of other helpers, because they, like so many of their customers, eventually migrated to Ex Libris and continued serving in similar capacities.
Finally, we think of Stacy Kowalczyk not so much from her NOTIS days as from her subsequent time as a colleague in OIS, a crucial member of the team evaluating systems as replacements for NOTIS, an exponent of an "everything will be just fine" attitude, and a baker of the most wonderful assortment of Scandinavian Christmas cookies.