If one picture is worth a thousand words, full-motion video fills volumes. Digital video, while an exciting media, carries a number of challenges in the arenas of storage, preservation, and access. If it can be harnessed, digital video can deliver content in ways that far exceed the capabilities of text, sound, or still images alone.
Digital video has been a big part of my work in recent months. I've been given the task of developing a strategy for converting a large collection of material that we own on analog videotape into digital form. It's been a major learning experience, challenging me to gain hands-on experience with various digital video applications as well as become familiar with the related standards, file formats, compression schemes, and the like. In this month's column I'll try to pass on some of the basics that I've learned.
Storage issues and options
One of the challenges related to digital video that becomes apparent very quickly involves the vast amounts of storage it demands. From text, to still images, to sound, to full-motion video, multimedia options form a continuum of increasing needs for storage and bandwidth. Video, by far, is the media with the largest appetite for storage. In its most simple form, digital video consists of a series of bitmapped images strung together in rapid succession. Without compression, an hour of raw video might consume as much as 100GB of disk space. With only 10 hours of content per Terabyte, storing raw video quickly proves itself as unpractical. Fortunately, compression can make a dramatic difference, squeezing video files down as much as a hundred-fold. But even at 1GB per hour, a large collection of digital video still requires a significant amount of storage.
Recent advancements in storage technologies make it much easier to accommodate the requirements of digital video. With 80GB hard drives commonplace, it becomes possible to work with fairly large amounts of video data on a computer workstation. Storage area networks (SANs) and large-scale Network Attached Storage (NAS) devices are well suited for managing large collections of video content on an enterprise-wide network. Through a NAS or a SAN, storage capacities of dozens, or even hundreds of Terabytes can be accomplished—though not necessarily inexpensively.
DVD offers another alternative for storing and distributing digital video content. In the same way that CD's revolutionized the packaging of sound, DVD is exploding as the preferred media for video content. Not only have commercially-pressed DVD's made a tremendous headway toward displacing VHS tapes, they have become much more accessible for distributing locally produced content or archiving digitized video material. DVD-R drives have come down in price from well over $1,000 a year ago to just a few hindered dollars. Prices of DVD media have also declined, with some types available for just over a dollar per disc.
When creating your own DVD's you need to be aware that many different formats prevail. DVD comes in both authoring and general formats. Authoring discs allow full access to the media, allowing the creation and copying of commercial content with copy protection. The cost of authoring media is quite high, making it impractical as a means of duplicating commercial DVD's. General media does not allow access to the part of the disc needed for copy protection, but is much less expensive. Also keep in mind that there are DVD-R drives and media that can write to the media once, and DVD-RW that can write and re-write thousands of times. DVD-ROM drives and cartridges treat the media almost like a hard drive, allowing hundreds of thousands of write and re-write cycles. Make sure that your media type and drive are matched. For most video digitizing projects in libraries or archives, DVD-R on general media is likely the best choice. The use of write-once technology ensures that the digital content cannot be easily overwritten or altered once burned onto the disc. The most common DVD formats offer about 4.7 GB per disc, enough space to accommodate about two hours of relatively high quality compressed video.
Compression basics
I've already noted the value of compression of video files. Compression makes video manageable in terms of its storage requirements and in the bandwidth needed to transfer it across a network from one computer to another. The main problem with compression lies in mitigating the amount of information lost in the process. While the ideal compression scheme would be loss-less, in reality, the process is “lossy.” Most of the processes available for compressing video data end up loosing a certain amount of content that can't be restored. Done well, however, compression can be accomplished in such a way that there is no perceptible deterioration of picture and sound quality, despite any theoretical loss.
The compression schemes that apply to video have been standardized by the Motion Picture Experts Group, and go by the name MPEG. The original MPEG-1 was designed for video-CD, and is designed around fairly low resolution, and a bit-rate of only about 1.5 mb/second. Given the relatively low-quality video it produces MPEG-1 finds little use today.
MPEG-2 rules as the dominant video compression standard. It achieves excellent results in reducing the size of files while producing very high-quality video. Commercial DVD's, satellite-based television transmission systems, and many other applications rely on MPEG-2. While the details are quite complex, the basic idea of MPEG-2 compression lies in the compression of the bitmap of pixels that comprise each frame of the video, and in storing only information that changes in each subsequent frame. Some frames are considered reference frames and are stored in their entirety, using the same type of compression as a single still image. These reference frames are called I-Frames, using only intra-frame compression. Subsequent frames are derived from the I-Frames and can be either P-Frames (predictive) or B-Frames (bi-directional) and contain only information that has changed relative to an I-Frame. A sequence consisting of an I-Frame and its associated B and P frames are called a Group of Pictures, or GOP. While we can leave the technical details of the compression scheme to the die-hard engineers, it is helpful to understand in general terms how the compression is performed.
The compression of video requires significant computational resources. While software does exist that can perform MPEG-2 compression, the most practical approach makes use of specialized hardware. An MPEG-2 encoding card can be installed in a computer workstation to accept input from a video camera, tape deck, or other video signal source and compress it into an MPEG-2 file in real time. The first generation of MPEG-2 encoding cards could cost as much as $20,000. Today, reasonably capable MPEG-2 cards range from $500 - $2000. One of the MPEG-2 cards that I have been evaluating with good results is the Apollo Expert from DV Studio Technologies (www.dv-studio.com).
You can make selections in how an MPEG-2 file is created, balancing the quality of the video and the size of the resulting file. Variables include the horizontal resolution, the bit rate of the video stream, whither to encode at a variable bit rate or a constant bit rate. Our experience is that a bit rate of about 4,000 bits per second yields about the quality of VHS tape. The flexibility of MPEG-2 makes it well suited for projects that involve the preservation of video content. The bit rate and resolution of can be adjusted achieve the appropriate level of quality in the resulting video file.
While MPEG-2 works well for creating high-quality digital video for preservation, it may not always be the best choice for delivering video content to end users. One of the disadvantages of MPEG-2 lies in the need to acquire software to view the files. Most home and office computers do not come equipped with the viewers for MPEG-2. Until very recently the standard media viewers such as the Windows Media Player and the Real Player could not play MPEG-2 files. Elecard, Inc. (http://www.elecard.com) offers a shareware MPEG-2 Player in addition to their commercial version.
But the real problem with using MPEG-2 to deliver video content lies in bandwidth constraints. While a 6mb/second MPEG-2 file will play nicely on a 100mb/sec LAN, it exceeds the bandwidth available to most Internet users.
Streaming Video
For the delivery of video content over the Web, some sacrifices have to be made in quality due to the limitations of network bandwidth. While MPEG-2 video can be full-screen, video transmitted on the Internet tends to use much smaller formats and the number of frames per second can be much slower. This lower-quality video seems to be well accepted. Typical applications include news clips from national news networks and their local affiliates, movie trailers, and other entertainment content.
The preferred way of delivering video content over the Internet relies on streaming technologies. The key characteristic of this transmission method lies in the user's computer playing the content as the server transmits it. In a true streaming environment, the end user does not end up with a copy of the video, preventing any unauthorized re-use of the content. The advantages of streaming for the end-user lies in the immediacy of viewing the video without first waiting for the whole file to download. For content owners, streaming video helps them maintain control since they can provide access without distributing countless copies of their work.
Streaming video involves its own set of servers, file formats, and players. If you place a digital video file on a Web server, it will be available for download, but cannot be streamed. It takes a streaming server, which relies on special protocol called Real Time Streaming Protocol (RTSP) instead of the HTTP protocol used by Web servers. Digital video files must also be encoded into a format specifically designed for streaming. In most cases streaming video files are created from other formats such as MPEG-2 or DV through transcoding software. Streaming video files must be encoded according to the type of streaming server involved and according to the bit rate that will be used as its transmitted by the server. In many cases multiple versions must be created to accommodate different bit rates associated with the bandwidth capabilities. It's common to create a low-bandwidth version for dial-up users, a moderate version for DSL and cable modem users, and a fast version for those with access through high-performance networks.
While MPEG-2 is a standard that is widely deployed in for high-quality digital video, the realm of lower-bandwidth streaming video is dominated by a set of proprietary formats that compete with each other for dominance in the delivery of video content to the consumer audience. The three main contenders include Microsoft with its Windows Media File format, Apple with its QuickTime format, and Real Networks with its RealMedia format. Each of these vendors promotes its own proprietary streaming video format, its streaming media server products, and its own media player software.
This multiplicity of proprietary file formats and the need to encode for multiple bit rates to accommodate bandwidth options adds significant complexity to the process of delivering streaming video. As a provider, you have to decide if you want to offer your content in only one of the streaming formats or if you want to support multiple versions. Fortunately, the major media players from Microsoft, Real, and Apple can generally read each other's file formats, meaning that most users will be able to receive your content even if you settle on a single format.
Complex but manageable…
It wasn't that long ago that working with digital video would have been beyond the reach of most libraries. The equipment was expensive, the storage requirements were prohibitive, and the level of specialized expertise required was high. But today, advancements in several areas of technologies have converged to make digital video much more accessible. Though many complexities remain, such as the multiplicity of streaming formats, its easer now than it ever has been before to integrate digital video into a library's Web environment. Even without a large budget or an in-house specialist, a library can begin to experiment with digital video and undertake small-scale projects. It seems to me that video will steadily increase as one of the ways of conveying information on the Web. The more that libraries gain experience with this media, they will be better prepared for a future when digital video is considered a mainstay of the Web.
