Solving the Video Display Problem Through Hardware

This paper describes the range of hardware acceleration techniques used by Video Graphic Adapter (VGA) manufacturers to address the performance needs of today's graphics-intensive computer environments. Attention is focused on those acceleration issues that relate most directly to the display of digital video. Finally, we describe Sonic Solutions's methods for selection and optimization of the most effective of these techniques or use in digital media playback such as our DVD Decoder Pack .

VGA Acceleration is the enhancement of computer graphic display technology through the use of specific hardware (and software) solutions made resident on the Video Graphics Adapter (VGA) subsystem. To some degree, each manufacturer of VGA technology develops and implements its own proprietary methods of VGA acceleration, but the goal-a faster, more fluid display on the computer monitor-is always the same.

VGA stands for Video Graphic Adapter. The term comes from the earliest days of the IBM PC as the official name for a class of subsystems that drove computer display terminals. The Video Graphic Adapter specification defined a baseline performance of at least 640 horizontal screen dots by 480 vertical screen dots and 16 usable colors from a palette of 256.

The main distinguishing characteristic of the VGA adapter subsystem (as compared to the earlier Monochrome Display Adapter, or MDA) was the ability to address individual screen dots. Earlier display methodologies concerned themselves with painting ANSI characters in columns and rows.

With the VGA adapter, a new world of graphical display technology was enabled. And, as graphics-intensive software began to appear to take advantage of the new display technology, computers, everywhere, chugged to a halt under the massive new demands on their processing and RAM resources. While the world waited for faster CPUs and cheaper RAM, VGA adapter manufacturers began to look for ways to speed up graphics display within the adapter, itself.

The original IBM PC made very few demands of its display hardware: Put up ANSI characters on a screen in one color in regular columns and rows. It wasn't until the Apple Macintosh and several other "desktop-metaphor" systems popularized the use of graphical elements and "point-and-click" ways of interacting that the original PC graphics architecture began to show its weaknesses.

As graphical interfaces continue to multiply and multimedia applications grow in sophistication, new challenges and opportunities are created for the companies that manufacture the VGA components of computer systems. Each manufacturer in the PC market continues to learn that competitiveness means newer, faster, more efficient and lower cost subsystems. Often, market advantage for these companies comes from implementations that are deliberately proprietary and exclusive. PC systems now differentiate themselves in the market based on the VGA or audio subsystems they contain.

It's the proprietary and different nature of these VGA implementations that complicates the lives of software providers such as Sonic Technologies. Where it once was possible to write a single program that would play equally well (or poorly) on a "standard" PC, there now are dozens of different display systems. Many now offer sophisticated hardware assistance for digital video, 3D object manipulation and texturing, and input/output options for other home entertainment devices.

Suddenly, as suppliers of media technology, we find that we must support many different VGA technologies and, in order to support them, we must first understand something about them.

General

However different individual VGA subsystems may be, they all have the same primary function: To paint a 2-dimensional representation (an image) of the computing environment on a CRT or LCD display device; and update that image as often as necessary to reflect ongoing changes to the environment.

Figure 1. Block diagram of a typical VGA system.

Display Memory

Before it can be painted onto a screen, the computing environment must be mapped into an image. Whatever method the operating system designers choose to "envision" this environment, the mapping process is always the same: Assignment of pixel data in a discrete area of memory. The pixel data consists of digital Red, Green and Blue (RGB) values.

When PC displays were character based and total system memory was limited to 1 MegaByte, display data was stored in the upper region of main system memory (> 640 kB). Understandably, it wasn't long before display data requirements exceeded this tiny storage reserve. Logically, the place to add memory for display purposes was on the display adapter, itself. Early cards offered 16, 64 or even 128 kB of memory for display purposes! Current display adapters typically contain 16, 32 or 64 MB of display memory.

RAMDAC

The key element in any VGA subsystem is the Random Access Memory/ Digital to Analog Converter (RAMDAC). The RAMDAC is a special purpose integrated circuit that reads the digital values from display memory and translates them into analog voltages (signals) suitable to drive the display device (either CRT or LCD). The RAMDAC is the single component that determines graphics system performance; by virtue of the speed with which it can draw individual pixels, complete scan lines of pixels and full screens.

Hardware cursor

Our consideration of VGA technology presupposes a graphical user interface; a Microsoft Windows user interface, specifically. This windowing interface, where user mouse activities are translated into onscreen changes, carries with it a certain amount of overhead directly related to the image of the mouse cursor. Cursor movements and imagery can be managed entirely by the operating system or through brute-force application of software, but most VGA systems now include special hardware to track and image the mouse cursor.

Blitter

Many display changes involve large rectangular blocks of pixels - a window opens or closes, a menu is revealed, etc. - that must be updated all at once. Very simple VGA implementations have no choice but to draw these areas pixel-by-pixel. More sophisticated VGA subsystems implement a BLock-Transfer (called a Blit or BLT) method for drawing, moving or otherwise updating large areas of display memory in a single operation. In a system with hardware BLT support, windows and menus appear almost instantaneously, and the VGA subsystem is able to update its display faster, while doing much less work.

LineDraw, Area Fill

Virtually all graphical user interfaces rely on common graphical techniques, called "primitives" that are combined to produce more complex displays. For instance, a window may be realized by drawing a rectangle, filling it with the color "white," adding a contrasting dark border along the bottom and right edges and a contrasting light border along the left and top edges. Each of these contributing operations uses a graphic primitive (geometric shape, area fill, line draw, etc.) to build the complex display element of a window. Again, the operating system can perform all graphic primitive operations internally, or the VGA subsystem can provide hardware-level support that dramatically speeds up this type of drawing.

Dithering

The amount of display memory available determines the absolute number of pixels that can be represented (defined as an array of n-pixels wide by n-pixels high). It also determines how many colors the VGA system can represent at any given resolution. We are fortunate to have finally arrived at a stage where display memory is inexpensive enough to support an almost infinite number of colors, but this was not always the case.

In cases of very high screen resolutions or very low memory states, displays may be limited to 256, 64 or even fewer colors. Dithering is a method of simulating more colors by scattering isolated color dots in varying patterns to trick the human eye into perceiving more colors than are actually available. Some VGA systems can perform dithering in hardware.

Figure 2. Illustration of dithering.

Color depth conversion

A detailed description of how computers represent color is well beyond the scope of this document. We assume the reader has some knowledge of color representation and accept as given that VGA displays are primarily concerned with the faithful reproduction onscreen of the colors contained within the computer's memory and defined by the running software.

In the best of worlds, display memory is sufficient to contain and represent all possible colors. A "True Color" display provides 24 bits per pixel (8-bits of Red, 8-bits of Green, 8-bits of Blue) to allow for 16,777,216 unique colors. This is, of course, fewer colors than actually exist, but more than the typical human eye can distinguish.

So, a True Color display is said to have a "depth" of 24-bits. When display memory does not permit 24-bits per pixel, the system must reduce the number of colors it represents. Other common color depths include: 16- or 15-bit (High Color), 8-bit (256 colors), 4-bit (16 colors) and 1-bit (black & white). The previously discussed dithering technique is typically applied in low color depth situations to simulate the appearance of more colors.

Again, the operating system may be called upon to determine how to render colors in low-depth modes, but it's faster and less work if the VGA adapter can handle this task on its own.

Tiling, multi-monitor displays

The windowing operating system typically uses a desktop metaphor: The display represents a physical work area. Thus, we generally can do more work if we have a larger work area. Historically, this has meant increasing the size of the display. From its original dimensions of 640x480 pixels, the computer display has grown; to 800x600, 1024x768, 1280x1024, 1600x1280 and even higher.

Recently, an alternative method of increasing usable work space has emerged. Instead of driving larger and larger displays, systems are now beginning to support multiple displays. It's now possible to operate two monitors side by side with display elements that slide seamlessly from one to the other. This is achieved through the use of "tiling."

Memory access (Caching, Pipelining, Porting)

Assuming an effectively infinite RAMDAC speed (it isn't), display updates will still be limited by the speed and type of memory used. Historically, a number of different types of RAM, with significantly different performance characteristics, has been used. Typically, the selection of RAM type involves some compromise between performance requirements and its effect on the cost of the final VGA adapter.

Standard Dynamic RAM, of the type used for system main memory is the least expensive, but also slowest performing type of RAM. Due to its relatively slow refresh time, DRAM takes longer to read from or write to. It's also single-ported, meaning that a particular memory cell can only be read from or written to at any given time.

Extended Data Out RAM improves upon DRAM in that a new read operation can begin before the previous one has finished, but it's still only single-ported and still relatively slow.

Synchronous Graphics RAM is single-ported like DRAM, but able to operate at much higher data rates (i.e.: synchronously with the RAMDAC or main graphics processor).

Multibank DRAM is an implementation using ordinary single-ported DRAM, but arranged in smaller, separately addressable banks. Being able to address different areas of display memory at the same time, or to overlap the same set of addresses with interleaved read or write calls effectively increases the bandwidth and improves performance.

Video RAM is dual-ported: It can be written to and read from in the same operation, effectively doubling the performance as compared to standard DRAM. Why wouldn't VGA designers always use VRAM for their adapters? VRAM is significantly more expensive.

Window RAM is a dual ported design optimized for use in graphics applications. It directly supports block transfers and primitive drawing operations. WRAM was first introduced in the Matrox Millennium, establishing that VGA adapter as the performance leader of its day.

Everything that a VGA adapter knows how to do (color space conversion, blitting, graphic primitives) is contained in a Read Only Memory (ROM) chip. Typically, the ROM chip has a relatively slow access time and a comparatively narrow bus path. Literally every operation the VGA adapter performs can be accelerated by copying the contents of the ROM into a separate area of main memory. Graphics calls then access the RAM-based copy instead of the slower ROM. This process is called cacheing.

In typical memory access implementations, the RAMDAC will request the data from a specific memory address and then wait for the data to come back. Only after it obtains the data for the first request can it send another request. Pipelining is the ability to issue multiple read requests in sequence, without having to wait for each to return its data. Graphics system pipelining is a feature enabled by the AGP data bus specification.

Other Acceleration Technologies

A modern accelerated VGA subsystem will also contain a variety of other components. Those that bear upon the display of digital video will be discussed presently. Those that do not (3D support technologies, for instance), will not be covered to any significant detail.

We've already discussed how graphical interfaces and windowing systems place increased demands on VGA display systems. But if the VGA adapter, accelerated or not, is able to paint large-format screens at refresh rates exceeding 75 Hz, what's the big deal when it comes to digital video? Why shouldn't a DVD title play smoothly as a matter of course?

The simplest answer is in the nature of the medium. Even a very large format computer screen (1280x1024, 32-bit color, for instance) is a mostly static image. A window may open or a column of numbers in a spreadsheet may need to be rendered as inverted (indicating a selection process), but the vast majority of pixels in the display will not change for minutes at a time. When this is the case, the VGA adapter really has very little work to do: Update in its video memory those few pixels that do change, and continue pumping the video memory data through the RAMDAC at a rate high enough to avoid the appearance of flicker.

With digital video comes a significantly more complex and challenging set of tasks. In a typical video display, all the pixels are changing all the time. For full-screen NTSC, that means 30 frames per second (fps) of new data coming into and out of video memory, through the RAMDAC and to the screen. Typically, this data originates in a form that is completely unfamiliar to standard VGA subsystems.

Frames, Fields & Color Space

As mentioned, video typically runs at 30 fps. This should pose no problem to VGA subsystems that are designed to update a computer monitor anywhere from 35 to 135 times per second (Hz). Unfortunately, video frames are interlaced.

Due to historical factors limiting the performance of early television sets, video engineers chose to divide each frame into two fields of alternating horizontal scan lines. First, all the odd lines are transmitted (taking 1/60th of a second), then all the even lines (also taking 1/60th of a second). At the television receiver, the odd field is drawn, then the even field; completing a full frame in 1/30th of a second.

Figure 3. Interlaced Even and Odd Fields make a Frame.

Also, to save transmission bandwidth, video signals use a "color difference" model to represent the colors in the image, as opposed to the computer's RGB color definition. Briefly, the video color model, Y, B-Y, R-Y , contains as its components the luminance (basically old-style black & white television) as Y, a fractional value representing the amount of blue minus the luminance (B-Y) and a fractional value representing the amount of red minus the luminance. Television receivers contain circuitry that decodes these values and calculates the third, green component. Y, B-Y, R-Y encoding allows just 16 bits of data to represent all the same color information as 24 bits of digital RGB data. It just takes more work to recover that information.

So, in the case of preparing digital video for display on a computer monitor, the following steps must be accomplished:

1. Read a digital video bitstream from a storage medium or data source (network connection).
2. (If necessary) decrypt a content-protection scheme.
3. Decompress the video bitstream.
4. Convert film frame rate to video frame rate (if the video source is 24 fps to begin with).
5. Deinterlace fields into complete frames (if the video source is interlaced to begin with).
6. Convert Y, B-Y, R-Y into RGB.
7. Deliver RGB frames to the monitor.

As developers of digital video playback software, we must supply the necessary technology to perform each of these steps - which Sonic has already accomplished with remarkable success. Or, we can look to the VGA adapter to assist with some or all of them. The remainder of this paper concerns itself with the types of help VGA adapters can offer. For a discussion of the ways in which Sonic Technologies' products can, and do, operate independently of VGA assistance, please refer to the Sonic white paper on "VGA Independence."

Overlay

One way to simplify the display of motion video on a computer display screen - particularly when that video appears in a window - is to handle it in a separate area of memory. This area, called a "surface," can be much smaller than the full display (just the size of the window, for instance), and it can even directly use whatever colorspace the video requires. At play-time, the contents of this separate memory area can be inserted, or overlaid, on the main RGB display.

Figure 4. Video Overlay Overview.

The great benefit of overlay surfaces is that they are non-destructive of the main VGA display underneath (the "Primary display surface"). If an overlay window is moved, any display information that needs to be revealed is already present in the VGA adapter's main image buffer. Whatever background imagery is obscured by the overlay's new position remains unchanged in the primary surface.

To perform video windowing without overlays involves replacing all the pixels in the primary surface with the video window's contents. When that window is moved, the graphics system must recreate all the display elements necessary to fill the hole it left behind.

Overlay technology became a part of the Windows operating system with DirectX 5.0. Virtually all VGA adapters manufactured since 1997 include some method of overlay support. The key factor affecting the performance of overlays in a given VGA implementation is the amount of memory available for the preparation of overlay surfaces.

Colorspace Conversion

If the video data originates as Y, B-Y, R-Y (or YUV, as it's frequently referred to), it must be converted to RGB for display on a computer monitor. This may be done by the media player software, by the operating system (DirectDraw, in the case of Microsoft Windows), or it can be handled by the VGA adapter.

Generally, if the VGA adapter has implemented colorspace conversion in hardware, this is the best solution. First, the part of the circuit that does colorspace conversion won't be busy doing anything else, so it's available to do the work. And second, performing colorspace conversion in the circuit frees a significant amount of processor and memory resources that would otherwise have been used for this purpose.

Decompressing MPEG Video

Assuming that the video to be displayed is not coming directly from a digital camera, it's almost guaranteed to be compressed. Many forms of compression are used today (including Intel Indeo, Apple QuickTime, RealNetworks' Real Movie and many others). DVD and digital broadcast television use MPEG 2 compression, and that's the specific type we will discuss here.

MPEG compression uses a combination of intraframe compression and interframe compression techniques to arrive at a very high quality image result at compression ratios of up to 100:1. Intraframe compression involves analyzing each individual frame of video and selectively reducing its content in ways that are not typically noticeable. The primary method of intraframe compression is the discrete cosine transform.

Discrete cosine transform is a mathematical process used in signal processing to represent a waveform (or in our case, aggregations of pixel values) as a series of spatial frequency values or cosines. Results are weighted in such a way that the least significant details in a given frame can be eliminated. Intraframe DCT compression can usually yield a 5:1 savings in file size with little appreciable loss in image quality.

IDCT

The discrete cosine transform method of encoding video requires a reverse (properly, an inverse) application to restore the image from the supplied cosines. Again, the player application or operating system may provide this functionality as part of their video decoding infrastructure (as does Sonic), but it's usually preferable, when the VGA adapter includes inverse discrete cosine transform (IDCT) support, to use it.

Motion Compensation

Pixels, and blocks of pixels, that may move within the frame, but whose content doesn't change from frame to frame, can be coded in terms of the extent and direction of the move. This method, in the encoding process, is called Motion Estimation, and in the decoding process is called Motion Compensation. Since much less processing resource is required to track the motion of pixel blocks than to compress full or partial frames, Motion Compensation can provide a significant improvement in video playback performance.

Scaling (up and down)

Every digital video stream has a default size, determined by the pixel dimensions of its source data (e.g.: 640x480). As long as it's shown in a window with the same dimensions, there's no need to resize the video data. If the user wants to make the window larger or smaller, or to fill the entire screen, then some method of scaling must be used.

Figure 5. Overview of Up- and Down-scaling.

H & V pixel interpolation

When scaling an image up or down, new pixel boundaries are created in places where there weren't any before. Either the image has gotten bigger and there's a new pixel space between two previously known pixels, or the image has gotten smaller and two or more previously known pixels must somehow be represented into a single pixel space. Since any pixel space can only have one color, the VGA adapter must decide what color use. This process is called interpolation and involves consideration and averaging of the colors of the original pixels.

Some VGA solutions offer horizontal interpolation only. That is, pixels to the right and left of a new pixel space are averaged to derive the new color. Frequently, this results in some degree of horizontal banding or layered "blockiness." A more convincing method is to use pixels from the left and right (horizontal interpolation) and from the line above and the line below (vertical interpolation). Horizontal and Vertical pixel interpolation will produce the smoothest image with the fewest perceivable artifacts.

Edge smoothing & Anti-Aliasing

Any time a decision is made involving some degree of uncertainty (i.e.: should the new pixel be this color or that color), a degree of error is certain to arise. In the field of signal processing (which provides the mathematical infrastructure for all our pixel and image manipulation), this error is commonly referred to as aliasing distortion. Aliasing distortion arises when the mathematically correct pixel value can't be represented (due to lack of precision in the imaging system, for instance). The result is an unnatural hard edge, or visual artifact.

Hard edges can also appear any time two graphic elements are combined, as when subtitles are inserted into a video stream. In this case, there's no uncertainty about what color a given pixel should contain; but the appearance of the image to a given viewer may still be less than satisfactory.

Figure 6.  Example of an aliased subtitle overlay.

Methods exist to detect such displeasing edges and to smooth them away by averaging the values of adjacent pixels. Using anti-aliasing techniques sometimes means that the resulting pixel is less faithful, mathematically, to the original data, but more pleasing from the viewer's perspective. As with scaling and the other techniques, anti-aliasing can be performed in software by a player application, by the operating system, or it can be built into the VGA hardware.

Figure 7.  Example of the same subtitle using anti-aliasing or pixel blending.

Deinterlacing - "Bob"

There are many ways to perform de-interlacing, but the most visually successful is to display each field in succession, using an algorithm called "Bob." Because each field contains only half the vertical resolution of a full frame, it must be doubled in vertical size. This can be accomplished by simply repeating each scan line in the field, but is more successful when pixel interpolation is performed. In this case, new lines are synthesized, based upon analysis of the neighboring scan lines.

The Bob part comes when we realize that picture content in the even field is one line higher than that in the odd field. So, a line must be added to the even field to keep the picture content from 'bobbing' up and down on alternate frames. The result is a series of complete frames, delivered to a progressive scan monitor at a rate of 60 frames per second (fps).

In some cases, a VGA subsystem performs deinterlacing by simply pasting two fields together without any processing or smoothing. This can produce heavily distorted frames, especially if scaling is performed on the resulting frame. Proper combination of fields into frames involves smoothing and averaging of the content on a scan-line by scan-line basis and requires a lot of performance capability on the part of the VGA subsystem.

For additional information on ways to perform de-interlacing independently of any particular VGA subsystem, consult the Sonic white paper on "VGA Independence."

24 fps Deinterlacing - "Weave"

Most theatrical motion pictures are shot on film at a rate of 24 fps. When encoded to DVD, the 24 fps rate is preserved. Most computers, on the other hand, redraw their screens at least 60 times per second (and sometimes much more frequently). Typically, video subsystems convert the frame rate of 24 fps source material to 30 fps using a technique called 3:2 pulldown. In 3:2 pulldown, some frames are displayed as a 1/60th second field three times, and some frames are shown twice. As a result, some full output video frames may actually contain fields from two separate scenes.

The Weave deinterlacing method was developed to overcome this problem. Since the purpose of deinterlacing is to combine fields into frames and present a single, progressively scanned image to the display adapter, Weave detects the 3:2 frame content and groups them correctly back into their original sequence. The result is full-framed video displayed at 24 frames per second, just as it was originally sampled from film.

Figure 8. Frame rate restoration and deinterlacing using Weave.

For additional information on ways to perform de-interlacing independently of any particular VGA subsystem, consult the Sonic white paper on "VGA Independence."

Double-buffering (surface flipping)

We've already discussed overlay surfaces and how they are used to present video while preserving the underlying VGA display. Another benefit of the overlay surface mechanism is how easily it lends itself to techniques of double-buffering. Double-buffering is the use of two or more separate areas of memory for the preparation of video frames. These are then swapped into the overlay surface, increasing the apparent smoothness of the displayed video.

Figure 9.  Double-buffering through the use of multiple overlay surfaces.

Due to differences in VGA chipset design, memory arrangement and accessing methods, quite a number of different surface flipping methods have been developed. As a result, it's frequently necessary to develop different frame decoding strategies in order to take advantage of, or work around the limitations of, any particular VGA vendor's surface flipping implementation.

3D Acceleration

VGA performance technology tends to be driven by market forces dictated by the so-called "power users" of VGA products. Historically, the greatest number of power users have been computer games players. While many different gaming paradigms exist, the most popular have tended to be full-immersion virtual environments for role playing. This application has led most of the VGA manufacturers to concentrate their new technology development efforts in the area of 3-dimensional visualization and projection.

New developments in motion video presentation have temporarily taken a back seat while the VGA manufacturers have been rushing to implement fog, particle and lighting effects, and complex geometric modeling and manipulation engines for 3D navigation. As these technologies mature, however, they will surely require more traditional video capabilities.

There will be a growing need for faster and better motion video presentation techniques, as moving video content begins to play a more significant role in the texturing and environmental ambiance of 3-dimensional games.

By this point in our discussion, it should be fairly evident that significant benefits can be gained by using available VGA hardware capabilities to decode and display motion video. Unfortunately, due to the number of radically different implementation methods used by VGA manufacturers, Sonic has had to spend an enormous amount of resources developing custom versions of its code to gain these benefits. Historically, the effort has been worthwhile; yielding the highest performance video players in the marketplace.

But there obviously is a need for some set of common standards around which software and hardware manufacturers can build newer, higher-performance, video subsystems in a more cost-effective way. The benefits to be gained by such a standard extend to every part of the computer and home-entertainment industry; from the shareholder to the end-user.

Throughout the years, a number of efforts have been made to introduce, and sometimes enforce, standards for video presentation on PCs. Most, having to do with the Windows operating system, have come from Microsoft. The following paragraphs will briefly recap those efforts and conclude with the latest, and most far-reaching, standardization scheme: DirectX Video Acceleration.

Video for Windows

Video for Windows (VfW) was Microsoft's first offering of an application programming interface designed to allow full motion digital video to be added to an application. VfW used MCI (Media Control Interface) to provide tools for capturing, editing, compressing and viewing video. VfW also included the installable codec (short for COmpressor/ DECompressor) interface and video capture interface. Codecs included with VfW included the Cinepak codec from Radius, Intel's Indeo codec and Microsoft's general purpose and RLE codecs.

DirectX

DirectX was designed as an application programming interface for multimedia and game applications using graphics, sound, input, and force feedback. The DirectX environment contains several different subsystems, each concerned with an aspect of video or multimedia performance :

• DirectDraw - This is the VGA graphics rendering subsystem. DirectDraw comprises two layers: the Hardware Abstraction Layer (HAL) and the Hardware Emulation Layer (HEL). The HAL provides video and graphics functionality from the VGA hardware if it is available. If a particular function is not available in hardware, the HEL provides an equivalent function that uses host processor (CPU) resources to achieve the same result.
  
• DirectSound - What DirectDraw does for graphics, DirectSound does for audio. It, too, has a Hardware Abstraction Layer for mapping the supported audio features of individual sound cards into standard function calls and a Hardware Emulation Layer for producing, in software, those functions the hardware does not support. DirectSound supports audio capture as well as playback, 3D spatial effects and multi-input audio mixing.
  
• Direct3D - Direct3D provides low-level, device-independent, 3D graphics for visualization and games. It includes hardware acceleration and emulation features for advanced 3D effects such as depth-buffering, shading, light-source rendering, texturing, and object transforms.
  
• DirectPlay - DirectPlay is a high-level software interface between applications and communication services that makes it easy to connect games over the Internet, a modem link, or a network.
  
• DirectMusic - DirectMusic is the musical component of the DirectX API. Unlike the DirectSound API, which is for the capture and playback of digital sound samples, DirectMusic works with message-based musical data, which is converted into wave samples, either in hardware or in a software synthesizer.
  
• DirectInput - DirectInput provides a standard interface for a variety of input devices such as joysticks, multi-button mice, flight yokes, etc. as well as the new input-output devices known as force-feedback.
  
• DirectShow - DirectShow is a media-streaming architecture that enables capture and playback of multimedia streams. DirectShow is integrated with DirectX technologies so that it automatically takes advantage of any video and audio acceleration hardware to deliver the highest possible performance.

OpenGL (Open Graphics Library) is a multi-platform application programming interface developed for portable, interactive 2D and 3D graphics applications. OpenGL drivers exist for every major operating system including Mac OS, OS/2 UNIX, Windows 9X, Windows NT and Linux.

Hardware Video Acceleration (HVA)

Hardware Video Acceleration is an additional interface provided by DirectShow that enables a video decoder to access hardware video acceleration functionality during decoding. Currently this interface is only available in Windows2000. An application using HVA must use DirectShow and therefore must use DirectShow's overlay mixer to handle display of the video

DirectX Video Acceleration (DVA)

Supplemental to the Windows 2000 HVA specification, and now contained within the broader DirectX system of technologies, is the DirectX Video Acceleration (DVA) specification. Its intent is captured in the following Microsoft statement:

. . . an Application Programming Interface (API) for acceleration of video codec processing. It provides an interface definition focused on support of MPEG-2 "main profile" video (formally ITU-T H.262 | ISO/IEC 13818-2), but also intended to support other key video codecs (e.g., ITU-T Recommendations H.263 and H.261, and MPEG-1 and MPEG-4). . . . The interface is designed to extract the most basic computationally-intensive building-blocks of these various codec designs and support their acceleration in hardware. . . the intent of this specification is to establish a common interface to provide cross-vendor compatibility between software application programs and advanced graphics acceleration capabilities. The establishment of a common interface is expected to increase the capability of computing systems to support video, increase the demand for software applications that provide this capability, and increase the demand for high-performance graphics capabilities.

The DVA specification then goes on to detail a common set of interfaces and process flows to handle motion compensation, iDCT processing, subpicture rendering, deinterlacing, scaling, etc. Each step in the DVA specification allows for the use of hardware support, where available, with a fallback emulation, in software, where necessary.

Sonic's CinePlayer technologies offer complex and adaptable capabilities for displaying superior digital video in spite of any particular hardware configuration. But it's in the presence of specific VGA hardware acceleration features that the technology can really excel. Historically, support for these features has come at a huge cost of resources; a cost that has, to date, been well worth the investment.

However, as more VGA subsystem developers continue to add digital video hardware support and as Industry efforts continue to drive these implementations toward a common set of rules, we will find our development challenges greatly reduced and our performance goals much easier to attain.

Sonic Solutions is involved in this standardization effort, working with both Microsoft and key industry VGA subsystem providers, to make sure the new standards are effective, easy to implement, and deliver powerful results to consumers.