Hardware is the basis of understanding rendering

Hardware is the basis of understanding rendering. Not numerical problems, not geometric problems and certainly not memorizing OpenGL or Direct3D APIs.

One of the most first questions that needs to be asked when deciding to implement new graphical features is what needs to be implemented on the general purpose CPU and what can be computed on a GPU (or even a set of SPEs). This question is impossible to answer without a fundamental understanding of the hardware you are programming for. The abstraction of your favorite API does less to mask this as more programmatic options become available. Consider:

• What are the latencies and bandwidth of ports and buses in the graphics pipeline?
• What caching and swizzling operations exist that offset those latencies?
• What can or must be batched in order to reach expected performance targets?
• What is the performance and capacity of the target hardware framebuffer memory?

Once you can competently speak on these points, you can start to devise a hypothesis about how to best divide your hardware resources to render a typical scene for your game.

At this point, you have a shot at guessing what data needs to be where in your pipeline and when it needs to be there. This is the point when numerical and geometric issues move into focus.

Learning OpenGL to learn graphics now considered harmful

OpenGL (with the exception of OpenGL ES) is a pool of functions, many of which superficially achieve the same results but with different approaches to dealing with bus latency. As memory latency becomes more of an acute problem in paring down hardware implemented pipeline stalls, literature promoting immediate mode and display lists continues to be the most prominent information on OpenGL in spite of the strong need for sizable data batches.

I’m always going to be a fan of getting stuff up on the screen quickly and programming some sample apps in OpenGL does give you something to mentally referenece when learning theory, but you aren’t programming anything really interesting until you’ve understood the graphics pipeline and at least the timeline of how hardware acceleration has crept backwards through the graphics pipeline over the past ten years.

Recommended Reading

Jim Blinn’s Corner: A Trip Down the Graphics Pipeline - An old one that covers software rendering, but gives you a basis of understanding of the graphical pipeline.

Real-Time Rendering - Currently in 3rd edition. The modern replacement for Foley and VanDam. The chapters on performance and hardware combine with the fundamental understanding of the graphics pipeline to help you really understand what’s going on.

Technical briefs for hardware prepared by NVidia and ATI for target hardware. This information is made available on their respective sites.