Archive for July, 2009

Totally Unofficial Gamebryo Roadmap

Posted in Video Game Development with tags , , on July 23, 2009 by Dan Amerson

Did I catch you with my provocative title? There’s good news and bad. The bad news is that this truly isn’t official. It’s just my ideas that I’m presenting to folks internally. It’s also less of a roadmap and more of a cluster of features. The good news is that I’m working very hard on this right now. If you’re a Gamebryo LightSpeed licensee interested in where I think our rendering should go, this is an indication just not the official indication. Regardless, here’s the big picture as snapped from my whiteboard a few weeks back.

Rendering Enhancments Matrix

Rendering Enhancments Matrix

The black circles, excluding the LOD circle on the right, represent seven major areas that I’m hoping to rework in the Gamebryo renderers. Note that while I refer to renderers here, the work will really encompass other bits like scene graph management, animation system, etc. Those areas are:

  • Spatial Management Systems – Support for improved bounding volume hierarchies separate from the scene graph and support for spatial partitioning systems.
  • Parallel Resource Creation – Creation of resources from the non-rendering thread with minimal time on the main, rendering thread. Some of our renderers have this now, but there are areas that need improvement.
  • Occlusion Culling – An integrated occlusion culling system that combines a portalling system with our new tool plus software and hardware occlusion queries.
  • Property System Rework – Rework the Gamebryo property system to support a more modern set of render states and to store that data in a much more friendly manner for performance and parallelization.
  • Effect System Rework – Rework the Gamebryo dynamic effect system to add more effects and remove the need for some of the manual management required by the current system.
  • Parallel Rendering – Simple in premise and complicated in process. We need to get support for submitting draw calls from multiple threads and SPUs.
  • Pretty Pixels – It all adds up to pretty pixels. It’s actually labeled “Improved Rendering Techniques” in my docs, but it’s about getting the latest and greatest rendering techniques running out of the box for clients.

Obviously, that’s a lot of work. Those small numbers are estimates in man-months of effort. Right now, I’m working on breaking up those tasks into a resource plan. It’s pretty exciting because it’s a chance to rework some portions of the codebase that have been in place since I joined the company eight years ago. In addition to that, these features will help enable other important features like streaming content which you can see in the green circle in the middle. The hard part is that those red and blue arrows represent dependencies. Breaking these things down and delivering to market is going to be a challenge. It’s one that I hope to talk about in coming posts.

For now, I’ll leave it at the feature matrix above.

Someone changed Pretty Pixels on my board to Beautiful Pixels after I took this picture. Hrm… dba

Advertisements

Friends Can See Your Private Parts

Posted in Video Game Development with tags , , on July 17, 2009 by Dan Amerson

An interesting programming issue cropped up this week in Gamebryo. The following code had some thread-safety issues.  This code loads some pixel data from a NIF file. Note that I’ve removed some error handling code for brevity hence the totally unsafe cast at the end. However, the main issue is intact.

NiPixelData* NiNIFImageReader::ReadFile(efd::File& kIst, NiPixelData*)
{
    m_kReadCriticalSection.Lock();

    m_kStream.RemoveAllObjects();
    m_kStream.Load(&kIst);

    NiObject* pkPixelData = m_kStream.GetObjectAt(0);

    m_kReadCriticalSection.Unlock();
    return (NiPixelData*)pkPixelData;
}

It’s possible for another thread to enter ReadFile and remove the reference to pkPixelData before the function returns the pointer to some object in the calling context that will add a reference. Furthermore, NiImageReader subclasses, of which NiNIFImageReader is one, are expected to return an NiPixelData object with a reference count of 0. In this case, however, the reference count is 1. m_kStream holds an internal reference that is cleared by calling RemoveAllObjects. Since the image reader is static, few if any people have noticed this anomaly.

On top of this, things are complicated by this being very old, entrenched code. There are a number of other NiImageReader subclasses both internal to the engine and developed by our customers. Changing the assumptions or the interface is expensive if not out of the question. Barg!

The answer is that we need to remove the reference to pkPixelData from m_kStream without deleting the object. However, the internals of the reference counting base class are private (for good reason) meaning that we have to break encapsulation some how to make this happen. If we try to manually increment and decrement the reference count, then we’ll end up deleting the object. For example:

pkPixelData->IncRefCount(); // It's 2 now.
m_kStream.RemoveAllObjects(); // Back to 1.
pkPixelData->DecRefCount(); // Drops to 0 and calls dtor

What we really need is a method that would zero out the reference count without deleting the object. However, most of the team involved in the discussion really felt that would expose too much of the internal implementation in a public method that would be misused at a later date when ZeroRefCount showed up in autocomplete. After some discussion, I proposed that we simply declare NiNIFImageReader as a friend of the reference counting class and move on. It turns the fix into 4 lines of code, 1 to declare the friend and 3 to manage the count.

In one class:

friend class NiNIFImageReader;

In the offending function:

pkPixelData->IncRefCount();
m_kStream.RemoveAllObjects();
pkPixelData->m_uiRefCount = 0;

The interesting part of this proposal was the vehemence of the reaction that it got. You’d have thought that I was proposing wholesale slaughter of puppies. It was asserted that the fix was “the dirtiest code ever written” and that one should never use the friend keyword. Really? Never? Dirtiest? Perhaps the alternative of refactoring all the image readers is cleaner and preserves encapsulation. However, it would take hours if not days not to mention the support load when we break customer code in the next release. Courtesy of our friend the friend keyword, this issue is fixed in 1 hour or less. I won’t lie and claim that it’s good design, but as a professional programmer with deadlines you have to consider the cost sometimes.

The whole incident also got me to thinking about some of the assumptions that programmers make. If you do some reading, there’s some great nuances to the use of friend. The C++ FAQ Lite has some interesting points, but frankly just search for “C++ friend keyword” to get plenty of folks to opine on whether or not it’s evil. I’m of the opinion that 90% of the time there’s a better design decision. However, there’s always a time when it makes sense. Unfortunately, programmer education simply beat the good use of friend out of folks. It’s similar to many programmers believing that you should always have a virtual destructor whenever a class has a virtual method. You shouldn’t necessarily. Proof left to the reader.

In any case, the whole incident has been valuable. If nothing else, it prompted me to go read some interesting things about C++. Plus, I got to use the witty title of this post.

Aside

I mentioned this post and its title to another programmer at a gathering the other night. He chuckled and noted that we, as a group, have a habit of creating weird metaphors for programming concepts. His examples all involved terminating child and zombie processes. It made me think of Tron with zombies. 🙂

A foolish consistency is the hobgoblin of little minds. dba

SIGGRAPH 2009 – Coming Soon

Posted in Miscellany, Video Game Development, Video Games with tags , , on July 13, 2009 by Dan Amerson

I’m heading down to SIGGRAPH this year, and it’s shaping up to be a good time. It’s in New Orleans which never hurts. There’s also a ton of interesting content this year. Here are a few of the courses and talks that I’m interested in attending:

Also, I’ll be on the ground solo this year, so I’m looking for cool folks to hang out and keep dinners interesting. If you’re in town for the show and want to grab a bite, let me know.

I will not promise to live blog the conference at this point. dba