I agree 100% with Tim that using OpenGL directly as a rendering API for GTK+ makes no sense. The OpenGL 2D API is outdated and irrelevant; using the 3D API for acceleration of 2D is possible, but involves a lot of deep hackery; you want something like glitz to hide the details.

But does that mean there is no use for 3D rendering engines in the GTK+ world? I don't think that's the case. One idea I've been thinking about recently is that a 3D engine could provide a way of doing flexible theming in a way that is more accomodating of custom widgets than pixmap-based approaches. With a pixmap-based approach, if you know how to theme a button, that doesn't tell you how to to theme something that is somewhat like a button. For example, imagine that you had a button separated into two parts with a shallow groove in between, similar to what you often see for a drop-down on a browser's back button.

A theme that knows how to draw a button won't know how to draw a button with a groove. But if the custom widget calls the theme API not as "draw a button", but as "draw a button with this skeleton", where the skeleton is a two-and-a-half dimension shape like:

Then the theme can apply textures, pixel shaders, and possibly transformations to the shape to get the final appearance. The same approach that produces a button from the basic box shape should produce something reasonable from the custom skeleton.

(A poor quality mockup in the GIMP.) Note that with pixel shaders, you aren't restricted to pure 3D looks; it should be possible to get a flat high-contrast appearances if you want that. Another possible point of control for the theme engine would be modifying the skeleton a bit by, say, rounding off the corners.

This approach should allow a very wide range of themes; what it doesn't allow easily is themes that reproduce anything that a graphics designer could possibly dream up and draw in the GIMP or Photoshop. So, it would be very important to have interactive user-friendly tools for theming that allow the graphics designer to play around, get a sense of the possibilities and create themes without having to go through the intermediatary of a programmer.