Archive for August, 2009

Final report, part one: XCB and OpenGL

August 29, 2009

My original Kesäkoodi project was adding support for OpenGL 3 extensions to XCB-GLX, the XCB bindings to the GLX protocol. The goal of the project was to enable OpenGL-based graphics in XCB applications.

The GLX API is a standard programming interface to enable using OpenGL applications in the X Windowing System and the GLX protocol is the underlying protocol for X client/server communication. The GLX API is closely coupled with Xlib, the standard programming interface to the X Windowing System. While XCB is trying to be an alternative to using Xlib, XCB-GLX is not intended as a replacement for GLX. I had to learn it the hard way.

The GLX API is not implemented by a single software library, but instead different OpenGL implementations have different GLX implementations. The GLX implementation is responsible for two things: setting up client-local OpenGL state and negotiating with the X server using the GLX protocol. XCB-GLX on the other hand can only be used to negotiate with the X server but it does not (and can not) set up the private client-local OpenGL state and thus cannot be used to set up OpenGL rendering.

Although GLX and XCB-GLX do very different things, it’s very easy to be mistaken. Looking at the GLX and the XCB-GLX functions, one can easily get the impression that XCB-GLX is to GLX what XCB is to Xlib, that is, a replacement API. The GLX functions have a nearly one-to-one correspondance to the XCB-GLX functions, just the way XCB functions look almost like Xlib functions. It’s easy to make the same mistake I made, and looking at various discussions in the Internet, I am not the only one to make this mistake.

The start of my XCB project went pretty fluently, I was able to set up an XCB development environment pretty fluently and got hacking straight away. I was able to add some of the OpenGL 3 GLX protocol extensions to XCB-GLX and use them. I started to notice that things were wrong when I had a simple XCB + XCB-GLX + OpenGL app set up just like it’s supposed to be, but I was not get any drawing to happen.

With next to no documentation available on the subject, all I could do was reverse engineering. I started doing a side by side comparison of “equivalent” XCB/XCB-GLX and Xlib/GLX applications. I used the xtrace X client/server communication debuffer, and noticed that something’s fishy and that my Nvidia GLX implementation was doing something that I didn’t expect it to. I contacted several other XCB developers across the Internet (in mailing lists as well as private e-mail correspondance) to discuss my findings. I figured out that what I was trying to do was effectively impossible.

While struggling with the GLX vs. XCB-GLX impedance mismatch and trying to figure out a solution to set up OpenGL in an XCB application, I had a breakthrough. Although the GLX API is closely tied to the Xlib library, it is possible to use OpenGL with XCB. The solution is to use a hybrid XCB/Xlib setup, where Xlib is used to negotiate with GLX and XCB is used for windowing, event handling and everything else. This is possible if you use an XCB-based version of Xlib, which is likely if you have a modern unix-based operating system. New versions of Xlib are built using XCB for the network communication and it is possible to mix XCB and Xlib together. This provides a way to incrementally port Xlib applications to XCB, but it can also be used to work around the tight Xlib/GLX coupling.

Once I had realized that my project with XCB and OpenGL was impossible to complete, I wanted to finish the project as quickly as possible and move on to another project. I went on to document my findings the best I could. I wrote a draft documentation in this blog, complete with a few clarifying diagrams. I asked the XCB mailing list for feedback on the documentation and after recieving some encouragement, I posted the documentation in the XCB wiki. The final documentation can be found here: http://xcb.freedesktop.org/opengl/. Unfortunately, I could not upload any images to the XCB wiki and I had to leave out the diagrams I had spent lots of time and effort doing.

This concludes my one month ordeal with XCB and OpenGL. The rest of my Kesäkoodi I would go on and spend on another project, Libwm. What I achieved was not much. I wrote the XCB/OpenGL documentation to the XCB wiki, which does present the first working XCB/OpenGL solution I have found in the Internet, although it’s not Xlib-free like I wanted. I also posted a bug report to a bug I found while hacking on the project. All in all, I was very dissapointed with my work and embarassed of looking like a fool working on a project that was impossible to begin with.

Libwm 0.2.0

August 29, 2009

This week I released Libwm 0.2.0. It has plenty of new stuff like OpenGL 3 support, multisampling and extension function loading. Get it here:

http://libwm.googlecode.com/

In my original schedule, I had booked the entire week for writing my final Kesäkoodi documentation. However, next to coding, writing boring docs got the shorter straw. A surprising news article in OpenGL.org also pushed me to go for the next Libwm release ASAP. The post, titled “New libwm version released with support for OpenGL 3.0” was blatantly misleading, since I had not yet released a version with OpenGL 3 support but just pushed the OpenGL 3 code to the Mercurial development repository. Now the release is done and I still have my docs to write.

OpenGL 3 support for Libwm

August 21, 2009

After getting Libwm in release shape last week, I started adding new features for future releases. I finally got to the stuff I originally started the Libwm project for, OpenGL 3 support (and new pixel formats, etc).

I added OpenGL 3 support using GLX/WGL_ARB_create_context and create_context_profile. The coding went smoothly but I had some driver-related issues and problems finding suitable pixel formats. However, now the support is there and you can get an OpenGL 3 context set up.

However, I was not entirely satisfied. The problem is that it’s impossible to determine what features are supported by the underlying OpenGL implementation. Selecting suitable pixel formats is particularly annoying. You can’t tell beforehand whether or not a pixel format can be used with hardware accelerated OpenGL.

On Windows, I get a Microsoft GDI OpenGL 1.1 context if I select a pixel format that the Nvidia implementation cannot handle (like one with a 16 bit depth buffer). On OSX/GLX 1.2/Intel GMA I can’t tell if a pixel format will support direct rendering or not. The best solution is to try a pixel format and see if it fails to provide a context. Detecting failures on Xlib is hard too, Xlib calls an error handler function in case of failure. The error handling is asynchronous and it’s impossible to tell what went wrong. You can detect the request type (glXCreateContextAttribsARB) but if there are many different contexts set up at the same time (either in parallel or quickly in a sequence), you can’t really tell which of the requests failed. The error handlers do know the sequence number of the failed request but you don’t have a number to compare it to (you can’t get the sequence number of the create context request you send).

It seems like the best portable alternative is to write a launcher application/script that uses a simple libwm-based probe application that queries available confiugrations and checks what can be supported. If the probe application fails then try another one. Once a suitable configuration (a pixel format, direct/indirect rendering, OpenGL context version) has been found, the launcher script/app starts the actual application. It’s pretty much a hack but I couldn’t come up with a better solution given the limitations of the underlying platform API’s.

I’d really like to get support for multisampling done soon, it’s only a small addition to the pixel format setup. A slightly bigger change is supporting floating point and packed float pixel formats. Getting that done needs a bit of a change to pixel format context setup architecture, which is why I’d want it done rather sooner than later. Currently the floating point pixel formats are available only for off-screen PBuffers and Pixmaps which are not yet supported by Libwm, but it can’t hurt to have the functions ready for future floating point on-screen rendering or the addition of off-screen support to Libwm.

I’ll be very busy for the rest of the month, so not a lot of progress can be expected, but hopefully I have some coding time too.

Libwm 0.1.0 released

August 13, 2009

Today I released Libwm 0.1.0.

’nuff said.

Tutorial, building instructions and bug fixes for Libwm

August 7, 2009

My original goal for this week was to get Libwm in shape for release. While things are looking pretty good, I decided to wait a couple more days. I don’t want to rush it, since I still have a few weeks of time in buffer.

I wrote one new feature this week, asynchronous window repaint requests. The rest of the week I spent writing small bugfixes and improvements here and there, mostly on the Win32 side.

Most of this week’s work, however, was documentation. I wrote a tutorial and building instructions page to the Libwm Wiki in Google Code. I also configure CMake, Doxygen and TeX Live to create pdf API docs.

This weekend is the Assembly demoparty/fragfest weekend. Too bad they don’t have a compo for utility software libraries. My friends have been busy getting their productions in shape for Assembly release. Next week, I might get some help doing a little code review and testing before the release.