Kesäkoodi final report and OpenMind 2009 presentation

Here’s the final version of my final report. I hope I don’t have to make any changes to it any more.

Final report (.pdf)

I’ll be at OpenMind 2009 along with the other summer coders. This little video will be screened. It’s got a few slides, then a little demo of Libwm action and some more slides coupled with a nice tune by Kitkaliitto.

This concludes my summer code work and this blog. Work with Libwm will go on and new features will be added when I need them, bugs will be fixed when I find them (or someone reports them). Help is welcome and contributions will be accepted.

Final report, part two: Libwm

After coming to the conclusion that my original Kesäkoodi project, XCB and OpenGL had come to a dead end, I quickly documented my findings and published my XCB/Xlib/GLX hybrid setup example in the XCB wiki. After that I could move on to other projects. I made a few suggestions for new projects, my favorite suggestion being Libwm, an abstraction library for window and render context management for OpenGL applications. I was glad to find out that my supervisors approved of my choice. I got cracking right away.

First, I had to write a new project plan. I set my goal on making the first release of Libwm. I had seven weeks of time left to finish my goal. I decided to set my schedule so that I’d have four weeks of time to achieve the primary goal, the Libwm 0.1.0 release. The three remaining weeks I wanted to have as a worst case scenario-buffer time or if I’d make release in time, I could improve the library and add new features to it.

I had started the Libwm project early in the spring and it had about two months of work and 7000 lines of code under it’s belt. I felt that the project was about half-way from being a complete usable software library and coding-wise I knew pretty much what I had to do to get there. The bigger part of getting the library in release shape was documention and project management related. I decided to schedule my time half coding, half project management. I wanted to do the most useful project management tasks first, the ones that would help me with the coding process.

Once I had project plan and a schedule, I started off by creating task issues in the Libwm issue tracker in Google Code. I had a few dozen task TODO list written on a piece of paper earlier this year and I compiled the issues list based on that and comments I had previously added to the source code. I also set up CMake and Doxygen-based API doc generation and wrote the skeleton for the API docs by adding a brief annotation comment for all public classes, structures, functions and their parameters and return values. I had plenty of coding time on my schedule so I went to work.

I started off by splitting the source code in two parts, the windowing and the rendering context part. The windowing part would consist of Win32 and Xlib code and the context management would be done with their respective OpenGL-glue API’s WGL and Xlib. I had tried doing this earlier in the project, but I ended up having a simpler solution by merging them together. However, the lessons learned when struggling with XCB turned out to be very helpful for putting together a big picture about OpenGL and Windowing system communication. I also wanted to keep things well organized so adding EGL and OpenGL ES support would be easier to do in the future. Overall, the library design improved a whole lot. My overall design paradigm for Libwm has been trying to find the commonality in the underlying API’s and abstracting it without breaking the underlying fundamental concepts instead of providing a monolithic and intrusive framework like many of Libwm’s competitors do.

The next project management task I had was to improve the CMake build system and configure CPack to build distribution files. Everything didn’t quite work out as planned but I got a decent system set up to build and package Libwm. I couldn’t get proper distribution files built for .deb-based systems or Mac OS X, because of difficulties and little details in creating a distributable shared library package. Everything was pretty much following the 80-20 rule and I decided that I was happy with the result and not to waste any more time on CPack.

I did some small but important improvements to Libwm event handling code, but I quickly realized that I’m running out of small fixes, I have plenty of time on my schedule and some big coding tasks that I originally intended to postpone until after the release. They were the ones that I also had the most negative feedback about, adding support for full screen and backwards compatibility code for GLX 1.2. I took the rest of the week to get that done, and it went pretty fluently. The biggest issue was doing GLX 1.3 and GLX 1.2 code binary compatibility. To make Libwm-based applications run on a GLX 1.2-only version of the OpenGL library (libGL.so or equivalent), I used dlopen to load the GLX 1.3 functions dynamically at run-time if glXQueryVersion reported GLX 1.3 or higher.

The third week of the coding process was entirely spent coding. I did some refactoring and redesign behind the scenes and added small less-significant things like hiding the mouse cursor and creating windows that can not be resized. I felt that the library was almost complete, but needed some review and testing.

The fourth week was the planned release week. I spent most of the time writing user documentation in the Libwm Wiki. I also stumbled across a few bugs and overall I did not feel confident enough to make the release quite yet and decided to wait a little longer. The fifth week was spent on finalizing the library and on Thursday 13th of August I uploaded the official 0.1.0 release tarballs to the Libwm home page in Google code and added a version tag to the Mercurial source repositories.

After the release was done, I had a few weeks left. I had scheduled the last week for writing the final documentation but instead I decided to spend the entire period coding. When I started the Libwm project, the intention was to write it to provide a comfortable platform for writing OpenGL 3.x-based applications across platforms and provide backward compatibility workarounds. Adding support for OpenGL 3.0 (or the ARB_create_context extensions) didn’t make it to the first 0.1.0 release and that was intentional. Now that I had the release done, I didn’t hesitate. I quickly wrote support for OpenGL 3, multisampling and shared-exponent (sRGB) pixel formats, among other stuff. I released the second version of Libwm, 0.2.0 just a few weeks after the first release.

Now that Libwm is a full-featured and capable software library, the next step is to start using it. Some practical use and real world testing will surely improve the library and help to squish out bugs that may be there.

The next natural step for Libwm is to support a wider range of platforms, including new windowing and rendering context API’s. The most intresting API to support would be EGL, a new glue API that can be used to set up OpenGL ES, OpenVG and other graphics API’s on mobile platforms and I hear it’s coming to the desktop too. Adding native Mac OS X Cocoa support is also intresting and would enable OSX users to run Libwm applications without the X server and with EGL could run Libwm on the iPhone. There’s also plenty of new features that can be added to Libwm, including but not limited to multi display support, better input handling, support for multi-GPU systems and a lot more.

I thank COSS and the Kesäkoodi crew and all the sponsors for the financial support and making Libwm possible. I am very satisfied with the final product and it wouldn’t have been possible without the ability to work full time on this open source software library.

Final report, part one: XCB and OpenGL

My original Kesäkoodi project was adding support for OpenGL 3 extensions to XCB-GLX, the XCB bindings to the GLX protocol. The goal of the project was to enable OpenGL-based graphics in XCB applications.

The GLX API is a standard programming interface to enable using OpenGL applications in the X Windowing System and the GLX protocol is the underlying protocol for X client/server communication. The GLX API is closely coupled with Xlib, the standard programming interface to the X Windowing System. While XCB is trying to be an alternative to using Xlib, XCB-GLX is not intended as a replacement for GLX. I had to learn it the hard way.

The GLX API is not implemented by a single software library, but instead different OpenGL implementations have different GLX implementations. The GLX implementation is responsible for two things: setting up client-local OpenGL state and negotiating with the X server using the GLX protocol. XCB-GLX on the other hand can only be used to negotiate with the X server but it does not (and can not) set up the private client-local OpenGL state and thus cannot be used to set up OpenGL rendering.

Although GLX and XCB-GLX do very different things, it’s very easy to be mistaken. Looking at the GLX and the XCB-GLX functions, one can easily get the impression that XCB-GLX is to GLX what XCB is to Xlib, that is, a replacement API. The GLX functions have a nearly one-to-one correspondance to the XCB-GLX functions, just the way XCB functions look almost like Xlib functions. It’s easy to make the same mistake I made, and looking at various discussions in the Internet, I am not the only one to make this mistake.

The start of my XCB project went pretty fluently, I was able to set up an XCB development environment pretty fluently and got hacking straight away. I was able to add some of the OpenGL 3 GLX protocol extensions to XCB-GLX and use them. I started to notice that things were wrong when I had a simple XCB + XCB-GLX + OpenGL app set up just like it’s supposed to be, but I was not get any drawing to happen.

With next to no documentation available on the subject, all I could do was reverse engineering. I started doing a side by side comparison of “equivalent” XCB/XCB-GLX and Xlib/GLX applications. I used the xtrace X client/server communication debuffer, and noticed that something’s fishy and that my Nvidia GLX implementation was doing something that I didn’t expect it to. I contacted several other XCB developers across the Internet (in mailing lists as well as private e-mail correspondance) to discuss my findings. I figured out that what I was trying to do was effectively impossible.

While struggling with the GLX vs. XCB-GLX impedance mismatch and trying to figure out a solution to set up OpenGL in an XCB application, I had a breakthrough. Although the GLX API is closely tied to the Xlib library, it is possible to use OpenGL with XCB. The solution is to use a hybrid XCB/Xlib setup, where Xlib is used to negotiate with GLX and XCB is used for windowing, event handling and everything else. This is possible if you use an XCB-based version of Xlib, which is likely if you have a modern unix-based operating system. New versions of Xlib are built using XCB for the network communication and it is possible to mix XCB and Xlib together. This provides a way to incrementally port Xlib applications to XCB, but it can also be used to work around the tight Xlib/GLX coupling.

Once I had realized that my project with XCB and OpenGL was impossible to complete, I wanted to finish the project as quickly as possible and move on to another project. I went on to document my findings the best I could. I wrote a draft documentation in this blog, complete with a few clarifying diagrams. I asked the XCB mailing list for feedback on the documentation and after recieving some encouragement, I posted the documentation in the XCB wiki. The final documentation can be found here: http://xcb.freedesktop.org/opengl/. Unfortunately, I could not upload any images to the XCB wiki and I had to leave out the diagrams I had spent lots of time and effort doing.

This concludes my one month ordeal with XCB and OpenGL. The rest of my Kesäkoodi I would go on and spend on another project, Libwm. What I achieved was not much. I wrote the XCB/OpenGL documentation to the XCB wiki, which does present the first working XCB/OpenGL solution I have found in the Internet, although it’s not Xlib-free like I wanted. I also posted a bug report to a bug I found while hacking on the project. All in all, I was very dissapointed with my work and embarassed of looking like a fool working on a project that was impossible to begin with.

Libwm 0.2.0

This week I released Libwm 0.2.0. It has plenty of new stuff like OpenGL 3 support, multisampling and extension function loading. Get it here:

http://libwm.googlecode.com/

In my original schedule, I had booked the entire week for writing my final Kesäkoodi documentation. However, next to coding, writing boring docs got the shorter straw. A surprising news article in OpenGL.org also pushed me to go for the next Libwm release ASAP. The post, titled “New libwm version released with support for OpenGL 3.0” was blatantly misleading, since I had not yet released a version with OpenGL 3 support but just pushed the OpenGL 3 code to the Mercurial development repository. Now the release is done and I still have my docs to write.

OpenGL 3 support for Libwm

After getting Libwm in release shape last week, I started adding new features for future releases. I finally got to the stuff I originally started the Libwm project for, OpenGL 3 support (and new pixel formats, etc).

I added OpenGL 3 support using GLX/WGL_ARB_create_context and create_context_profile. The coding went smoothly but I had some driver-related issues and problems finding suitable pixel formats. However, now the support is there and you can get an OpenGL 3 context set up.

However, I was not entirely satisfied. The problem is that it’s impossible to determine what features are supported by the underlying OpenGL implementation. Selecting suitable pixel formats is particularly annoying. You can’t tell beforehand whether or not a pixel format can be used with hardware accelerated OpenGL.

On Windows, I get a Microsoft GDI OpenGL 1.1 context if I select a pixel format that the Nvidia implementation cannot handle (like one with a 16 bit depth buffer). On OSX/GLX 1.2/Intel GMA I can’t tell if a pixel format will support direct rendering or not. The best solution is to try a pixel format and see if it fails to provide a context. Detecting failures on Xlib is hard too, Xlib calls an error handler function in case of failure. The error handling is asynchronous and it’s impossible to tell what went wrong. You can detect the request type (glXCreateContextAttribsARB) but if there are many different contexts set up at the same time (either in parallel or quickly in a sequence), you can’t really tell which of the requests failed. The error handlers do know the sequence number of the failed request but you don’t have a number to compare it to (you can’t get the sequence number of the create context request you send).

It seems like the best portable alternative is to write a launcher application/script that uses a simple libwm-based probe application that queries available confiugrations and checks what can be supported. If the probe application fails then try another one. Once a suitable configuration (a pixel format, direct/indirect rendering, OpenGL context version) has been found, the launcher script/app starts the actual application. It’s pretty much a hack but I couldn’t come up with a better solution given the limitations of the underlying platform API’s.

I’d really like to get support for multisampling done soon, it’s only a small addition to the pixel format setup. A slightly bigger change is supporting floating point and packed float pixel formats. Getting that done needs a bit of a change to pixel format context setup architecture, which is why I’d want it done rather sooner than later. Currently the floating point pixel formats are available only for off-screen PBuffers and Pixmaps which are not yet supported by Libwm, but it can’t hurt to have the functions ready for future floating point on-screen rendering or the addition of off-screen support to Libwm.

I’ll be very busy for the rest of the month, so not a lot of progress can be expected, but hopefully I have some coding time too.

Libwm 0.1.0 released

Today I released Libwm 0.1.0.

• Libwm website
• Source code
• API Documentation
• Mac OS X Darwin binaries
• Windows binaries
• Building instructions
• Tutorial

’nuff said.

Tutorial, building instructions and bug fixes for Libwm

My original goal for this week was to get Libwm in shape for release. While things are looking pretty good, I decided to wait a couple more days. I don’t want to rush it, since I still have a few weeks of time in buffer.

I wrote one new feature this week, asynchronous window repaint requests. The rest of the week I spent writing small bugfixes and improvements here and there, mostly on the Win32 side.

Most of this week’s work, however, was documentation. I wrote a tutorial and building instructions page to the Libwm Wiki in Google Code. I also configure CMake, Doxygen and TeX Live to create pdf API docs.

This weekend is the Assembly demoparty/fragfest weekend. Too bad they don’t have a compo for utility software libraries. My friends have been busy getting their productions in shape for Assembly release. Next week, I might get some help doing a little code review and testing before the release.

New windowing and input features

In my original schedule, I set next week as a target for the first Libwm release. This week I intended to write the last remaining features that go in to the release and write some documentation. I didn’t get very far with the documentation but for the coding part I had good progress. One week remaining before the release, I’d say I’m doing pretty fine. If all goes well, the release will be next week.

I started the week by improving my Extended Window Manager Hints (EWMH) code on Xlib. I also discovered a bug that caused some key press events to be duplicated. The bug originated when I was refactoring the event reading code a while ago and had gone unnoticed for a while (I really do too little testing). It took two attempts to fix the bug but now the key events work correctly.

Next I went on to add some useful input features. I started by detecting repeated keyboard events. After that, I wrote functions for warping the mouse and hiding the mouse cursor. While working on the mouse cursor I also figured out how to detect when the mouse pointer enters a window on Win32. I used TrackMouseEvent, WM_MOUSELEAVE and WM_MOUSEMOVE and a simple state machine to send MouseOverEvents and hide/show the mouse cursor to limit the effect of cursor hiding to one Window. Doing all this I found another bug. All the sudden I started getting “class already exists” errors from Win32 RegisterClassExW. I used a simple generator to make unique string identifiers from pointer addresses. The code has a simple operator precedence mistake that made my hexadecimal string conversion give same strings for different pointsrs. A pair of parens fixed the problem.

Finally I went on to add size limits to windows. Now a window can have a minimum and a maximum size and the window style is switched to make the window non-resizable if the lower and upper size limits are equal and non-zero.

Next week there won’t be any new features, only small additions and clean up. I will probably rename or add a function or two, but the coding stuff will be minimal. Most of next week’s work will probably be doing the documentation. I need to write a tutorial, building instructions for the library and a howto on using Libwm in projects. The API docs are already complete, but there’s a lot of room for improvement.

Last week’s blog post

Last week was pretty solid coding time. I made some good progress, added a few new features, fixed some bugs I found and tweaked the CMake build system.

I started out with hacking my CMakeFiles to have configure options for controlling the build type (Debug/Release/etc, the default is Debug) and handle building shared libraries properly. I added a config header file to sort out __declspec(dllexport) issues when building shared libraries on Windows, using a config.hpp.in template file and CMake’s configure_file-command to generate the compile time configuration file, GNU Autoconf style.

I also experimented creating Debian .deb packages with CMake and CPack. However, I soon figured out that building separate .deb packages for runtime and development files was hard and decided not to make .deb packages at all. I might do Mac OS X framework bundles later on.

Next, I went on to write non-blocking event reading/handling/queueing code. For Xlib I used XCheckMaskEvent and XCheckTypedEvent for ClientMessages, on Windows I used PeekMessageW with PM_NOREMOVE. A new test application was needed to test the non-blocking event i/o, so I whipped up a small new test app with a little rotating triangle. I used Boost threads and date_time to do timing for the animation. I tried to have a small model-view-controller-type design with a game-like update main loop with a fixed frame rate, independent for redrawing and model updates.

I also added an issue about synchronizing the event code for multithread applications. I’ve already separated the per-display and per-window event handling using a queue and a dispatcher utility that calls the event handlers. The event queue and the dispatcher should be easy to synchronize, but the hard part is getting along with the Xlib and Win32 API’s that are not exactly multithread friendly. I hope a little select(2)-based unix hackery will make the problem go away. Using XCB instead of Xlib would solve a lot of problems too. However, the synchronization will not make the first 0.1 release and is not my concern at the moment, for now we’ll be fine off single threaded.

Next I decided to tackle two issues I’ve had the most negative feedback about. First up was supporting GLX 1.2 and second was adding full screen windows. There’s still a lot of platforms that don’t support GLX 1.3, so backward compatibility for older API versions was needed. I used dlopen(NULL, RTLD_NOW) to dynamically link to the GLX 1.3 functions and #ifdefs to leave out GLX 1.3 stuff if it’s not available in the compile time environment. I then pieced together the configuration query code using XVisualInfos instead of GLXFBConfigs in a builder pattern-type design I also used with WGL_ARB_pixel_format/DescribePixelFormat on Windows. The rest was pretty easy, there was only about 6 GLX function calls to begin with, the pixel format query being the hardest part. For other minute GLX1.2/GLX1.3 api differences I just added a few good old if’s. I was very glad to see libwm running on a 2007 MacBook with only GLX 1.2 available, now I can test with OSX/X11 again.

Next up was full screen support. On Xlib I used Extended Window Manager Hints (EWHM) to switch to full screen and back. It worked fluently on my Linux desktop but on the MacBook it failed leaving the window unusable. I decided not to try to fix the MacBook bug because it seemed like it just failed trying to do what I wanted it to do, so it’s probably a graphics driver (go Intel!) or an X implementation bug. On Windows I used window styles, extended styles and moving and resizing the window to make it borderless and fill the entire screen. It worked pretty nice. Currently the window does not return to it’s original size when switched back from fullscreen to windowed, but I decided it’s not an issue.

I’ve had good experiences using Google Code with Mercurial. The on-line issue tracker is quite simple and perfect for little projects. I really like the Project Updates page, which is particularly helpful when writing long blog entries about coding progress.

This week, I intend to focus on small and useful features like handling keyboard repeat and hiding the mouse cursor. It’s also time for getting the documentation in shape for next week’s scheduled Release 0.1.

Rewriting and bug squishing

I started this week by adding all coding tasks I could come up with to the libwm issue tracker in Google Code. I could only come up with a few dozen issues to start with, but I’ve been adding new ones as I come up with them. I found it hard to convert some ideas to issue tickets and trying to select which tasks to complete before the upcoming release. So far, I’ve only marked the critical ones with the “Release 0.1” milestone label.

I had originally scheduled setting up Doxygen and CMake to generate API documentation, but I had already done this last week, so I had nothing but coding time at my hands.

I started out with adding support for keyboard modifiers (shift, alt, ctrl, num lock, etc)  and mouse motion events, I spent monday and tuesday doing that and everything went pretty fluently. When hacking with the event reading routines I realized that they are in need of a clean up. So after I had added the keymods and motion events I spent a day re-organizing and rewriting the event reading code that listens to event messages from the windowing system, interprets the events and passes them on to the libwm event queue.

When I was fixing the event reading parts I found a bug that had gone unnoticed for quite a while. To work around an incompatibility in window initialization-configuration query order between Xlib and Win32, I had used an extra subwindow in the Xlib implementation. When setting up a drawing surface I added a child window that was the X window that actually got drawn into. When clicking the client area of my window, I got X events that informed the mouse cursor leaving the parent window, entering the subwindow, then a click and leave/enter from the child to the parent. I decided to get rid of the subwindow hack altogether and use a dummy window to query the device configuration on Windows. A third alternative would have been to use some kind of creational design pattern to do delayed window initialization in the Xlib implementation.

So I went on and rewrote the window initialization, configuration query, drawing surface and context set up. On Windows I used a dummy window and device context when using DescribePixelFormat or WGL_ARB_pixel_format to query available pixel formats. On Xlib/GLX no hacks are required. The only hard part was trying to arrange the internal communication between the Window and PixelFormat class. I try to keep Xlib separated from GLX (and WGL separated from the Win32 stuff), so that adding support for EGL and OpenGL ES would be easier in the future.

Sometime during the week I realized I had another bug. I had previously used boost::scoped_ptr to handle the cleanup of my pimpl idiom pointers. I figured out that using scoped_ptr in a .dll on Windows was difficult because of __declspec(dllexport) issues so I switched to using plain old pointers and delete in destructors to get a dynamic library built on Win32. I forgot to take care of the destruction of the “pimpl objects” in case of exceptions in constructors. So I wired an std::auto_ptr inside the constructor to take care of the clean up. I release() the auto_ptr in the end of the constructor and let the destructor handle the final clean up.

Next week I’ll get writing some example applications that use libwm. I’ll focus on trying to set up a testing platform that I can use to debug non-blocking event i/o. I’ll also try to improve my build system to generate package files and handle building static libs optionally.