Monday, May 23, 2011

Abusing texel blur to smooth out DXT5 textures used by user interfaces

If the game has the memory for it, I generally prefer to use uncompressed textures for 2D UI elements because a compressed texture format such as DXT5 will almost certainly degrade the image quality.

An important part of making a good looking UI is supporting pixel-accurate rendering - this is especially important on smaller screens such as mobile devices where space is limited and small fonts & controls are common. If the UI is drawn a half-texel off or anything other than 1:1 pixel:texel, there will be blur that will make the UI appear less crisp. In some situations it may not be noticeable, but in some situations it's completely unacceptable; for instance small 8 point fonts that would have been clear become a blurry mess when the one-pixel-wide parts of a character texel lie on a screen pixel boundary. Even worse is when you have small pixels moving at sub pixel distances; there is simply no avoiding terrible strobing artifacts in this case as characters alternate between sometimes clear and mostly blurry while they move (I was reminded of this recently when working on the credit roll for Star Ninja). Not having pixel accurate rendering for UI is pretty much the same as telling the artist that no matter what they make, they have to put a blur filter on it before they save to make sure it is usable in sub pixel positions which of course would be ridiculous - but that's how it will look much of the time if you don't have pixel accurate rendering. I suspect this is why many games have UI elements that are bigger than they really need to be - it's just easier to ignore pixel accuracy and compensate by making everything big. Sometimes however, when UI elements are small, there is no avoiding the need for pixel accurate rendering.

Of course, when pixel accurate rendering is used the UI shows exactly what the textures contain, pixel for pixel. Uncompressed textures rendered pixel accurately look great, and it seems to usually be the best solution if your game has the memory for it. What if your UI elements are so numerous that it's out of the question to save them all as uncompressed textures? Or perhaps other factors are putting pressure on your overall memory use and you have to squeeze everywhere possible? It can & does happen. So, just compress it and be done with it, why not? The problem is, pixel accurate drawing of compressed textures only serves to exaggerate the artifacts caused by the compression. Sometimes these artifacts can cause the resulting texture to fail to meet quality requirements.

The good news is that the final result actually tends to benefit if compressed textures are drawn a half pixel off because the filtering that occurs for free will smooth out the artifacts caused by compression. It still won't look as good as uncompressed textures, but it will often be an improvement over pixel accurate rendering. By maintaining a consistent half-pixel offset in rendering these elements, you continue to benefit from the filtering and also eliminate strobing if the UI moves.

Here's an example using the level picker button in Star Ninja. Click for a zoomed in view of the button. The uncompressed and DXT5 versions are the direct output of the content pipeline. I had to make the bilinear filter version in photoshop due to time constraints but it does reasonably match the results I was seeing in the game engine when I was originally doing these tests.

The uncompressed obviously looks the best. The DXT5 compressed version looks the worst because of how the gradients don't respond well to the compression, particularly on the top and bottom edge. The last version is the same DXT5 texture shifted 0.5 pixels which has the effect of blurring out the compression artifacts.

Another option for crisp UI with DXT5 compression is to make sure the artists carefully inspect the compressed results of their work and modify the texture until the artifacts are not a problem. This sounds reasonable, and can sometimes be depending on the artist and toolchain, but if the assets are going to be procedurally packed into a sprite sheet (like this tool does) then it may be difficult to guarantee the artifacts for one sprite sheet layout will be the same as another. This is because the DXT5 compression is based on a compressing 4x4 pixel blocks; if the sprite sheet is built where the elements can shift within the 4x4 block the artifacts will be different. This page has a good explanation of how the compression works.

While uncompressed textures are generally preferred for user interfaces, the use of compressed textures is sometimes necessary but will result in compression artifacts. By rendering these elements pixel accurately but offset one half pixel in x & y, a consistently smoothed version of the texture will be rendered which will mask some of the compression artifacts and can improve the final image quality.