Tuesday, May 10, 2011

Adding ImageMagick to Star Ninja's XNA content pipeline

This week, I've been working on Star Ninja's high score system. The underlying UI system is the same one created for Atomic Sound and Moonlander, which uses a custom content pipeline that does a lot of things to prepare data for the UI system. One of the features is to process fonts and incorporate them into a sprite sheet, saving various bits of metadata required to render these fonts later. Since our content pipeline does a lot of different things well beyond the scope of this post, I'm going to limit the example code to the parts related to integrating ImageMagick into the bitmap font generator tool found on the XNA App Hub.

Back to the problem at hand.

During development of the high score screen, I found myself looking at this (which is populated with random data for now):

Not bad, I was thinking to myself, fairly pleased to be able to make something I didn't consider terrible. Something wasn't great though, which is the actual high score table font. Plain white and boring, it really needed something to stand out a bit more because it seemed to blend with the background too much. I wanted a better looking font, one that had shadows and perhaps other features. So I thought about it for a bit, realizing what a hassle it would be to create color fonts in Photoshop and making the existing font rendering pipeline use that data instead of the existing font rendering technique. Doable, but not ideal. Time is short and that seemed like a terribly cumbersome process especially when I considered the inevitable "can you make the shadow a little bigger" or other change requests that might come in. Custom images per character is an entirely unacceptable way to solve the problem - error prone and difficult to track font metadata (spacing, mainly). I don't mind creating one-off assets, but if there's any real chance of having to iterate then a system to automate the process is often justified.

So I looked into a couple things, the first being Photoshop Scripting. Rejected this because Photoshop scripting is more of a UI automation and not a background process suitable for a content building script. The second one I looked at was ImageMagick, which turned out to not only be pretty cool, but well suited for this task. It's an image processor that is typically used as a console command in batch files, but it includes an OLE component which allows me to use it slightly more easily within the content pipeline. It wouldn't have been much trouble to start up a batch job and use the console command, but the OLE component makes it all a bit cleaner. I couldn't find any examples of using C# with ImageMagick's OLE component, so it seemed like a good idea to write up a little bit about how it can be used within the context of XNA content processing.

The font processor we use is similar to what is found in bitmap font generator found here. To gain access to ImageMagick, just add a reference to the ImageMagick OLE object to the bitmap font generator project (or your content pipeline).

Around line 190 of MainForm.cs in the XNA bitmap font generator, you will see the bitmap that is generated by rasterizing a character from a font.

                        // Rasterize each character in turn,
                        // and add it to the output list.
                        for (char ch = (char)minChar; ch < maxChar; ch++)
                            Bitmap bitmap = RasterizeCharacter(ch);

That's where we hook in. To do something useful with ImageMagick, you will need to set it up and pass it a command line. Because the OLE component takes the arguments as an array of objects, each object being a string with each argument, I wrote a helper function to split a standard string into this array:

object[] GetImageMagickArgs(string args)
   if (args == null || args.Length == 0)
    return null;
   string[] args1 = args.Split(' ');
   object[] result = new object[args1.Length+2];
   for (int i = 0; i < args1.Length; i++)
    result[i+1] = (object)args1[i];
   return result;

You may notice how the string[] is copied to the object[]. This is because the ImageMagic API requires the type of the array to be exactly an array of objects and string[] doesn't match so it will throw an exception if you don't do this.

I get the ImageMagicArgs using a custom parameter to our content pipeline, so you'll need to find a suitable way to get the arguments to your content pipeline or to the bitmap font generator if you are using that. Once the bitmap and arguments are ready, this function can be called to process the character:

private Bitmap ProcessCharacter(object[] args, char ch, Bitmap bitmap)
   if (args == null || args.Length == 0)
    return bitmap;
   int chInt = (int)ch;
   var src = "c:\\temp\\char-" + chInt.ToString() + ".png";
   var dest = "c:\\temp\\char-" + chInt.ToString() + "-output.png";
   bitmap.Save(src, ImageFormat.Png);
   var m = new ImageMagickObject.MagickImage();

   args[0] = src;
   args[args.Length - 1] = dest;
   var r = m.Convert(args);
   var bitmap2 = Bitmap.FromFile(dest);
   return (Bitmap)bitmap2;

Finally, add the processing of the character to the point mentioned above:

var imageMagicArgs = GetImageMagickArgs(ImageMagickString);
                        for (char ch = (char)minChar; ch < maxChar; ch++)
                            Bitmap bitmap = RasterizeCharacter(ch);
bitmap = ProcessCharacter(imageMagicArgs, ch, bitmap);

The end result as you probably can see is that each character is rasterized as normal, then fed to ImageMagick as a temporary file and then later re-read back into the bitmap.

This technique is not without shortcomings. Ideally, I would have used the ImageMagickObject.Stream method to feed the data in directly without the use of a temporary file. However, the docs for this are sorely lacking and it wasn't worth the time to figure out - the temporary files blast through so fast I really don't see the need to spend more time on that. For whatever reason, ImageMagick was keeping a write lock on each file until the component was finalized so I had to create a different file for each character (there is no Dispose method to control this, unfortunately). The biggest shortcoming of this however is that the processed bitmap is the same size as the input bitmap which means processing that spills over the edge will leave the rasterized/processed font with a visible edge. Ideally I would resize the bitmap to contain any effects that might be created for the font and then crop it and adjust font/spritemap metadata accordingly when it was done. I may do this at some later time, but for my specific needs today, this works. I just needed a small effect, something that fits within the existing bitmap.

By feeding in the ImageMagick string "-alpha on ( +clone -channel A -blur 0x1.5 -level 0,50% +channel +level-colors black ) compose Over +swap", feeding the fonts through the pipeline and then running the game, I was rewarded with this image:

Much better!

Here's a closeup of the letter 'A' with and without processing:

The text is much more clear against the background and we now have a system which can be used for any font in any of our games moving forward. As a huge bonus, it's pretty much automated and we can tweak settings and regenerate the textures without dealing with individual letter image files. With a little more work to support larger output bitmaps, more dramatic effects could be used but this is a good solid step in the right direction.

In other news, Star Ninja is going to have local & global high scores tracked across four different game modes! :)