Ditch the Emulators: How to Code Your Own J2ME Simulator

There are few things worse for a programmer than the delay between altering some lines of code and seeing those changes working in game. Developing J2ME games is especially prone to build process delay, with many steps required between compilation and final jar file output. Add to this the complication of many different device emulators with their slow start up times, poor keyboard response and limited debugging support. Mobile Java programming can be sped up and even made a pleasurable experience with the help of some J2SE code which simulates the J2ME APIs. This article will lead programmers through the creation of a deceptively simple J2ME simulator which speeds up programming, improves debugging, facilitates rapid prototyping, aids the creation of device independent code and frees you from build processes that may take minutes to complete.

What's wrong with emulators?

I'm not going to list point by point all the things I personally dislike about using device manufacturer's emulators for J2ME development. I'm sure if you've spent any time developing J2ME applications you've felt there must be something better. Firstly, there are just so many SDKs and emulators available. Installing, maintaining, learning and integrating them into your development process is a time consuming and cumbersome task. Secondly, actually using them is all too often frustrating and slow, when they should be saving you from the even slower task of testing your game on a real device. They are slow to invoke, offer little or no feedback when errors occur, and debugging support is usually unreliable or functionally limited.

Figure 1. What you usually do.

The one area where a good emulator is essential is gauging the memory usage of your game. You can be pretty much assured that if your game runs in the Nokia 7210 MIDP SDK v1.0, it will work on a real 7210. However, you can't say the same for all emulators, and quite often you're left in the dark regarding memory errors.

If you are an experienced J2ME developer who knows how to write memory efficient, jar size and performance conscious code, I would suggest that your primary need for the majority of the programming work on a project is simply a J2ME environment that supports arbitrary screen sizes and all the relevant APIs your code utilises. You can get this in a tool that can be invoked in the blink of an eye both for normal execution and source level debugging, much more simply than you might think.

Figure 2. What you could do.

Compare the steps in figures 1 and 2, which you would potentially perform thousands of times during the programming of a J2ME game. I'd take the three steps over five any day.

Simulating J2ME with Desktop Java

Implementing the necessary J2ME APIs using pure Java turns out to be quite simple. The classes in CLDC 1.0 and 1.1 (java.io, java.lang and java.util) are all derived from their J2SE counterparts, so literally no wrapping or re-writing is required for the base standard library. The javax.microedition.midlet package is simple, well defined and easy to implement. The javax.microedition.lcdui package contains classes that (for the main part) have AWT counterparts, and those that don't, you probably never use anyway, so there's no need to implement them! You can probably do without the rest too, although javax.microedition.rms comes in handy, it's just file storage and again, easy to implement.

Figure 3. Simulator Class Overview

Figure 3 shows all but one class (MIDletStateChangeException) that needs to be implemented to create a functioning J2ME environment. The boxes in red are the main framework of the application and provide the entry point, window to display the midlet upon and keyboard input. The boxes in grey are the J2ME API classes that midlets use. These hook into each other and the simulator classes. Image, Graphics and Font are implemented using their AWT equivalent classes, shown in blue.

Start Coding

The MIDlet class contained within the javax.microedition.midlet package is the starting point of all midlet development and is the obvious place to look first when creating a MIDP execution environment. All midlets have to provide a class that extends MIDlet, and the 3 abstract methods that need to be overridden are the almost completely self explanatory startApp, pauseApp and destroyApp.

To get off the ground, our MIDP simulator needs an implementation of the MIDlet class with some public methods that call the abstract midlet lifecycle methods. Since these refer to the trivial MIDletStateChangeException, we should add this too.

Then we'll need a ‘main' application class for command line invocation that loads the concrete midlet class and invokes the public method that calls startApp. The following shows the code needed to load and invoke a midlet.

// Create our AWT main frame (more on this later)
frame = new FrameAWT (midlet_class_name);

// Load and instantiate the midlet
midlet = (MIDlet) Class.forName(midlet_class_name).newInstance();

// Call the midlet startApp method
midlet.amsStartApp ();

The main class should probably be in a package of its own, with a suitable name. I've chosen Jammy as the name of my J2ME simulator. I prefer single words over compound words or complex names that aren't very catchy. With just these 3 classes in place we are able to compile a midlet and execute it by invoking it with Jammy. If you have simple midlet built into a jar, you can invoke the simulator using a command line like Figure 5.

java -classpath bin;BasicMidlet_v1.0.0.jar com.longsteve.jammy.Main BasicMIDlet

Resource loading

J2ME uses the same mechanism to load resources as J2SE, the Class.getResourceAsStream() method. If you are running a midlet from a jar file, with the jar on the classpath, any midlet calls to getResourceAsStream will simply work in the simulated environment. You don't need to run your game code from a jar file however. One of the speed improvements of using a simulator comes from having to avoid the jar packaging step. Your classpath can simply include the directory containing your compiled classes, and your game resource directory (with PNG images in for example). Your compile - run cycle is then reduced to simply that, without the need for preverification and jar packaging.

Adding a display

Just running a basic midlet is ok, but we need to add a display. Since we're writing our simulator in Java, we can use one of the windowing toolkits available with J2SE. We could choose Swing and use JFrame and its companion classes, but I've often had trouble with threads when using Swing. As far as our midlets are concerned too, all we need is a window and a canvas, we don't need all the fancy widgets of Swing, so I'm going to keep it simple and use the AWT.

In J2ME, midlets use classes in the javax.microedition.lcdui package to interface with a display. The Display class provides the static setCurrent method, which a midlet uses to set a Displayable object visible on the screen. Canvas is the primary Displayable subclass a game will use. The midlet is coded with its own subclass of Canvas, which draws the game frames in its paint method, using a Graphics context.

We're going to need to create the javax.microedition.lcdui package and add Displayable, Canvas and Graphics to the simulator. This will mean adding AWT Frame and Canvas classes, and interfacing these with the MIDP classes. Later we'll add the MIDP Image and Font classes and expand the basic midlet even further into something resembling a game.

Main window

FrameAWT extends the AWT Frame class and provides the main window for the midlet display. In order to know how big the window should be, there's a command line argument to the main class that specifies the dimensions for the midlet canvas size. There's nothing to stop these dimensions from being cleverly worked out by referring to known device details, or by inferring the size from some metadata left over from a build process. Command line arguments are convenient for now though.

The canvas

CanvasAWT extends the AWT Canvas class and handles the screen drawing and keyboard/mouse input. CanvasAWT is linked to the current midlet Displayable object by containing a reference to it. A public method on Displayable called invokePaint() is used to actually invoke the midlets Canvas.paint() method. CanvasAWT maintains a MIDP Image object, which is used as the midlet screen. Each time the AWT paint method in CanvasAWT is called, it calls the midlet Canvas paint() method with a MIDP Graphics object derived from the screen Image. When the MIDP Canvas paint() method returns, the screen image is painted to the AWT Graphics context passed to the CanvasAWT paint() method.

The following summarises the code needed in the CanvasAWT.paint() method. The invokePaint() method is part of our implementation of the MIDP Displayable class. When a midlet calls Display.setCurrent(), our Display class actually sets the ‘current' object in CanvasAWT, so all windows paint requests get forwarded to the midlet.

public javax.microedition.lcdui.Displayable current;
public javax.microedition.lcdui.Image midp_screen;

public void paint (java.awt.Graphics g)
{
 javax.microedition.lcdui.Graphics midp_graphics =
  midp_screen.getGraphics ();

 // Call the midlet paint method with a J2ME graphics context
 current.invokePaint (midp_graphics);

 // Draw the midlet screen image to this canvas, scaled up
 g.setClip(0,0,awt_canvas_width,awt_canvas_height);
 g.drawImage(midp_screen._image,
  0,0,getWidth(),getHeight(),
  0,0,midp_screen.getWidth(),midp_screen.getHeight(),
  this);
}

The AWT drawImage() method also handily scales up the MIDP screen image to the required size, so you can resize the main window and see a zoomed in view of the midlet. I hate squinting at tiny emulator windows on my desktop, even the 2x zoom offered by some device emulators isn't enough. By default, Jammy creates its canvas 3x the required size, but a command line parameter could be used to set this as you like.

Keyboard input

Keyboard input is handled by CanvasAWT in combination with the Displayable class. CanvasAWT implements the AWT KeyListener interface, and handles keyboard events, passing the J2SE key code straight to the invokeKeyPressed() method of the current Displayable. 

Our Displayable class contains a table of J2SE key codes and their mappings to MIDP key codes and game actions. It would have been easy to simply re-use the J2SE key codes here, instead of inventing (or borrowing) J2ME codes. However, the J2ME Canvas class definition actually specifies the constant values for the codes (e.g. KEY_NUM0 is 48), so using the same ones will make the simulator directly compatible with existing midlets in this respect.

Image and Graphics

MIDP Image objects are implemented fairly simply by using a java.awt.BufferedImage. BufferedImage objects can be created with a width and height, like MIDP Image objects, or created from input streams using one of a number of image support methods or libraries. The Java 5 ImageIO class contains support for PNG images, as does the Advanced Imaging (javax.media.jai) package. I've found these methods of reading PNG data unreliable though. By far the fastest and most robust PNG library for Java (that I've used) is the library from sixlegs.com. The library jar is under 50k, licensed with a library exception to the GPL and succeeds in reading optimised and crushed PNGs where the built in Java support fails.

MIDP Graphics objects are created directly from Image objects and are implemented using a reference to a java.awt.Graphics object, created from the java.awt.Image. Most of the J2ME functionality is simply forwarding on method calls to the underlying AWT Graphics object, e.g. drawLine(). There are some differences though, which include the image anchors commonly used with J2ME for the drawImage() and drawString() methods, see below for an example. Things do get more complex if you start implementing any enhanced MIDP 2.0 methods like drawRegion(), which includes image transformations like rotation and mirroring. These are by no means impossible though, and some investigation into the AWT Graphics2D and AffineTransform classes should provide all the methods required.

public void drawImage(Image img, int x, int y, int anchor)
{
 // default anchor
 if (anchor == 0)
 {
  anchor = TOP | LEFT;
 }

 // Work out the x and y offsets given specific anchors
 switch (anchor & (TOP|BOTTOM|BASELINE|VCENTER))
 {
  case BASELINE:
  case BOTTOM:
  y -= img.getHeight();
  break;

  case VCENTER:
  y -= img.getHeight() >> 1;
  break;

  case TOP:
  default:
  break;

 }

 switch (anchor & (LEFT|RIGHT|HCENTER))
 {
  case RIGHT:
  x -= img.getWidth();
  break;

  case HCENTER:
  x -= img.getWidth() >> 1;
  break;

  case LEFT:
  default:
  break;
 }

 // Draw the AWT image within the MIDP image to our AWT
 // graphics context
 _graphics.drawImage(img._image,x,y,null);
}

Font

Closely related to the Graphics class is the MIDP Font class, and any implementation of Graphics cannot be completed without it. By using java.awt.Font and java.awt.FontMetrics objects, all of the functionality is fairly simple to code. Some elements aren't quite perfect, like underlined text support, but if your game uses a bitmap font, you're not going to worry too about any text limitations.

Extras

Implementing just eight J2ME classes and three application classes is enough to start developing games using your simulator. You have all the components you need, and you may even find that you can run some existing games you already have. However, if you try running anything complex, you're likely to run into class and method not found exceptions. What's great though, is you can immediately see any components you need to add in order to flesh out the simulator.

Assuming you try a MIDP 1.0 compliant game you'll almost certainly find it needs an implementation of the javax.microedition.rms record store package. Stub this out with some dummy code and you'll be surprised at how much will now run. Adding actual file storage isn't hard either using Java's excellent file and stream IO classes.

Following that, it will probably be the additional classes in the javax.microedition.lcdui package that need adding. Command, and CommandListener should be the first, allowing access to device ‘softkeys'. You'll probably not need any of the forms classes, since most games don't use these. Figure 4 shows Jammy running our old X-Change game.

Figure 4. Jammy running X-Change

Following on from the basic MIDP 1.0 functionality, MIDP 2.0 additions slot in fairly obviously, I already mentioned the graphics transformations. If you really want to get a wide range of existing games working, then adding the Nokia UI would be good too. If you do MIDP 2.0 first, then the Nokia UI can be implemented completely on top of it.

Audio

The javax.microedition.media package can be added for audio support. You can use the javax.sound.midi and javax.sound.sampled J2SE packages if you're using Java 5 or above. Again, stub out the classes first so that any midlet you are attempting to run actually works without crashing, then you can implement the functions and get real sound playing.

Bluetooth

Developing midlets with Bluetooth can be awkward since none of the manufactures J2ME emulators support actual wireless transmission. They all simulate it between running instances of emulators. Any actual integration with a real Bluetooth device is going to need testing on a phone, which slows the development and debug cycle immensely. There are javax.bluetooth (JSR-82) packages for J2SE available today. One GNU LGPL implementation is called BlueCove and can be dropped into your classpath for development. Another JSR-82 implementation is called avetana and is available for free trial, with a very reasonable fee for continued use.

3D

Once you've experienced the benefits of developing with a J2SE simulator, you'll want it to support all the common APIs your games utilize. The Mobile 3D Graphics API (JSR-184) is increasingly important to J2ME games. Hybrid Graphics have a development product called Rasteroid which is a J2SE implementation of JSR-184. Adding this to your simulator won't quite provide binary compatibility with midlets due to it's direct dependency on AWT classes, but it will allow you to develop 3D midlets with only a couple of minor source code tweaks.

The Benefits

You might wonder if programming your own J2ME simulator is worth the effort, in the face of existing emulators from device manufacturers. My own simulator application grew out of a level design tool for a J2ME game. I wanted to use my game engine within the Java based design tool to provide rapid feedback to the designer. The simplest way to do this was to include the necessary J2ME support classes and methods within my design tool so the game engine class could be dropped in without any source code changes. While coding the design tool, the speed of standard Java development (compile and run) along with instant source code debugging made me wish I could carry on doing it for the rest of the game code.
In the end, I re-factored the J2ME support code into a stand alone application that ran my complete midlet. Then, over new projects, I've gradually implementing more supporting J2ME code as it was needed.

When I first thought about my design tool requirements, had I known of Mpowerplayer I probably would have used it, rather than write my own tool. Mpowerplayer is a great development tool and the company have built a quality service offering around it. There are additional benefits to owning your own in house tool however, many of which weren't immediately obvious.

Really Rapid Development

An ant build process, with its compile, obfuscate, preverify and jar steps will take several seconds. You can set up a project in your IDE that includes your simulator classes, game classes and resource path, hit the ‘Run' button and see you code running immediately. If you use Eclipse you won't even need to compile first, it's like developing any desktop Java application. If your standard studio project environment includes any sort of meta data about your current device target, you can build this into the simulator and have it start automatically at the correct screen dimensions. No need to switch emulators to perform the bulk of the porting work, retargeting a game at a different screen size.

Debugging

With a J2SE simulator you're essentially coding your midlets in a pure Java development environment. As well as hitting the ‘Run' button in your IDE (did I mention that already?), you can hit the ‘Debug' button, and step through the source code. How long have you been developing J2ME games without source code debugging? This feature alone is probably the single best reason to ditch traditional emulators. I know they are supposed to support debugging, and some of the modern ones don't do a bad job, but source level Java debugging in your IDE and debugger of choice is much more mature. You can also take advantage of numerous Java profiler tools.

Game Design/Prototyping

Having nailed the rapid development cycle, you're going to be able to knock up prototypes faster. These builds can be given to designers, testers and anyone else to play with on their PCs, and all they need is a Java Virtual Machine installed. No need for numerous emulators, all to run different builds of the game.

How about a design tool that incorporates a build of your game using your simulator as the runtime? Designers can code levels, or artists can change tile sets and instantly see the results. You could add another AWT window to a special build with some sliders or checkboxes that allow game engine adjustments, so testers can tune difficulty parameters in real time. Artists can drop PNG images directly into the resource directory to see real time changes in the game, rather than fiddling with zip files and updating the JAD files.

Post Development

Having control of the environment your games run in brings further benefits after a game is complete. A lot of post development work is involved in obtaining submission material for operators, screenshots and video footage. Adding a screenshot facility is simply a matter of taking the MIDP screen buffer and writing it to a PNG file (inside CanvasAWT):

java.io.File f = new File ("screenshot.png");
javax.imageio.ImageIO.write((BufferedImage)midp_screen._image,
 "png", f);

Another potential time saver would be the ability to output video files of a game session. This would benefit QA too, with testers being able to record details of a bug. The Java Media Framework contains functionality to write AVI files, and could be hooked into the paint output of CanvasAWT, like the screenshot code.

Having a set of code not constrained to a J2ME execution environment opens up countless possibilities for running your games. You can demonstrate J2ME games to potential clients using a laptop and a projector, rather than hand them a phone to squint at, they get a full screen presentation. You can plan for future developments like VGA screen size phones simply by setting your simulator display to 640x480. Running your games as applets on the web is another obvious possibility, as are Pocket PC and PDA type devices. Java native compilers might also offer a quick route to alternative executable versions of your games for non-Java platforms.

Conclusion

I hope this article has shown how simple it is to write your own J2ME simulator using pure J2SE Java. By becoming (mostly) free of manufacturers device emulators, your full game development cycle can be positively enhanced. Stable source level debugging alone should have J2ME programmers running! Taking control of your midlet execution environment might also open up surprising new avenues for your games.

Source Code

[Steve Longhurst is a lead programmer at Infospace's Iomo Studio.]