Getting Started With the Mobile 3D Graphics API for J2ME

This tutorial introduces the Mobile 3D Graphics API for the Java 2 Platform, Micro Edition (J2ME), defined by the Java Community Process (JCP) in JSR 184. The API is an optional package to be used with profiles like the Mobile Information Device Profile (MIDP). After an overview of the API, the article discusses potential application areas, the differences between JSR 184 and two related APIs, the classes in the new optional package, the programming model, the reference implementation, and some programming examples.

Overview of the Mobile 3D Graphics API for J2ME

JSR is the first Java-specific standard for three-dimensional graphics on mobile devices. The JSR's 26-member expert group includes all the major players in the mobile arena, including Sun Microsystems, Sony Ericsson, Symbian, Motorola, ARM, Cingular Wireless, and specification lead Nokia. The API takes the form of an optional package expected to be used with MIDP and version 1.1 of the Connected Limited Device Configuration (CLDC). It defines low- and high-level programming interfaces that bring efficient, interactive 3D graphics to devices with little memory and processing power, and with no hardware support for 3D graphics or floating-point operations. As new phones with diverse functionality appear, however, the API can scale up to higher-end devices that have color displays, 3D graphics hardware, and support for floating-point operations.

Application areas that will benefit from a 3D graphics API include games, map visualization, user interfaces, animated messages, and screen savers. Each of these areas requires simple content creation, some require high polygon throughput, and others require high-quality still images with special effects. To meet this wide spectrum of needs, the API supports both high-level and low-level graphics features, with a footprint of only 150 KB. In the high-level implementation (called retained mode), the developer works with scene graphs, and the world renders itself based on the positions of virtual cameras and lights. The low-level access ( immediate mode) allows applications to draw objects directly. You can use either mode, or both at the same time, depending on the task at hand.

The features of immediate mode are aligned with OpenGL ES standardization by Khronos. OpenGL ES (from "OpenGL for Embedded Systems") is a low-level, lightweight API for advanced embedded graphics using well-defined subset profiles of OpenGL. It provides a low-level interface between applications and hardware or software graphics engines. This standard makes it easy and inexpensive to offer a variety of advanced 3D graphics and games across all major mobile and embedded platforms. Because OpenGL ES is based on OpenGL, no new technologies are needed, which ensures synergy with, and a migration path to, the most widely adopted cross-platform graphics API, full OpenGL.

Requirements

The JSR 184 expert group has agreed on a set of capabilities that the API must support:

• The API must support both retained-mode access (scene graphs) and immediate-mode access (the OpenGL ES subset or similar), and allow mixing and matching of the two modes in a unified way.
• The API must have no optional parts; all methods must be implemented.
• To reduce the amount of programming required, the API must include importers for certain key data types, including meshes, textures, and scene graphs.
• Data must be encoded in a binary format for compact storage and transmission.
• It must be possible to implement the API efficiently on top of OpenGL ES, without floating-point hardware.
• The API must use the float data type of the Java programming language, not introduce a custom type.
• Because using integer arithmetic is difficult and error-prone, floating-point values should be used wherever feasible.
• The ROM and RAM footprint must be small; the API should be implementable within 150KB on a real mobile terminal.
• The API must provide minimal garbage collection.
• The API must interoperate properly with other Java APIs, especially MIDP.

JSR 184 requires version 1.1 of CLDC for its floating-point capability. Because most mobile devices do not actually have floating-point hardware, the API designers struck a balance between the speed obtained through integer operations and the ease of programming provided by floating-point operations. Calculations that require the fastest processing accept 8- or 16-bit integer parameters, and other calculations are performed with floating-point math for easier application programming. Figure 1 shows how the relevant APIs relate:  

Figure 1: The Mobile 3D Graphics API Software Stack

Contrasting JSR 184 With Other APIs

Newcomers to the Mobile 3D Graphics API for J2ME might find it all too easy to confuse it with two other interfaces. Note these distinctions:

• The Mobile Media API (JSR 135) is a high-level control API for different media types; it doesn't address interactive 3D graphics. JSR 184 was defined to provide core 3D graphics functionality.
• Java Bindings for OpenGL ES (JSR 239), which aims to provide Java bindings to the OpenGL ES native 3D graphics library, is still in its early stages. JSR 239 aims to provide the same functionality as JSR 184, but it caters to those developers who are already familiar with OpenGL.

The API

The Mobile 3D Graphics API for J2ME is defined in the package javax.microedition.m3g, which provides an easy-to-use API for rendering 3D graphics in retained mode and immediate mode.

In addition to the APIs, the package defines a scene graph structure and a corresponding file format for managing and deploying 3D content efficiently, along with all other necessary data: meshes, scene hierarchies, material properties, textures, animation keyframes, and so on. This data is written to a file using content-creation tools and loaded into the API through the Loader class. The most important class is Graphics3D, because all rendering is done there. The World class serves as the root of the scene graph structure. Object3D is the base class of all objects that can be rendered or loaded from a file, as well as the place where animations are applied. Table 1 briefly describes all the classes in the javax.microedition.m3g package.

Table 1: Classes in the m3g Package

Creating 3D Applications

An application that uses the Mobile 3D Graphics API for J2ME must be a valid MIDP application that follows the conventional MIDlet life-cycle. Use existing MIDP user interface classes such as Canvas and CustomItem to access the screen. To render the user interface, follow these steps:

1. Create a Graphics3D object.
2. Bind it to a 2D graphics object by linking the Graphics3D object to a 2D buffer.
3. Render the desired World, Node, or sub-mesh.
4. Release the graphics object, which will flush the rendered 3D view to the 2D buffer.

Complete functional examples appear later in this article, but this quick sample shows you the basics:

    public class MyCanvas extends Canvas     Graphics3D g3d;

     World world;     int currentTime = 0;
   public MyCanvas() { g3d = Graphics3D.create();
Object root[] = Loader.load("world.m3g");
     world = root[0];
  }      protected void paint(Graphics g) { g3d.bindTarget(g); 
      world.animate(currentTime); 
  currentTime += 50; 
  g3d.render(world);
       g3d.releaseTarget();
} } 

JSR 184 Support in the J2ME Wireless Toolkit

Version 2.2 of the J2ME Wireless Toolkit provides support for mobile 3D graphics by implementing JSR 184, including three demonstration MIDlets. To try these out, open the Demo3D project and run it. You'll see three samples listed, as in Figure 2:

• Life3D implements the Game of Life in three dimensions.
• PogoRoo shows a kangaroo bouncing up and down on a pogo stick.
• retainedmode plays back a scene file that shows a skateboarder.

Figure 2: Sample Mobile 3D Graphics MIDlets in the J2ME Wireless Toolkit

Figure 3 is a snapshot of PogoRoo in action.

Figure 3: The PogoRoo Example

The Reference Implementation From Nokia

The JSR 184 Reference Implementation (RI) was developed by Nokia. To get access to it and use it you must sign a limited license agreement with Nokia. The process is fairly straightforward and fast – it took me just a couple of days.

The RI, which is based on CLDC 1.1 and MIDP 2.0, comes in binary format. Note that you don't need to install CLDC or MIDP, as the RI includes them. It uses the open source Mesa 3D graphics library for rasterization. Mesa is a 3D graphics library similar to OpenGL. The RI has been developed and tested on Windows 2000 only, and doesn't come with the DLL files needed for Windows 2000. You'll need to download and build the Mesa DLL libraries yourself, especially because the RI works only with Mesa version 5.0.2, which was the release available at the time the RI was developed. Basically you need two DLL files: MesaGL.dll and osmesa.dll. You can either download the source code and build the libraries from scratch – you'll need Visual C++ – or build it following these steps:

1. Go to SourceForge.net's Mesa3D file list.
2. Download MesaLib-5.0.2.zip and MesaDemos-5.0.2.zip.
3. Unpack the archives in the same directory.
4. Follow the instructions in the readme.win32 file in the MesaLib package to build the DLLs. The process will generate MesaGL.dll and osmesa.dll and write them to the lib directory of your Mesa installation.
5. Move the Mesa DLLs, MesaGL.dll and osmesa.dll, to the system folder c:\winnt\system32.

Once you have installed the RI and copied the DLL files to the right directory, change to the bin directory of your reference implementation and type midp. If all goes well, you'll see the emulator as in Figure 4.

Figure 4: Nokia's Mobile 3D Graphics API Emulator

The RI comes with a sample rendering MIDlet. To run it, use the following command:

midp -classpath ..\examples\jar\RenderTarget.jar RenderTargetMIDlet

Figure 5 is a snapshot of the running animation.

Figure 5: The RI's Sample Rendering MIDlet

Sample Applications

1. Retained Mode

In the first full-length example, a MIDlet uses the high-level API (retained mode) to play back a ready-made animation that it downloads over an HTTP connection. Note that the application plays the pogoroo.m3g file that comes with the J2ME Wireless Toolkit 2.2, and can be found at \WTK-HOME\Demo3D\res\com\superscape\m3g\wtksamples\pogoroo\content. Once you've located it, you'll need to modify the parameter to the call Loader.load("pogoroo.m3g"), in the following code.

    import javax.microedition.midlet.MIDlet;

 import javax.microedition.midlet.MIDletStateChangeException; 
  import javax.microedition.lcdui.Graphics;
   import javax.microedition.lcdui.Display; 
   import javax.microedition.lcdui.Displayable;
    import javax.microedition.lcdui.Command; 
    import javax.microedition.lcdui.Canvas;
     import javax.microedition.lcdui.CommandListener;
       import java.util.Timer; import java.util.TimerTask; 
import javax.microedition.m3g.*; 

 public class JesterTestlet extends MIDlet implements CommandListener { 

     private Display myDisplay = null;
 private JesterCanvas myCanvas = null;
       private Timer myRefreshTimer = new Timer();
private TimerTask myRefreshTask = null;
      private Command exitCommand = new Command("Exit", Command.ITEM, 1);
private World myWorld = null; 
     /**      * JesterTestlet - default constructor.   

        */     public JesterTestlet() { 

        // Set up the user interface. myDisplay = Display.getDisplay(this); 

 myCanvas = new JesterCanvas(this);  
 myCanvas.setCommandListener(this); 
myCanvas.addCommand(exitCommand);
 }      /**      * startApp()    

   */     public void startApp() throws MIDletStateChangeException {  myDisplay.setCurrent(myCanvas);

   try { // Load a file. Object3D[] roots =      Loader.load("pogoroo.m3g"); 

// Assume the world is the first root node loaded. myWorld = (World)roots[0];   

   // Force a repaint so that we get the update loop started. myCanvas.repaint(); 
       } catch(Exception e) { e.printStackTrace(); 
   }     }      /**      * pauseApp()     

    */     public void pauseApp() { 
    // Release resources. myWorld = null; 

       }     
        /**      * destroyApp()      

        */     public void destroyApp(boolean unconditional) throws MIDletStateChangeException { 

         myRefreshTimer.cancel(); 

   myRefreshTimer = null;  // Release resources. myWorld = null;  

      }      /**      * MIDlet paint method.   

         */     public void paint(Graphics g) { // We are not fully initialised yet; 
   just return. 

   if(myCanvas == null || myWorld == null) return; 

    // Delete any pending refresh tasks. if(myRefreshTask != null) { myRefreshTask.cancel(); 
myRefreshTask = null; 
}  // Get the current time. long currentTime = System.currentTimeMillis();  

   // Update the world to the current time. int validity = myWorld.animate((int)currentTime);  

   // Render to our Graphics. Graphics3D myGraphics3D = Graphics3D.getInstance(); 
       myGraphics3D.bindTarget(g);   
 myGraphics3D.render(myWorld);  
myGraphics3D.releaseTarget();
      // Subtract time taken to do the update. 

      validity -= System.currentTimeMillis() - currentTime; 
   if(validity < 1) { 

   // The validity is too small; 
   allow a minimum of 1ms. validity = 1;    
}  // If the validity is not infinite schedule a refresh task. if(validity 

< 0x7fffffff) { // Create a new refresh task. myRefreshTask = new RefreshTask();

// Schedule an update. myRefreshTimer.schedule(myRefreshTask, validity); 
    }     }      /**      * Handle commands.    

      */     public void commandAction(Command cmd, Displayable disp) 

      { if (cmd == exitCommand) { try {     destroyApp(false);  
     notifyDestroyed();   
   } catch(Exception e) {     e.printStackTrace();   
 } }     } 
  /**      * Inner class for refreshing the view.   

     */     private class RefreshTask extends TimerTask { public void run() {

      // Get the canvas to repaint itself. myCanvas.repaint(); 
  }     }      /**      * Inner class for handling the canvas.   

     */     class JesterCanvas extends Canvas { JesterTestlet myTestlet; 
 /**  * Construct a new canvas. 
 */ JesterCanvas(JesterTestlet Testlet) { myTestlet = Testlet; }  

 /**  * Initialize self.  */ void init() { }  

 /**  * Clean up and destroy.  */ void destroy() { }
   /**  * Ask myTestlet to paint itself. 
    */ protected void paint(Graphics g) { myTestlet.paint(g); \
}     } } 

To experiment with this MIDlet, create two subdirectories under your JSR 184 RI installation directory. You can call them whatever you like; for this illustration I name them mysrc and myclasses. All .java and .class files will be written to mysrc, and all verified .class files will be written to myclasses. Now do the following:

1. Copy the above MIDlet and save it in mysrc.
2. Compile it:

   javac -classpath ..\classes.zip JesterTeslet.java

3. Preverify the .class files using the preverify command that comes with the RI. From the bin directory:

   preverify -classpath ..\classes.zip -d ..\myclasses ..\mysrc.

   This command verifies all classes in the mysrc directory and writes the verified classes to myclasses.

4. Run the MIDlet. From bin:

midp -classpath ..\myclasses JesterTestlet

You can also test JesterTestlet in the J2ME Wireless Toolkit 2.2; it should work just fine.

2. Immediate Mode

The second full example uses the low-level interface (immediate mode) to display a rotating, texture-mapped cube. The MIDlet initializes a 3D graphics context and binds it to a MIDP canvas. It then illustrates how to create a Mesh object manually, and set up the coordinates, triangle connectivity, texture maps, and materials. In normal practice, however, you'll create Mesh objects with a 3D modeling tool rather than programmatically, then retrieve them using the load() method.

   import javax.microedition.lcdui.*;

    import javax.microedition.m3g.*; 
     public    class MyCanvas extends Canvas {      private Graphics3D      iG3D; 
 private Camera  iCamera;
      private Light   iLight; 
  private float   iAngle = 0.0f;  
     private Transform       iTransform = new Transform(); 
 private Background      iBackground = new Background(); 
     private VertexBuffer    iVb;  
       // positions, normals, colors, texcoords     private IndexBuffer     iIb;
   // indices to iVB, forming triangle strips     private Appearance      iAppearance; 
   // material, texture, compositing, ...     private Material iMaterial  = new Material(); 
       private Image   iImage;  
   /**      * Construct the Displayable.   
      */     public MyCanvas() { 
      // set up this Displayable to listen to command events setCommandListener(new CommandListener() 

       { public void commandAction(Command c, Displayable d) { 
          if (c.getCommandType() == Command.EXIT) { 
          // exit the MIDlet MIDletMain.quitApp(); 
       } } }); 
       try { init();  
      } catch(Exception e) {  e.printStackTrace(); }     }    
        /**      * Component initialization.     
         */     private void init() throws Exception  {
          // add the Exit command addCommand(new Command("Exit", Command.EXIT, 1)); 

       // get the singleton Graphics3D instance iG3D = Graphics3D.getInstance(); 
// create a camera iCamera = new Camera(); 
iCamera.setPerspective( 60.0f, 

// field of view (float)getWidth()/ (float)getHeight(),

  // aspectRatio 1.0f,     

   // near clipping plane 1000.0f );
 // far clipping plane 

 // create a light iLight = new Light(); iLight.setColor(0xffffff);

// white light iLight.setIntensity(1.25f);  
// overbright     
     // init some arrays for our object (cube)   
    // Each line in this array declaration represents a triangle strip  
   // for one side of a cube. The only primitive we can draw with is the  
  // triangle strip so if we want to make a cube with hard edges we   
// need to construct one triangle strip per face of the cube.
 // 1 * * * * * 0   
       //   * *     *   
     //   *   *   *     
 //   *     * *     
     // 3 * * * * * 2      
// The ASCII diagram above represents the vertices in the first line    
     // (the first tri-strip) short[] vert = { 10, 10, 10,  -10, 10, 10,   10,-10, 10,  -10,-10, 10,  
      // front-10, 10,-10,   10, 10,-10,  -10,-10,-10,   10,-10,-10, 
        // back-10, 10, 10,  -10, 10,-10,  -10,-10, 10,  -10,-10,-10, 
// left 10, 10,-10,   10, 10, 10,   10,-10,-10,   10,-10, 10, 
  // right 10, 10,-10,  -10, 10,-10,   10, 10, 10,  -10, 10, 10, 
    // top 10,-10, 10,  -10,-10, 10,   10,-10,-10,  -10,-10,-10 }; // bottom   
 // create a VertexArray to hold the vertices for the object VertexArray vertArray = 

 new VertexArray(vert.length / 3, 3, 2); vertArray.set(0, vert.length/3, vert); 

  // The per-vertex normals for the cube; these match with the vertices  

     // above. Each normal is perpendicular to the surface of the object at       
       // the corresponding vertex. byte[] norm = {   0, 0, 127,    0, 0, 127,    0, 0, 127, 

          0, 0, 127, 0, 0,-127,    0, 0,-127,    0, 0,-127,    0, 0,-127,-127, 0, 0, 

            -127, 0, 0,   -127, 0, 0,   -127, 0, 0, 127, 0, 0,    127, 0, 0,  
              127, 0, 0,    127, 0, 0, 0, 127, 0,    0, 127, 0,    0, 127, 0,

                  0, 127, 0, 0,-127, 0,    0,-127, 0,    0,-127, 0,    0,-127, 0 };   
      // create a vertex array for the normals of the object VertexArray normArray = new VertexArray(norm.length / 3, 3, 1);    
 normArray.set(0, norm.length/3, norm);   
// per vertex texture coordinates short[] tex = {   1, 0,       0, 0,       1, 1,       0, 1, 1, 0, 
      0, 0,       1, 1,       0, 1, 1, 0,       0, 0,       1, 1,       0, 1, 1, 0,   
          0, 0,       1, 1,       0, 1, 1, 0,       0, 0,       1, 1,       0, 1, 1, 0,   

              0, 0,       1, 1,       0, 1 };  
// create a vertex array for the texture coordinates of the object VertexArray texArray = new VertexArray(tex.length / 2, 2, 2);   
      texArray.set(0, tex.length/2, tex);   
     // the length of each triangle strip int[] stripLen = { 4, 4, 4, 4, 4, 4 }; 

      // create the VertexBuffer for our object VertexBuffer vb = iVb = new VertexBuffer();   
   vb.setPositions(vertArray, 1.0f, null);     
    // unit scale, zero bias vb.setNormals(normArray);   
  vb.setTexCoords(0, texArray, 1.0f, null);  
    // unit scale, zero bias
      // create the index buffer for our object (this tells how to   
    // create triangle strips from the contents of the vertex buffer). iIb = new TriangleStripArray( 0, stripLen );
      // load the image for the texture iImage = Image.createImage( "/texture.png" );      
  // create the Image2D (we need this so we can make a Texture2D) Image2D image2D = new Image2D( Image2D.RGB, iImage );      
      // create the Texture2D       
 // texture color is to be modulated with the lit material color Texture2D texture = new Texture2D( image2D );  
texture.setFiltering(Texture2D.FILTER_NEAREST,      Texture2D.FILTER_NEAREST); 
       texture.setWrapping(Texture2D.WRAP_CLAMP,     Texture2D.WRAP_CLAMP);   
     texture.setBlending(Texture2D.FUNC_MODULATE);     
  // create the appearance iAppearance = new Appearance();  
 iAppearance.setTexture(0, texture);   
       iAppearance.setMaterial(iMaterial);   
     iMaterial.setColor(Material.DIFFUSE, 0xFFFFFFFF); 
       // white iMaterial.setColor(Material.SPECULAR, 0xFFFFFFFF); 
// white iMaterial.setShininess(100.0f); 
 iBackground.setColor(0xffffcc);
  // set the background color     }    
    /**      * Paint the scene.      */     protected void paint(Graphics g) {      
// Bind the Graphics of this Canvas to our Graphics3D. The       
  // viewport is automatically set to cover the entire clipping     
      // rectangle of the Graphics object. The parameters indicate    
   // that z-buffering, dithering, and true color rendering are    
// enabled, but antialiasing is disabled. iG3D.bindTarget(g, true, Graphics3D.DITHER | Graphics3D.TRUE_COLOR);    
      // clear the color and depth buffers iG3D.clear(iBackground);  
      // set up the camera in the desired position Transform transform = new Transform();  
     transform.postTranslate(0.0f, 0.0f, 30.0f); 
       iG3D.setCamera(iCamera, transform);   
      // set up a "headlight": a directional light shining    
   // from the direction of the camera iG3D.resetLights(); 
   iG3D.addLight(iLight, transform);    
 // update our transform (this will give us a rotating cube) iAngle += 1.0f; iTransform.setIdentity();  
iTransform.postRotate(iAngle,       // rotate 1 degree per frame1.0f, 1.0f, 1.0f);  
// rotate around this axis       
   // Render our cube. We provide the vertex and index buffers  
  // to specify the geometry; the appearance so we know what    
       // material and texture to use; and the transform to tell 
       // where to render the object iG3D.render(iVb, iIb, iAppearance, iTransform);
 // flush iG3D.releaseTarget();     } }  
  import javax.microedition.midlet.*; 
  import javax.microedition.lcdui.*;
   import java.util.*;  public    class MIDletMain extends MIDlet {     static MIDletMain instance; 
       MyCanvas displayable = new MyCanvas(); 
   Timer iTimer = new Timer(); 
/**      * Construct the midlet.   
   */     public MIDletMain() { this.instance = this;     }  
    /**      * Main method.     
     */     public void startApp() { Display.getDisplay(this).setCurrent(displayable);  
       iTimer.schedule( new MyTimerTask(), 0, 40 ); 
   }      /**      * Handle pausing the MIDlet.   
      */     public void pauseApp() {     }    
/**      * Handle destroying the MIDlet.   
   */     public void destroyApp(boolean unconditional) {     }       
      /**      * Quit the MIDlet.   
 */     public static void quitApp() { instance.destroyApp(true); 
      instance.notifyDestroyed();      
 instance = null;     }      /**      * Timer task for providing animation.  
     */class MyTimerTask extends TimerTask {      
public void run() { if( displayable != null ) {     displayable.repaint(); 
    } }     } }  

To play with this example, you can follow the same steps as with the retained-mode example – or you can run it using the J2ME Wireless Toolkit 2.2:

1. In the toolkit create a new project and give it any name you like – I call it 3d – but make sure the name of the MIDlet is MIDletMain. After you click on the Create Project button, in the Settings window be sure you include the Mobile 3D Graphics API.
2. Copy the two example classes, MyCanvas and MIDletMain, from this article and save them as .java files in \WTK-HOME\apps\3d\src.
3. Download the texture file texture.png and save it in \WTK-HOME\apps\3d\res.
4. Build and run the project; you'll see something like Figure 6.

< span class="dkcaption1">Figure 6: Rendering in Immediate Mode

Conclusion

The emerging Mobile 3D Graphics API for J2ME provides an efficient interface for rendering 3D graphics on CLDC/MIDP devices. It supports devices with little memory and processing power and no hardware support for 3D graphics or floating-point operations, but the API can scale up to higher-end devices that have a color display, floating-point unit, and 3D graphics hardware, as they come along. The applications of this API are many, including games, screen savers, map visualization, and animated messages.

The code samples in this article demonstrate how easy it is to integrate 3D graphics into your MIDlets – but remember that this API is an optional package, and may not be supported on all devices.

For More Information

• JSR 184: Mobile 3D Graphics API for J2ME
• OpenGL
• OpenGL ES
• Mesa 3D
• VRML/X3D: an open file format standard for 3D visual effects, behavior modeling, and interaction.

Acknowledgments

Special thanks to Nicholas Lorain and Vincent Hardy of Sun Microsystems, whose feedback helped me improve this article.

About the Author

Qusay H. Mahmoud provides Java consulting and training services. He has published dozens of articles on Java, and is the author of Distributed Programming with Java (Manning Publications, 1999) and Learning Wireless Java (O'Reilly, 2002).