The J2ME Mobile Media API

Abstract

The Mobile Media API (MMAPI) is an optional package that supports multimedia applications on J2ME-enabled devices. This standard Java specification, defined by the Java Community Process (JCP) in JSR 135, is highly flexible. It has been designed to run with any protocol and format; for example, it doesn't specify that the implementation must support particular transport protocols such as HTTP or Real-Time Transport Protocol (RTP), or media formats such as MP3, MIDI, or MPEG-4

This article provides a technical overview of MMAPI's architecture and APIs, followed by a tutorial in which sample code demonstrates how MMAPI can be used to build multimedia-rich wireless Java applications. A complete media player is developed, and steps for testing it are provided.

Overview of MMAPI

MMAPI has been designed to run on any J2ME-based virtual machine, including the CDC and CLDC VMs. Sun's reference implementation runs on CLDC/MIDP for Windows 2000. The J2ME Wireless Toolkit comes with the MMAPI.

MMAPI's developers designed into it the following features:

• Support for Tone Generation, Playback, and Recording of Time-Based Media: The package supports any time-based audio or video content.
• Small Footprint: MMAPI works within the strict memory limits of CLDC devices.
• Protocol- and Content-Agnostic: The API is not biased towards any specific content type or protocol.
• Subsettable: Developers can limit support to particular types of content, basic audio for example.
• Extensible: New features can be added easily without breaking older functionality. More importantly, additional formats can be easily supported, and the framework is in place for additional controls.
• Options for Implementers: The API offers features for different purposes. The API is designed to allow implementers to leave some features unimplemented if they cannot be supported.

Multimedia Processing

There are two parts to multimedia processing:

• Protocol Handling: reading data from a source such as a file or a streaming server into a media-processing system.
• Content Handling: parsing or decoding the media data and rendering it to an output device such as an audio speaker or video display.

To facilitate these operations, the API provides two high-level object types:

• DataSource encapsulates protocol handling by hiding the details of how the data is read from its source. This object's utility methods enable the Player object to handle the content.
• Player reads the data from DataSource, processes it, and renders it to an output device. This object provides methods to control media playback, including methods for type-specific controls to access features for specific media types.

MMAPI specifies a third object, a factory mechanism known as the Manager, to enable your application to create Players from DataSources, and also from InputStreams. The overall architecture of MMAPI is shown in Figure 1:

Figure 1: The MMAPI Architecture

The Manager object provides the method createPlayer(), which is the top-level entry point into the API. Here's an example:

...
Player player = Manager.createPlayer(String url);
...

The url specifies the protocol and the content, using the format <protocol>:<content location>.

The application uses the methods of the returned Player to control the retrieval and playback of time-based media.

The player's life-cycle includes five states: UNREALIZED, REALIZED, PREFETCHED, STARTED, and CLOSED. Six of its methods result in state transitions:

realize()
prefetch()
start()
stop()
deallocate()
close()

When a player is created, it is in the UNREALIZED state. Calling realize() moves it to the REALIZED state and initializes the information the player needs to acquire media resources. Calling prefetch() moves it to PREFETCHED, establishes network connections for streaming data, and performs other initialization tasks. Calling start() causes a transition to the STARTED state, where the player can process data. When it finishes processing (reaches the end of a media stream), it returns to the PREFETCHED state. Calling close() moves the player to the CLOSED state.

A Player provides controls specific to the particular types of media it processes. The application uses getControl() to obtain a single control, or getControls() to get an array of them. As an example, if a player for MIDI media invokes getControl() it gets back a MIDIControl.

The MMAPI Packages

MMAPI comprises three packages:

• javax.microedition.media provides some interfaces, an exception, and the Manager class, which is the access point for obtaining system-dependent resources such as Players for multimedia processing.
• javax.microedition.media.control defines the specific control types that can be used with a Player: VolumeControl, VideoControl, and others.
• javax.microedition.media.protocol defines the protocols for handling custom controls. For example, it includes the DataSource class, which is an abstraction for media-control handlers.

The classes, interfaces, and exceptions contained in these packages are shown in Tables 1, 2, and 3, respectively.

Table 1: MMAPI Classes

Table 2: MMAPI Interfaces

Table 3: MMAPI Exceptions

Note: MMAPI doesn't mandate particular media types or protocols. These are to be determined by the profile used. An implementation must, however, guarantee support of at least one media type and one protocol. Because optional features are organized as controls, your application can use the methods Manager.getSupportedContentTypes() and Manager.getSupportedProtocols() to query for supported media types and protocols. If you attempt to create a Player for a type or protocol that's not supported, a MediaException will be thrown.

The choice of functionality is left to the implementation as well. It may support media playback but not recording, some devices may support volume control while others may not, and so on.

Using MMAPI

This section demonstrates how to use the multimedia APIs, but provides only short snippets of sample code. To see working examples, look at the mmademo project that comes with the J2ME Wireless Toolkit. You'll find it in <j2me-toolkit>/wtk20/apps/mmademo.

Tone Generation

Tone generation is a characterized by frequency and duration. This type of media is important for games and other audio applications, especially on small devices, where it might be the only form of multimedia capability available.

The Manager.playTone() method generates tones. Its implementation can be mapped to the hardware's tone generator. You specify the note, duration, and volume:

...
try {
   // play a tone for 4000 milliseconds at volume 100
   Manager.playTone()(ToneControl.C4, 4000, 100);
} 
catch(MediaException me) {
}
...

You can also create a player for synthesizing tone sequences:

...
Player player =
  Manager.createPlayer(Manager.TONE_DEVICE_LOCATOR);
...

This type of player provides a ToneControl that can be used to program a tone sequence. This feature is available only on the more powerful devices.

MP3 Playback

This snippet shows how to play simple media with no fine-grained playback control:

...
try {
   Player p = Manager.createPlayer
     ("http://server/somemusic.mp3");
   p.setLoopCount(5);
   p.start();
} 
catch(IOException ioe) {
} 
catch(MediaException e) {
}
...

This one adds a little control:

...
Player p;
VolumeControl vc;
try {
   p = Manager.createPlayer("http://server/somemusic.mp3");
   p.realize();
   // get volume control for player and set volume to max
   vc = (VolumeControl) p.getControl("VolumeControl");
   if(vc != null) {
      vc.setVolume(100);
   }
   // the player can start with the smallest latency
   p.prefetch();
   // non-blocking start
   p.start();
} 
catch(IOException ioe) {
} 
catch(MediaException e) {
}
...

Because any InputStream can be passed to the Manager.createPlayer() method, your application can play back media from a MIDP Record Management System (RMS), or from a JAR file. Here's how to get an InputStream from an RMS and play media stored in it:

...
RecordStore store;
int id;
// play back from a record store
try {
   InputStream is = new ByteArrayInputStream
     (store.getRecord(id));
   Player player = Manager.createPlayer(is, "audio/X-wav");
   p.start();
} 
catch (IOException ioe) {
} 
catch (MediaException me) {
}
...

...And here's how to the same thing with media stored in a JAR:

...
try {
   InputStream is = 
     getClass().getResourceAsStream("audio.wav");
   Player player = Manager.createPlayer(is, "audio/X-wav");
   p.start();
} 
catch(IOException ioe) {
} 
catch(MediaException me) {
}
...

Video Playback

This snippet shows you how to play back an MPG video:

...
Player p;
VideoControl vc;  
try {
   p = Manager.createPlayer("http://server/somemovie.mpg");
   p.realize();
   // get video control
   vc = (VideoControl) p.getControl("VideoControl");
   ....
   p.start();
} 
catch(IOException ioe) {
} 
catch(MediaException me) {
}
...

Camera

MMAPI includes support for a camera, with a special locator capture://video used to create its Player. An application can use the VideoControl to display a viewfinder on the screen, then take a picture using VideoControl.getSnapshot(String imageType). The default image format is PNG. You can use the imageType parameter to select any other supported format, and query the system property video.snapshot.encodings to find out what formats are supported.

The J2ME Wireless Toolkit and MMAPI

The J2ME Wireless Toolkit 2.0 supports MMAPI. It provides demo applications and a MediaControlSkin – an emulator capable of playing audio and video content. To experiment with the demos, open the mmademo project and run it. You should see a list like the one shown in Figure 2.

Figure 2: J2ME Wireless Toolkit MMAPI Demo Project

Supported Formats

MMAPI supports several audio and video formats, of which the J2ME Wireless Toolkit supports:

• Audio: PCM and WAV
• MIDI: Type 0 (single track), Type 1 (multiple tracks), and SP-MIDI
• Video: MPEG-1

Limitations on Using MMAPI in the Toolkit

Keep in mind that the J2ME Wireless Toolkit has the following limitations – and that every device will have its own, different limitations:

• An application can create multiple players, but can realize only one MIDI player or one tone-sequence player. Any attempt to realize a second player of either type will throw a MediaException. In other words, only one MIDI or tone sequence player can be in use at any time.
• The toolkit allows any number of simultaneous playTone() calls for a single note, but guarantees only that at least four will actually work. The playTone() method can be called while a MIDI or tone-sequence player is playing.
• The number of video players is limited by the size of the heap, as specified in the -Xheapsize option of the emulator command if you are using the command line utilities, or from KToolBar by choosing Edit -> Preferences -> Storage.
• The number of WAV audio players is also limited by the heap size.
• Only one audio capture can be in use at one time.

Note also that MIDP limits the number of HTTP connections open simultaneously to four.

Example: Simple Media Player

To give you an example of using the Mobile Media APIs I've developed a simple media player. PlayerMIDlet allows the user to enter the URL of an audio or video file and plays it back:

Code Sample 1: PlayerMIDlet.java

import javax.microedition.lcdui.*;
import javax.microedition.media.*;
import javax.microedition.media.control.*;
import javax.microedition.midlet.*;

public class PlayerMIDlet extends MIDlet implements 
  CommandListener, PlayerListener, Runnable {
   private Display display;
   private Form form;
   private TextField url;
   private Command start = new Command("Play", 
     Command.SCREEN, 1);
   private Command stop = new Command("Stop", 
     Command.SCREEN, 2);
   private Player player;
      
   public PlayerMIDlet() {
      display = Display.getDisplay(this);
      form = new Form("Demo Player");
      url = new TextField("Enter URL:", "", 100, 
            TextField.URL);
      form.append(url);
      form.addCommand(start);
      form.addCommand(stop);
      form.setCommandListener(this);
      display.setCurrent(form);
   }

   protected void startApp() {
      try {
         if(player != null && player.getState() == 
                   Player.PREFETCHED) {
            player.start();
         } else {
            defplayer();
            display.setCurrent(form);
         }
      } 
      catch(MediaException me) {
         reset();
      }
   }

   protected void pauseApp() {
      try {
         if(player != null && player.getState() == 
                   Player.STARTED) {
            player.stop();
         } else {
            defplayer();
         }
      } 
      catch(MediaException me) {
         reset();
      }
   }

   protected void destroyApp(
      boolean unconditional) {
      form = null;
      try {
         defplayer();
      } 
      catch(MediaException me) {
      }
   }

   public void playerUpdate(Player player, 
     String event, Object data) {
      if(event == PlayerListener.END_OF_MEDIA) {
         try {
            defplayer();
         } 
         catch(MediaException me) {
         }
         reset();
      }
   }

   public void commandAction(Command c, Displayable d) {
      if(c == start) {       
         start();
      } else if(c == stop) {
         stopPlayer();
      }
   }

   public void start() {
      Thread t = new Thread(this);
      t.start();
   }

   // to prevent blocking, all communication should 
   // be in a thread
   // and not in commandAction
   public void run() {
      play(getURL());
   }

   String getURL() { 
     return url.getString();
   }

   void play(String url) {
      try {
         VideoControl vc;
         defplayer();
         // create a player instance
         player = Manager.createPlayer(url);
         player.addPlayerListener(this);
         // realize the player
         player.realize();
         vc = (VideoControl)player.getControl(
                   "VideoControl");
         if(vc != null) {
            Item video = (Item)vc.initDisplayMode(
                          vc.USE_GUI_PRIMITIVE, null);
            Form v = new Form("Playing Video...");
            StringItem si = new StringItem("Status: ", 
                          "Playing...");
            v.append(si);
            v.append(video);
            display.setCurrent(v);
         }
         player.prefetch();
         player.start();
      }
      catch(Throwable t) {
         reset();
      }
   }

   void defplayer() throws MediaException {
      if (player != null) {
         if(player.getState() == Player.STARTED) {
            player.stop();
         }
         if(player.getState() == Player.PREFETCHED) {
            player.deallocate();
         }
         if(player.getState() == Player.REALIZED || 
                    player.getState() == Player.UNREALIZED) {
            player.close();
         }
      }
      player = null;
   }

   void reset() {
      player = null;
   }

   void stopPlayer() {
      try {
         defplayer();
      } 
      catch(MediaException me) {
      }
      reset();
   }
}

The play() method performs the most interesting tasks:

• Creates a Player instance
• Registers the MIDlet as a PlayerListener, because the MIDlet implements the PlayerListener interface by defining a playerUpdate() method
  When a player event occurs, all player listeners registered are notified. An event is denoted by a string constant such as PlayerListener.END_OF_MEDIA.
• Realizes the player
• Checks to see whether a VideoControl is present; if so, initializes an Item display component and adds it to a form to be displayed
• Prefetches the player and starts it

You can test the simple player MIDlet using the J2ME Wireless Toolkit 2.0. Create a new project ( SimplePlayer for example), and copy the PlayerMIDlet source code to <j2me-toolkit>/apps/SimplePlayer/src. Build the project, and run the MIDlet using the following URLs:

null

After entering each URL, select Play from the menu, as in Figure 3:

Figure 3: Testing PlayerMIDlet

Conclusion

This article described the architecture and features of the Mobile Media API and showed you how to use MMAPI to develop multimedia-rich wireless Java applications. A sample MIDlet created a simple player for audio and video media files from remote network servers. You can enhance PlayerMIDlet yourself use your imagination! For working sample applications, see the mmademo project in the J2ME Wireless Toolkit 2.0.

For more information:

• MIDP 2.0 (JSR 118)
• CLDC 1.1 (JSR 139)
• Mobile Media API (JSR 135)

About the Authors: Qusay H. Mahmoud provides Java consulting and training services. He has published dozens of articles on Java, and is the author of Distributed Programming with Java (Manning Publications, 1999) and Learning Wireless Java (O'Reilly & Associates, 2002).