Introduction to OTA Application Provisioning

The term over-the-air provisioning (OTA) describes the ability to download and install content over a wireless network, typically on demand. In this article we'll go over what OTA is, how it works, and how it affects you, for both MIDP 1.0 and MIDP 2.0. We'll also look at the server impacts - how any web site can be used as a download server, and how the new J2EE Client Provisioning specification looks at a high level. Note that even though this article will concentrate on OTA provisioning of applications, specifically Mobile Information Device Profile (MIDP) applications (MIDlets), some of the concepts covered also apply to the provisioning of other types of content.

OTA Overview

From the mobile client's perspective, in concept OTA is simply a matter of finding an interesting application on the Web and initiating its download over the wireless network. In its simplest form, OTA is pretty straightforward, as Figure 1 illustrates:

Figure 1 - Simplified View of an OTA Provisioning Architecture

Note the participants in this process:

• Client device with a discovery application - The client device must have software that allows the user to locate applications at a particular provisioning portal on the network, and to choose which applications to download - this software is referred to as a discovery application (DA). The discovery application may be browser-based or a native application, so long as it shares a common provisioning protocol with the download server; for example, for MIDP OTA this protocol is HTTP.
• The Network - This is any appropriate wireless network, which may include a radio network and a WAP gateway, for example.
• The Download Server - Also called a provisioning portal, or server, or vending machine, this is a host, visible in the network, that typically runs a web server and has access to a content repository - in its simplest form any Web site can serve as a provisioning portal. It has two main functions: it provides properly formatted menus, often written in WML or HTML, that list the applications currently available for download, and it provides access (sometimes controlled) to the applications. A single download server might support one or many OTA protocols.
• The Content Repository - As the name implies, this is the repository of all the application descriptors and applications that are available for download.

In actual use, OTA is not quite so simple. An OTA provisioning system typically encompasses content publication and management, access control, installation (and upgrading) of applications, and tracking the use of applications (content) for billing purposes. Figure 2 introduces these complexities:

Figure 2 - Detailed View of an OTA Provisioning System

As you can see, the process behind the scenes is more involved than what the user actually sees:

• A. Content Management - Server-side software manages the repository, typically a database, and supports content versioning, ways for third-party developers to drop their applications, and so on. Some carriers require applications be certified before making them available for OTA.
• B. Content Discovery - As you've already seen, the user directs the discover application to a download portal, which accesses the application repository and provides a properly formatted menu of the available content and applications.
• C. Authenticate - If the provisioning server supports an authentication module, the user is authenticated before gaining access to the repository.
• D. Application retrieval and installation - Once any authentication is complete, downloading the application is a two-part operation, handled by the application management system (AMS), the software in the device that manages the download, installation, execution, and removal of applications and other resources on the device. If an application description (in the form of a JAD file) exists, the AMS downloads it from the provisioning server's repository. Based on information found in the downloaded application descriptor, the AMS automatically downloads the application (the MIDlet suite JAR) from the repository. If required, the user is re-authenticated. If the application is retrieved successfully, installation is automatic.
• E. Confirmation - The AMS sends a confirmation status to indicate whether the installation succeeded or failed.
• F. Tracking - The confirmation status can be used to track the use of the application, for example for billing purposes. A billing system is often integrated into the provisioning server.

In addition to the basic features described above, some OTA provisioning servers add value in the form of features such as:

• Personalization - the ability to offer selected applications based on user preferences
• Device-based classification - limiting offerings to only those applications that will run on the user's device
• Security - strong user authentication and access control
• Alerts - notices to users that new applications fitting their preferences are available

You now have a high-level perspective of OTA application provisioning. The following sections will turn to the details, first on the client side, where we'll look at MIDlet provisioning specifically, and then on the server side.

MIDP OTA Specification Overview

Let's take a look at the OTA of MIDP applications. The purpose of this section is not to duplicate the information in the OTA specification (see the resources section), but to provide you with a good starting point, then encourage you to spend some time reading the actual specification.

The MIDP OTA specification defines how MIDP devices interact with provisioning servers, including how they download applications. The MIDP OTA specification defines a user-initiated OTA process - not a server-initiated "push" OTA mechanism.

The first version of the MIDP OTA specification appeared after the MIDP 1.0 specification itself. It was not a true specification, only a recommended practice, but it provided a good foundation for the protocol for OTA. Many devices currently support the MIDP1.0 recommendation.

MIDP 2.0 improved the MIDP 1.0 recommendation and made it part of the specification. Because it is now part of the standard, all future MIDP devices will support OTA in a consistent, standard way. It's a great benefit to be able to distribute applications to many different mobile devices all in the same way.

The two versions are very similar, but there are a few differences, such as a new JAD property used by the AMS to report when MIDlet suites are removed, and more stringent checks when updating MIDlet suites. The use of cookies has also been replaced by URL rewriting. The MIDP OTA specification can be broken down into the following areas of interest:

• Expected device functionality (in support of OTA)
• The OTA provisioning life-cycle
• Changes to the Java Application Descriptor in support of OTA
• Interactions between the AMS and the provisioning server

Let's consider each of these now.

Expected Device Functionality

There is certain functionality that MIDP-compliant devices must provide in their support of OTA:

• Support for version 1.1 of the Hypertext Transport Protocol (HTTP) - or a protocol that implements the functionality of HTTP 1.1, including support for HTTP Basic Authentication (RFC 2617) and the ability to prompt for (and resend) user credentials.
• Support for a discovery application, as described in the "OTA Overview," that allows the user to locate applications on the network, and to choose which applications to download.
• Support for an application management system, the piece of software in the device that manages the application's life-cycle - download, installation, execution, and removal. (We'll look at life-cycles in the next section.) Another name for the AMS on J2ME-enabled devices is the Java application manager (JAM).

OTA Provisioning Life-cycle

You already have a high-level view of some elements of the OTA life-cycle. Let's look at discovery, installation, update, execution, and removal of applications in some detail. The OTA life-cycle is summarized graphically in Figure 3.

Figure 3 - OTA Application Provisioning Life-cycle

It all starts with the user instructing the device (using the DA) to search for an application of interest on a particular provisioning portal on the network. On finding one, the user selects the application to download and install. After a successful installation, the application can be executed, updated, and removed from the device - again, all these stages of the life-cycle are managed by the AMS.

Most of the OTA life-cycle is transparent to us - the network protocols, the user interface details, and the DA and AMS implementations. Besides initiating the installation, update, or removal of the application, the things that are not transparent revolve around configuration, such as changes to the JAD file, .configuring the DA to point to the right server, and configuring the web server to serve content appropriately, as I will describe shortly.

Discovery

Application discovery is about locating an application of interest on the wireless network - pointing the device's DA to a provisioning server, downloading a menu of applications available for OTA, and selecting an application of interest. Application discovery might also entail downloading JAD files of applications that have them.

In some devices, including those from Nokia and Siemens, the discovery application exploits a WAP browser, but it can be a native application as well. How the discovery application is configured for OTA is vendor-specific - consult your device documentation on how to configure the OTA settings.

The DA initiates discovery with an HTTP request to get the list of available applications / MIDlet suites; for example:

GET /ota/index.html HTTP/1.1

Host: www.j2medeveloper.com:80
:
:

The response is a list or menu of available applications, with a description and a URL for each. Here is an example of a response that contains a menu in WML:

HTTP response with application menu in WML:

HTTP/1.1 200 OK

Content-Type: application/vnd.wap.wml, text/x-wap.wml, text/vnd.wap.wml
Content-Length: .....
:
:
<?xml version="1.0"?>
<!DOCTYPE wml PUBLIC "-//WAPFORUM//DTD WML
1.1//EN" "http://www.wapforum.org/DTD/wml_1.1.xml">
<wml>
<head>
<meta forua="true" http-equiv="Cache-Control"
content="must-revalidate, no-store"/>
</head>
<template>
<do type="prev" label="Back">
<prev/>
</do>/
</template>
<card title="Available MIDlets" id="main">
<p>Available MIDlets:</p>
<p align="left">
<a href="
/mobility/midp/articles/ota/<b>http://www.j2medeveloper.com/ota/demos.jad</b>">
WTK104 demos</a></p>
<a href=
"/mobility/midp/articles/ota/<b>http://www.j2medeveloper.com/ota/weather.jar</b>">
Weather</a></p>
</card>
</wml>

Here is an example of a response that contains the same menu in HTML:

HTTP response with application menu in HTML:

HTTP/1.1 200 OK

Content-Type: text/html
Content-Length: .....
:
:
<html>
<head>
<title>J2MEDeveloper.com MIDlet Menu</title>
</head>
<body>
<p>Welcome to J2MEDeveloper.com</p>
<p>
Available MIDlets: <br>
<a href=
"/mobility/midp/articles/ota/www.j2medeveloper.com/ota/demos.jad">
WTK104 demos.jad</a>
<a href=
"/mobility/midp/articles/ota/www.j2medeveloper.com/ota/weather.jar">
Weather</a>
</p>
Enjoy!
</body>
</html>

Listing 1 - Discovery Response - an Application Menu

The received menu is then rendered so the user can select an application; Figure 4 shows how Listing 1 could be rendered.

Figure 4 - A Menu of Available Applications

In Listing 1 you can see that the application's URL can point to either a JAD or a JAR - but note that MIDP 1.0 OTA (and thus version 1.0.4 of the J2ME Wireless Toolkit) only supports JAD files, while MIDP 2.0 supports both JADs and JARs. If the URL refers to a JAR file, the AMS uses that URL to download the JAR. If the URL in the menu refers to a JAD file, the AMS downloads the JAD, which contains the JAR's URL, then uses that URL to download the JAR. Either way, the AMS begins installation as soon as download is complete.

It is important to mention that, even though the MIDP 2.0 specification does not require developers to provide a JAD file (the URL can point to the MIDlet suite directly), it does recommend providing one, because it helps the AMS determine, before the download ever takes place, whether the device has sufficient resources for the application to be installed successfully.

Some OTA download servers implement a presentation layer that takes care of formatting the menu appropriately depending on the device's characteristics, returning the menu in HTML or WML or XML, for example. The download server can also customize the discovery phase to the characteristics of the user's device by analyzing the HTTP headers that come from it. For example, the User-Agent provides information on the device model and the software version, the Accept header provides information on types of content the device is willing to accept from the server, and the Accept-Language header provides information on the language (localization). Using this information, the server can restrict the menu of available applications to only those applications the connecting device supports.

Assuming that all goes OK, the AMS proceeds directly to installation.

Installation and Update

The installation process consists of downloading a MIDlet suite JAR and making it available to the user for execution.

If the application has an associated JAD file, the AMS has downloaded it during the discovery process, and will now download the MIDlet suite from the OTA portal indicated by the JAD property MIDlet-Jar-URL. If no JAD file is provided, the AMS uses the URL to the MIDlet suite that was indicated in the menu of applications that came from the download server - as shown in Listing 1.

If it's updating an existing installation, the AMS must compare version numbers and notify the user whether it is installing an older, newer, or identical version. It also must preserve the contents of the Record Management System (RMS), the persistent storage. MIDP 2.0 adds some security checks to preclude unwanted access to the RMS by the new MIDlet suite: Both the JAD and JAR must come from the same source as the original MIDlet suite. For signed MIDlet suites, both the original and the new MIDlet suite must have the same cryptographic signer. Last but not least, the AMS will not allow an unsigned MIDlet suite to replace a signed one.

The AMS will report back to the download server any errors encountered during the installation process, as I describe in the "Status Report" section.

Now we'll examine the interactions between the AMS and the provisioning portal.

AMS-Provisioning Portal (Client-Server) Interaction

Let's briefly survey the interactions between the client device and the download server during OTA provisioning. As with any network communications between two endpoints, there is a protocol that specifies the rules and dialect used between the two ends. In MIDP, the OTA protocol comprises HTTP requests and responses, the downloading of JAD files and MIDlet suites JARs, HTTP Basic Authentication, and status reporting.

The following listing illustrates the use of MIDlet-Jar-URL to describe the remote location of the MIDlet suite:

                    JAD File:


Created-By: 1.3.1 (Sun Microsystems Inc.)
MIDlet-1: Colors, /icons/ColorChooser.png, example.chooser.Color
MIDlet-2: Properties, /icons/App.png, example.PropExample
MIDlet-3: Http, , example.http.HttpTest
MIDlet-4: FontTestlet, , example.fonts.FontTestlet
MIDlet-5: Stock, /icons/Stock.png, example.stock.StockMIDlet
MIDlet-6: Tickets, /icons/Auction.png, example.auction.TicketAuction
MIDlet-7: ManyBalls, /icons/ManyBalls.png, example.manyballs.ManyBalls
MIDlet-Description: Technical demonstration programs for MIDP
MIDlet-Jar-Size: 144445
MIDlet-Jar-URL:
 http://www.j2medeveloper.com:80/ota/demos.jar
MIDlet-Name: SunSamples - Demos
MIDlet-Vendor: Sun Microsystems, Inc.
MIDlet-Version: 1.0.3
Manifest-Version: 1.0
                

Listing 2 - The MIDlet-JAR-URL JAD property

Assuming a MIDlet-JAR-URL value of www.j2medeveloper.com/ota/demos.jad, the interactions between the client and the server to download the JAD and JAR files are as in Figure 5.

Figure 5 - OTA Client-Server Interactions

If something goes wrong during download or installation, the AMS will respond with the appropriate status code - see "Status Reports."

Execution

Once the application has been downloaded and installed, it can be executed. The AMS allows the user to select a MIDlet suite, then the desired MIDlet within that suite. Once the MIDlet is selected, the AMS instantiates it, which starts its life-cycle in the Paused state. The AMS transitions the MIDlet into the Active state (invokes the MIDlet's startApp() method) almost immediately. Figure 6 depicts the MIDlet life-cycle:

Figure 6 - MIDlet Life-cycle

Now let's see what happens during application removal.

Removal

The AMS allows the user to select an application for removal. On the user's confirmation, it then removes the selected MIDlet suite - note that all MIDlets associated with the selected suite are removed and the associated RMS storage reclaimed. If a MIDlet-Delete-Confirm JAD property is defined, the AMS uses its value in the confirmation prompt - see "Java Application Descriptor and OTA" later in the article. After the removal of the MIDlet suite, the AMS reports the status code 912 back to the server - see the "Status Reports" section, next.

Status Reports

Provisioning portals may take advantage of status reports to track the use of an application - for example, for billing purposes or to prioritize their content repository.

If the status-reporting JAD properties are defined, as explained shortly, whenever a MIDlet suite is installed, updated, or removed, the AMS informs the provisioning server via an HTTP POST. When reporting status, the AMS uses the value of the appropriate Notify JAD property as the URL to POST to - see Table 2. To report the status of an installation or update, the AMS uses the value of the JAD entry MIDlet-Install-Notify, if one is defined. For deletions, the AMS uses the value of the JAD entry MIDlet-Delete-Notify, if one is defined. Figure 7 shows the interaction that occurs when the client reports status:

Figure 7- Status Report Interaction

The status report's HTTP POST body is one of the status codes in Table 1 followed by a status message. The server responds with 200 OK, indicating the end of the status report interaction. Table 1 summarizes the status codes that the AMS must support.

Table 1 - Status Codes

Note that some OTA provisioning systems might go beyond the basic status reporting defined by the OTA specification. By instrumenting MIDlets before they are downloaded, systems can build in additional levels of tracking. The MIDlet life-cycle methods startApp(), destroyApp(), and pauseApp() can be wrapped with code that records and transmits each use of the application, not just installation and removal.

Java Application Descriptor and OTA

To support OTA, you have to make a few changes to the Java Application Descriptor (JAD) file. Four JAD properties relate to OTA, as described in Table 2.

Table 2 - JAD properties for OTA

Let's look at an example of a JAD file containing these properties:

                    JAD File:


Created-By: 1.3.1 (Sun Microsystems Inc.)
MIDlet-1: Colors, /icons/ColorChooser.png, example.chooser.Color
MIDlet-2: Properties, /icons/App.png, example.PropExample
MIDlet-3: Http, , example.http.HttpTest
MIDlet-4: FontTestlet, , example.fonts.FontTestlet
MIDlet-5: Stock, /icons/Stock.png, example.stock.StockMIDlet
MIDlet-6: Tickets, /icons/Auction.png, example.auction.TicketAuction
MIDlet-7: ManyBalls, /icons/ManyBalls.png, example.manyballs.ManyBalls
MIDlet-Description: Technical demonstration programs for MIDP
MIDlet-Jar-Size: 144445 MIDlet-Jar-URL: http://www.j2medeveloper.com:80/ota/demos.jar 
MIDlet-Install-Notify: http://www.j2medeveloper.com:80/ota/status?action=I&key=1234&user=abcd 
MIDlet-Delete-Notify: http://www.j2medeveloper.com:80/ota/status?action=D&key=1234&user=abcd
MIDlet-Delete-Confirm: 
Are you sure you want to delete the WTK demos?
MIDlet-Name: SunSamples - Demos
MIDlet-Vendor: Sun Microsystems, Inc.
MIDlet-Version: 1.0.3
Manifest-Version: 1.0
                

Listing 3 - Using the OTA JAD Properties

Note in Listing 3 that you can apply URL rewriting to the properties MIDlet-Install-Notify and MIDlet-Delete-Notify to encode information that helps track application use, as in: MIDlet-Install-Notify: http://www.j2medeveloper.com:80/ota/ status?action=I&key=1234&user=abcd

Where parameters can be appended to the URL to help the provisioning server track application usage. In our example, " action=I&key=1234&user=abcd", we have action that indicates "application installed", and a key with an unique application identifier that maps to the installed application, and a user ID indicating the user that is installing the application. Advanced provisioning systems with content management and usage tracking capabilities typically generate these URLs, and possibly the whole JAD file, automatically.

The Server Side of OTA

OTA can be implemented very simply, using a standard web server, but a new standard for the Java 2 Platform, Enterprise Edition (J2EE) is becoming available that will allow for more complex provisioning servers, as I described in the OTA Overview section.

Minimal Configuration

All a minimal OTA configuration needs is a web server that is properly configured to serve content, in our case JAD and JAR files. Configuring the server consists of defining the appropriate MIME types for JAD and JAR files, as indicated in Table 3.

Table 3 - Required MIME types for OTA

The download server identifies these files by setting the HTTP Content-Type header for JADs to text/vnd.sun.j2me.app-descriptor, and for JARs to application/java-archive. The client software (AMS) uses the Content-Type header during the application discovery and installation phases. Also, the OTA MIME types in Table 3 help some gateways deal with content segmentation and reassembly (SAR).

How these values are configured depends on the web server. On Apache Tomcat, for instance, you can either update the global configuration file found at the $CATALINA_HOME/conf/web.xml or you can update your application's web.xml. Just make sure the following two entries are defined:

<!-- JAD file -->

<mime-mapping>
<extension>jad</extension>
<mime-type>text/vnd.sun.j2me.app-descriptor</mime-type>
</mime-mapping>

<!--JAR file -->
<mime-mapping>
<extension>jar</extension>
<mime-type>application/java-archive</mime-type>
</mime-mapping>

Listing 4 - Defining OTA MIME-Types In Tomcat's web.xml

Refer to the article Deploying Wireless Java Applications by Qusay H. Mahmoud, for more information on application deployment, including information on using the Wireless Toolkit to test OTA provisioning.

J2EE Application Server-based Configuration

A web server can support OTA on a small scale, but you'll need a lot more if you want to implement a complete OTA provisioning system like the one described in Figure 2, one that can serve thousands of concurrent users, and that is scalable and expandable. For such a system you will want to exploit J2EE to create a scalable transactional environment and secure access to a database and other back-end resources.

Currently the Java Community Process (JCP) is finalizing a standard for J2EE-based client provisioning. Known as The J2EE Client Provisioning Specification (JSR 124), this JSR is standardizing the way provisioning portals and their clients interact by defining a framework to enable J2EE application servers to serve as client provisioning portals. JSR 124 allows for a provisioning portal like the one illustrated in Figure 2, a robust system that includes content packaging and publishing, authentication and policies, support for many devices (including OTA), and a plug-in architecture that ensures future expandability. Table 4 describes the packages proposed for JSR 124. Note the JSR is not yet final.

Table 4 - J2EE Client Provisioning Core Java Packages

These packages include APIs for access to the application repository and the delivery process, for customizing the server, and for integrating services such as billing. It also defines the concept of an Adapter that abstracts provisioning models and protocols, and an Adapter SPI so that many different provisioning models and protocols can be supported, including the J2ME MIDP OTA. The JSR also standardizes a new packaging format, the Provisioning Archive ( PAR), for content to be provisioned. Figure 8 illustrates J2EE client provisioning, its packages, and interactions.

Figure 8 - J2EE Client Provisioning

Security

OTA application provisioning raises security concerns. These include privacy, access control, and even piracy (unauthorized distribution of unlicensed applications).

Privacy concerns can be addressed by encrypting the communication channel. Luckily, MIDP OTA (and JSR 124/MIDP OTA) interactions are based on HTTP, and major web servers, J2EE application servers, and a number of devices support HTTP and SSL.

Access control is addressed by using authorization and access policies. By exploiting HTTP Basic Authentication and the J2EE-based application server's security features, provisioning servers can implement user management, authentication, and access control.

Piracy is a two-part problem. Content providers want to eliminate both unauthoried downloading, and redistribution of the application after it's downloaded, from the device itself. OTA application provisioning is mainly concerned with the first part of the problem. At this level, piracy prevention is loosely associated with access control - the provisioning portal must properly authenticate users and apply access-control policies before applications are downloaded. Once the application has been downloaded, however, it might be impossible to prevent redistribution, if the application is downloaded to a personal computer or a PDA, for example. The provisioning server could take certain measures to minimize this risk; for example, based on the connecting device type obtained from the User-Agent header, it might refuse to download to devices of concern such as PCs. The application itself can use licensing techniques to protect from illegal redistribution. Note that MIDP 2.0 supports an application-signing model, but this feature is used to minimize infiltration by rogue applications, and provides no means to prevent unauthorized use of applications.

Summary

This article has covered OTA application provisioning - what it is, how it works, and how it affects you. OTA application provisioning is a very important technology, a facility to download applications on the fly that is ideally suited to the mobile space. OTA is officially supported in J2ME MIDP 2.0, and a number of J2EE provisioning servers are available today. JSR 124, which is currently in public review, will soon standardize the way provisioning portals are written and used. You will have your choice of provisioning servers to use when you run your portal, and your portal will be able to provision many different kinds of devices.

Resources

• MIDP 2.0 specification: see http://www.jcp.org/jsr/detail/118.jsp
• J2EE Client Provisioning: see http://www.jcp.org/jsr/detail/124.jsp

Acknowledgements

Many thanks to Danny Coward and Roger Riggs for their contributions to this article.

About the Author: C. Enrique Ortiz, Director of Engineering, Mobile Applications for Aligo. He is an active participant in the wireless Java community, and is the co-author of the Mobile Information Device Profile for J2ME published by John Wiley and Sons. Enrique holds a B.S. in Computer Science from the University of Puerto Rico and has over 13 years of industry experience.