An Introduction to Real-Time Java Technology: Part 1, The Real-Time Specification for Java (JSR 1)

Based on a white paper by Brian Goetz and edited by Robert Eckstein, July 2008

Real-time computing is often associated with high speed, but this is only one part of the picture. At its core, real-time computing is about predictability -- the knowledge that the system will always perform within the required time frame. The deadlines involved need not be very short -- though they sometimes are -- and the consequences of missing a deadline may not be dire -- though they sometimes are. The key to whether an application is a real-time one has to do with whether its requirements include temporal constraints.

The developers of the Real-Time Specification for Java (RTSJ), JSR 1, use this definition of real-time computing:

The programming environment must provide abstractions necessary to allow developers to correctly reason about the temporal behavior of application logic. It is not necessarily fast, small, or exclusively for industrial control; it is all about the predictability of the execution of application logic with respect to time.

Real-time applications are partitioned into a set of tasks, some of which have deadlines. The goal is for all tasks to be scheduled in such a way as to complete before their deadline, if any.

Definitions: Hard-Real-Time and Soft-Real-Time

Real-time systems may be classified differently depending on the system's requirements. A hard-real-time system is one in which the system must meet all deadlines without fail. Typically, such systems also have short latencies, the time between when a triggering event occurs and when the response must begin or complete, often measured in microseconds or milliseconds.

Many hard-real-time systems are further classified as safety-critical systems. These systems are used to protect humans from injury or danger. Safety-critical systems must go through exhaustive testing and a line-by-line code review before certification. Note that safety-critical Java technology is being studied through JSR 302 and is not addressed by the RTSJ at the time of this writing.

A soft-real-time system, on the other hand, is one that will still function correctly, according to its specification, if the system occasionally misses deadlines. For example, a cellular telephone switching system is a soft-real-time system: Customers expect all calls to be completed, but an occasional deadline miss may cause an acceptable failure, such as audio dropouts or delayed call setup. Soft-real-time systems typically specify what percentage of deadlines can be missed or how often this is acceptable.

Unpredictability in a Java Technology-Based Application

A number of factors may render the timing of execution unpredictable and therefore may cause a standard Java task to miss its deadline. Here are the most common.

  • Operating-system scheduling. In Java technology, threads are created by the JVM * but are ultimately scheduled by the operating-system scheduler. In order for the JVM to provide guarantees of temporal latency, that is, the time delay in reaction after an action, the operating system must provide scheduling-latency guarantees as well. Hence, the operating system must offer capabilities such as a high-resolution timer, program-defined low-level interrupts, and a robust priority-based scheduler.
     
  • Priority inversion. One hazard in an application in which threads can have different priorities is priority inversion. If a lower-priority thread shares a resource with a higher-priority thread, and if that resource is guarded by a lock, the lower-priority thread may be holding the lock at the moment when the higher-priority thread needs it. In this case, the higher-priority thread is unable to proceed until the lower-priority thread has completed its work -- and this can cause the higher-priority thread to miss its deadline.
     
    In addition, if some other task with a medium priority that does not depend on the shared resource attempts to run in the interim, it will take precedence over both the low- and the high-priority tasks. The effect of priority inversion is to "downgrade" the priority of the higher-priority thread to that of the lower-priority thread.
     
  • Class loading, initialization, and compilation. The Java language specification requires classes to be initialized lazily -- when an application first uses them. Such instantiations might execute user code, creating jitter, a variance in latency, the first time that a class is used. In addition, the specification allows classes to be lazily loaded. Because loading of classes may require going to disk or across the network to find the class definition, referencing a previously unreferenced class can cause an unexpected -- and potentially huge -- delay. In addition, the JVM has some latitude to decide when, if ever, to translate a class from byte code into native code. Typically, a method is compiled only when it is executed frequently enough to warrant the cost of compilation.
     
  • Garbage collection. The primary source of unpredictability in Java applications is garbage collection (GC). The GC algorithms that standard JVMs use all involve a stop-the-world pause, in which the application threads are stopped so that the garbage collector can run without interference. Applications with hard response-time requirements cannot tolerate long GC pauses. Despite a large amount of work in recent years on reducing GC pauses, a so-called low-pause collector is still not enough to guarantee predictability and still may require significant tuning and testing.
     
  • The application. Another major source of unpredictability is the application itself, including any libraries that it uses. Most applications consist of multiple computational activities that compete equally for CPU resources. Java applications typically do not use thread priorities, partly because the JVM offers such weak guarantees for thread priorities. In a properly provisioned application, enough CPU cycles should be available to go around, but completing a task may sometimes take longer than expected -- which may cause other tasks to have to wait for CPU resources.
     
    Most developers treat this problem iteratively: They look for the most serious bottleneck, improve it, then find the next bottleneck and improve it, repeating these steps until the application reaches acceptable performance. However, because this development process is itself unpredictable, it can often affect delivery schedules or require significant additional testing.
     
  • Other activities in the system. Other high-priority activities that occur in the system -- such as hardware interrupts, other real-time applications, and so forth -- can cause jitter in the application, thereby affecting determinism.

The Real-Time Specification for Java (RTSJ)

The Real-Time Specification for Java (RTSJ), or JSR 1, specifies how Java systems should behave in a real-time context and was developed over several years by experts from both the Java and real-time domains.

The RTSJ is designed to seamlessly extend any Java family -- whether the Java Platform, Standard Edition (Java SE); Java Platform, Micro Edition (Java ME); or Java Platform, Enterprise Edition (Java EE) -- and has the requirement that any implementation pass both the JSR 1 technology compatibility kit (TCK) and the TCK of the platform -- Java SE, Java ME, or Java EE -- on which it is based.

The RTSJ introduces several new features to support real-time operations. These features include new thread types, new memory-management models, and other newly introduced frameworks. Let's start with how the RTSJ views tasks and time.

Tasks and Deadlines

The RTSJ models the real-time part of an application as a set of tasks, each with an optional deadline. The deadline of a task is when the task must be completed. Real-time tasks fall into several types, based on how well the developer can predict their frequency and timing:

  • Periodic: tasks that run on a fixed schedule, such as reading a sensor every five milliseconds
     
  • Sporadic: tasks that do not run on a fixed schedule but that have a maximum frequency
     
  • Aperiodic: tasks whose frequency and timing cannot be predicted

The RTSJ uses this task-type information in several ways to ensure that critical tasks do not miss their deadlines. First, the RTSJ allows you to associate with each task a deadline miss handler. If a task does not complete before its deadline, this handler is called.

Deadline-miss information can be used in deployment to take corrective action or to report performance or behavioral information to the user or to a management application. By comparison, in non-real-time applications, failures may not become apparent until secondary or tertiary side effects arise -- such as request timeouts or memory depletion -- at which point it may be too late to recover gracefully.

Deadline-miss handling is deferred to a deadline miss handler, like this:

ReleaseParameters.setDeadlineMissHandler(
                AsyncEventHandler handler);

Or if there is no handler, it can be performed by the thread itself:

if (waitForNextPeriod() == false) {
                handle_deadline_miss();
}

Thread Priorities

In a true real-time environment, thread priorities are extremely important. No system can guarantee that all tasks will complete on time. However, a real-time system can guarantee that if some tasks are going to miss their deadlines, the lower-priority tasks are victimized first.

The RTSJ defines at least 28 levels of priority and requires their strict enforcement. However, as this article mentioned earlier, RTSJ implementations rely on a real-time operating system to support multiple priorities, as well as on the ability of higher-priority threads to preempt lower-priority ones.

The problem of priority inversion can undermine the effectiveness of a priority facility. Accordingly, the RTSJ requires priority inheritance in its scheduling. Priority inheritance avoids priority inversion by boosting the priority of a thread that is holding a lock to that of the highest-priority thread waiting for that lock.

This prevents a higher-priority thread from being starved because a lower-priority thread has the lock that it needs but cannot get adequate CPU cycles to finish its work and release the lock. This feature also prevents a medium-priority task that does not depend on the shared resource from preempting the higher-priority task.

In addition, the RTSJ is designed to allow both non-real-time and real-time activities to coexist within a single Java application. The degree of temporal guarantees provided to an activity depends on the type of thread in which the activity is executing: java.lang.Thread or javax.realtime.RealtimeThread thread types.

  • Standard java.lang.Thread (JLT) threads are supported for non-real-time activities. JLT threads can use the 10 priority levels specified by the Thread class, but these are not suitable for real-time activities because they provide no guarantees of temporal execution.
     
  • The RTSJ also defines the javax.realtime.RealtimeThread (RTT) thread type. RTTs can take advantage of the stronger thread priority support that the RTSJ offers, and they are scheduled on a run-to-block basis rather than a time-slicing basis. That is, the scheduler will preempt an RTT if another RTT of higher priority becomes available for execution.

Memory-Management Extensions

One of the problems with automatic memory management in standard virtual machines (VMs) is that one activity may have to "pay for" the memory-management costs of another activity. Consider an application with two threads: a high-priority thread (H) that does a small amount of allocation, and a low-priority thread (L) that does a great deal of allocation.

If H is unlucky enough to run at a time when L has consumed almost all the available memory in the heap, the garbage collector may kick in and run for a long time when H goes to allocate a small object. Now, H is paying -- in the form of an incommensurately long delay -- for L's enormous memory consumption.

The RTSJ provides a subclass of RTT called NoHeapRealtimeThread (NHRT). Instances of this subclass are protected from GC-induced jitter. The NHRT class is intended for hard-real-time activities.

To maximize predictability, NHRTs are allowed neither to use the garbage-collected heap nor to manipulate references to the heap. Otherwise, the thread would be subject to GC pauses, and this could cause the task to miss its deadline. Instead, NHRTs can use the scoped memory and immortal memory features to allocate memory on a more predictable basis.

The following two NHRT examples demonstrate this.

NHRT -- Example 1

public class PeriodicThread {
    static NoHeapRealtimeThread nhrt;
    static {
        int prio = PriorityScheduler.instance().getMaxPriority();
        RelativeTime period = new RelativeTime(20,0);
        nhrt = new NoHeapRealtimeThread(
                                new PriorityParameters(prio),
                                new PeriodicParameters(
                                        period),
                                        ImmortalMemory.instance()) {
                public void run() {
                        . . . . .
                };
      };
    }
    . . . . .
}

NHRT -- Example 2

public class MyNHRT extends NoHeapRealtimeThread {
    public MyNHRT() {
       super(new PriorityParameters(5), ImmortalMemory.instance());
       setReleaseParameters(
            new PeriodicParameters(
                   new RelativeTime(0, 0),
                    new RelativeTime(200, 0),
                    new RelativeTime(50, 0), null, null, null));
    }
}

Memory Areas

The RTSJ provides for several means of allocating objects, depending on the nature of the task doing the allocation. Objects can be allocated from a specific memory area, and different memory areas have different GC characteristics and allocation limits.

  • Standard heap. Just like standard VMs, real-time VMs maintain a garbage-collected heap, which can be used by both standard (JLT) and RTT threads. Allocating from the standard heap may subject a thread to GC pauses. Several different GC technologies can balance throughput, scalability, determinism, and memory size. NHRTs cannot use the standard heap in order to be protected from this balancing, which is a source of unpredictability.
     
  • Immortal memory. Immortal memory is a non-garbage-collected area of memory. Once an object is allocated from immortal memory, the memory used by that object will never, in principle, be reclaimed. The primary use for immortal memory is so that activities can avoid dynamic allocation by statically allocating all the memory they need ahead of time, and managing it themselves.
     
    Managing immortal memory requires greater care than managing memory allocated from the standard heap, because if immortal objects are leaked by the application, they will not normally be reclaimed. In this case, there is no easy way to return memory to the immortal memory area. Note: If you are familiar with C/C++, you will probably recognize immortal memory as being similar to malloc() and free().
     
  • Scoped memory. The RTSJ provides a third mechanism for allocation called scoped memory, which is available only to RTT and NHRT threads. Scoped-memory areas are intended for objects with a known lifetime, such as temporary objects created during the processing of a task. Like immortal memory, scoped-memory areas are uncollected, but the difference is that the entire scoped-memory area can be reclaimed at the end of its lifetime, such as when the task finishes.
     
    Scoped-memory areas also provide a form of memory budgeting. The maximum size of the scoped area is specified when it is created, and if you attempt to exceed the maximum, an OutOfMemoryError is thrown. This ensures that a rogue task does not consume all the memory and thereby starve other -- perhaps higher-priority -- tasks of memory.

Why Scoped Memory?

In Java programs, it is common to allocate temporary objects in the course of a computation. For example, concatenating two strings generates a StringBuffer object that is thrown away as soon as the result string is available.

The standard programming style for Java programs encourages producing a certain amount of garbage as a side effect of nearly any computation, and many libraries do just that. Because not being able to use library code from hard-real-time tasks would be unacceptably limiting, there needs to be a mechanism to make the costs of allocation and deallocation of such temporary objects predictable.

Scoped memory does just that. It provides the task with a means to say "I'm going to use this much temporary memory during the course of this task, and when the task is over, reclaim it all."

Because the memory is preallocated when the task starts, the allocations will not fail. Because all the memory in the scoped area is freed when the task completes, deallocation is fast and predictable. Therefore, as long as real-time tasks can accurately estimate how much memory they'll need to accomplish their work, scoped memory can eliminate the unpredictability of temporary object allocation.

However, one downside of scoped memory is that, because all the objects in it will disappear after the task completes, you cannot store a reference to an object in a scoped-memory area into the immortal or heap areas. If you need to create an object with a longer lifetime, you will have to create it in a different scope. You can create a hierarchy of scopes, with each child having a shorter lifetime than its parent.

In a procedure similar to the checks for null pointers or array bounds that any JVM performs, the real-time JVM dynamically performs assignment checks to guarantee that an object is never referenced by an object with a longer lifetime.

For each thread, there is always an active-memory area, called the current allocation context. All objects allocated by a thread are allocated from this area. The current allocation context changes when you execute a block of code with a specific memory area.

The memory area API contains an enter (Runnable) method that causes the specified task to be executed using that area as the current allocation context. Therefore, even if your code uses third-party libraries that allocate memory, you can still use that code with scoped-memory areas -- and all the temporary objects allocated by that code will go away when the current allocation context is finalized.

Advanced Communication Between Threads

One of the advantages of the RTSJ is that it allows real-time and non-real-time activities to coexist within a single VM. However, communication between threads involves memory, and this poses a challenge. The obvious mechanism for communication between threads is a queue, where one thread is putting data onto the queue and the other thread is removing data from the queue.

Imagine that an RTT is putting data onto a queue every 10 milliseconds, and a non-RTT is consuming the data from the queue. What happens if the non-RTT does not get enough CPU time to drain the queue? The queue will grow, and you have a choice of three bad options:

  • The queue could be allowed to grow without bound, potentially running out of memory.
     
  • Data from the queue must be discarded.
     
  • The RTT will have to block.

The first option is impractical, and the last option is unacceptable. The predictability of real-time activities should not be affected by the behavior of lower-priority activities. The RTSJ supports several types of non-blocking queues for communicating between threads. When they are used between real-time and non-RTTs, there are some restrictions on the memory area in which elements must reside.

* As used on this web site, the terms "Java Virtual Machine" and "JVM" mean a virtual machine for the Java platform.

Acknowledgments

The editor of this article wishes to thank David Holmes and Antonia Lewis for their work on the white paper on which this article is based. And thanks to Carlos Lucasius for his valuable technical review comments.

For More Information

About the Authors

Brian Goetz, a senior staff engineer at Sun, has been a professional software developer for 20 years. He is the author of over 75 articles on software development, and his book, Java Concurrency in Practice , was published in May 2006 by Addison-Wesley. He serves on the JCP Expert Groups for JSRs 166, concurrency utilities; 107, caching; and 305, annotations for safety analysis.

Robert Eckstein has worked with Java technology since its first release. In a previous life, he has been a programmer and editor for O'Reilly Media, Inc. He has written or edited a number of books, including Java Swing, Java Enterprise Best Practices, Using Samba, XML Pocket Reference, and Webmaster in a Nutshell.

Oracle Chatbot
Disconnected