RESTful GlassFish Monitoring and Management

by Adam Bien
Published December 2013

Exploit the built-in monitoring and management capabilities of GlassFish to automate application deployment and gain insight into application performance.

IT is all about streamlining and automation, so it is somewhat ironic that we developers still tolerate repetitive and boring manual tasks, such as deployment. Also, our ignorance about easily accessible information for application servers, such as monitoring data, is surprising.

Application servers have emitted useful monitoring data and provided basic management capabilities for 10 years, ever since J2EE 1.4 was released in November of 2003. For unknown reasons, both capabilities have been ignored for years, but the DevOps movement is making these built-in monitoring and management capabilities interesting again.

This article describes the GlassFish management API, which allows you to manipulate the GlassFish application server's configuration, including its monitoring capabilities. It also describes the GlassFish monitoring API, which provides a read-only facility for accessing the Java Virtual Machine (JVM), the GlassFish application server, and the GlassFish application server's metrics and counters.

The Forgotten Standard

JSR 77 (J2EE Management) is a useful but forgotten standard. Starting with J2EE 1.4, all application servers were forced to expose standardized management and monitoring APIs via a dedicated Management EJB component (MEJB) and Java Management Extensions (JMX), for example:



Context context = new InitialContext();
Object obj = context.lookup(ejbName);
ManagementHome home = (ManagementHome)PortableRemoteObject.narrow(obj,ManagementHome.class); 
Management mejb = home.create();

With a reference to the Management remote interface, you can query the MBeans, search for JMX attributes, or invoke MBeans methods. Fortunately, most application servers expose the same beans via alternative channels, such as HTML (for example, through a JMX/HTML connector), SNMP, the command line, or REST interfaces.

A J2EE or Java EE application server has to provide at least the following interfaces representing deployable component types: EJBStats, EntityBeanStats (which are no longer interesting, now that we have Java Persistence API [JPA] entities), JavaMailStats, JCAStats, JDBCStats, JMSStats, JTAStats, JVMStats, MessageDrivenBeanStats, ServletStats, SessionBeanStats, StatefulSessionBeanStats, StatelessSessionBeanStats, and URLStats. With the JSR 77 specification, not only are component types—as well as how you access them—standardized, but data structures are also standardized.

The javax.management.j2ee.statistics package comprises predefined data structures provided as interfaces: BoundaryStatistic, BoundedRangeStatistic, CountStatistic, RangeStatistic, and TimeStatistic. Each of these interfaces exposes monitoring data as getters, for example:



public interface TimeStatistic extends Statistic {
    public long getCount();
    public long getMaxTime();
    public long getMinTime();
    public long getTotalTime();
}

You can access the monitoring and management APIs on any Java EE–certified application server directly from your application using EJB/JMX; however, you will more likely access the metrics through alternative channels such as administrative consoles, command-line clients, or REST interfaces.

HTTP over JMX

JMX is powerful, but Java is required in order to access JMX remotely. The requirement for Java on the client introduces superfluous friction and, therefore, negatively affects usability. HTTP, on the other hand, is ubiquitous and easy to access with simplistic tools. Tools such as cURL, Wget, and even UI clients provide frictionless access to HTTP-based resources. Luckily GlassFish exposes its monitoring and management APIs not only via JMX, but also via HTTP. These interfaces are explorative and follow basic REST principles.

The RESTful monitoring interface is generic and consistent. Hence, it is a generic view of the underlying management and monitoring infrastructure. The URI design is not always intuitive and requires some tinkering to be fully understood. Just start in the browser and append .json or .xml to the URI to get a machine-readable view of the management API.

GlassFish comes with monitoring disabled out of the box. You can enable it from the GlassFish administration console, the command line, or the JMX console, or by using the RESTful administration interface. Regardless of which channel you use, behind the scenes, the state of the same MBean is modified and monitoring for the given module is activated. GlassFish monitoring can be activated most easily, but in the least automated way, using the GlassFish administration console, which is shown in Figure 1.

GlassFish administration console

Figure 1. GlassFish administration console.

The command-line tool, asadmin (which is located in <GLASSFISH_DIR>/bin/asadmin), provides a more-powerful and scriptable interface to GlassFish's monitoring and administration capabilities. The command get server.monitoring-service.module-monitoring-levels produces the output shown in Listing 1:



asadmin> get server.monitoring-service.module-monitoring-levels
server.monitoring-service.module-monitoring-levels.cloud=OFF
server.monitoring-service.module-monitoring-levels.cloud-account-manager=OFF
server.monitoring-service.module-monitoring-levels.cloud-elasticity=OFF
server.monitoring-service.module-monitoring-levels.cloud-orchestrator=OFF
server.monitoring-service.module-monitoring-levels.cloud-virt-assembly-service=OFF
server.monitoring-service.module-monitoring-levels.connector-connection-pool=OFF
server.monitoring-service.module-monitoring-levels.connector-service=OFF
server.monitoring-service.module-monitoring-levels.deployment=OFF
server.monitoring-service.module-monitoring-levels.ejb-container=OFF
server.monitoring-service.module-monitoring-levels.http-service=OFF
server.monitoring-service.module-monitoring-levels.jdbc-connection-pool=OFF
server.monitoring-service.module-monitoring-levels.jersey=OFF
server.monitoring-service.module-monitoring-levels.jms-service=OFF
server.monitoring-service.module-monitoring-levels.jpa=OFF
server.monitoring-service.module-monitoring-levels.jvm=OFF
server.monitoring-service.module-monitoring-levels.orb=OFF
server.monitoring-service.module-monitoring-levels.security=OFF
server.monitoring-service.module-monitoring-levels.thread-pool=OFF
server.monitoring-service.module-monitoring-levels.transaction-service=OFF
server.monitoring-service.module-monitoring-levels.web-container=OFF
server.monitoring-service.module-monitoring-levels.web-services-container=OFF
Command get executed successfully.

Listing 1

After running the command, you can use the set command to activate monitoring for any module shown in Listing 1, for example:



asadmin> set server.monitoring-service.module-monitoring-levels.transaction-service=HIGH
server.monitoring-service.module-monitoring-levels.transaction-service=HIGH
Command set executed successfully.

The asadmin command translates the command-line input into remote HTTP calls. All "online" commands are also available directly via the RESTful interface. You can specify what module to read or manipulate after the /management/domain/configs/config/server-config prefix. To list the monitoring levels, just access the following URI from the browser: http://localhost:4848/management/domain/configs/config/server-config/monitoring-service/module-monitoring-levels.

The browser will render a web form for you in a simple HTML page. Executing the same URI from the command line using cURL or Wget will lead to the same response rendered as an XML document. You can re-execute the command-line argument setting the Accept HTTP header to application/json, as follows, and then you will get the response as a JSON document:



curl -H "Accept: application/json" 
http://localhost:4848/management/domain/configs/config/server-config/monitoring-service/module-monitoring-levels

All the monitoring and management APIs are available as JSON, XML, and HTML at the same time.

So you can also choose the desired format by appending the .json, .xml, or .html ending to the URL. Consequently, the following URI returns the monitoring levels in JSON format:

http://localhost:4848/management/domain/configs/config/server-config/monitoring-service/module-monitoring-levels.json

GlassFish Monitoring Capabilities

The GlassFish management API allows you to manipulate the server's configuration via the REST, command line, JMX, and administration console channels. A specific example is the activation of module monitoring mentioned in the previous section. Monitoring is a read-only capability for accessing the Java Virtual Machine (JVM), the application server, and the application server's metrics and counters.

Consequently, access to monitoring facilities can be obtained via the same channels as for accessing the management API. Monitoring information is exposed through the monitoring resource, http://localhost:4848/monitoring/domain/server, as shown in Figure 2.

Exposed monitoring information

Figure 2. Exposed monitoring information.

Similarly, you can access all the monitoring data from the asadmin command line by executing the following command:

asadmin> get --monitor server.*

Be careful with this command, though. It will return all available server and application monitoring data—you will get at least several hundred lines of text. Of course, you can limit the amount of data by restricting the request to a specific module, for example:



asadmin> get --monitor server.transaction-service.committedcount-count
server.transaction-service.committedcount-count = 5797
Command get executed successfully.

Exactly the same amount of information is available through the REST interface: http://localhost:4848/monitoring/domain/server/transaction-service/committedcount. Usually, it is preferable to choose the JSON or XML representation by appending the .json or .xml ending or by setting the Accept HTTP header accordingly. Compared to plain HTML, having monitoring data in JSON or XML format makes the data more easily accessible to computer programs.

Systematic Monitoring

Monitoring data becomes even more interesting when it is gathered periodically and stored in a database. Having historical monitoring data available allows you to compare current application performance with any past data points. Particularly valuable is data captured during stress tests, because you can easily measure the demand for memory and CPU resources under heavy load, while measuring application performance at the same time. With historical monitoring data, you can easily compare the performance of any applications that are already deployed to identify performance regression or to estimate system requirements.

With Java, you can use the Java API for RESTful Web Services (JAX-RS) client to access GlassFish's monitoring and management interfaces directly. The fetchMethods method shown in Listing 2 fetches performance metrics for EJB methods:



import javax.json.JsonObject;
import javax.ws.rs.client.Client;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;

public JsonObject fetchMethods(String applicationName, String ejbName) {
        WebTarget target = client.target(getUri() + "{application}/{bean}/bean-methods/");
        Response response = target.resolveTemplate("application", applicationName).
                resolveTemplate("bean", ejbName).
                request(MediaType.APPLICATION_JSON).get();
        if (response.getStatus() == 404) {
            return null;
        }
        return response.readEntity(JsonObject.class);
    }

Listing 2

JsonObject is a java.util.Map, so the return value of the fetchMethods method could be transformed into a JPA entity and directly persisted in a database using only a few lines of code. Having the metrics available as Java classes introduces a whole set of new possibilities, such as proactive monitoring or the implementation of alerts. In fact, the open source LightFish project gathers and persists a wide range of metrics and stores them in a database (usually an Apache Derby database).

Management and Deployment

Although the JSR 77 spec allows us to manage and monitor applications, application deployment is out of its scope. In fact, deployment was standardized by another specification: JSR 88: Java EE Application Deployment. Unfortunately, JSR 88 became optional in Java EE 7, so you cannot rely on a standard deployment interface any more.

GlassFish also offers a REST-based deployment API, which allows you to deploy, undeploy, list, and query applications, and also to change the configuration of applications. The interface is accessible through the URL http://localhost:4848/management/domain/applications/application, and it can be best explored by accessing it via a browser first.

According to REST principles, you can list the deployed applications using HTTP GET, deploy an application using POST, and undeploy a deployed application using DELETE.

GlassFish offers the application upload and deployment capability via multipart encoding: multipart/form-data. The deployment URI in the browser—http://localhost:4848/management/domain/applications/application—is rendered as a simplistic form: <form action="http://localhost:4848/management/domain/applications/application/" method="post" enctype="multipart/form-data">. The most important element in the entire form is the id element (see Figure 3), which carries the content of the deployment archive:

Application deployment form

Figure 3. Application deployment form.

You are not limited to using the browser. You can also deploy an archive using the same form from command line:



curl -X POST \
    -H 'X-Requested-By: loadr' \
    -F id=@/[PATH]/coffeebeans.war \
    -F force=true http://localhost:4848/management/domain/applications/application

The X-Requested-By header is necessary plumbing; without the header, GlassFish would reject the request. The force field allows the server to quietly override any already deployed application.

Tools such as cURL or Wget are great for scripting but are harder to seamlessly integrate with applications written Java. However, you can easily interact directly with the REST interface using Java and, in particular, JAX-RS.

The Deployer class (from the open source loadr utility) uses the same HTTP interface to deploy an archive remotely. Unfortunately, multipart support was not standardized in the JAX-RS 2.0 specification, so you will have to depend on JAX-RS provider-specific extensions for the archive upload, as shown in Listing 3:



import javax.json.JsonObject;
import javax.json.JsonValue;
import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.Entity;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import org.glassfish.jersey.media.multipart.FormDataMultiPart;
import org.glassfish.jersey.media.multipart.MultiPart;
import org.glassfish.jersey.media.multipart.MultiPartFeature;
import org.glassfish.jersey.media.multipart.file.FileDataBodyPart;
public class Deployer {

    public static final String MANAGEMENT_URI = "/management/domain/applications/application";
    private final Client client;
    private final WebTarget applicationTarget;

    public Deployer(String serverLocation) {
        this.client = ClientBuilder.newClient().register(MultiPartFeature.class);
        this.applicationTarget = client.target(serverLocation + MANAGEMENT_URI);
    }

    public boolean deploy(String archiveFile) {
        FileDataBodyPart archive = new FileDataBodyPart("id", new File(archiveFile));
        MultiPart form = new FormDataMultiPart().
                field("force", "true").bodyPart(archive);
        Response response = applicationTarget.request("application/json").
                header("X-Requested-By", "loadr").
                post(Entity.entity(form, form.getMediaType()));
        return (response.getStatus() == 200);
    }
...
}

Listing 3

The Deployer#deploy method provides the same amount of information as the cURL request. However, the convenient @ functionality of cURL, which automatically sends the file as a "part," needs to be implemented in Java with the org.glassfish.jersey.media.multipart.file.FileDataBodyPart class. Deployment is the most complicated interaction. Status queries and application undeployment are trivial to implement, as shown in Listing 4:



   public boolean isDeployed(String appName) {
        Response response = this.applicationTarget.path(appName).request().get();
        return response.getStatus() == 200;
    }

    public boolean undeploy(String applicationName) {
        Response response = this.applicationTarget.path(applicationName).
                request(MediaType.APPLICATION_JSON).
                header("X-Requested-By", "loadr").
                delete();
        return response.getStatus() == 200;
    }

Listing 4

Naturally, the same URL, [SERVER_LOCATION:4848]/management/domain/applications/application, with the appended application name is used in a GET request to check the deployment status of an application. Consequently, a DELETE request URI undeploys an application. In both cases, the server confirms a successful operation using the 200 HTTP status. As expected, a plain GET request without anything appended will return a list of all deployed applications. Because the applications are not returned as an array, but rather as a nested JSON object, the data extraction requires a little bit more post processing, as shown in Listing 5:



    public Set<Application> applications() {
        Set<Application> retVal = new HashSet<>();
        JsonObject answer = this.applicationTarget.
                request().
                accept(MediaType.APPLICATION_JSON).
                get(JsonObject.class);
        JsonObject props = answer.getJsonObject("extraProperties");
        JsonObject kids = props.getJsonObject("childResources");
        Set<Map.Entry<String, JsonValue>> entrySet = kids.entrySet();
        for (Map.Entry<String, JsonValue> application : entrySet) {
            retVal.add(new Application(application.getKey(),
                    application.getValue().toString()));
        }
        return retVal;
    }

Listing 5

The Deployer class covers only basic create, read, update, and delete (CRUD) use cases and, therefore, covers only a tiny subset of the actual management functionality. With additional form elements and a few additional lines of code, you could easily change the context root, enable or disable applications, or enable load balancing using the REST interface.

The most popular build and Continuous Integration (CI) tools—such as Ant, Maven, Gradle, and Jenkins—are written in Java. Wrapping the REST interface with a thin Java class makes it directly and robustly accessible to, for example, Maven plug-ins or Ant tasks.

Conclusion

Using nice-looking web interfaces to deploy applications is appealing the first few times, but the process is brittle. It is very easy to choose a wrong archive or misinterpret the results of deployment. I suspect every Java EE developer has at one time or another spent a considerable amount of time debugging the wrong version of an application. Automation creates confidence. Using direct management interfaces allows developers to streamline the deployment process and makes the process reliable and repeatable.

The case is similar for monitoring. It is nice to browse a library of nice-looking charts and tables in the admin console to get a general feeling for an application's performance and robustness. During stress testing and in production, persistent data is very important. Only then do you get the opportunity to put your current performance data into historical context and identify possible regressions. An API that is accessible through Java makes any further processing trivial. In minutes, you can build JPA-based persistence from a few performance counters, and then use JavaScript (Oracle Nashorn) or any other scripting language running on the JVM to evaluate the results.

However, a REST-based monitoring and management interface becomes even more important for non-Java clients. IT automation tools, such as Puppet or Chef, are not based on Java and, therefore, a REST interface becomes the interface of choice for automation and monitoring.

Unfortunately, the Java EE application deployment specification (JSR 88) became optional in Java EE 7, and so it could disappear in future Java EE releases. JSR 77 is still available and useful but would require a general overhaul.

I am hoping future Java EE releases will focus more on nonfunctional requirements and provide modern and standardized management and monitoring interfaces for application servers. Automation is the first step towards...clouds.

See Also

About the Author

Consultant and author Adam Bien is an Expert Group member for the Java EE 6 and 7, EJB 3.X, JAX-RS, and JPA 2.X JSRs. He has worked with Java technology since JDK 1.0 and with Servlets/EJB 1.0 and is now an architect and developer for Java SE and Java EE projects. He has edited several books about JavaFX, J2EE, and Java EE, and he is the author of Real World Java EE Patterns—Rethinking Best Practices and Real World Java EE Night Hacks—Dissecting the Business Tier. Adam is also a Java Champion, Top Java Ambassador 2012, and JavaOne 2009, 2011, and 2012 Rock Star. Adam occasionally organizes Java EE workshops at Munich's airport (airhacks.com).

Join the Conversation

Join the Java community conversation on Facebook, Twitter, and the Oracle Java Blog!