Gerard Davison's Blog

wadl 1.1.5 released

2013-05-24T10:19:00.000+01:00

I should have done this a while ago but last month we released a new version of the wadl2java tools, primarily to deal with issues seen whilst writing a previous blog. The headline change was a move to the latest version of the jsonschema2pojo in order to resolve some class generation issues. Otherwise if you are just using XML Schema based WADL there is nothing new in this release.

Persistent breakpoints every java developer should have

2013-05-23T15:43:00.001+01:00

When a developer is working in Java there are a few failure cases you always want to know about even if you were trying to debug something else. Here is a list of the persistent breakpoints that should be enabled in every IDE. (IMHO of course)

Yes in theory you should be able to just get this from a good logging implementation; but often these problems are deep down in somebody else's library caused by changes that are beyond your control..... or just hidden in a large number of currently executing threads.

Java these days doesn't have that much of an overhead to be running in debug mode, generally I never run anything I am working on without a debugger attached as HotSwap is such a productivity boost.

I would be interested to heard of other people's must have breakpoints. They should be failures that should never come up in the normal running of your application that you would want to know about right away.

Deadlock detection

The VM is getting better and better at recognising deadlocks from low level java contracts, you debugger should have an option to break on deadlock. You should make sure it is turned on by default.

On java.lang.ExceptionInInitializer

This one can be a right pain to track down because further down the line it can become a ClassNotFoundException and often if it is more than a few levels down there is no obvious reason why the indirectly referencing class has failed to load.

Often caused by people initialising public/final/static variables with methods/constructors that throw RuntimeExceptions. If you are not sure use a static{} block with proper error handling, never do something in there that depends on something external to the current jar file.

On java.lang.NoClassDefFoundError

Always a popular when working in OSGi, normally you have forgotten a dependant jar file and you see this type of failure. Interestingly you sometimes only get a cause for the first time this exception is thrown for a particular class. So if you have a breakpoint then you can more quickly track this down.

I have found that sometimes that the breakpoint is not hit when behind some reflective code. In that case I often have breakpoints in the constructor of this class just to make sure.

On java.lang.NoSuchMethodError

You will of course generally see this if you haven't built locally properly or you are using miss-matched versions of libraries. You always want to know about this as soon as it happens.

On java.lang.LinkageError

This normally only turns up when things really go bad, for example when you have two versions of the same class hanging about even though they are from the same jar file. (Gotta love multiple class loaders) Be thankful these don't turn up very often, normally timed for a day when I have a really bad headache already.

On java.lang.OutOfMemoryError / java.lang.StackOverflowException

If you are very lucky the VM will breakpoint for these; but unfortunately as a lot of time this will happen in native code you will just have to deal with the fallout.

Getting the full stack for the latter is a bit of a pain; but not impossible. See this older post on getting the full stack trace.

On java.lang.AssertionError

A surprising number of people use assertions and then get upset when you complain about there exceptions when debugging or running tests. (You are doing this with -ea aren't you?)

The downside is that you want to turn this off when debugging certain testing frameworks, I am looking at you JUnit, as it is not entirely helpful to hit a breakpoint when you just want to run all of your tests.

Finally every now and again I will run across the following code, which causes an exception just to test if assertions are enabled.

boolean assertionsEnabled = false;
    try
    { assert false; }
    catch (AssertionException ae)
    { assertionsEnabled = true }

Update: It has been pointed out by a helpful colleague at that I should have provided some alternative suggestions as to what this code should be replaced with. The first option would be to call Class.desiredAssertionStatus but if you want to be totally sure you can use the following code that uses assignment to capture the assertion status without throwing an exception.

boolean assertionsEnabled = false;

    assert isAssertionsEnabled = true;

A little command line Web Socket client tool from Tyrus

2013-05-22T14:14:00.002+01:00

The Tyrus project, the RI implementation of the Web Socket JSR, has a little sub project added by Pavel Bucek that provides a very useful command line testing tool. (Along with a couple of bug fixes of my own) This blog post gives a quick introduction.

To get hold of the tool at the moment you are going to have to do a little bit of work with svn and maven. So you need to check out the head of https://svn.java.net/svn/tyrus~source-code-repository/trunk and then perform a mvn package on the etc/client-cli sub project. Depending on the state of the maven repository you might need to do a mvn install -DskipTests=true on the root project first.

You can now find the executable jar in .../tyrus/ext/client-clit/target called something like tyres-client-client....jar. This has been shaded so it contains all of the required deps to run. Simples.

The tool currently allows you to send text and ping messages. It can receive binary messages but it just displays the hex values of up to the first 1024 bytes.

So you can either pass in the ws:// URL as the single command line argument or you can connect using the open command once the tool is running. Here are some very simple commands along with a response from a simple echo web socket service.

gdavison@gbr10460 ~]$ java -jar ~/software/tyrus-client-cli-1.0-SNAPSHOT.jar ws://localhost:7101/Project1/echo
# Connecting to ws://localhost:7101/Project1/echo...
# Connected in session 689607b4-7bd8-465a-8d84-e0dca54fa3e8
session 6896...a3e8> 
session 6896...a3e8> send Hello!
# text-message: Message out Hello!
session 6896...a3e8> send
# End multiline message with . on own line
send...> Some
send...> Long
send...> Message
send...> .
# text-message: Message out Some
# Long
# Message
# 
session 6896...a3e8> close
# closed: CloseReason[1000,no reason given]
# Session closed
tyrus-client> quit


gdavison@gbr10460 ~]$ java -jar ~/software/tyrus-client-cli-1.0-SNAPSHOT.jar 

tyrus-client> 
tyrus-client> open ws://localhost:7101/Project1/echo
# Connecting to ws://localhost:7101/Project1/echo...
# Connected in session 59e501a9-a793-4321-ba6a-1c73c13bd822
session 59e5...d822>
session 59e5...d822> ping
# pong-message
session 59e5...d822> help
# 
    open uri : open a connection to the web socket uri
    close : close a currently open web socket session
    send message : send a text message
    send : send a multiline text message teminated with a .
    ping : send a ping message
    quit | exit : exit this tool
    help : display this message
    
session 59e5...d822>

I can see this being very helpful when trying to debug a web socket service and you want to remove the web browser out of the equation. It is implemented using JLine2 so it behaves like a native command line tool so is pleasant to use.

Using Java WebSockets, JSR 356, and JSON mapped to POJO's

2013-04-30T14:07:00.001+01:00

So I have been playing around with Tyrus, the reference implementation of the JSR 356 WebSocket for Java spec. Because I was looking at test tooling I was interested in running both the client and the server side in Java. So no HTML5 in this blog post I am afraid.

In this example we want to sent JSON back and forth and because I am old fashioned like that I want to be able to bind to a POJO object. I am going to use Jackson for this so my maven file looks like this:

<dependencies>
    <dependency>
        <groupId>javax.websocket</groupId>
        <artifactId>javax.websocket-api</artifactId>
        <version>1.0-rc3</version>
    </dependency>

    <dependency>
        <groupId>org.glassfish.tyrus</groupId>
        <artifactId>tyrus-client</artifactId>
        <version>1.0-rc3</version>
    </dependency>

    <dependency>
        <groupId>org.glassfish.tyrus</groupId>
        <artifactId>tyrus-server</artifactId>
        <version>1.0-rc3</version>
    </dependency>

    <dependency>
        <groupId>org.glassfish.tyrus</groupId>
        <artifactId>tyrus-container-grizzly</artifactId>
        <version>1.0-rc3</version>
    </dependency>
    

    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-databind</artifactId>
      <version>2.2.0</version>
    </dependency>

    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-annotations</artifactId>
      <version>2.2.0</version>
    </dependency>

    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-core</artifactId>
      <version>2.2.0</version>
    </dependency>

  </dependencies>

So the first things we need to do is to define an implementations of the Encode/Decoder interfaces to do this work for us. This is going to do some simple reflection to workout what the bean class is. Like with JAX-WS it is easier to put them on the same class. Note that we use the streaming version of the interface and are only handling text content. (Ignoring the ability to send binary data for the moment)

package websocket;

import com.fasterxml.jackson.databind.ObjectMapper;

import java.io.IOException;
import java.io.Reader;
import java.io.Writer;

import java.lang.reflect.ParameterizedType;
import java.lang.reflect.Type;

import javax.websocket.DecodeException;
import javax.websocket.Decoder;
import javax.websocket.EncodeException;
import javax.websocket.Encoder;
import javax.websocket.EndpointConfig;

public abstract class JSONCoder<T>
  implements Encoder.TextStream<T>, Decoder.TextStream<T>{


    private Class<T> _type;
    
    // When configured my read in that ObjectMapper is not thread safe
    //
    private ThreadLocal<ObjectMapper> _mapper = new ThreadLocal<ObjectMapper>() {

        @Override
        protected ObjectMapper initialValue() {
            return new ObjectMapper();
        }
    };
    

    @Override
    public void init(EndpointConfig endpointConfig) {
    
        ParameterizedType $thisClass = (ParameterizedType) this.getClass().getGenericSuperclass();
        Type $T = $thisClass.getActualTypeArguments()[0];
        if ($T instanceof Class) {
            _type = (Class<T>)$T;
        }
        else if ($T instanceof ParameterizedType) {
            _type = (Class<T>)((ParameterizedType)$T).getRawType();
        }
    }

    @Override
    public void encode(T object, Writer writer) throws EncodeException, IOException {
        _mapper.get().writeValue(writer, object);
    }

    @Override
    public T decode(Reader reader) throws DecodeException, IOException {
        return _mapper.get().readValue(reader, _type);
    }

    @Override
    public void destroy() {
        
    }

}

The bean class is really quite simple with a static subclass of the Coder that we can use later.

package websocket;

public class EchoBean {
    
    
    public static class EchoBeanCode extends
       JSONCoder<EchoBean> {
        
    }
    
    
    private String _message;
    private String _reply;


    public EchoBean() {
        
    }

    public EchoBean(String _message) {
        super();
        this._message = _message;
    }


    public void setMessage(String _message) {
        this._message = _message;
    }

    public String getMessage() {
        return _message;
    }


    public void setReply(String _reply) {
        this._reply = _reply;
    }

    public String getReply() {
        return _reply;
    }

}

So new we need to implement our server endpoint, you can go one of two way either annotating a POJO or extending Endpoint. I am going with the first for the server and the second for the client. Really all this service does is to post the message back to the client. Note the registration of the encode and decoder. The same class in this case.

package websocket;

import java.io.IOException;

import javax.websocket.EncodeException;
import javax.websocket.EndpointConfig;
import javax.websocket.OnMessage;
import javax.websocket.OnOpen;
import javax.websocket.Session;
import javax.websocket.server.ServerEndpoint;
import static java.lang.System.out;

@ServerEndpoint(value="/echo",
                encoders = {EchoBean.EchoBeanCode.class},
                decoders = {EchoBean.EchoBeanCode.class})
public class EchoBeanService
{
    
    @OnMessage
    public void echo (EchoBean bean, Session peer) throws IOException, EncodeException {
        //
        bean.setReply("Server says " + bean.getMessage());
        out.println("Sending message to client");
        peer.getBasicRemote().sendObject(bean);
    }

    @OnOpen
    public void onOpen(final Session session, EndpointConfig endpointConfig) {
        out.println("Server connected "  + session + " " + endpointConfig);
    }
}

Lets look at a client bean, this time extending the standard Endpoint class and adding a specific listener for a message. In this case when the message is received the connection is simply closed to make our test case simple. In the real world managing this connection would obviously be more complicated.

package websocket;

import java.io.IOException;

import javax.websocket.ClientEndpoint;
import javax.websocket.CloseReason;
import javax.websocket.EncodeException;
import javax.websocket.Endpoint;
import javax.websocket.EndpointConfig;
import javax.websocket.MessageHandler;
import javax.websocket.Session;

import static java.lang.System.out;

@ClientEndpoint(encoders = {EchoBean.EchoBeanCode.class},
                decoders = {EchoBean.EchoBeanCode.class})
public class EchoBeanClient 
  extends Endpoint
{
    public void onOpen(final Session session, EndpointConfig endpointConfig) {

        out.println("Client Connection open "  + session + " " + endpointConfig);
        
        // Add a listener to capture the returning event
        //
        
        session.addMessageHandler(new MessageHandler.Whole() {

            @Override
            public void onMessage(EchoBean bean) {
                out.println("Message from server : " + bean.getReply());
                
                out.println("Closing connection");
                try {
                    session.close(new CloseReason(CloseReason.CloseCodes.NORMAL_CLOSURE, "All fine"));
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        });
        
        // Once we are connected we can now safely send out initial message to the server
        //
        
        out.println("Sending message to server");
        try {
            EchoBean bean = new EchoBean("Hello");
            session.getBasicRemote().sendObject(bean);
        } catch (IOException e) {
            e.printStackTrace();
        } catch (EncodeException e) {
            e.printStackTrace();
        }

    }
}

Now running the WebSocket standalone is really quite straightforward with Tyrus, you simple instantiate a Server and start it. Be aware this starts daemon threads so you need to make sure if this is in a main method that you do something to keep the JVM alive.

import org.glassfish.tyrus.server.Server;

Server server = new Server("localhost", 8025, "/", EchoBeanService.class);
server.start();

So the client is relatively simple; but as we are doing the declarative method we need to explicitly register the encoders and decoders when registering the client class.

import javax.websocket.ClientEndpointConfig;
import javax.websocket.Decoder;
import javax.websocket.Encoder;
import javax.websocket.Session;

import org.glassfish.tyrus.client.ClientManager;


// Right now we have to create a client, which will send a message then close
// when it has received a reply
//

ClientManager client = ClientManager.createClient();
EchoBeanClient beanClient = new EchoBeanClient();

Session session = client.connectToServer(
        beanClient, 
        ClientEndpointConfig.Builder.create()
         .encoders(Arrays.<Class<? extends Encoder>>asList(EchoBean.EchoBeanCode.class))
         .decoders(Arrays.<Class<? extends Decoder>>asList(EchoBean.EchoBeanCode.class))
         .build(),
        URI.create("ws://localhost:8025/echo"));
        
        
// Wait until things are closed down
        
while (session.isOpen()) {
    out.println("Waiting");
    TimeUnit.MILLISECONDS.sleep(10);
}

Now the output of this looks like the following:

Server connected SessionImpl{uri=/echo, id='e7739cc8-1ce5-4c26-ad5f-88a24c688799', endpoint=EndpointWrapper{endpointClass=null, endpoint=org.glassfish.tyrus.core.AnnotatedEndpoint@1ce5bc9, uri='/echo', contextPath='/'}} javax.websocket.server.DefaultServerEndpointConfig@ec120d
Waiting
Client Connection open SessionImpl{uri=ws://localhost:8025/echo, id='7428be2b-6f8a-4c40-a0c4-b1c8b22e1338', endpoint=EndpointWrapper{endpointClass=null, endpoint=websocket.EchoBeanClient@404c85, uri='ws://localhost:8025/echo', contextPath='ws://localhost:8025/echo'}} javax.websocket.DefaultClientEndpointConfig@15fdf14
Sending message to server
Waiting
Waiting
Waiting
Waiting
Waiting
Waiting
Waiting
Waiting
Waiting
Waiting
Sending message to client
Message from server : Server says Hello
Closing connection
Waiting

Interestingly the first time this is run the there is a pause, I suspect this is due to Jackson setting itself up but I haven't had time to profile. I did find that this long delay on occurred on the first post - although obviously this is going to be slower than just passing plain text messages in general. Whether the different is significant to you depends on your application.

It would be interesting to compare the performance of the plain text with a JSON stream API such as that provided by the new JSR and of course the version that binds those values to a JSON POJO. Something for another day perhaps.

Building a simple java client for forecast.io

2013-04-25T09:27:00.000+01:00

I like to be outdoors when life allows so weather is always an interest for me. So i was very pleases to see forecast.io becoming available with global coverage after being jealous of the US only "DarkSkys" app. It was also nice to see that there is a nice REST api for the service which I could play with.

Unfortunately there was only a textual description of service, nothing I could consume with the Java client tools I have been working with. But I though it would be an interesting experiment to try to document an existing service from scratch.

The first thing I did was to get an example of the data that the service would respond with and run it through a JSON->JSON Schema tool. I used JSONSchema.net which gave me a rough v3 JSON Schema; but it wasn't able to take into account elements that are reused. A little bit of hand editing later I ended up with three JSON Schema files, one for the forecast itself and one for a list of weather details and and one for each particular data point.

From there is was pretty simple to read the developer documentation and write up a simple WADL to describe the two resources provided by the service. Luckily Jason LaPort from the DarkSkys was on hand to fix a few mistake and generally verify my description.

Once I had this all in place I of course found a couple of bugs in my own code, so it was time to update and fix wadl.java.net and release a 1.1.5 version with a revised depdency on the jsonschema2pojo project. This is why a little bit of adhoc expert testing pay dividends.

So taking the WADL from github it is possible to generate a nice clean Java API using the wadl.java.net generator. I won't reproduce the steps to generate the client here, but this is the end result and the kind of code you can get out of it.

// Create a Jersey client and setup for POJO JSON mapping

    ClientConfig cc = new DefaultClientConfig();
    cc.getFeatures().put(JSONConfiguration.FEATURE_POJO_MAPPING, true);
    Client c = Client.create(cc);


    // Get current weather for location and check for any alerts 
    
    Forecast f = ApiForecastIo_Forecast
        .apikey(c, "xxxxxxx")                  // Get own key from developer.forecast.io
        .latitudeLongitude(51.32d, -1.244d)    // Some location near where I live
           .getAsForecast(
              null,                            // Not JSONP
              Units.SI,                        // Return values in SI units
              "hourly,minutely,daily,flags");  // Just return current weather and any alerts
    
    // Output a summary

    System.out.println(
        f.getCurrently().getSummary() + " " +
        f.getCurrently().getTemperature() + " C");

    // Output any alerts
  
    List<Alert> alerts = f.getAlerts();
    for (Alert altert : alerts) {
        System.out.println(
            altert.getTitle() + " " + altert.getUri());
    }

The nice thing here was just how pleasantly easy describing a service that already exists was, yes I could have written it by hand in Java; but then when I wanted to move to Jersey 2.x I would have to re-write by hand rather that just regenerating from the metadata to a new technology target. (Also did the same for a BBC REST Schedule API, that is in the same git project)

wadl2java 1.1.4 released

2013-03-22T16:05:00.003Z

We have released a new version of the wadl2java tool today with a bunch of new features and bug fixes. Again thanks for Pavel Bucek from the Jersey team for his continual help, and for the contribution from my JDeveloper team mate Michael Bachand.

New Features:

Added support for generating binding classes for WADL & JSON Schema decribed services.
Refactored upgrade transform to wadl-xslt module and added a WADL->HTML transform.
Added support for constructors that take a URI, this require use of an internal class UriTemplate in Jersey 2.x to have this work in the RS 2.0 version breaking general compatibility.
Fixes with regards to the requirement to use a later version of JAX-B than provided with JDK 6, some examples now required endorsed directory to run / build.
The default message for this exception with just a status was just an empty string, so we generate some boilerplate to fix this. (See JAX_RS_SPEC-312)

WADL Bugs:

WADL-65 Exception if non-required query parameters are passed in as null.
WADL-66 If the XML Schema didn't start with <?xml it would be treated as a JSON Schema.
WADL-52 UriBuilder impl dependency missing.

Oracle Bugs:

16234340 matrix parameters were not showing up in the test URL because of inconsistent usage of the resourceTypes and the resourceObjects structures.
16429792, where you would get duplicate methods if there were matrix parameters. These are dealt with like template parameters and as such are dealt with differently and needed to be filtered from the optional list.
16069855 we are modifying the WADL stylesheet (wadl_2009-02.xsl) since it's resulting XHTML document is now properly depicting non ascii characters (such as a french accent) the fix involves adding a element specifying UTF-8 as the charset.
16061180 Cannot process WADL with regex in template parameters, templates needed to be take out for code gen.
15878417 Generate template method to make it easier for IDE tooling to customize client creation
14831111 The client was no longer throwing an exception for a server side failure.
14710107 where the generated javadoc would would not be rendered properly do a XML copying issue.

Future Work:

Going forward we are looking to generate "java.lang.reflect.Proxy" based clients based on the Jersey 2.x module or a future part of the JAX-RS 2.x specification. We are also going to look at transforming other REST service descriptions into XML, for example Swagger, so that they can be fed into the client generation tool.

Using UriTemplate to change path parameters in a URI

2013-02-22T12:24:00.001Z

I have been recently working on a client generator in the wadl.java.net project and I was solving a bug where I wanted to change path parameter on a URI. This turned out to be quite easy with existing classes that are part of the Jersey project.

First of all you need a instance of UriTemplate, depending on the very of Jersey this is in a slightly different package - you IDE's automatic import will do the work for you.

  String template = "http://example.com/name/{name}/age/{age}";
  UriTemplate uriTemplate = new UriTemplate(template);

Then you use match(...) to extract them:

  String uri = "http://example.com/name/Bob/age/47";
  Map<String, String> parameters = new HashMap<>();

  // Not this method returns false if the URI doesn't match, ignored
  // for the purposes of the this blog.
  uriTemplate.match(uri, parameters);

Then you can replace the parameters at will and rebuild the URI:

  parameters.put("name","Arnold");

  UriBuilder builder = UriBuilder.fromPath(template);
  URI output = builder.build(parameters);

A little temporal feedback when running tests

2013-02-22T12:05:00.000Z

I have been working a lot recently with a bunch of test jobs that are intermittently getting stuck. Now it is possible to work out the point where the problem is occurring by looking at the date labels on the LOG output; but it is hard visually to pick these discontinuities out.

In order improve the output in a way that my brain can really quickly parse and identify I thought I would try my hand a bit of simple ASCII animation. I wrote a simple fixture that writes out a line every 1 minute while the test is running, here is a very simple implementation - I am sure there are better ways to do the animation!

import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.TimeUnit;

public class TickTockFixture
{
   private Timer timer;
   

   @Before
   public void setUp()
   {
      // Clean out any old timers
      tearDown();
      
      // Start a new timer
      
      timer = new Timer();
      timer.scheduleAtFixedRate(new TimerTask()
      {
         int counter = 0;
         
         @Override
         public void run()
         {
            StringBuilder sb = new StringBuilder(". TickTock : ");
            int animation = counter++ % 10;
            boolean direction = animation < 5;
            int count = direction ?  animation : 9 - animation;
            sb.append("     ", 0, count);
            sb.append(direction ? "/" : "\\");
            sb.append("     ",0, 5-count);
            sb.append(counter);
            sb.append(" min(s)");
            System.err.println(sb);
         }
      }, 0, TimeUnit.MINUTES.toMillis(1));
      
   }

   @After
   public void tearDown()
   {
      if (timer!=null)
      {
         timer.cancel();
         timer = null;
      }
   }
}

So does this help, well take a look at the - made up example - output below and try to spot the section of the log output where everything slows down. It is a lot easier with the ticking output which I think sticks out better because of the animation.

22-Feb-2013 11:54:30 Items in drs, going to tidy up
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/Project1WebApp.war/WEB-INF/weblogic.xml
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/Project1WebApp.war/WEB-INF/web.xml
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/Project1WebApp.war/WEB-INF/classes/project1/Hello.class
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/Project1WebApp.war/WEB-INF/classes/project1/
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/Project1WebApp.war/WEB-INF/classes/
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/Project1WebApp.war/WEB-INF/
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/Project1WebApp.war/
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/.module_marker
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/META-INF/weblogic-application.xml
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/META-INF/application.xml
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/META-INF/
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/.adrs-module.properties
22-Feb-2013 11:54:40 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/
22-Feb-2013 11:54:53 oracle.jdevimpl.webservices.tcpmonitor.config.AnalyzerInstance stop
WARNING: AnalyzerInstance aborting a stop as already appears to be stopped
22-Feb-2013 11:54:53 weblogic.logging.ServerLoggingHandler publish
WARNING: Container weblogic.wsee.jaxws.WLSContainer$BasicContainer@1232e17 doesn't support class com.sun.xml.ws.api.server.Module
22-Feb-2013 11:54:53 weblogic.logging.ServerLoggingHandler publish
WARNING: Container weblogic.wsee.jaxws.WLSContainer$BasicContainer@1232e17 doesn't support class com.sun.xml.ws.api.server.Module
trigger seeding of SecureRandom
. TickTock :  \    9 min(s)
. TickTock : \     10 min(s)
. TickTock : /     11 min(s)
. TickTock :  /    12 min(s)
. TickTock :   /   13 min(s)
22-Feb-2013 11:59:50 done seeding SecureRandom
22-Feb-2013 11:59:53 weblogic.logging.ServerLoggingHandler publish
WARNING: Container weblogic.wsee.jaxws.WLSContainer$BasicContainer@1232e17 doesn't support class com.sun.xml.ws.api.server.Module
22-Feb-2013 12:00:01 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/Project1WebApp.war/WEB-INF/classes/project1/Hello.class
22-Feb-2013 12:00:01 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/Project1WebApp.war/WEB-INF/classes/project1/
22-Feb-2013 12:00:01 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/Project1WebApp.war/WEB-INF/classes/
22-Feb-2013 12:00:01 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/Project1WebApp.war/WEB-INF/
22-Feb-2013 12:00:01 Removing file:/scratch/gdavison/view_storage/gdavison_lt/oracle/jdeveloper/system12.1.2.0.40.65.92/o.j2ee/drs/Application1/Project1WebApp.war/

Proxying a request to localhost in Java

2013-02-11T16:05:00.002Z

It can be very convenient when developing to server based application to run them using "localhost" in order to maintain consistency between developer machines. This is normally a good idea but there is a small case where this can cause problems.

Consider if you are running a local http proxy on your machine in order to capture your HTTP traffic. (Cough perhaps even the one in JDeveloper I work on). Then you might run into Java bug 6737819. Basically by default JDK 1.6 was hard coded not to send any request to localhost via a proxy which of course was a bit of a pain. Luckily a workaround was put in where you could put the string "~localhost" in your nonProxyHosts entry to turn of this feature:

java -client -classpath classes -Dhttp.proxyHost=localhost -Dhttp.proxyPort=8099 -Dhttp.nonProxyHosts=~localhost -Dhttps.proxyHost=localhost -Dhttps.proxyPort=8099 client.Example

Now moving forward to JDK 1.7 this workaround no longer works; but you need to take care to define nonProxyHosts as an empty string:

java -client -classpath classes -Dhttp.proxyHost=localhost -Dhttp.proxyPort=8099 -Dhttp.nonProxyHosts= -Dhttps.proxyHost=localhost -Dhttps.proxyPort=8099 client.Example

If you define this any anything other than an empty string the DefaultProxySelector though beware because internally it will append / or use the http.nonProxyHosts value from ../jre/lib/net.properties".

Just a minor complication that is not obvious from the published API.

JSON-Schema generation in Jersey

2012-11-19T15:00:00.000Z

So in my previous post I talked about a proposal to allow the use of JSON-Schema in a WADL, this post looks at how to get this working with a recently build of Jersey. You are going to have to download / reference 1.16SNAPSHOT until 1.16 is released.

If you are using Maven this should be quite straight forward to update your dependencies assuming you already have jersey and jersey-json. You are just going to need to add a dependency on the "jersey-wadl-json-schema" artefact from the "com.sun.jersey.contribs" group to get the new feature.

If you are outside of Maven the easiest thing to do would be to download the latest jersey-archive and then the jersey-wadl-json-schema jar. How you deploy these is tool specific, but if you are using WLS then here are some specific notes on how to upgrade the version of Jersey.

Once you have this working, you need to create a WadlGeneratorConfig class in order to enable this new grammar generation:

package jersey;

import com.sun.jersey.api.wadl.config.WadlGeneratorConfig;
import com.sun.jersey.api.wadl.config.WadlGeneratorDescription;
import com.sun.jersey.wadl.generators.json.WadlGeneratorJSONGrammarGenerator;

import java.util.List;

public class JsonGeneratorConfig extends WadlGeneratorConfig {
 
    @Override
    public List configure() {
        return generator(WadlGeneratorJSONGrammarGenerator.class).descriptions();
    }
}

This can then be registered in a variety of ways, here is an example using a servlet init param. Note also that to make this example simple that we are using the Jersey POJO mapping; but whilst writing this blog I noticed that this setting affected the format of the JSON version of the WADL in case you try this.

<?xml version = '1.0' encoding = 'ISO-8859-1'?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         version="3.0">
  <servlet>
    <servlet-name>jersey</servlet-name>
    <servlet-class>com.sun.jersey.spi.container.servlet.ServletContainer</servlet-class>
    <init-param>
      <param-name>com.sun.jersey.config.property.WadlGeneratorConfig</param-name>
      <param-value>jersey.JsonGeneratorConfig</param-value>
    </init-param>
    <init-param>
      <param-name>com.sun.jersey.api.json.POJOMappingFeature</param-name>
      <param-value>true</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>jersey</servlet-name>
    <url-pattern>/resources/*</url-pattern>
  </servlet-mapping>
</web-app>

So I put together a really simple echo service, just to check this is all working:

package jersey;

import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;

@Path("/echo")
public class EchoResource {
     

    @GET
    @Produces("application/json")
    public EchoBean echo() {
        EchoBean bean = new EchoBean();
        bean.setMessage("Hello");
        return bean;
    }

    @POST
    @Consumes("application/json")
    @Produces("application/json")
    public EchoBean echo(EchoBean echo) {
        return echo;
    }
    
}

and

package jersey;

public class EchoBean {
    public EchoBean() {
        super();
    }
    
    private String message;

    public void setMessage(String message) {
        this.message = message;
    }

    public String getMessage() {
        return message;
    }


}

This very simple example results in the following WADL with the JSON-Schema elements referenced:

<?xml version = '1.0' encoding = 'UTF-8'?>
<ns0:application xmlns:ns0="http://wadl.dev.java.net/2009/02">
   <ns0:doc xmlns:ns1="http://jersey.java.net/" ns1:generatedBy="Jersey: 1.16-SNAPSHOT 11/19/2012 12:59 AM"/>
   <ns0:grammars/>
   <ns0:resources base="http://localhost:7103/Jersey/resources/">
      <ns0:resource path="/echo">
         <ns0:method id="echo" name="GET">
            <ns0:response>
               <ns0:representation mediaType="application/json" xmlns:ns2="http://wadl.dev.java.net/2009/02/json-schema" ns2:describedby="application.wadl/echoBean"/>
            </ns0:response>
         </ns0:method>
         <ns0:method id="echo" name="POST">
            <ns0:request>
               <ns0:representation mediaType="application/json" xmlns:ns3="http://wadl.dev.java.net/2009/02/json-schema" ns3:describedby="application.wadl/echoBean"/>
            </ns0:request>
            <ns0:response>
               <ns0:representation mediaType="application/json" xmlns:ns4="http://wadl.dev.java.net/2009/02/json-schema" ns4:describedby="application.wadl/echoBean"/>
            </ns0:response>
         </ns0:method>
      </ns0:resource>
   </ns0:resources>
</ns0:application>

The URI application.wadl/echoBean contains this simple JSON-Schema definition:

{
    "type" : "object",
    "properties" : {
        "message" : {
            "type" : "string"
        }
    },
    "name" : "echoBean"
}

Now there are a number of limitations in the current design, not least that the generated schema doesn't take into account any notation settings. But I thought this would be enough to provoke feedback on whether this feature would be useful generally. There appears to be a growing interest in JSON-Schema both around the net and internally to Oracle, so it would be interesting to see whether this description becomes more common.

JSON-Schema in WADL

2012-11-16T13:01:00.000Z

In between other jobs I have been recently been reviewing the WADL specification with a view to fixing some documentation problems with a view to producing an updated version. One of the things that because apparent was the lack of any grammar support for languages other than XML - yes you can use a mapping from JSON<->XML Schema but this would be less than pleasant for a JSON purist.

So I began to look at how one would go about attaching a JSON-Schema grammar of a JSON document in a WADL description of a service. This isn't a specification yet; but a proposal of how it might work consistently.

Now I work with Jersey mostly, so lets consider what Jersey will current generate for a service that returns both XML and JSON. So the service here is implemented using the JAX-B binding so they both use a similar structure as defined by the XML-Schema reference by the include.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<application xmlns="http://wadl.dev.java.net/2009/02">
    <doc xmlns:jersey="http://jersey.java.net/" jersey:generatedBy="Jersey: 1.16-SNAPSHOT 10/26/2012 09:28 AM"/>
    <grammars>
        <include href="xsd0.xsd">
            <doc title="Generated" xml:lang="en"/>
        </include>
    </grammars>
    <resources base="http://localhost/">
        <resource path="/root">
            <method id="hello" name="PUT">
                <request>
                    <representation xmlns:m="urn:message" element="m:requestMessage"  mediaType="application/json" />
                    <representation xmlns:m="urn:message" element="m:requestMessage" mediaType="application/xml" />
                </request>
                <response>
                    <representation xmlns:m="urn:message" element="m:responseMessage" mediaType="application/json"/>
                    <representation xmlns:m="urn:message" element="m:responseMessage" mediaType="application/xml" />
                </response>
            </method>
        </resource>
    </resources>
</application>

So the first thing we considered was re-using the existing element property, which is defined as a QName, on the representation element to reference an imported JSON-Schema. It is shown here both with and another an arbitrary namespace to it can be told apart from XML elements without a namespace.

<grammars>
        <include href="xsd0.xsd" />
        <include href="application.wadl/responseMessage" />
    </grammars>


    <representation element="responseMessage" mediaType="application/json"/>

Or
    xmlns:json="http://wadl.dev.java.net/2009/02/json" 

    <representation 
        element="json:responseMessage" mediaType="application/json" />

The problem is that the JSON-Schema specification as it stands doesn't have a concept of a "name" property, so each JSON-Schema is uniquely identified by it's URI. Also from my read of the specification each JSON-Schema contains the definition for at most one document - not the multiple types / documents that can be contained in XML-Schema.

So the next best suggestion would be to just use the "filename" part of the URI as a proxy for the URI; but of course that won't necessarily be unique. I could see for example the US government and Yahoo both publishing there own "address" micro format.

The better solution to this problem is to introduce a new attribute, luckily the WADL spec was designed with this in mind, that is of type URI that can be used to directly reference the JSON-Schema definitions. So rather than the direct import in the previous example we have a URI property on the element itself. The "describedby" attribute name comes from the JSON-Schema proposal and is consistent with the rel used on atom links in the spec.

xmlns:json="http://wadl.dev.java.net/2009/02/json-schema" 
    xmlns:m="urn:message" 


    <grammars>
        <include href="xsd0.xsd" />
    </grammars>

    <representation 
        mediaType="application/json"
        element="m:responseMessage" 
        json:describedby="application.wadl/responseMessage" />

The has the secondary advantage in that this format is backwardly compatible with tooling that was relying on the XML-Schema grammar. Although this is probably only of interesting to people who work in tooling / testing tools like myself.

Once you have the JSON-Schema definition then some users are going to want to do away with the XML all together, so finally here is a simple mapping of the WADL to a JSON document that contains just the JSON-Schema information. It has been suggested by Sergey Breyozkin the JSON mapping would only show the json grammars and I am coming around to that way of thinking. I would be interested to hear of a usecase for the JSON mapping that would want access to the XML Schema.

{
   "doc":{
      "@generatedBy":"Jersey: 1.16-SNAPSHOT 10/26/2012 09:28 AM"
   },
   "resources":{
      "@base":"http://localhost/",
      "resource":{
         "@path":"/root",
         "method":{
            "@id":"hello",
            "@name":"PUT",
            "request":{
               "representation":[
                  {
                     "@mediaType":"application/json",
                     "@describedby":"application.wadl/requestMessage"
                  }
               ]
            },
            "response":{
               "representation":[
                  {
                     "@mediaType":"application/json",
                     "@describedby":"application.wadl/responseMessage"
                  }
               ]
            }
         }
      }
   }
}

I am currently using the mime type of "application/vnd.sun.wadl+json" for this mapping to be consistent with the default WADL mime type. I suspect we would want to change this in the future; but it will do for starters.

So this is all very interesting but you can't play with it unless you have an example implementation. I have something working for both the server side and for a Java client generator in Jersey and wadl2java respectively and that will be the topic of my next post. I have been working with Pavel Bucek on the Jersey team on these implementations and the WADL proposal, thanks very much to him for putting up with me.

Using Hudson/Jenkins to diagnose that intermittent failure

2012-10-11T11:29:00.000+01:00

I have been working on one of those intermittent bugs that just won't reproduce on my machine; but will reproduce intermittently on other machines while they are running automated testing. I filled the code with trace statements, now I suspect the problem is in code that I don't control and doesn't appear to have much in the way of diagnostics in the place I am working on.

So I did the obvious thing which is to run the tests on a loop on my machine overnight, 12 hours later and 8 test iterations later, no test failures and I am no further forward.

Since the tests are failing in the Hudson farm, it make sense to try to connect the debugger up to those jobs; but I don't want to hang around to attach the remove debugger to each. Thankfully there is a workaround that will allow me to set suitable breakpoints and manage the debugger connection for me.

First of all you need to configure you IDE to accept incoming debugger connections, here are some notes on configuring JDeveloper for a listening debugger, in Netbeans you need to use the Debug->Attach menu item and select "SocketListen" as the connector and configure as per JDeveloper. In Eclipse you need to configure the debug type as "Socket Listen".

The second step is modifying your build system so that there is a target you can call that will start the test cases in debug mode. This is an example of the parameters for one of our CI jobs that passes in the right information. Note of course the blacked out text the the name of then machine you are trying to connect back to. (The java tests are started with the parameter -agentlib:jdwp=transport=dt_socket,address=xxxx.oracle.com:5000,server=n) Make sure that you don't have any firewalls running on that machine that will block the in-coming connections.

You probably will want to run to run multiple jobs at the same time if you have the nodes available, so consider checking this concurrent build box. Always a good idea to bring cakes / cookies into the office if you are going to tie up all the preflight nodes for the day.

And then all that remains is to run a bunch of jobs and wait for your breakpoint to be hit, might take a little while; but it is going be quicker than running these jobs in series on your own machine. And if your farm is Heterogeneous so much the better for reproducing intermittent failures.

You can sit back and then wait for your code to fail..... may I suggest some sessions from JavaOne while you wait?

wadl2java 1.1.3 released

2012-10-04T11:42:00.001+01:00

So after a little bit of a pause we now have a new release of the wadl2java client generation tool. This is a wide range of improvements in this release; but the main feature is the new support for generating JAX-RS 2.0 client code.

This generated code is nearly identical to the Jersey 1.x interface so most simple code should compile straight after a swap without too many problems. If the client is making use of ClientResponse these references will have to be replaced with Response; but otherwise the interface will be consistent. I would welcome suggestions as to how I could improve what is generated.

So here is a list of items of changes in the release:

WADL-25, a patch to allow the passing in of argument to xjc, as provided by Brian Chapman thanks for that..
There is a problem in when there was not response content type, this type of method now returns the relevant Response object.
Classes that are @XmlType were being wrapped incorrectly with JAXBElement and the information was being taken from the return class.
The Proxy objects are now immutable, modifying any property created a new instance.
There is a now a programmatic interface to override the base URI at each level. (Suggested by Luigi Tagliamonte)
Fix for Oracle Bug 13804542, generation would fail if inner and outer class names match
JAX-RS 2.0 generation support available in all tool modes
Fix for Oracle Bug 14534583, where a fault element was being incorrectly upgraded from a '2006 WADL. This was causing problems with the examples SOAPUI were providing.
Fix for Oracle Bug 1462282, where the generated exception classes were not actually used in the generated code. All exceptions now extend WebAppilcationException to make migration from 1.x to 2.x easier.
Improved method generation, removed "application" from media types, removed duplication when method takes and returns the same type (putApplicationXmlAsApplicationXml -> putXml), moved "As" to correct location when we have a type, removed excessive method combinations where we have symmetrical media types. (eg xml,json->xml,json)
Initial work looking at how to support JSON-Schema type generation, non functional
Adding functional testing for generated clients as was previously only examining the generated classes. Now they are run for both 1.x and 2.x type clients.

Thanks again as usual help from Pavel Bucek in getting this software out the door.

So what credentials do you need?

2012-09-20T16:55:00.000+01:00

One of the features that I am very proud of in the HTTP Analyzer in JDeveloper is the ability to test a SOAP service with the minimum of fuss. Rather than having to configure security headers and understand WS-Policy we make use of the WLS JAX-WS client to read and interpret the policies defined in the WSDL and then apply abstracted credentials.

It turns out that this feature is very popular; but there is some confusion about just what information you might need to provide for a particular policy. So here is a simple summary of what is required for what type of policy.

Username Token

The simplest page just ask for a username and password. You are going to need this for policies that contains the phrase usernametoken, username, UNT or even HttpBasic and HttpDigest. The policy dictates where the values end up whether they be in an HTTP header or a WS-Security SOAP header.

X509 Certificates

The X509 panel is the most powerful and the most misunderstood. On this panel you can define a set of certificates and keys for both the client and the server.

The simplest version of this panel would be one that just contains the server key store. This is sufficient in most policy to be able to send a server an encrypted message and receive a message back. (In the most common case the server certificate is used to encrypt a symmetrical key, most likely AES, that can be used to decrypt the message being received.)

You will also need this to verify messages that are signed by the server.

If you need to server to trust that you send the message with signing then you have to provide a identity in the form of a Public/Private key that can be used to encrypt the outgoing hashes used to sign the message.

This client identity can also be used in leu of the username token described earlier. In the most common policy I have worked with the CN of the public certificate is used as the user identity.

Http Keystores

Some policies will make use of a Https channel to send messages, and like the x509 certificate panel the settings come in both client and server side store.

In the simplest case this panel will just be configured with a "Trusted Certificates Keystore", so this will list with certificates or certificate signing authorities are going to be trusted by the analyser. In order to make this as easy as possible by default we append the "CA" certificates that come with Java so you can by default connect to most publish websites without any further configuration.

This will normally be enough unless the policy is defined as 2-Way HTTPS, most of the OWSM https policies I have worked with are 2-Way by default. The WLS ones tend to specify this explicitly. In the 2-Way case you again need to prove the identity of the client, and the only way of doing this is going to be with a Public/Private key that the server trusts.

STS Configuration

The final panel regards working with policies that require SAML tokens; JDeveloper is unable to issue SAML tokens itself so instead it need to get hold of a token from a STS store that is trusted by the server you are in contact with.

In this case you need to specify the URL of the endpoint of the STS service along with a security policy that will match that on the sever side. This is unfortunately a static list and has to be figured out manually. Perhaps this is something that can be improved on in a later release.

On important feature to note is that when connecting to the STS store the analyser will be using values from the other three panels that we have discussed. This might be important if you are having trouble setting up a connection.

Changing delay, and hence the order, in a DelayQueue

2012-09-05T15:09:00.001+01:00

So I was looking at building a simple object cache that expires the objects after a given time. The obvious mechanism for this is the use the DelayedQueue class from the concurrency package in Java; but I wanted to know if it way possible to update the delay after an object has been added to the queue. Looking at the Delayed interface there didn't seem to be a good reason not to in the docs so I thought it was time to experiment.

So first of all you need to to create an instance of Delayed, this is a very simple implementation that with the switch of a flag you can basically invert the timeout order in the list. (And add a suitable offset so things happen in the right order)

static int COUNT=100;


   class DelayedSwap implements Delayed, Comparable<Delayed> {

       int index = 0;
       volatile boolean swap = false;
       long starttime;

       public DelayedSwap(int index, long starttime) {
           super();
           this.index = index;
           this.starttime = starttime;
       }

       private long getDelay() {
           return (swap ? starttime + (2*COUNT - index) * 100 :
               starttime + index * 100) - System.currentTimeMillis();
       }


       public String toString()
       {
           return index + " swapped " + swap + " delay " + getDelay();
       }

       @Override
       public long getDelay(TimeUnit unit) {
           return unit.convert(getDelay(), TimeUnit.MILLISECONDS);
       }

       @Override
       public int compareTo(Delayed delayed) {
           if (delayed == this)
               return 0;

           return (int)(getDelay(TimeUnit.MILLISECONDS) - delayed.getDelay(TimeUnit.MILLISECONDS));
       }
   }

So to test this I created a method that would create a bunch of the DelayedSwap objects and half way through processing the list switch the flag so altering the order of expiration.

public static void main(String[] args) throws InterruptedException {

       long start = System.currentTimeMillis();
       final List delayed = new ArrayList();
       for (int i = 1; i < COUNT; i++) {
           delayed.add(new DelayedSwap(i, start));
       }

       final DelayQueue dq = new DelayQueue();
       dq.addAll(delayed);

       new Thread(new Runnable() {

           @Override
           public void run() {
               try {
                   TimeUnit.SECONDS.sleep(5);
               } catch (InterruptedException e) {
               }
               for (DelayedSwap d : delayed) {
                   d.swap = true;
               }
           }
       }).start();

       while (!dq.isEmpty()) {
           System.out.println(dq.take());
       }

   }

So what I was expecting was the elements 1-50 ish written out in the correct order but instead after the swap over the elements are coming out in an arbitrary order quite far away from the request delay time.

1 swapped false delay -19
2 swapped false delay -4
3 swapped false delay -4
4 swapped false delay -4
5 swapped false delay -4
6 swapped false delay -4
7 swapped false delay -4
8 swapped false delay -4
9 swapped false delay -4
10 swapped false delay -4
11 swapped false delay -4
12 swapped false delay -4
13 swapped false delay -4
14 swapped false delay -4
15 swapped false delay -4
16 swapped false delay -4
17 swapped false delay -4
18 swapped false delay -4
19 swapped false delay -4
20 swapped false delay -4
21 swapped false delay -4
22 swapped false delay -4
23 swapped false delay -4
24 swapped false delay -4
25 swapped false delay -4
26 swapped false delay -4
27 swapped false delay -4
28 swapped false delay -4
29 swapped false delay -4
30 swapped false delay -4
31 swapped false delay -4
32 swapped false delay -4
33 swapped false delay -4
34 swapped false delay -4
35 swapped false delay -4
36 swapped false delay -4
37 swapped false delay -4
38 swapped false delay -4
39 swapped false delay -5
40 swapped false delay -4
41 swapped false delay -4
42 swapped false delay -5
43 swapped false delay -4
44 swapped false delay -5
45 swapped false delay -5
46 swapped false delay -5
47 swapped false delay -5
48 swapped false delay -5
49 swapped false delay -5
50 swapped false delay -5
51 swapped true delay -6
94 swapped true delay -4306
96 swapped true delay -4506
87 swapped true delay -3606
91 swapped true delay -4006
97 swapped true delay -4606
95 swapped true delay -4406
98 swapped true delay -4706
92 swapped true delay -4106
82 swapped true delay -3106
80 swapped true delay -2906
90 swapped true delay -3906
93 swapped true delay -4206
74 swapped true delay -2306
99 swapped true delay -4806
70 swapped true delay -1906
69 swapped true delay -1806
66 swapped true delay -1506
83 swapped true delay -3206
62 swapped true delay -1107
61 swapped true delay -1007
58 swapped true delay -707
71 swapped true delay -2007
89 swapped true delay -3807
85 swapped true delay -3407
78 swapped true delay -2707
86 swapped true delay -3507
81 swapped true delay -3007
88 swapped true delay -3707
84 swapped true delay -3307
79 swapped true delay -2807
76 swapped true delay -2507
72 swapped true delay -2107
68 swapped true delay -1707
65 swapped true delay -1407
60 swapped true delay -907
57 swapped true delay -608
55 swapped true delay -408
75 swapped true delay -2408
77 swapped true delay -2608
73 swapped true delay -2208
63 swapped true delay -1208
67 swapped true delay -1608
64 swapped true delay -1308
59 swapped true delay -808
56 swapped true delay -508
54 swapped true delay -308
53 swapped true delay -208
52 swapped true delay -108
Process exited with exit code 0.

So the trick is when you know you are going to modify the delay is to remove and then re-add the element to the queue.

// Replacement swap loop
    for (DelayedSwap d : delayed) {
        if (dq.remove(d))
        {
            d.swap = true;
            dq.add(d);
        }
    }

This run produces a more sensible set of results:

1 swapped false delay -4
2 swapped false delay -8
3 swapped false delay -14
4 swapped false delay -8
5 swapped false delay -4
6 swapped false delay -4
7 swapped false delay -4
8 swapped false delay -4
9 swapped false delay -4
10 swapped false delay -4
11 swapped false delay -4
12 swapped false delay -4
13 swapped false delay -4
14 swapped false delay -4
15 swapped false delay -4
16 swapped false delay -4
17 swapped false delay -4
18 swapped false delay -8
19 swapped false delay -4
20 swapped false delay -4
21 swapped false delay -4
22 swapped false delay -4
23 swapped false delay -4
24 swapped false delay -4
25 swapped false delay -4
26 swapped false delay -4
27 swapped false delay -4
28 swapped false delay -4
29 swapped false delay -4
30 swapped false delay -4
31 swapped false delay -4
32 swapped false delay -4
33 swapped false delay -4
34 swapped false delay -4
35 swapped false delay -4
36 swapped false delay -4
37 swapped false delay -4
38 swapped false delay -4
39 swapped false delay -5
40 swapped false delay -5
41 swapped false delay -5
42 swapped false delay -4
43 swapped false delay -4
44 swapped false delay -5
45 swapped false delay -5
46 swapped false delay -5
47 swapped false delay -5
48 swapped false delay -5
49 swapped false delay -5
50 swapped false delay -5
99 swapped true delay -5
98 swapped true delay -5
97 swapped true delay -11
96 swapped true delay -1
95 swapped true delay -5
94 swapped true delay -9
93 swapped true delay -5
92 swapped true delay -5
91 swapped true delay -5
90 swapped true delay -5
89 swapped true delay -5
88 swapped true delay -5
87 swapped true delay -5
86 swapped true delay -5
85 swapped true delay -5
84 swapped true delay -5
83 swapped true delay -5
82 swapped true delay -5
81 swapped true delay -5
80 swapped true delay -5
79 swapped true delay -5
78 swapped true delay -5
77 swapped true delay -5
76 swapped true delay -5
75 swapped true delay -5
74 swapped true delay -5
73 swapped true delay -5
72 swapped true delay -6
71 swapped true delay -5
70 swapped true delay -5
69 swapped true delay -5
68 swapped true delay -5
67 swapped true delay -5
66 swapped true delay -5
65 swapped true delay -5
64 swapped true delay -5
63 swapped true delay -6
62 swapped true delay -5
61 swapped true delay -6
60 swapped true delay -6
59 swapped true delay -6
58 swapped true delay -6
57 swapped true delay -6
56 swapped true delay -6
55 swapped true delay -6
54 swapped true delay -6
53 swapped true delay -6
52 swapped true delay -6
51 swapped true delay -6
Process exited with exit code 0.

I don't think this is a bug in the object itself, as you wouldn't expect a HashTable to orders it's self when the key changes, but I was a little bit surprise by the behaviour.

Catch a StackOverFlowError by its tail

2012-06-28T11:57:00.000+01:00

One of the more annoying situations you might have to deal with when working with a Java program is a StackOverFlowError, if you have a nice producible test case then there are few options with regard to playing with the stack size, or setting a conditional breakpoint / trace of some kind.

But if you have a test case that might fail once in a 100 times, perhaps a race condition in AWTMulticaster as in my case, then you want to improve the diagnostics. The problem is that by default the VM won't any elements in a stack trace after the first 1024 entries. (At least for JDK 6) So if you run the following trivial example:

package other;

public class Overflow {


   public static final void call(double a, double b, double c, double d) {
       call(a,b,c,d);
   }


   public static void main(String[] args) {
       call(0,0,0,0);
   }

}

The output will stop before you get to the cause of the problem, making it very hard to resolve the issue.

> java other.Overflow 
Exception in thread "main" java.lang.StackOverflowError
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
        [ lots of lines removed ]
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
Process exited with exit code 1.

The good news is that this limit is one of the many official and unofficial things you can tweak when starting a VM.

> java -XX:MaxJavaStackTraceDepth=1000000 other.Overflow 
Exception in thread "main" java.lang.StackOverflowError
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
        [ lots of lines removed ]
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.call(Overflow.java:7)
 at other.Overflow.main(Overflow.java:12)
Process exited with exit code 1.

Notice that end of the stack now contains the root of the problem which will be incredibly useful when trying to diagnose the issue.

You probably don't want to leave this property set long term on a production servers because I am not entirely sure about the impact of it. Looking at the C++ code it appears that this is just a maximum value and won't affect most other stack traces as they are allocated in a linked list in segments of 32 entries.

try/with/resources refactoring that isn't

2012-05-30T15:35:00.000+01:00

So I was looking at refactoring some code to use the new try/with/resources construct and found that perhaps it is not as simple as it first appears.

class Close implements AutoCloseable
   {
       @Override
       public void close() {
           System.out.println("Close");
       }
   }


   System.out.println("Normal version");


   Close close = new Close();
   try {
       throw new Error();
   }
   catch (Error e) {
       System.out.println("Catch");
   }
   finally {
       close.close();
   }

So the output for this is pretty easy to predict; but lets write it out just to be clear.

Normal version
Catch
Close

So this is pretty easy, lets throw some JDK 7 goodness on the code, this is an obvious refactoring right?

System.out.println("JDK 7 version");

   try (Close close2 = new Close()) {
       throw new Error();
   }
   catch (Error e) {
       System.out.println("Catch");
   }

So you can be pretty sure that given I am writing this up that the output is not going to be the same.

JDK 7 version
Close
Catch

So in this case the close is called before the catch, this might not be a problem but in some cases the catch block might need to make use of the resource before everything is shut down. If you want to perform a "refactoring" the code must behave the same before and after the changes. The following code is equivalent to the JDK 7 version without the try/with/resources

System.out.println("JDK 6 equivalent of JDK 7 version");

   Close close3 = new Close();
   try {
       try {
           throw new Error();
       }
       finally {
           close3.close();
       }
   }
   catch (Error e) {
       System.out.println("Catch");
   }

Just for completeness here is one possible refactoring of the original code which is functionally equivalent. Note quite as pretty I am afraid.

System.out.println("JDK 7 equivalent of original try/catch/finally");

   try (Close close4 = new Close()) {
       try {
         throw new Error();
       }
       catch (Error e) {
           System.out.println("Catch");
       }
   }

Now I ran this past my colleague Mark Warner and like me was surprised by the result. Mark pointed out that this nuance is noted in the java tutorial....

Note: A try-with-resources statement can have catch and finally blocks just like an ordinary try statement. In a try-with-resources statement, any catch or finally block is run after the resources declared have been closed.

...but it does feel like this rather important distinction is part of the small print that some people might at first glance assumed is related to the SQL example above it. (Yeah okay I didn't read the SQL example down to the end)

Again not something that will be a problem all the time; but it might be an important consideration in some quite important use cases.

try/with/resource context class loaders

2012-05-30T14:48:00.001+01:00

For various reasons I seem to end up writing a lot of code that fiddles with the context class loader in order to get non-module code running in the OSGi environment that JDeveloper runs in. This leads to a whole bunch of code that looks like this:

 public void doSomething() {

       ClassLoader context = Thread.currentThread().getContextClassLoader();

       try {
           Thread.currentThread().setContextClassLoader(Example.class.getClassLoader());

           // Class class that required the context class loader
           Endpoint.publish(null, null);            
       }
       finally {
           Thread.currentThread().setContextClassLoader(context);

       }  
  }

Now it occurred to me that the try/with/resources feature in JDK 7 isn't just for the nasty things in life, well resources, you can use it for any operation that might previously used a try/finally for.

   public void doSomething() {

       try (CloseableContext c = contextClassLoader(Example.class)) {

           // Class class that required the context class loader
           Endpoint.publish(null, null);            
       }
   }

It would have been nice to just call the method an not to allocate any variables as in the following example but it isn't allowable in the spec.

   public void doSomething() {

       try (contextClassLoader(Example.class)) { // Compilation errors

           // Class class that required the context class loader
           Endpoint.publish(null, null);            
       }
   }

Still the implementation of this is rather trivial and does still tidy up the original code. The only wrinkle is the need to have a public class/ interface subtype of AutoCloseable so that we can narrow the throws clause on the close() operation. If you don't do this then the original code has to deal with "throws Exception".

   public static CloseableContext contextClassLoader(Class loader) {
       return contextClassLoader(loader.getClassLoader());
   }

   public static CloseableContext contextClassLoader(ClassLoader loader) {
       final Thread currentThread = Thread.currentThread();
       final ClassLoader ocl = currentThread.getContextClassLoader();
       currentThread.setContextClassLoader(loader);
       return new CloseableContext(currentThread, ocl);
   }

   public static class CloseableContext implements AutoCloseable {
       private Thread _currentThread;
       private ClassLoader _ocl;

       private CloseableContext(Thread currentThread, ClassLoader ocl) {
           this._currentThread = currentThread;
           this._ocl = ocl;
       }

       @Override
       public void close() {
           this._currentThread.setContextClassLoader(this._ocl);
       }
   }

Ignoring the return value of a command in Hudson/Jenkins sh task

2012-05-26T17:42:00.001+01:00

So I was was trying to run pylint and nosetest running on a Jenkins instance; but these python commands tend to return a non zero status code if they see a non zero return code. This is a problem as it means that any further steps are ignored. The most obvious thing to try was this:

pylint src/openhea -f parseable ; true

The problem is that both statements are treated as separate steps so when the first one fails it never does the return true. Instead, and pointed out by a helpful chap in the #efdhack2012 room was to do the following

pylint src/openhea -f parseable || true

This correctly ignores the false status code, the only wrinkle is that you need to be careful when piping the output, it has to be to the left of the bars.

pylint src/openhea -f parseable > output.report || true

Common try-with-resources idiom won't clean up properly

2012-05-10T12:02:00.000+01:00

So I was re-reading some blogs on the JDK 7 language changes and I noticed the following idiom seems popular:

//
try (BufferedReader br = new BufferedReader(
        new FileReader("/tmp/click.xml"))) {
   System.out.println(br.readLine());
}

That seems all well and good; but consider what happens should something goes wrong constructing the BufferedReader. The contract for try-with-resources will only try to close and tidy up the declared field, if something goes wrong then FileReader is never closed.

So consider this example where the constructor of BufferedReader throws an Error, it could well be an OutOfMemoryError or any number of other failure modes.

//
package client;

import java.io.*;

public class BufferedFileExample {

    public static void main(String[] args) throws FileNotFoundException, 
                                                  IOException {

        try (BufferedReader br = new MyBufferedReader(
            new MyFileReader("/tmp/click.xml"))) {
            System.out.println(br.readLine());
        }
    }
    
    public static class MyFileReader extends FileReader 
    {

        public MyFileReader(String in) throws FileNotFoundException {
            super(in);
        }
        
        public void close() throws IOException {
            super.close();
            System.out.println("Close called");
        }
    }
    
    
    
    public static class MyBufferedReader extends BufferedReader 
    {

        public MyBufferedReader(Reader in) {
            super(in);
            throw new Error();
        }
    }
}

This gets us the following output and as such the close is not performed on the FileReader which until the finaliser is called. You are stuck until the gc wakes up in the future to tidy it up.

Exception in thread "main" java.lang.Error
 at client.BufferedFileExample$MyBufferedReader.(BufferedFileExample.java:38)
 at client.BufferedFileExample.main(BufferedFileExample.java:12)
Process exited with exit code 1.

Luckily there is an easy fix for this, the language change supports multiple entries in try statement, so you can modify this code to be:

//
try (FileReader fr = new MyFileReader("/tmp/click.xml");
     BufferedReader br = new MyBufferedReader(
        fr)) {
   System.out.println(br.readLine());
}

So if you run the code with the modification above then you find that close is now called:

Close called
Exception in thread "main" java.lang.Error
 at client.BufferedFileExample$MyBufferedReader.(BufferedFileExample.java:38)
 at client.BufferedFileExample.main(BufferedFileExample.java:12)
Process exited with exit code 1.

So problem solved, well unfortunately there is another wrinkle here, if we remove the code that throws the Error then we will see the following output:

Close called
Close called
Process exited with exit code 0.

So this is fine for classes that implement the existing java.io.Closeble interface because calling close for a second time is fine; but the more generic interface that is used for other closeable resources unfortunately doesn't have this restriction, so quote the spec

Note that unlike the close method of Closeable, this close method is not required to be idempotent. In other words, calling this close method more than once may have some visible side effect, unlike Closeable.close which is required to have no effect if called more than once. However, implementers of this interface are strongly encouraged to make their close methods idempotent.

So in order to using a split-try-with-resources idiom you first need to make sure that the resources have a idempotent close method. I suspect that most API will do this; but you need to check first and perhaps need to further wrap your objects to make this the case.

One final point with try-with-resource is the handling of exceptions when the close and construction fails. JDK 7 introduced the concept of suppressed exceptions, so you it make the split version of the code throw an exception both in the constructor or the MyBufferedReader and in the close of MyFileReader you get to see the following novel stack trace.

Close called
Exception in thread "main" java.lang.Error
 at client.BufferedFileExample$MyBufferedReader.(BufferedFileExample.java:38)
 at client.BufferedFileExample.main(BufferedFileExample.java:12)
 Suppressed: java.lang.Error: close
  at client.BufferedFileExample$MyFileReader.close(BufferedFileExample.java:27)
  at client.BufferedFileExample.main(BufferedFileExample.java:14)
Process exited with exit code 1.

This might not be what you calling code is expecting - certainly it might differ slightly from what the try/finally code you are replacing would produce, worth know thing that exceptions are handled slightly differently. For example if there is just an Error thrown during the close method would would see this odd self-referential stack trace.

Close called
Close called
Exception in thread "main" java.lang.Error: close
 at client.BufferedFileExample$MyFileReader.close(BufferedFileExample.java:28)
 at java.io.BufferedReader.close(BufferedReader.java:517)
 at client.BufferedFileExample.main(BufferedFileExample.java:15)
 Suppressed: java.lang.Error: close
  at client.BufferedFileExample$MyFileReader.close(BufferedFileExample.java:28)
  ... 1 more
Process exited with exit code 1.

So the lesson here is that the impact of some of the new JDK 7 constructs are going to a little bit subtler than we might expect. Many thanks to my valued colleague Mark Warner for talking this one through with me.

Off-loading test execution using Hudson part 3, skipping the queue

2012-04-03T18:00:00.004+01:00

Previously I had looked at preflighting, (1, 2) with Hudson (And Jenkins) but forgot to mention a very important plugin that is essential if you want to convince your developer to preflight there work. The problem is that is the slave allotment/smallholder/farm is busy running other jobs then they are forced to wait before merging.

The easy solution to this problem is the priority sorter plugin. This allows you to specify that some jobs are more or less important then others. The default priority is 100, so seeing your preflight job to 1000 should see it jump to the front of the queue.

It is interesting how the length of tests affects my commuting behaviour, now that is take around an hour to run the long tests just for our components. It makes sense to start a work at home, kick of a job and you should have you nice shiny test results by the time we get into work. One day I might even get a blue ball if I am so very lucky with our dependencies. :-)

Catalog support for WADL client generation

2012-03-16T09:55:00.004Z

I have just been fixing a few bugs in the WADL client generator that I have been working on recently. In particular one bug relates to being able to control the server the client uses at deploy time. (WADL-44)

So for example you have a standard packaged ?ar file that contains a client to a production service, in order to run automated testing you want to be able to use the same archive; but redirect the request to an internal server. Using a deployment plan, for example, you can update or introduce an extra deployment descriptor called META-INF/jax-rs-catalog.xml.

Only the base URI for each root resource in the WADL is updated; but this should be enough for most cases. (Would be interested to hear where people though this was limiting). This simple example redirects the client from a production service to a qa server.

<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">
  <uri name="http://production.example.com/resource"
          uri="http://qa.example.com/resource"/>
</catalog>

If you want to play with this you are going to have to check out the tip of the WADL client project for the moment; but in the coming weeks it will be part of the 1.1.2 release. Feedback always welcome.

Off-loading test execution using Hudson part 2

2012-01-23T09:36:00.007Z

Previously I have written about using Hudson to perform pre-flights using branches on source control systems; but sometimes you just have a patch file that you want to run your tests against.

It turns out this can be quite simple, you just need to make use of a job parameter that takes a File as it's input, this is then written to the specified location in the workspace. In my example I have the following two parameters, the Title allow me to name the job and this property is picked up later by the job Description plugin.

Once you know where the patch is located you can just apply the patch before you perform the required build steps, here is an example using SVN so we use the unix patch command, git users can use the git command instead for this.

And there you are, you can test a user submitted patch for testing without tying up your development machine. You can also run the tests more than once to check for any intermittent tests which can be very valuable in concurrent environments.

x!, a very simple xml library

2011-11-25T11:28:00.007Z

A couple of weeks ago, having gotten annoyed about the amount of boilerplate you need in Java in order to do some very simple XML actions I decided to put something in place that would get me close to the fluency of the Groovy and JScript XML extensions with a very small interface. Indeed the original proposal can be contained in a tweet and the only methods I have added since are parent() and prefix() which doesn't quite fit in the standard 140 characters.

So there is basically only one class to worry about X, this allows you to read and write values, select children and perform the normal input and output operations. Everything throws unchecked exceptions, and where possible Attr and Elements are treated the same so for example set(val) and get() will work for both in a consistent manner.

You can find simpler examples in the unit tests in the github project, but the most interesting use cases for me are when used with JAX-RS to access and program REST resources that accept XML. XPath makes Duck Typing easy and if you are careful your code can ignore minor schema changes, this is in stark contrast to static JAX-B binding for example.

Take this example, it looks at a remote resource and sets it as being offline using the Jersey client. In the simplest case you need to explicitly tell Jersey about the X message body readers and writers; but other than that the code is quite straight forward.

   ClientConfig clientConfig = new DefaultClientConfig(
      XMessageBodyWriter.class,XMessageBodyReader.class);    
   Client client = Client.create(clientConfig);
   client.addFilter(new HTTPBasicAuthFilter(...));
        
   WebResource resource = client.resource("http://hudson...");

   resource.put(
      resource.get(X.class)
         .set("//n:offline", "true"));
        
   System.out.println(resource.get(X.class).get("//n:offline"));

Note in the last line we use the pre-defined namespace prefix n which is always populated by the first node in the tree. You can also perform the select, or selectList for a list, explicitly to get hold of the value:

   resource.put(
      resource.get(X.class)
         .select("//n:offline").set("true"));
        
   System.out.println(resource.get(X.class)
     .select("//n:offline").get());

You can also use x! in JAX-RS resources, here is a very simple hello world example using the same message body readers and writers as before. Since they are marked as @Provider your container should be able to pick them up for you.

@Path("/hello")
public class HelloMessage {
    @POST
    @Produces("text/xml")
    @Consumes("text/xml")
    public X hello(X input) {

        String name = input.select("//name").get();

        X response = X.in("http://www.example.com", "message");
        response
            .children().create("name")
                .set(name).parent()  // Think of .parent() like a CR
            .children().create("message")
                .set("Hello " + name)
                .set("@lang", "en");

        return response;
    }
}

I am not entirely sure about the flow to create new objects, feedback is always appreciated of course. The use of @ to set attributes is quick as internally this doesn't result in a XPath query. Currently you can only work on direct children when creating new attributes, for more complex paths you need to select the direct parent to create new attributes. You can still set values on attributes that do exist with complex xpath expressions though.

    // @attr exists
    x.select("....@attr").set("value");
    or 
    s.set("...@attr", "value");

Here is a simple request and response from this service just for comparison you can get the code to reflect the xml quite closely.

// Example input message
<message>
   <name>Bob></name>
</message>

// Example response

<message xmlns="http://www.example.com">
   <name>Bob></name>
   <message lang="en">Hello Bob></message>
</message>

This really was a thought experiment that got out of control; but I would welcome feedback, suggestions, and of course as it is on GitHub patches.

One interesting direction is replacing the current DOM implementation with a streaming version which might be possible for certain read and write options depending on the ordering of the data.

A little script to update job descriptions on Matrix jobs

2011-10-19T13:25:00.005+01:00

We have started to use Matrix jobs in Hudson to try and reduce the amount of clutter on the front page; but we have run into problems in that the job description plugin we used to add the LABEL name to the job id only works for the subordinate tasks. This leaves the top level job without a description.

There are of course other ways you can do this; but since I was playing with the new REST api yesterday, I am going to use this. You could of course use the old RESTy API or indeed the command line tool to perform similar feats. This week I mostly just have a hammer.

Before you use the proxy classes you need to first create a client with the right security credentials configured:

   public static Client createClient() {
       Client client = Client.create();
       client.addFilter(new HTTPBasicAuthFilter("xxxxx@oracle.com","xxxxxxx"));
       return client;
   }

The first part of the job is going to be to create proxies for both the parent project and of the child variant:

   public static void main(String[] args) {

       if (args.length!=2) {
           System.err.println("Expecting two parameters project and variant");
       }

       String project = args[0];
       String variant = project + "%2F" + args[1]; 

       System.out.printf("Will copy missing description from %s to %s\n", variant, project);

       // Create a Jersey client
       Client client = createClient();

       // Create a suitable proxy using the client we configured
       Projects hudsonProjects = projects(client);

       // Handles to the right resources
       Projects.ProjectNameBuilds projectBuilds = hudsonProjects.projectNameBuilds(project);
       Projects.ProjectNameBuilds variantBuilds = hudsonProjects.projectNameBuilds(variant);

Then we can make a quick map of the variant build number to descriptions by getting the first build object:

       // Copy the descriptions out from the variant
       Map<Integer, String> variantToDescription = new HashMap<Integer,String>();
       BuildsDTO variantBuildsDTO = variantBuilds.getAsApplicationXml(BuildsDTO.class);
       for (BuildDTO build : variantBuildsDTO.getBuild()) {
           if (build.getDescription()!=null)
           {
               variantToDescription.put(
                   build.getNumber(), 
                   build.getDescription());
           }
       }

Now it turns out that the current version of the API doesn't allow you to update build descriptions; but we can easily read them using this API and then update them using a simple form POST using the same client.


       // Update the main project descriptions
       BuildsDTO projectBuildsDTO = projectBuilds.getAsApplicationXml(BuildsDTO.class);
       for (BuildDTO build : projectBuildsDTO.getBuild()) {

           String description = build.getDescription();
           // Update description if it has not already been set
           if (description == null || description.length()==0) {

               String variantDesc = variantToDescription.get(build.getNumber());
               if (variantDesc!=null && variantDesc.length()!=0) {

                   // We need to set the description; but nothing in the REST API
                   // so we can use the url property to perform a form submit that will
                   // update the description for us.

                   MultivaluedMap map = new Form();
                   map.add("description", variantDesc);
                   map.add("Submit", "Submit");
                   WebResource buildResource = client.resource(
                       build.getUrl() + "submitDescription");
                   ClientResponse response = buildResource.type(MediaType.APPLICATION_FORM_URLENCODED_TYPE)
                       .post(ClientResponse.class, map);
                   System.out.printf("Updating build %d with description %s gave response %d\n",
                                     build.getNumber(), variantDesc, response.getStatus());

               }
           }
       }