Monthly Archives: March 2014

Understanding Multi-thread, Jersey and Jetty

I think it belongs to the basic understanding between request, thread and web container. Although similar concept could apply to other web  containers or REST frameworks, I like to be modest about my understanding and the post here is limited to the discussion of Jersey and Jetty.

There is a common doubt on how requests interact with web servers through REST. I want to know how each client request is being handled by a thread spawned by web container and what are those configurations I can play with to tune my REST service performance. What I did is simply:

  1. maven a web app project and use jetty-maven-plugin for my test. It is relatively easy since I just issue maven jetty:run then I can test the service without any hassle.
  2. write my simple service by listening @Path defined and write web.xml files to define my servlet. In the service, I wrote a method getDoc() to retrieve a local file and return a Response with the local file. I want to know what is going on when multiple client requests hit my REST. I purposely to delay the service execution to make it longer by adding Thread.sleep(a processing time).
  3. use curl to simulate my naive client by calling curl --connect-time --max-time -G -s -w %{time_total}:%{time_connect}\\n http://localhost:8080/rest/get. The GET request will waiting X seconds for getting the connection, and Y seconds for entire operation, otherwise a timeout will interrupt the client request. I tried with 3 such clients simultaneously.

The first finding is that Jetty takes in 3 clients, and processing them one by one simultaneously (i.e. less than 3 clients x processing time), and they are interleaved with each other. Each clients are blocked for slightly more than the processing time. That means the default jersey service is a Synchronous Web Service. If the client is going to be held too long and client is better off to carry out other tasks, an Asynchronous Service is better. Jersey supports this.

So how to play with connections and what is the default behavior of jersey service? We need to look into the configuration of jersey. Jersey configuration can be put in the pom.xml within the jersey plugin configurations.

<Configure id="Server" class="org.mortbay.jetty.Server">
  <!-- required configuration -->
  <!-- connectors -->
  <!-- handlers -->
  <!-- webapps/contexts -->
  <!-- optional configuration -->
  <!-- threadpool -->
  <!-- session id manager -->
  <!-- authentication realms -->
  <!-- request logs -->
  <!-- extra server options -->
</Configure>

I played with thread pool to limit the max and min pool size, but the number of connections seems not controlled by that. What I did was limit the pool size to min and max as 1, and 3 clients still got processed simultaneously. The next step is to know more about the http connections and jetty configurations.

Interesting list:  Quora, Newbie Guide to Jetty, Jetty Optimization Guide

Jersey Client Discovery

Jersey Client has two different implementations for Synchronous and Asynchronous resource.

    • For Synchronous resource, the key invocation call lies in the portion of createDefaultClientHandler() which returns a new instance of URLConnectionClientHandler. The new returned handler has a method handle(ClientRequest ro) which essentially opens a connection by java.net.URL. This means each time a request is called like in webResource.get() a new URL connection is opened. So for Synchronous resource, the number of threads holding a http connection is equal to the number of requests.
    • For Asynchronous resource, the key invocation call lies in the portion of AsyncWebResource$handle returning a FutureClientResponseListener. The constructor of AsyncWebResource calls a getExecutorService() which follows a singleton pattern and retrieves the only instance of a Executor pool by either a newFixedThreadPool or a newCachedThreadPool. Before return, AsyncWebResource$handle calls executorService.submit() and sumbit a runnable to the executor pool. So for Asynchronous resource, the number of threads holding a http connection is determined by the pool.

This is what will happen when concurrent threads invoke Jersey Client, either one http connection per request or connection threads controlled by executor service.

Studying Builder Pattern with Inheritance

Just to pay some tech debts.

Builder Pattern alone is easy to understand, but not so much if inheritance is concerned.

(Part I) understand the problem

First, we need to understand how inner class works.

Given A <— B (B is subclass of A), and A.Builder is inner class of A, and B.Builder is inner class of B. If we have A.Builder <— B.Builder as well, what is the consequence of following calls?

  1. new B.Builder(): A and B are not created, but A.Builder() is called and then B.Builder() is called. new B.Builder().getClass()returns B$Builder;
  2. new B.Builder().Bmethod(): call B’s Bmethod (who returns B$Builder); getClass() returns B$Builder;
  3. new B.Builder().Bmethod().Amethod(): call A’s Amethod (who returns A$Builder); but getClass() still returns B$Builder();
  4. new B.Builder().Bmethod().Amethod().build(): call B’s build() (who returns new B(B$Builder)) because Amethod return “this”, by no means compiler knows who is “this”; Once B(B$Builder) is called, first B’s super(B$Builder) is called, and then B’s rest is called. So finally, B is created and returned. However, this only happens in runtime. In compile time, the build() that is involved is A’s build() . So casting has to happen so that compiler can pass.

Compile time follows the definition/ signature strictly, but in runtime, jvm finds the nearest method to call. Method getclass() only returns the runtime class, and down cast (from parent to child) is only possible if in runtime parent type is actually a child object. Compiler does not know what is “this” it is referring, so runtime cast must be done.

Therefore, as Eamonn McManus mentioned in the post,

new B.Builder().Bmethod().Amethod().build() will be compiled, but the sequence requirement is not a pleasant requirement, and a down cast must be there in order to do sth. like

B b = (B) new B.Builder().Bmethod().Amethod().build();

(Part II) explore to solve the problem

In summary, the problem happens because in parent class A$Builder we have a method Amethod who returns this, but in compile time, compiler correctly thinks “this” is A$Builder, but what we hope is that “this” is B$Builder. So we need a trick to cheat compiler, here we go,
Continue reading