We’ve been working with performance in our REST API a while back. We’ve been considering different options for measuring our API in production, but we had an idea to use the design of the API itself to handle the measuring.

Part of the inspiration came from the Changlelog’s coverage of CodeConf and Coda Hale’s Metrics talk. (Slides). 

Using our pipeline approach to constructing a REST-API with conventions, we added a pipe that starts a stopwatch and stops it when the request is finished. Then we can log statistics for each individual part of the API, closely coupled with the code that is executed for that particular request. I pitched the idea to my good colleague Johan Andersson, and he did a fantastic job at implementing the solution!

The output looks something like this:

metrics output

This allows us to see how our API is performing in production, and where we need to focus our optimization efforts to make the API go faster. We can even add some flags to the request to get more detailed output.

I got a Tellstick for Christmas together with some Nexa remote controllers for the lighting at home.

I knew that this type of project needed to be started immediately or else it would just loose it self in a drawer somewhere. So I added some tasks to Track and Shuffle and got to it.

I wanted to stick installed in my Linux server that sits in my closet and acts as my home’s central information system. So I was natural to install it there.

Tellstick has more software available for Mac and Windows, but they do have software for Linux as well.

Install the tell stick software on the Linux machine according to the following guide: http://developer.telldus.se/wiki/TellStickInstallationUbuntu

Configuration of devices can be found here: http://developer.telldus.se/wiki/TellStick_conf

Software page seems promising: http://developer.telldus.se/wiki/Software

The install for Linux is just a C library, so I’ll have to set up some services to use the stick, but first. How to set up a device?

Apparently all you have to do is edit the configuration in /etc/tellstick.conf. This is how I did it.

deviceNode = "/dev/tellstick"
device {
  id = 1
  name = "Spotligh bokhylla"
  protocol = "arctech"
  model = "selflearning-switch"
  parameters {
    house = "1"
    unit = "1"
  }
}
device {
  id = 2
  name = "Taklampor bokhylla"
  protocol = "arctech"
  model = "selflearning-switch"
  parameters {
    house = "1"
    unit = "2"
  }
}

I spent a long time to figure out how to find out which house code to use. It turned out that all you do is select on and teach it to the Nexa receiver.

Now for the services?

I want to have it as a webpage, and some way I’m used to programming against. a Rest service comes to mind since I enjoy that model. So I need to create a wrapper for the C library. So what to use: Python? NodeJS?

Bingo, Apparently there is a restful web service already available.

https://github.com/pakerfeldt/remotestick/wiki/rest-api

http://remotestick.org/

I even has a Android app for the server.

With this setup all I had to do was to write a little JavaScript page that turns the lights on or off.

So I did just that, here is the result of the first version.

image

It has support for dimmers, however I haven’t been able to try that out fully due to lack of well… dimmers.

I tried to get it contributed to the remotestick repro, however the author pointed out that it was better to have this as an add-on to the remotestick-server. So the end result is that you can find the GUI here: https://github.com/morkeleb/remotestick-webgui

The all you need to set it up is to download a release available under downloads, and then extract the static directory to the static directory under remotestick-server.

Now I wanted the webpage to be accessible through port 80 without interrupting any of the services already running on my box, luckily this holiday I configured my Linux box to run with NGINX to front all the different services available through HTTP on my box.

Configuring NGINX to run proxy remotestick and handle my JavaScript page

        location /remotestick/ {
                proxy_pass http://localhost:8422/;
                proxy_redirect    off;
                proxy_set_header  Host             $http_host;
                proxy_set_header  X-Real-IP        $remote_addr;
                proxy_set_header  X-Forwarded-For  $proxy_add_x_forwarded_for;
        }

There is a problem with hosting it under NGINX since it changes the url and the Android app for Remotestick currently only works with IP-addresses, however I haven’t tried the latest version which was released just a few days ago. It should be able to handle more general addresses to the server.

I hope to add a simple guide for adding new devices trough the web interface, since it would greatly improve the setup experience of adding receivers to my home.

Currently the greatest problem with the TellStick is that it’s not quite easy to install and setup, however I hear that TellStick is working on a service called TellStick Live! that is in beta for Windows and Mac users.

We’ll see how this evolves :)

I ran into another problem with Grails just now.

I’m starting to separate the REST implementation of my service. I wanted to keep the default generated behaviour of the Grails views, at least for now. But the interaction of the web-application is foremost using the REST api.

Thus I wanted to add Url mappings to a /rest/ “sub-directory” of the web-app. So I have all the rest functionality using the rest/ prefix. Thus I updated my Urlmappings as such:

static mappings = {
    "/rest/ServiceProvider/$id?"(controller:"ServiceProvider", action="rest")
    "/$controller/$action?/$id?" {
      constraints {
        // apply constraints here
      }
    }
}

There is a problem with this, I named my domain class with a capital letter. As such my controller also starts with a capital letter. Apparently the url-mappings can’t find my controller when its specified using  a capital letter in the name. If I enter the domain-class name using lower-case letters in the url-mapping it works.

static mappings = {
    "/rest/ServiceProvider/$id?"(controller:"serviceProvider", action="rest")
    "/$controller/$action?/$id?" {
      constraints {
        // apply constraints here
      }
    }
}

Jira ticket: http://jira.codehaus.org/browse/GRAILS-4488

Update: I made some assumptions about the error that were untrue, you do not need to rename the controller to get the custom url-mappings to work. You just need to start the domain-class name with a lowercase letter.

I had an idea today while working with a performance issue on the REST-service of RemoteX Applications.

In theory having a performance counter with the average execution time of SQL-queries over time. Will allow monitoring of system wide average cost for queries in the system.

That means that overtime its possible to track the changes of the execution times as they grow or shrink during normal production environment conditions. The cost for calculating the average could be relatively cheap to calculate compared to the round trip to the servers, and also the calculation could be threaded of to a different thread, so as not to disturb the worker thread that handles the request to the web server to begin with.

The idea is to examine the changes of the average query time over time, to see if performance of data access is changed between version upgrades, or hardware upgrades for that matter. There are some performance counters built into MSSql that can give data that is somewhat like this. However most of the require heavy load to see any performance upgrade or downgrade. This instead logs an average of the actual execution time for the application.

A benefit with this is that the performance is individual per application, but the actual code or counter could be reused between different applications.

So I have my Grails environment setup. The point is to build a REST-service that will work as a back end to a highly dynamic JavaScript page. I chose Grails, since it builds on Hibernate and Spring.

Two frameworks Id like to examine for the purpose of seeing the differences between Castle and nHibernate in the .Net world.

Building domain-classes and having the represented as XML is straight forward in Grails, but I need to take care of the login scenario. If you’d like to know more about building REST web services with Grails, I recommend reading the Mastering Grails: RESTful Grails article. Login seems like a good place to start, to ensure that my domain-controllers can dictate login as I develop the domain.

Now Grails.org has a list of tutorials, one of them, focusing on Login. It’s a good beginners tutorial showing how to use interceptors to execute user verification before showing the actual view requested. For my scenario however it has a few problems. Lets use the tutorial as a start of point.

The problems I have with the solution is with the following code:

def checkUser() {
  if(!session.user) {
    // i.e. user not logged in
    redirect(controller:'user',action:'login')
    return false
  }
}

For my scenario this has one problem, in two places. Its stateful. First of it always directs the user to the same place, of course it can be different per controller. I need it to simple say access denied.

def checkUser() {
  if(!session.user) {
    // i.e. user not logged in
    response.sendError(401)
    return false
  }
}

The above code is changed to instead of redirecting the user, it simply sets the response code to Access Denied. Now there’s just one issue left, the session code.

I don’t want to use session, as it requires my server to remember each session, preferably I want the client to remember the state.

Another issue is that I want to use Mor.ph AppSpace, if I pay for the most basic of subscriptions at Mor.ph I get two Cube, i.e. two servers with a load balancers in front. As a result I can’t be sure that the same server will service the same client all the time.

If we instead choose to check for the logged in user using the authorization header. The state is always provided by the client, instead of keeping the state on the server.

def checkUser() {
  if(!UserController.checkAuthorization(request, response)) {
    // i.e. user not logged in
    response.sendError(401)
    return false
  }
  return true
}

I let the UserController do the checking of the user, using a static method. This will let me keep the authorization code in the same place I handle users to begin with. It also allows me to change the authorization method later by changing the code in the checkAuthorization method. We could use Dependency Injection here to inject a real instance of the UserController, or even better an object with the sole responsibility to check user access. However DI is out of scope for this post.

You can implement any way of checking the user login you want. If you want a simple solution and stick with the HTTP standard you could use Basic auth. Here is a good explaination of how to implement Basic auth for Grails. Another way is too use a cookie to hold authorization information.

For this example were just sending username:password in the authorization header. My final implementation will be Basic Auth with the password hashed to a checksum. The point of the hash is to never send plain-text password across the net.

Here is the checkAuthorization method in the UserController for this example.

static boolean checkAuthorization(HttpServletRequest request, HttpServletResponse response) {
  def authString = request.getHeader('Authorization')

  if (!authString) {
    return false;
  }

  def credentials = authString.split(':')
  if( credentials.length != 2 || credentials[0] == null || credentials[0] == "" || credentials[1] == "" || credentials[1] == null)
    return false;

  def user = User.findByEmailAndPassword(credentials[0], credentials[1])

  if (user) {
    return true;
  }
  return false;
}

Now we can create some JQuery based JavaScript code to handle login.

function RestClient() {
    var self = this;
    var auth;

    function ShowLoginUI(afterLoginCallback) {
        var dialog = document.createElement('div');
        $(dialog).html("<table><tr><td>Email</td><td><input id='email' type='text'/></td></tr><tr><td>Password</td><td><input id='pass' type='password'/></td></tr></table>");
        $(dialog).dialog({ buttons: {"Login": function() {
            auth = $(this).find("#email").val() + ":" + $(this).find("#pass").val();
            afterLoginCallback();
            $(this).dialog("close");
        }, "Cancel":function() {
            $(this).dialog("close");
        }}, modal:true});
    }

    this.Get = function(url) {
        var data = null;

        data = $.ajax({
            url : url,
            method : 'GET',
            beforeSend : function(req) {
                req.setRequestHeader('Authorization', auth);
            }, cache : true,
            error: function(request, textStatus, error) {
                if (request.status == 401) {
                    ShowLoginUI(function() {
                        data = self.Get(url);
                    });
                }
            }});

        return data;
    };
}

The above code does Ajax Get-calls to the specified address. If the call failed due to Access Denied, it shows a login form and tries to execute the Ajax request again. The class remembers the authorization string, so as long as the user uses the same RestClient-object the state is maintained at the client side. Ill leave instructions on how to wire up Grails with JQuery to someone else.

Here is a simple usage example:

var a = new RestClient();
$("#test").click(function() {
  var data = a.Get("plant/create");
  $(document).append(data);
});

Good luck writing scalable login scenarios using Ajax instead of page reloads.