Archive

Posts Tagged ‘Design’

Working at the right level of abstraction

May 25th, 2010 2 comments

At RemoteX we have been a bit unhappy with the code produced to create our REST-service using WCF.

The problem is that you have these contracts that you specify. These contracts then build up a web-service end-point using the WebGet and similar attributes, to create a REST-style endpoint. The contracts however often look like a SOAP contract, or an interface wrapping behavior. A typical contract often includes the following operations for a specific type of document:

  • Get Single (web get on href entity-type/id)
  • Get List (web get on href: entity-type)
  • Get List with modified since (entity-type with query-string)
  • and so on

The trouble is the URI’s. A single contract often specifies several endpoint’s and several behaviors. There is no Single-responsibility principal at work, or rather the single responsibility might be that a contract handles a specific entity. End result, a lot of code for simple things and a REST-service that is unnecessary hard to extend.

This setup also makes it tricky to see the structure of the REST-service, the URI’s are spread out in the code contracts, forcing you to look at each method to figure out the different URIs.

After working with Grails and Django, and finally when I read the documentation for the Kayak web server. I realized what we were doing wrong. A reference in the Kayak documentation was the wakeup call:

The server component is an intuitive, unobtrusive, no-gimmicks implementation of HTTP, and the framework simply maps HTTP transactions to C# method invocations with minimal syntax and configuration.

We’re not working on entities as an abstraction, we’re building a REST-based service. Our level of abstraction needs to be HTTP.

So I discussed this with a colleague Johan, the day after. After a while Johan set to work on a piece of the REST-server that needed rewriting and after a week of TDDing. He came back with this wonderful piece of infrastructure, that worked along side the “old style” infrastructure.

The idea is that we have one contract. This contract has 4 methods:

  • Get
  • Put
  • Post
  • Delete

Essentially all the methods we want to implement. We then have a route configuration, similar to that of Grails or Django. That routes requests to Controllers that will take care of them.

Johan set things up so there is a single server, that looks at the route-configuration when a request arrives and executes a command on the controller object. The controller object doesn’t have to add methods for the operations doesn’t support. In fact, the controller object is a plain POCO with no inheritance or interface on it. As long as it has a method that matches the operation it will be called.

Serialization is handled in the framework and by following conventions when writing the controller you can take different input automatically processed by the infrastructure.

Lets see an example:

public class LogController
{ 

    readonly DataContextFactory<RMXAppsDataContext> _contextFactory;  

    public LogController(DataContextFactory<RMXAppsDataContext> contextFactory)        {
        _contextFactory = contextFactory;   

    }

    public void Post(LogEntry entry, IIdentity user)
    {  

        PostMessage( user, entry.Message );
    }

        public void PostMessage( IIdentity user, string href, string entry )
    {            using (var context = _contextFactory.CreateContext())
        {                var dbUser = context.GetTable<User>().Where(u => u.UserName == user.Name).FirstOrDefault();
            var dbEntry = new Applications.Data.LogEntry                    {
                    Message = entry,                        Date = DateTime.UtcNow,
                    User1 = dbUser                    };
            context.GetTable<Applications.Data.LogEntry>().InsertOnSubmit(dbEntry);                context.SubmitChanges();
        }        }         [return: Xml]
    public Log Get()        {
        Collection<LogEntry> entries;
        using (var context = _contextFactory.CreateContext())

        {                var e = (from l in context.GetTable<Applications.Data.LogEntry>()

                    orderby l.Date descending
                    select Mapper.Map<Applications.Data.LogEntry, LogEntry>( l )).Take( 50 );

                      entries = new Collection<LogEntry>(new List<LogEntry>(e));
        }            return new Log { Href = "log/", LogEntries = entries };
    }}

It’s not the prettiest of examples (and I apologize if the code doesn’t look that great).

What your looking at is a simple log controller. It handles the log/ endpoint. The endpoint has a Get and a Post operation. Post is used to add LogEntries, and Get gives the latest 50 entries.

You will notice that the Get operation is marked with [return: Xml] this means that the operation will serialize the return value using XML.

The Post operation takes two inputs. LogEntry and a IIdentity. Both these parameters are set by conventions.

IIdentity is the user, as authenticated by the login-module, the infrastructue simply gives the user to any IIdentity parameter in a web-method.

The LogEntry is the deserialized body of the request. The infrastructure looks at any parameter that doesn’t fit a convention, checks if its serializeable and if the body of the request can be deserialized into such an object.

Now a route configuration is another object, that specifies which urls which controllers are registered too. These structures are set up using Castle, and loaded from any assembly in the web directory. Allowing us to extend the REST-service by dropping in assemblies that implement our structure.

Since this was added the development of the REST-service has really taken off. We add roughly one-two endpoints per week, designed to solve specific pains and give views of the domain specific to the task at hand.

By working at the HTTP level of abstraction instead of the contract abstraction, we have reduced the cost for extending the rest-service tremendously. Even better is that the controllers are easy to test, the code displays its intent clearly, and you can see what each specific Uri in the REST-service does.

Special thanks to Johan, great work!

Tags: ,

Inheritance of features

January 14th, 2010 2 comments

Given my recent combining of Tracks and Shuffle. An interesting thing happened. Something that happens when you combine two products so that they inherit features from each other.

On Shuffle’s issue list there are a few items currently regarding repeating tasks, and the ability to hide a task for a period of time. Tracks has these features, since Shuffle can synchronize with Tracks, Shuffle now also have these features. That is if you choose to synchronize with Tracks.

I’ve done a few integration projects thus far. And the ones that are truly successful are when they take the aspects and features of both applications into consideration, to create a win-win scenario between the two applications.

Here is another recent example. At RemoteX, we recently integrated a system that handles the ordering of materials, goods, equipment etc. In essence this integration feature means that you can go to a retailer, order goods, specify the order they should tag the invoice with. End result, RemoteX automatically adds the goods to the application and if possible attaches a copy of the invoice to the case. Ready to be billed to the customer. Since the other system has integrated with almost all resellers of service equipment in Sweden, the daring customers running this set-up can, at least in theory, eliminates the need for keeping stocks of equipment or goods in house.

If you understand where these synergy effects can show up you can also align the integration so that both applications maximizes the gain of integrating with the other system. This is important to think of when designing an integration, how do we achieve the maximum win-win?

The guessing user interface

January 13th, 2010 No comments

My girlfriend has downloaded a solitaire application to her HTC Tattoo. While solitaire isn’t anything revolutionary it has an interface feature which got me thinking.

Se what the interface does is move those cards that it is sure that the next step is to move. So if I for example get an Ace, it moves it to it’s correct place at the top four card stacks.

This feature got me thinking about mobile phone interfaces. In applications in general, but even more so in the mobile context, you want to reduce the amount of interaction done with the user interface.

What if we can “guess” the input of the users input in the interface even in business applications?

image

We have hints in the domain, such as if you select this type of project you cannot register this type of time on it. You don’t register vacation time on a customer’s bill if your a consultant. Well, at least not that often.

These hints are common constraints, you might not want to enforce them by disabling the alternatives available. But you can check them and see if for a given input if there are any “assumptions” you can make about the next input to be made.

image

To take this one level further. We have the application data. We can order things in the UI according to a statistical analysis. Given this value in a combo box, this other value has more than 80% chance to be set in the other combo box. In such a case we can “guess” that this is the input the user would want to enter, and worst case is that he has to change it.

The cost of changing a value in a combo box is equal to setting one in the first place.

Now statistical analysis of data can often be costly, and typically not something you would want to do in a UI scenario. But what you can do instead is to calculate the statistical values into a decision graph (I think of it as similar to a Markov process for those whom have studied them). We can calculate this graph every evening and have a graph available for all input for the next day. Traversing the graph is now a simple algorithmic exercise.

Tags: , ,

Rethinking design desicions

November 21st, 2009 1 comment

I read a blog post at Three Rivers Institute that lead me to the following presentation.

The presentation is about startups and search engine optimization. How to drive visitors to your website.

Dharmesh discusses how to adapt your webpage to generate visitors, also promoting his hubspot seo grader.

Now this got me thinking of a project I’ve been working on. It’s a highly dynamic JavaScript based page, hosted on Google AppEngine. (currently not in a state I’d like to show it off since its still lacking the basic features).

The page is being designed with a few ideas in mind:

  • All dynamic data i served to the page through REST
  • All static pages are served with long expiration, most content is static
  • The main page never reloads, instead JavaScript does AJAX to update the content
  • The JavaScript is designed using SOLID design principles
  • The JavaScript is tested
  • Back and forwards buttons still work
  • Marketing Content is served using Google Sites, to reduce bandwidth but especially to shorten roundtrips when working with Marketing material

Now the idea here is to give a similar feeling as to what GMail feels like. A common trend, in web application development I think. However, after watching the presentation the following thought occurred to me. The GMail application needs to be found by search-engines.

Google has a signup page with content that is searchable, but the main JavaScript client is not designed to be found by Search Engines.

I want my page to be found by search engines, and this means that there are a few of the basic Ideas behind the page that I need to reconsider. Especially the following:

  • The main page never reloads, instead JavaScript does AJAX to update the content
    • Meaning that the page looks “empty” to search engine crawlers since all content is loaded using JavaScript
  • Marketing Content is served using Google Sites, to reduce bandwidth but especially to shorten roundtrips when working with Marketing material
    • Right now this is implemented using IFrames, which probably cause trouble for search-engines (yeah yeah, I know frames suck especially IFrames… but there were some cross site issues I didn’t want to fix right now),
  • I need some of my content pages to be searchable, but not the AJAX-driven application in it self. So now I have to rethink some of my decisions which were made from a pure coding view-point, to adapt to marketing viewpoints (and search engine algorithms so its not all bad).

The importance of teaching

July 1st, 2009 No comments

I have a good friend whom after university focused on first communication and then management. But now he has turned to development. We’ve been having a few discussion over the past few days about SOLID and REST and Grails. IoC containers and what not.

He hasn’t seen these kinds of concepts in his development so far. But he finds the concepts interesting and he is eager to learn about it. Which has given me a unique opportunity to learn how to communicate these concepts.

Now I’ve been an assistant teacher at the university. But I found that some of the benefits of these concepts are hard to explain, simply because to me their very abstract and I usually don’t discuss them at this level. I’d also like to mention that my friend is great at asking the right questions.

So this is a shout out to Joel. Thanks for teaching me how to communicate these concepts and their benefits.

Tags: , , , ,

Refactoring how to open views

April 18th, 2009 No comments

We had a problem in RemoteX applications, not a big one, but an old structure that had to be refactored. Our open method only accepted URIs as arguments. This forced most of our commands that create new entities, to first save an empty new version of the entity to make it accessible by using a URI for it.

This in turn caused unnecessary round trips to the data access layer, but also caused problems with potentially empty data.

So this Wednesday this code died. In the old implementation; the general OpenView command accepted a URI. Now it can both take a URI but also an actual object.

To allow this I had to change the targeted end-point urls to activate a class instead of a delegate, as it was before. This has two great benefits.

  • By using classes I open up for taking advantage of Castle for setting up the different code for opening. Previously all the setup was made in one big wiring class, resulting in excessive class coupling CA warnings on that class.
  • Each class that opens an entity view conforms to SRP. It has one purpose, to open a view for a specific entity.

As I implemented the ViewOpener I noticed that all the delegates for opening entities had one thing in common. The first thing they did was to open the entity from the data storage.

By breaking out this behaviour into an abstract base class I could reduce the amount of code, simply by having the base class fetch the entity and then let the more specific view opener setup the UI.

This resulted in the structure illustrated below.

ViewOpener structure

ViewOpener structure

So what are the insights of this refactoring?

Having delegates for a specified behaviour is a code smell. By having simple short classes with a single behaviour instead, we make code easier to refactor. Its easier to simply add another simple class to the Castle wire up, than it is to implement the same behaviour in a large complex class that handles all UI setup.

Performance monitoring of a deployed application

April 14th, 2009 No comments

I had an idea today while working with a performance issue on the REST-service of RemoteX Applications.

In theory having a performance counter with the average execution time of SQL-queries over time. Will allow monitoring of system wide average cost for queries in the system.

That means that overtime its possible to track the changes of the execution times as they grow or shrink during normal production environment conditions. The cost for calculating the average could be relatively cheap to calculate compared to the round trip to the servers, and also the calculation could be threaded of to a different thread, so as not to disturb the worker thread that handles the request to the web server to begin with.

The idea is to examine the changes of the average query time over time, to see if performance of data access is changed between version upgrades, or hardware upgrades for that matter. There are some performance counters built into MSSql that can give data that is somewhat like this. However most of the require heavy load to see any performance upgrade or downgrade. This instead logs an average of the actual execution time for the application.

A benefit with this is that the performance is individual per application, but the actual code or counter could be reused between different applications.

Grails: Ajax Login using Basic Auth

April 7th, 2009 No comments

So I have my Grails environment setup. The point is to build a REST-service that will work as a back end to a highly dynamic JavaScript page. I chose Grails, since it builds on Hibernate and Spring.

Two frameworks Id like to examine for the purpose of seeing the differences between Castle and nHibernate in the .Net world.

Building domain-classes and having the represented as XML is straight forward in Grails, but I need to take care of the login scenario. If you’d like to know more about building REST web services with Grails, I recommend reading the Mastering Grails: RESTful Grails article. Login seems like a good place to start, to ensure that my domain-controllers can dictate login as I develop the domain.

Now Grails.org has a list of tutorials, one of them, focusing on Login. It’s a good beginners tutorial showing how to use interceptors to execute user verification before showing the actual view requested. For my scenario however it has a few problems. Lets use the tutorial as a start of point.

The problems I have with the solution is with the following code:

def checkUser() {
  if(!session.user) {
    // i.e. user not logged in
    redirect(controller:'user',action:'login')
    return false
  }
}

For my scenario this has one problem, in two places. Its stateful. First of it always directs the user to the same place, of course it can be different per controller. I need it to simple say access denied.

def checkUser() {
  if(!session.user) {
    // i.e. user not logged in
    response.sendError(401)
    return false
  }
}

The above code is changed to instead of redirecting the user, it simply sets the response code to Access Denied. Now there’s just one issue left, the session code.

I don’t want to use session, as it requires my server to remember each session, preferably I want the client to remember the state.

Another issue is that I want to use Mor.ph AppSpace, if I pay for the most basic of subscriptions at Mor.ph I get two Cube, i.e. two servers with a load balancers in front. As a result I can’t be sure that the same server will service the same client all the time.

If we instead choose to check for the logged in user using the authorization header. The state is always provided by the client, instead of keeping the state on the server.

def checkUser() {
  if(!UserController.checkAuthorization(request, response)) {
    // i.e. user not logged in
    response.sendError(401)
    return false
  }
  return true
}

I let the UserController do the checking of the user, using a static method. This will let me keep the authorization code in the same place I handle users to begin with. It also allows me to change the authorization method later by changing the code in the checkAuthorization method. We could use Dependency Injection here to inject a real instance of the UserController, or even better an object with the sole responsibility to check user access. However DI is out of scope for this post.

You can implement any way of checking the user login you want. If you want a simple solution and stick with the HTTP standard you could use Basic auth. Here is a good explaination of how to implement Basic auth for Grails. Another way is too use a cookie to hold authorization information.

For this example were just sending username:password in the authorization header. My final implementation will be Basic Auth with the password hashed to a checksum. The point of the hash is to never send plain-text password across the net.

Here is the checkAuthorization method in the UserController for this example.

static boolean checkAuthorization(HttpServletRequest request, HttpServletResponse response) {
  def authString = request.getHeader('Authorization')

  if (!authString) {
    return false;
  }

  def credentials = authString.split(':')
  if( credentials.length != 2 || credentials[0] == null || credentials[0] == "" || credentials[1] == "" || credentials[1] == null)
    return false;

  def user = User.findByEmailAndPassword(credentials[0], credentials[1])

  if (user) {
    return true;
  }
  return false;
}

Now we can create some JQuery based JavaScript code to handle login.

function RestClient() {
    var self = this;
    var auth;

    function ShowLoginUI(afterLoginCallback) {
        var dialog = document.createElement('div');
        $(dialog).html("<table><tr><td>Email</td><td><input id='email' type='text'/></td></tr><tr><td>Password</td><td><input id='pass' type='password'/></td></tr></table>");
        $(dialog).dialog({ buttons: {"Login": function() {
            auth = $(this).find("#email").val() + ":" + $(this).find("#pass").val();
            afterLoginCallback();
            $(this).dialog("close");
        }, "Cancel":function() {
            $(this).dialog("close");
        }}, modal:true});
    }

    this.Get = function(url) {
        var data = null;

        data = $.ajax({
            url : url,
            method : 'GET',
            beforeSend : function(req) {
                req.setRequestHeader('Authorization', auth);
            }, cache : true,
            error: function(request, textStatus, error) {
                if (request.status == 401) {
                    ShowLoginUI(function() {
                        data = self.Get(url);
                    });
                }
            }});

        return data;
    };
}

The above code does Ajax Get-calls to the specified address. If the call failed due to Access Denied, it shows a login form and tries to execute the Ajax request again. The class remembers the authorization string, so as long as the user uses the same RestClient-object the state is maintained at the client side.
Ill leave instructions on how to wire up Grails with JQuery to someone else.

Here is a simple usage example:

var a = new RestClient();
$("#test").click(function() {
  var data = a.Get("plant/create");
  $(document).append(data);
});

Good luck writing scalable login scenarios using Ajax instead of page reloads.

Working with a domain model

March 4th, 2009 No comments

We just took a new turn on the work with our Domain Model in RemoteX Applications.

In recent months, mostly due to the increased number of customers using the system. There had been some innovative ways in using the product. Some for which we hadnt intended the product to be used. Also requests were coming in that were requesting information to be ordered in a way that wasn’t possible using the existing domain model.

This combined with new team-members putting a slight strain in the communication, since the ubiquitous language was evolving based on new experiences and people. And the Domain Model wasn’t evolving with it.

As a result we had a meeting today, where we discussed the Domain Model, as is. The goal was to introduce it, and the reasons for it, to the new people on the team. But also to find a way to keep it more in line with the rest of the organization and our customers.

We came up with a few actions. One of them is to find a file based wiki to keep the documenation of the domain model in. That way, we can have the domain model checked in under source control, and update it as we update the product. The goal here is to not let the domain model get out of sync with the application, causing it to either become out-dated or as in our previous case risking it coming down with a bad case of YAGNI.

This change also makes it more apparent that the Wiki is for the developers to maintain, ensuring that the ubiquitous language alive in the development team.

Tags: ,

Every time you downcast a kitten breaks a rib

February 19th, 2009 No comments

This is a snippet taken from our design guidelines document.

Most code doesn’t require casting. Avoid casting, especially if you cast to a more specialized class.

Tags: