Posts Tagged RemoteX
The PropertyChangedCallback on a DependecyProperty is never called on NotifyPropertyChangedEvents for certain properties
Posted by Morten in Uncategorized on July 11, 2011
The user interface of RemoteX Applications has a concept of linking objects to each other. This is described in the UI very similar to a link on a webpage. Some of these links can be changed using an autocomplete like scenario.![]()
These links have had us create user controls to handle them. We’re doing some changes to the interaction to these controls as such we had an issue where the controls never updated their content. This was due to the PropertyChangedCallback was never fired on their dependency property.
The backing field for a link looks something similar to this:
public IRef Contract { get { return _contract; } set { _contract.Href = value.Href UpdateContract(); OnPropertyChanged( Properties.Contract ); } }
As you can see the actual object is never changed, we just change the link on the object.
Apparently WPF optimizes its bindings using ReferenceEquals. The PropertyChangedCallback event is never fired if the bound property is still bound to the same object. so the user control never gets the changed object. This can cause problems for other scenarios as well.
The solution for us was to always return a new copy of the link. That way the user control’s PropertyChangedCallback is always fired. Example below:
public IRef Contract { get { //We need to return a new instance of an IRef here since WPF does ReferenceEquals to check for changes return new Ref<IContract>(_contract); } set { _contract.Href = value.Href UpdateContract(); OnPropertyChanged( Properties.Contract ); } }
Making it easy to work with performance
Posted by Morten in Uncategorized on June 13, 2011
We’ve been working with performance in our REST API a while back. We’ve been considering different options for measuring our API in production, but we had an idea to use the design of the API itself to handle the measuring.
Part of the inspiration came from the Changlelog’s coverage of CodeConf and Coda Hale’s Metrics talk. (Slides).
Using our pipeline approach to constructing a REST-API with conventions, we added a pipe that starts a stopwatch and stops it when the request is finished. Then we can log statistics for each individual part of the API, closely coupled with the code that is executed for that particular request. I pitched the idea to my good colleague Johan Andersson, and he did a fantastic job at implementing the solution!
The output looks something like this:
This allows us to see how our API is performing in production, and where we need to focus our optimization efforts to make the API go faster. We can even add some flags to the request to get more detailed output.
Deploying updates for RemoteX Applications using Hudson
Posted by Morten in Uncategorized on January 24, 2011
We’ve been doing some work with Hudson over the past weeks. Mainly We’ve managed to create the last part of RemoteX delivery pipeline on a hudson build-server.
Since the first release of RemoteX Applications we’ve been releasing each release as a “package” containing everything needed to setup applications for a customer. I briefly mentioned this in my previous post.
In the beginning the final delivery was done by our operations dude. However as we’ve deployed more and more versions it quickly became apparent that he could not keep up without assistance. So we start to create more and more elaborate scripts to set up each customer. Today everything is done through the execution of a single script that sets everything up using other modular scripts.
What we’ve done over these last week is to set up a Hudson server to do the final delivery automatically. I calls this central script with its arguments and deploys our upgrades to all our customers.
We set up some builds that will connect to RemoteX Applications using PowerShell and create deployment jobs to upgrade different installations based on different criteria.
Next we started to categories the different installations for different targets. We have three different targets that does automatic deployment:
- A testing target that sets up the release for manual testing
- A “early adopters” target who get the version that passes the manual testing
- The bulk, which consists of all customers who will be upgraded automatically
As we work through a release we will have many deploys to the testing installations, a few more to the early adopters, and once we have a final release it will be rolled out on the bulk of our customers. All this is done with the click of a button.
With a single Hudson service deploying a new version of RemoteX Applications to the bulk should take roughly 3 hours. Each customer should have only a few seconds of service interruption during the process, but might be required to upgrade their clients. This process can be parallelized by adding Hudson slaves to run several deployments at once. Effectively cutting the deployment time by O(1/N) where N is the number of build executors available.
We can make it faster but that requires some infrastructure work, and this will work sufficiently for the coming months.
Doing Continuous Delivery with Hudson and Git
Posted by Morten in Uncategorized on January 17, 2011
Right now I’m reading the book Continuous Delivery. Excellent reading and even though I’ve only read around 46% percent of it, it’s quite thought provoking.
I’ve written earlier on how we at RemoteX have set up our build environment with a concept I then called The Independent Trunk (terrible name, right?). Continuous Delivery describes a similar approach of handling your source-control and build system, though in greater detail.
Recently we’ve been considering switching build environments to Hudson, since we’ve been quite happy with the results Hudson has been given us when we’ve tried it out for different scenarios.
Hudson however does not have its configuration checked in under source-control. Most people seem to be happy with just doing backups of the configuration with the motivation that it seldom changes. However I firmly believe that the configuration of the build-environment follows the same lifecycle as the software, and it seems the authors of the Continuous Delivery book do as well.
So I set out to do some experiments with Hudson to see if I could model an arbitrary delivery pipeline using Hudson and yet have it checked in.
I wanted the following to hold:
- All configuration is checked in
- Developers run the same builds locally as they do on the build-server
- It’s easy to run the complete pipeline on a developer machine
- It should be fast to get up and going
I started thinking about this in the middle of December and about a week ago I started to set things up. The idea I got was to simply check the entire Hudson system into source-control. I put the war-file that is executed in source-control along with Hudson’s home directory. I spent quite some time tinkering with the .gitignore file to get things set up how I wanted it.
The result is available on git-hub at: https://github.com/morkeleb/continuous-delivery-with-hudson feedback is highly appreciated. Also if anyone wants to use it as a boilerplate for new projects go right ahead.
The only difference in the pipeline I’ve created and the one that the book describes is that I’ve added a packaging step that is supposed to take all artifacts from the CI builds and add them up into a deliverable package. There are two stub CI builds creating files and shows the intended structure.
To make it easy to start Hudson on a developer machine I created a .bat file that starts the CI engine locally. All a developer has to do is get the latest version and run the script to have Hudson setup locally.
What I’ve found thus far is that this approach seems to work quite well. Sure it takes some disc space but that is affordable.
Upgrading Hudson can be done easily and have the changes replicated across developer machines as the fetch the latest version from the origin.
Now there are some things that differ when you run Hudson locally and on the main build-server. The main build-server will have all its artifacts and fingerprints backed up, as well as following the source-control version.
There are however some things that I’ve yet to try out. Changes to the configuration needs to be updated on the build machine, this can be accomplished by logging on to the machine and the do a git pull. However since this is tiresome a web hook approach could be interesting for automatically restarting Hudson and getting the latest version of the configuration from source-control. This however is yet to be experimented with.
Feel free to check it out on git-hub and any feedback is great. Especially considering that this might seem a bit crazy.
Refactoring of logging rules
Posted by Morten in Uncategorized on November 8, 2010
I’ve been working on a logging system for RemoteX Applications, the system is going Live any day now. (In fact since this post will be scheduled it might already have).
So I started out doing the code as straight forward as possible, but it quickly started to nest if-statements as the different logging conditions grew. I got something similar to this:
public void HandleNotification(NotificationEventArgs notificationEventArgs) { if(notificationEventArgs.Type == typeof(Case) ) { var c = getCase(notificationEventArgs.Reference) AddForLogging("Case status changed to "+c.Status); } if(notificationEventArgs.Type == typeof(Resource)) { AddForLogging("Resource changed"); } if(notificationEventArgs.Type == typeof(Cost)) { var p = getProduct(notificationEventArgs.Reference); AddForLogging("Cost updated for product" + p ); } }
Now I wanted the if statements to execute after one anther since they would eventually build up to a single entry in the Log. However as the information that should be logged increased. This method became increasingly hard to maintain.
Also this point is a natural extension point for adding new rules for what should be logged and when. Thus I wanted it to be easy and simple to add things for logging.
By extracting an interface using a variation of the Specification pattern, I get the ILoggingRule.
Each if-statements body is moved to and implementation of the Execute method, and the Boolean expression is expressed in the AppliesFor methods execution. With this and a simple foreach-loop. The above code becomes the following:
public void HandleNotification(NotificationEventArgs notificationEventArgs) { foreach( var loggingRule in _rules.Where( loggingRule => loggingRule.AppliesTo( notificationEventArgs ) ) ) { loggingRule.Execute( notificationEventArgs, AddForLogging); } }
Now this is simple enough to extend using Castle, and it moves the complexity away from the generic log handling. Thus it allows for more complex logging-rules as well.
Here is an example of a logging rule:
public class CaseStatusLoggingRule : ILoggingRule { const string CaseStatusChangeTemplate = "$L{{Log.Case}} $L{{Log.StatusChange}} $L{{Log.Case.Status.{0}}} $L{{Log.To}} $L{{Log.Case.Status.{1}}}."; /// <summary> /// returns true if this rule applies to the notification /// </summary><param name="notificationEventArgs"></param><returns></returns> public virtual bool AppliesTo( NotificationEventArgs notificationEventArgs ) { return notificationEventArgs.EntityType == typeof( Case ) && notificationEventArgs.NotificationType == NotificationType.PropertyChanged; } /// <summary> /// Executes the logging rule /// </summary><param name="notificationEventArgs"></param><param name="logCallback">A callback that will execute the actual logging, first parameter is href, second is message</param> public virtual void Execute( NotificationEventArgs notificationEventArgs, Action<string, string> logCallback ) { var caseHref = "cases/" + notificationEventArgs.EntityRestId; string message = String.Format( CultureInfo.InvariantCulture, CaseStatusChangeTemplate, notificationEventArgs.Changes[0].OldValue, notificationEventArgs.Changes[0].NewValue); logCallback(caseHref, message); } }
As you can see the if this would have been placed in the first code the if-statements and their bodies would wreak havoc on the readability of the code. Now instead its moved to it’s own class, and the logging can naturally be extended without adding complexity other than what’s required for the specific logging criteria.
What’s more funny is that about a month ago I did something similar to how Shuffle handles its serialization of Track entities. It uses the XmlPullParser since I’ve read that it’s the highest performing parser on the Android Platform (please let me know if I’m wrong).
Now parsing a single entity from the Tracks-service follows this procedure:
public ParseResult<E> parseSingle(XmlPullParser parser) { EntityBuilder<E> builder = createBuilder(); E entity = null; boolean success = true; try { int eventType = parser.getEventType(); while (eventType != XmlPullParser.END_DOCUMENT && entity == null) { String name = parser.getName(); switch (eventType) { case XmlPullParser.START_DOCUMENT: break; case XmlPullParser.START_TAG: Applier applier = appliers.get(name); if(applier != null) { success &= applier.apply(parser.nextText()); } break; case XmlPullParser.END_TAG: if (name.equalsIgnoreCase(mEntityName)) { entity = builder.build(); } break; } eventType = parser.next(); } } catch (IOException e) { Log.d("Exception", "IO EXception", e); return new ParseResult<E>(); } catch (XmlPullParserException e) { Log.d("Exception", "pullparser exception", e); return new ParseResult<E>(); } return new ParseResult<E>(entity, success); }
As you can see there is nothing entity specific about this code, it’s a template method. The parsing is instead controlled through the dictionary containing something I badly called Appliers. An Applier takes a xml tag and applies it to the Shuffle entity builder.
All parsing is done using this method and the implementing template method classes instead create a list of appliers that can react to an expected xml tag.
An applier looks something like this:
appliers.put("name", new Applier(){ @Override public boolean apply(String value) { specificBuilder.setName(value); return true; } });
I’m using Java’s direct implementation of an interface to allow create the different appliers. Thus the parser just specifies which tag-names it can handle and how to handle them. The parsing is naturally extensible and quite easy to test as well.
I’d appreciate feedback, I’m quite certain there’s a pattern lurking here and I don’t know if the Specification pattern is the correct one.
Working at the right level of abstraction
Posted by Morten in Uncategorized on May 25, 2010
At RemoteX we have been a bit unhappy with the code produced to create our REST-service using WCF.
The problem is that you have these contracts that you specify. These contracts then build up a web-service end-point using the WebGet and similar attributes, to create a REST-style endpoint. The contracts however often look like a SOAP contract, or an interface wrapping behavior. A typical contract often includes the following operations for a specific type of document:
- Get Single (web get on href entity-type/id)
- Get List (web get on href: entity-type)
- Get List with modified since (entity-type with query-string)
- …
- and so on
The trouble is the URI’s. A single contract often specifies several endpoint’s and several behaviors. There is no Single-responsibility principal at work, or rather the single responsibility might be that a contract handles a specific entity. End result, a lot of code for simple things and a REST-service that is unnecessary hard to extend.
This setup also makes it tricky to see the structure of the REST-service, the URI’s are spread out in the code contracts, forcing you to look at each method to figure out the different URIs.
After working with Grails and Django, and finally when I read the documentation for the Kayak web server. I realized what we were doing wrong. A reference in the Kayak documentation was the wakeup call:
The server component is an intuitive, unobtrusive, no-gimmicks implementation of HTTP, and the framework simply maps HTTP transactions to C# method invocations with minimal syntax and configuration.
We’re not working on entities as an abstraction, we’re building a REST-based service. Our level of abstraction needs to be HTTP.
So I discussed this with a colleague Johan, the day after. After a while Johan set to work on a piece of the REST-server that needed rewriting and after a week of TDDing. He came back with this wonderful piece of infrastructure, that worked along side the “old style” infrastructure.
The idea is that we have one contract. This contract has 4 methods:
- Get
- Put
- Post
- Delete
Essentially all the methods we want to implement. We then have a route configuration, similar to that of Grails or Django. That routes requests to Controllers that will take care of them.
Johan set things up so there is a single server, that looks at the route-configuration when a request arrives and executes a command on the controller object. The controller object doesn’t have to add methods for the operations doesn’t support. In fact, the controller object is a plain POCO with no inheritance or interface on it. As long as it has a method that matches the operation it will be called.
Serialization is handled in the framework and by following conventions when writing the controller you can take different input automatically processed by the infrastructure.
Lets see an example:
public class LogController
{
readonly DataContextFactory<RMXAppsDataContext> _contextFactory;
public LogController(DataContextFactory<RMXAppsDataContext> contextFactory) {
_contextFactory = contextFactory;
}
public void Post(LogEntry entry, IIdentity user)
{
PostMessage( user, entry.Message );
}
public void PostMessage( IIdentity user, string href, string entry )
{ using (var context = _contextFactory.CreateContext())
{ var dbUser = context.GetTable<User>().Where(u => u.UserName == user.Name).FirstOrDefault();
var dbEntry = new Applications.Data.LogEntry {
Message = entry, Date = DateTime.UtcNow,
User1 = dbUser };
context.GetTable<Applications.Data.LogEntry>().InsertOnSubmit(dbEntry); context.SubmitChanges();
} } [return: Xml]
public Log Get() {
Collection<LogEntry> entries;
using (var context = _contextFactory.CreateContext())
{ var e = (from l in context.GetTable<Applications.Data.LogEntry>()
orderby l.Date descending
select Mapper.Map<Applications.Data.LogEntry, LogEntry>( l )).Take( 50 );
entries = new Collection<LogEntry>(new List<LogEntry>(e));
} return new Log { Href = "log/", LogEntries = entries };
}}
It’s not the prettiest of examples (and I apologize if the code doesn’t look that great).
What your looking at is a simple log controller. It handles the log/ endpoint. The endpoint has a Get and a Post operation. Post is used to add LogEntries, and Get gives the latest 50 entries.
You will notice that the Get operation is marked with [return: Xml] this means that the operation will serialize the return value using XML.
The Post operation takes two inputs. LogEntry and a IIdentity. Both these parameters are set by conventions.
IIdentity is the user, as authenticated by the login-module, the infrastructue simply gives the user to any IIdentity parameter in a web-method.
The LogEntry is the deserialized body of the request. The infrastructure looks at any parameter that doesn’t fit a convention, checks if its serializeable and if the body of the request can be deserialized into such an object.
Now a route configuration is another object, that specifies which urls which controllers are registered too. These structures are set up using Castle, and loaded from any assembly in the web directory. Allowing us to extend the REST-service by dropping in assemblies that implement our structure.
Since this was added the development of the REST-service has really taken off. We add roughly one-two endpoints per week, designed to solve specific pains and give views of the domain specific to the task at hand.
By working at the HTTP level of abstraction instead of the contract abstraction, we have reduced the cost for extending the rest-service tremendously. Even better is that the controllers are easy to test, the code displays its intent clearly, and you can see what each specific Uri in the REST-service does.
Special thanks to Johan, great work!
Measuring defects
Posted by Morten in Uncategorized on May 3, 2010
I was working on performance tuning for the mobile client of RemoteX Applications when I found a tool called EQATEC Analytics. Six month later I added it to our clients for the first time.
EQATEC Analytics is quite interesting, for RemoteX it solves an interesting issue. Error reports are hard to collect from Compact Framework devices, especially since all error messages state that “we’re missing the translation pack to translate these errors, thus you get this crappy errors message instead”.
EQATEC Analytics captures all unhandled exceptions and report them back to a server application for analysis. This means that all crashes are reported so they can be analyzed by us, without involving the user in the information collection process. (no personal information is sent to the server)
The benefits of this enormous. We can see the quality trends of our applications (at least when it comes to crashes), and fix the errors that occur the most.
Traditionally the errors are fixed based on the information received by customers, manual error reports and what not. Relying on this type of information means that you fix the errors that most people report, or the errors you can reproduce. With EQATEC Analytics you, you can get an exact stack trace indicating the problem area, and depending on how you configure it to work you can also request information regarding what was happening during the crash.,
It also provides an excellent way to see if our efforts of improving the quality of the product is helping or not.
Now it also has features to collect information about what kind of system our applications are run on, and also collect statistics of which features are used in the applications.
We’ve used EQATEC Analytics in production since January 2010 and so far we are very happy with the results.
An interesting note here is that the next version of Shuffle will use a similar tool for Android, called Flurry.
Inheritance of features
Posted by Morten in Uncategorized on January 14, 2010
Given my recent combining of Tracks and Shuffle. An interesting thing happened. Something that happens when you combine two products so that they inherit features from each other.
On Shuffle’s issue list there are a few items currently regarding repeating tasks, and the ability to hide a task for a period of time. Tracks has these features, since Shuffle can synchronize with Tracks, Shuffle now also have these features. That is if you choose to synchronize with Tracks.
I’ve done a few integration projects thus far. And the ones that are truly successful are when they take the aspects and features of both applications into consideration, to create a win-win scenario between the two applications.
Here is another recent example. At RemoteX, we recently integrated a system that handles the ordering of materials, goods, equipment etc. In essence this integration feature means that you can go to a retailer, order goods, specify the order they should tag the invoice with. End result, RemoteX automatically adds the goods to the application and if possible attaches a copy of the invoice to the case. Ready to be billed to the customer. Since the other system has integrated with almost all resellers of service equipment in Sweden, the daring customers running this set-up can, at least in theory, eliminates the need for keeping stocks of equipment or goods in house.
If you understand where these synergy effects can show up you can also align the integration so that both applications maximizes the gain of integrating with the other system. This is important to think of when designing an integration, how do we achieve the maximum win-win?
Testing level + 1
Posted by Morten in Uncategorized on November 30, 2009
I learned a new way to test objects the other day.
We had a consultant over to talk about a TDD course he will be holding for RemoteX internally. During the discussion the suggestion to test an object by mocking the object itself came up.
What you do is, you use Rhino mocks (or similar) to create a partial mock of the object under test. What you override is the methods that will result in external communication.
This requires you to put all communication code in a single method, which you then override using the partial mock. The test then becomes, is the method for external communication called.
This reduces the amount of mocking code needed for more coupled objects, and also reduces the “IoC container for testability” need. Allowing the IoC focus to be focused on interchangeable parts.
Functional TagEditor
Posted by Morten in Uncategorized on November 29, 2009
I notice now that I cannot use the tag editor I’ve created from RemoteX Applications on my personal laptop. In requires a larger screen than what I have.
Anyway, this is a working version of the design I showed to mock-up earlier
Each row has a sub-class handling each option in the drop down (Strategy pattern). Selecting an option changes the available controls for editing the tag.