Building Blocks

I just set up a build server for my Grails project. I chose Hudson since it was the one mentioned to work well, and it did. I’m quite happy with Hudson thus far. It hasn’t crashed (actually it crashed my browser once, more on that later), and been very easy to work with.

First of I installed it on my Ubuntu machine in the closet. It was straight forward installed using apt-get as per instruction. This was a clean install and the first use of that Ubuntu installation for any kind of development work.

With Hudson installed I started going through its configuration. I knew there was a Grails plug-in so I figured I had to install a lot more software to get things going.

I found the plug-in page, marked the Grails plug-in and installed it.

I created a job, configured it to fetch the code from my subversion repository. Already I could run the job, and watch the output from my normal development machine.

I love the console out page, compared to watching the build-logs and build progress in MSBuild this page gives good feedback, doesn’t crash that much, and works as both progress and log all wrapped up in one.

With the job running I pinpointed the missing parts in the job. Running the grails targets. So it was time to configure the Grails Hudson plug-in. It required the Grails installation path, so time to set up the libraries. Download and install Grails, update the plug-in with the path.

When running a Grails job the first time you need to run the “grails compile –non-interactive” command. This will install all other plug-ins as it is the Grails way. This will make plug-in specific commands available.

I ran it first without the –non-interactive flag, which caused the app-engine plug-in installation to ask questions. Hudson didn’t mind, but my browser in which I was looking at the console out didn’t like it. It kept updating the page with “do you want to use JPA or JDO” questions, until it finally crashed.

In my case I wanted my build to upload the compiled application to Google AppEngine. I added the APPENGINE_HOME environment variable to Hudson’s main configuration. With the APPENGINE_HOME variable set to my newly downloaded SDK.

This is what my grails build target looks like:

To get the deploy command to work I needed to add the login details for app-engine. After some research I found that the details are configurable. All you have to do is add the following to you Config.groovy file in the grails project.

google.appengine.email = "email"
google.appengine.password ="password"

But with the correct email and password of course. In my case I used the user I’ve created for sending emails as the user that will deploy using the build.

I now have a working build that deploys to AppEngine. What I’m currently missing are running tests, as I’m still searching for a functional testing framework that will run as a Grails plugin with the AppEngine plugin. None of the two (easyb, functional-testing) Grails plugins I’ve tried so far have worked.

I just added a Cron job to “ping” my application every 1 minute. After a short discussion with Paul Cusch, whom commented on one of my previous posts regarding the load request issues Grails on AppEngine is seeing. I decided to give it a try. This solution is entirely based on Paul’s idea I’m just writing it up.

Grails AppEngine plugin is missing a feature to use cron out of the box. I hope to submit a patch for it. This is how it works. You need to have a cron.xml file next to your appengine-web.xml file. In the cron file you describe which url and how often a GET operation should be done on the specified url.

Details can be found here.

I simply added a request to a specified url, which I added to my URL mapping:

"/today"(controller: "user", action = "ping")

Then added a cron file to ping it every minute:

<?xml version="1.0" encoding="UTF-8"?>
<cronentries>
    <cron>
        <url>/today</url>
        <description>Keep the app alive by ping every 1 minutes</description>
        <schedule>every 1 minutes</schedule>
    </cron>
</cronentries>

The cron.xml file I keep in my grails-app/conf directory, following the conventions used by the AppEngine plugin, for similar files.

The user controller unfortunately got the honor for hosting the ping request.

def ping = {
  render "ping ${new Date().toString()}"
}

And there it is. Next time that application is uploaded to the AppEngine, the ping method on the user controller will be executed every 1 minute. Helping to keep the application loaded.

This method comes with a few drawbacks however. The application will still hit loading requests every now and then, and those requests will take from your quota. This setup can easily eat up all your free quota.

Also, if everyone sets up this cron job. Then a lot of quota is wasted in general, which might affect the service for other applications. This workaround tries to fix a problem that should be fixed in the inner workings of AppEngine it self (not excluding changes to Grails initialization).

There is however a query that is ongoing. Google is considering the possibility to allow people to pay for keeping their application in memory. If you would consider such an option then star the this issue. I myself starred it, I think it’s quite acceptable.

I turned the cron loading task off in my application, instead seeing it as an experiment with Cron on Google AppEngine. The payoff for this solution wasn’t that great and for me it comes with the expense of some of my code quality.

While it is too early to say anything about actual performance gains. After upgrading to the 1.3.0 AppEngine SDK, I’m still suffering from loading requests timing out.

I did some experiments with lazy-initialization, causing my grails deployment to crash in many wonderful ways.

I’m going to let pingdom measure the application for a while to see if there are any CPU savings.

In the 1.2.8 version of the AppEngine SDK there is an option to precompile your solution as you deploy it to the AppEngine. Apparently precompilation can save up to 30% runtime on loading requests.

I added the option to turn on the precompilation-enabled setting in appengine-web.xml to the AppEngine plug-in this morning. Hopefully I can submit a patch with change, I’ll get back to that in a bit.

As I’m writing this I’m deploying to AppEngine for the first time. What happens is that during the deployment to Google AppEngine, the appcfg.cmd compiles the solution locally.

Showing the following lines in the input:

37% Cloned 600 files.
40% Uploading 4 files.
52% Uploaded 1 files.
61% Uploaded 2 files.
68% Uploaded 3 files.
73% Uploaded 4 files.
77% Precompiling.
80% Precompiling. 174 files left.
82% Precompiling. 94 files left.
84% Precompiling. 22 files left.
90% Deploying new version.

Now it’s deployed, I’m going to let it run for a while while I fix something else.

I have Pingdom set up for checking the AppEngine project every minute. That way I can see the changes in response time, and from the dashboard I can see if the CPU use goes down.

As far as I can tell, the precompilation doesn’t help with the timeout issues when starting a grails application hosted on AppEngine, it might help, but it doesn’t affect the loading time noticeably. However I for one am going to have my application precompiled.

This SDK release makes Morten (that would be me) a happy camper. Now I have something to do when I get home from work!

Release information

http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-released-including.html

Why am I looking forward to this? It’s the first time I see a mention of performance updates for dynamic languages in AppEngine. I’m upgrading later to night to see what my actual performance boost is.

The blob service is also a welcomed scenario. We’re using something similar in our REST-API in RemoteX Applications.

If you use the app-engine plug-in deploy command using AppEngine SDK version 1.2.8 it will fail with the following error:

[java] Bad argument: email requires an argument, for example, "email=FOO"
[java] usage: AppCfg [options] <action> <app-dir> [<output-file>]
[java] Action must be one of:
[java]   help: Print help for a specific action.
[java]   request_logs: Write request logs in Apache common log format.
…
[java]                         request logs are returned.
[java]   -n NUM_RUNS, --num_runs=NUM_RUNS
[java]                         Number of scheduled execution times to compute
[java] Java Result: 1

Google changed so deployment is targeting a directory instead of a war file.

What you need to do to deploy is instead to use the AppEngine built in command to deploy

%APPENGINE_HOME%/bin/appcfg.cmd update c:\users\morten\.grails\1.2.0.RC1\projects\<projectname>\stage

After this all I had to do was to update my deployment scripts, and the stage is set for some serious Grailsing.

I should point out that all previous workarounds for working on Windows still need to be in place.

Google just announced their attack plan for handling the long startup cost for, among others, Grails applications.

Apparently what I can do as a developer is to enable pre-compilation of my application. I’ll experiment with this at the end of the week I hope.

Read the full post on the AppEngine blog

As far as I can see there is no mention of startup performance. Doesn’t have to mean they haven’t worked on it. Ill check it out to night to see how the application behaves.

I got this on twitter last night. Google is doing a prerelease of the next AppEngine SDK.

Details can be found here:

http://groups.google.com/group/google-appengine-java/browse_thread/thread/fe334c9e461026fa

Only thing that worries me is that there is no mention of application startup times.

Regarding the logging issue with Grails and Google AppEngine, I’ve just installed the latest build of Grails1.2 and deployed to AppEngine, and it works.

No change to the loading time of grails however, I’m still getting timeouts on my initial requests.