development, node, coffeescript, performance

So we have a new Node app that we're developing. We have a fairly involved CI process which involves Amazon Beanstalk deployment and one step is load testing against our staging environment.

Mid way through development the load test started failing, it's not been the most stable of build steps so it got ignored for a few days (bad I know). But I decided it was now legit failing so decided to investigate.

Turns out it was a pretty simple issue. We did a bit of profiling which shows that hbs.js was taking the largest amount of time to render the templates. Each request to origin was requiring a compile of the templates.

I wanted to add a simple caching mechanism to this that would cache the response for X amount of time.

This is the module we created.

cache = []  
  expiry = (duration) ->
    Date.now() + (duration * 1000)

  module.exports = (action, duration) ->
    (request, response) ->
      key = request.url.split('?')[0]
      render = response.render

      if cache[key] and cache[key].expires > Date.now()
        return response.send cache[key].html

      response.render = (view, viewData) ->
        render.call response, view, viewData, (err, html) ->
          cache[key] =
            html: html
            expires: expiry duration

          return response.send html

      action request, response

We cache the HTML result of the redering in a simple array and we store an expiry time. If there's a cache hit we simply return the HTML otherwise we process and re-cache.

module.exports =  
  register: (app) ->
    app.get '/', cache home.action, 30

This is how we register this cache. We put it in the route registration so that we can choose which action to cache. We also provide a duration in seconds.

The resulting load test changes

Before it started to fail; the results are more spread out and 59% of the results took 21ms to load which was the quickest.

Before

After these changes, 49% of the requests took 1ms and 48% took 10ms. A huge improvement and far more consistant.

After

development, tools

All developers have their favourite tools. I'm going to list the current tools that I use. My current computer is a MacBook Pro and has been since I moved to Australia last year.

I like to keep the dev tools I use to a reaonably low number (I'm awful at remembering keyboard shortcuts). I still consider myself mostly a .NET developer but I do a lot of purely front end work (and a bit of Node) which I do in OSX.

OSX

  • IntelliJ IDEA - One of the best IDE's I've used, great for Node, PHP and Ruby development.
  • Sublime Text 2 - Great all round text editor. I love the keyboard shortcuts in this thing. Worth the licence fee considering how much time I spend in this editor.
  • Source Tree - I generally prefer to use version control from the command line but this tools is great for reviewing commits and history.
  • Photoshop Creative Cloud - I'm pretty poor with this, I have next to no graphical abilities. Very useful to have around for freelance work however.
  • Homebrew - Basically means I don't need to install Linux.

Windows

  • Visual Studio 2012/2013 - The only realistic choice for .NET development. It's generally pretty good if a little bit heavy.
  • ReSharper - Makes Visual Studio actually useable for me. I like the consistancy between this and IntelliJ. Means I don't have to remember many more shortcuts.
  • Chocolatey - Makes installing things in Windows a little bit nicer.
  • Fiddler - Amazing for debugging and the automatic responses make testing CSS and JS much easier.

Web

  • AppHarbor - Great .NET hosting for simple applications. Really good for rapid deployment. My .NET blog was running here.
  • Digital Ocean - Great for a really simple VPS. I have git and this web site running from here.
  • Cloudimage.io - I use this for resizing and hosting my blog images. It redirect all my ghost image assets through this.
nginx, development

Ghost currently has no image resizing functionality. And I'm far too lazy to resize all of my images before I upload them. I was thinking about writing a ghost plugin but that doesn't seem to be possible yet. Then I thought about using an image resizing service but then I would have had to hack my template to change my image urls which I also didn't want to do.

As I'm running Ghost on Digital Ocean I have full control over the whole stack. So I decided just to use Nginx to redirect all my image requsts through http://cloudimage.io.

It's been a whle since I configured any web server and i've never done any Nginx configuration so it took a bit of trial and error.

This is eventually the solution I came up with;

server {  
  listen 80;
  server_name garethrhugh.es;
  location ~ /content/images([^.]+).(jpg|png|PNG|JPG) {
      return 301 http://safpzj.cloudimage.io/s/resize/1000/http://asset.garethrhugh.es$request_uri;
  }
}

server {  
    listen 80;
    server_name asset.garethrhugh.es;
    root /var/www/ghost;
    deny all;

    location /content {
            allow all;
    }
}

I originally redirected to the same domain which I think caused a bit of an infinite loop (oops). After adding the asset domain to allow cloudimage to load the original file everything was good!

So at the start of 2013 I moved to Australia, I had no real plans other than maybe spend a few months there and see what kind of work I can find.

I ended up getting work fairly quickly and eventually getting sponsored at Ninemsn. I'm just entering my second year in Australia and decided I need to move on from Ninemsn. I will be leaving in the next few days to go somewhere where I will hopefully be allowed to do more development, specificalky more C# development.

Coming up early to mid 2014 I will be in New Zealand and then flying my Mum out to Sydney in May, following this with a relaxing holiday to Hawaii in June to recover.

Here's a few highlights from the first year in Australia.

Opera House

Harbour Bridge

Melbourne

At Suzuka

Skydive Weekend

Miyuki

Slide

At the beach - pre skydive

Updates, Blog, Ghost

I've had my own fully bespoke blog since the early 2000s in some shape or form. I decided it was more trouble than it's worth now and am in the process of switching to Ghost. It gives me markdown to use instead of Windows Live Writer. I'm hosting it on Digital Ocean instead of AppHarbor. I use OSX a lot more than Windows in the last couple of years so this switch makes sense.

I'm still in the process of switching over. I have issues with some of the blog posts I've imported and I want to maintain old urls so will need to look at adding rewrite rules to redirect.

Hopefully this will be live in the next couple of days.

It’s been a while since I last updated so I’m going to post about something that happened a few months ago now. I’ll hopefully get into the habit of more regular blog posts but I’ve been saying that for a while. Oh well! There will be some development related updates coming soon.

Because this happened a few months ago this is probably just going to be mostly pictures. In November I took a trip to Malaysia with AirAsia. It was a pretty hectic trip, I nearly doubled the amount of flights I’d done total in one week.

First up we headed to Kota Kinabalu via Kuala Lumpur. (I also went via Melbourne so I could return to Melbourne for the colour run.) This part of the trip was probably my favourite. We got to see Orangutans, eat plenty of the local food and visit the markets, as well as plenty of sight seeing.

Untitled

Untitled

Untitled

Untitled

Next up we spend a night in Brunei, just for the hell of it. We obviously didn’t get to see much but we went on an awesome boat ride around one of the water villages and into the wild to see monkeys. 

Untitled

Untitled

Untitled

Next up we went back to Kuala Lumpur, after deciding pretty quickly we didn’t want to spend any more time than necessary in our hostel we checked into a much nicer hotel then we spent some time relaxing, checking out the city and the indoor roller coasters. We also checked out the Petronas towers and science “museum” and then the DC shop obviously.

Untitled

Untitled

Untitled