120 individual PNGs for one jigsaw

As mentioned before, Jigosaurus is a multiplayer jigsaw puzzle that is played through a web browser. When Aleksei, Yasser and I joined as part of the Caplin Graduate Scheme, we were given the task of getting it running and improving it, and today we’ll be looking at the client facing end.

Following the route of the jigsaw from server to client, we quickly found a limitation. Each jigsaw that we wanted to be played had to be generated from an image with another tool before it was ready to host on the server side. This meant cutting the full image into around 200 individual pieces and saving them as PNG files ready to transfer over the internet. Then with many <img> tags placed in the right position, the clients browser could give Jigosaurus piece elements their correct look.

Variety is the spice of life

Clogging up the network with hundreds of requests

This “chop up once, serve many times” method may be good in a situation where everyone plays the same kind of jigsaw, but as soon as someone wants to try the same puzzle that has more or fewer pieces not only would we have to run the same image through the chopping tool again, but we would waste space on additional PNG files that represent pieces of the same image.

Even if we were ok having no variety in our puzzles (boo, hiss), we still have the problem of actually sending multiple images over the network. These piece images will rarely be sent over the network in complete parallel and so have to queue behind each other. Combined with the overhead of sending each file, this means more time waiting and less time puzzle solving.

CSS is more than a pretty face

Sprites for the Gmail Inbox page are held in one image file

The problem seems to be a server side limitation of pre-processing things for an individual, so we could try doing things with a bit of extra code on the client side. We could use Javascript as a work horse to cut up each user’s jigsaw, but this would be slow and there is a cleaner way.

Browsers are optimised to handle styling through CSS well, so we can take advantage of this and get the browser to cut up our jigsaw natively. This technique is known as CSS Sprites.

This technique is normally used for pages that contain multiple images that are usually used all at once. Icons, logos and other pretty pieces of image could be served up individually like our jigsaw pieces, but are instead mashed together and sent all in one go. Each relevant element references this large mashed together image, but by using the background offset CSS property, the element only shows a window of this one large image. This means only dealing with the resource overhead of one file and taking advantage of a browsers optimisations for CSS styling.

This technique is perfect for Jigosaurus, so we ditch the <img> tags for each jigsaw element and just add an individual background-image style that references a small window of the full image that would normally have to be cut up.

Before: 72ms page load - the majority being images

After: 8ms page load - a dramatic improvement

Not only do we see a dramatic drop in loading times, but we also pave the way for jigsaws of different shapes and even jigsaws that aren’t just static images…

Using Ant For Your Build Scripts

First, I’d like to describe where, for me, Ant really goes wrong, over and above the points that Jess makes. With Ant, out of the box, you end up with a build system that is inefficient, and is only reliable provided you correctly define the dependencies of all tasks. Given that requires you have a knowledge of the entire build system, and requires lots of testing, that in itself isn’t a trivial task.

Once you do have all your dependencies defined correctly, and your build has become reliable (but still very slow) the next task you are likely to undertake is to make it perform better. You will likely do this using the <uptodate/> task to only perform targets if the source files have been updated since the output files were last generated, and/or the <fileset/> and <modified/> tasks to limit the files your target processes to only those files that have been changed since the last time you ran the target.

This works provided you don’t have too many source files (they will all needed to be stat-ted or MD5ed before Ant can determine whether it needs to perform the task or not), and creates two new problems:

1. Your build system becomes much harder to comprehend and much more time consuming to test.
2. Your build system starts breaking every time you modify it because Ant doesn’t check-sum the targets, and ignores your build restrictions if the targets have changed.

Every time your developers sync and get a change to scripts, they also get a new build problem to debug … oh Joy! You can also fix this by treating all Ant build files as inputs to each conditional target, but there are even subtleties here, like the fact that developers may be using the -D flag to pass in properties that change whether things need rebuilding, or that you need to keep an eye out for each new .properties and .xml file that is build related and ensure it’s a matched as being a source file, or that your <uptodate/> tasks will keep breaking as developers change the inputs and outputs to tasks without also updating the from and to patterns, which doesn’t get immediately noticed, but just ends up creating another subtle build bug to debug! You can work around all of these problems, but you’re going to end up with a build system that is unwieldy and unreliable, and will be the constant focus of criticism in your retrospectives.

The Maven And Ivy Approach

Maven and Ivy try to avoid these problems entirely by encouraging you to divide your build up into small parts that can be checked into a repository, but ultimately by denying you first class access to your own source code, and by introducing a versioning problem you never previously even had, they just create a bigger problem than they solve in the first place.

Prebake To The Rescue

All of this is what intrigues me about prebake. At a cursory glance, it seems to solve all of these problems and many more, and it does this while still providing first class access to all source code. It tantalizes by seeming to promise that creating efficient and reliable build systems will become a near trivial operation. An example of this is how explicit dependencies are done away with in preference to what is essentially the parallel of Ant’s from and to properties from it’s <uptodate/> task. The genius here is that it becomes impossible to forget a dependency, and if you don’t define your from and to properties correctly you’ll know about it immediately. This is coupled with built in check-summing of each target, so that the build doesn’t break when you update it, but only the absolute minimum of targets are rebuilt.

Admittedly, I haven’t even downloaded this still alpha software, nor I have I even read all the documentation, and am still unclear of how well, in reality, prebake will handle dependencies which are not intrinsically file related, but everything I have read to date inspires me with confidence. If this isn’t the build system that ends up winning the battle, then I still have a hunch it will end up being considered a seminal work in that direction.

if you don’t have a good name for it, give it a bad name. a really, really bad name so you’ll fix it later.

Good naming is hard

Sometimes when writing a class, or occasionally even a method, I struggle to find a good name for it. After a while I give up trying to find a good name for it and chose something that is just about “OK”, a working title if you like. Unfortunately this tends to create poor generic names, such as “ConnectionManager”, that vaguely make sense, but can obfuscate the true purpose of the class. I’ve lost count of the number of times that I’ve heard developers say something like, “Oh, so that’s what that class does!”

The problem with these “OK” names is that they tend to have a strong inertia to change. Typically I have the intention of going back and renaming it, however the longer I leave it the less likely this is. If the name is adequate, but not great, then why go through the effort of renaming it? Sure, some IDEs, such as Eclipse, do a superb job at refactoring, however a lot of the code we write here at Caplin is in JavaScript and the tool support for this isn’t as good. Regardless of this, even where there is good tool support, there are time pressures, backward compatibility and other considerations that lead to “OK” names sticking.

Overcoming renaming inertia

This brings me back to Kent Beck‘s excellent suggestion. If you provide something with a really bad name the inertia to change it will be non-existent. You will have to change it before it becomes part of your public API or other developers have to use it.

The great thing with this is that it allows you to write your class and see what it responsibilities are before you provide its proper name. This should lead to meaningful names that define exactly what a class does, ensuring no one is surprised when they try to use it.

How to achieve a better User Experience on Portable Devices…

As much as everyone loves big pictures for blog backgrounds or lots of  fancy
widgets or flash animations, these tend to not be appropriate on Portable Devices
(the ones i’m thinking of are;  iPhone, iPad, Android or BlackBerry).

These features which are quite neat on a desktop computer can quickly become
irritating on small screens. Yes, we can zoom in & out but that just slows us
down as we wait for the page to increase in size, and then have to scroll left and right
to read sentences that are too long to fit onto the screen. This means that often
we are just waiting for the page to load. Even on a 3G network this can be boring.

What can be done?

As usual, there are a number of solutions. Here are my favourite two:

1.) Optimise your existing site to be more fluid and adapt to users’ resolution
2.) Create an alternative theme for Portable Devices.

We have opted to go for the latter (2). Optimising your existing site to work better
on mobiles can be frustrating as you feel that you are cutting out functionality just
to be able to appease the lowest common denominator.

Other pitfalls include super-compressing your images to have a better load time on mobile networks,
stripping out any flash bits you may have to keep Steve Jobs happy, (I’m joking
of course, let’s not get into that debate again) removing drop-downs or any
interaction that works on hover. This can be painful and often leaves everyone
unhappy as lots of  “nice things” have been left out.

Having an alternate theme for Mobile Users can be very effective in increasing
the number of eyeballs on your blog. By creating an experience which is relevant
to it’s environment  (for example someone standing in the train just wants to quickly
scan headlines and get the “gist of it” and not spend time waiting for pages to load on
an already flaky connection because of tunnels etc.) you can greatly increase traffic.  An other upshot of this approach is you do not feel the site has been compromised as the “full experience” is still available.

Implementation

A great way to achieve this is a plugin called WpTouch. This plugin can really help
to improve your loading times on mobile networks.

Customising

With WpTouch’s options a bit of  CSS-ing and designing your own assets
you can customise this plugin, bringing it in line with your existing branding and
“desktop theme” and make it work for your needs. It’s very quick to implement
will make a world of difference to users browsing your posts on their phones by
increasing readability, adding intuitive touch control and keeping important
functions such as search, comments & contact forms.

So why don’t you go check out this post on your mobile?
Let us know what you think.

When an end to end acceptance test fails, the failure could be due to any one of a large number of source lines or expectations changes across a large body of code.  Debugging may be quick, but often isn’t.

Small tests run faster too! While a typical small test may take 10 milliseconds to run, a large acceptance test may take twenty minutes.

Just a small back of the envelope calculation tells us that if a developer checks in twice a day and saves an hour each time because potential issues had been found before check-in with fast small tests, and the faster feedback, or has reduced their time spent debugging – that is two hours a day per developer that could be put to better use.

2 hours a day * thirty developers * two working weeks = 600 additional hours per two week iteration available.

This is equivalent to adding seven and a half developers to the team!

What value do you get from tests of different sizes?

Ignoring this testing just generates projects which will fail – not may fail – will fail, categorically will fail!

There are two sets of tests that are critical here…

Good unit test coverage and a TDD approach to development ensure that the rest of the organisation is supplied with product that works as developers expect it to.

Component tests and functional acceptance tests ensure that the product does what the product owner expects it to.

• Practically, the only way to do solid testing on new features is to have a stable product.
• Practically, the only way to assure a stable product is to either have solid regression testing on each release or to not make any changes.
• Practically, the only way to make sure that adequate regression testing is done is to automate the regression tests (unit, component, functional).

Without these tests in place, iterating quickly places an impossible to meet burden of verification demand on the testers and product owners, which just gives you a fragile process not an agile one.

Is your process Agile or Fragile?

The purpose of the Observer design pattern is to allow one object to notify other objects of events – usually to do with changes to its state. One common problem I have encountered however is what to do when an Observer  is first registered, since the object that it is observing may already be in a particular state.

There are two obvious approaches to this problem.

1. The object registering the Observer must check what the state of the object it wants to Observe is after the registration and handle the implications of this (e.g. enabling/disabling parts of the GUI). The main issue I have with this is that it can mean that I end up with two code paths, one to do something before the Observer has been registered and one for the event notifications. In the case where there are lots of different types of Observers for the same object this will introduce a lot of extra code paths – and eventually someone is bound to forget to handle the initial state and thus introduce a bug. In a multi-threaded environment this may become more complicated, particularly if you want to ensure that you only receive an event once.
2. When the Observer is registered it is notified of the current state of the object it is observing. Although this is not a standard part of the design pattern, I have found it useful since you only have one code path within the Observer for handling the events.

The following example demonstrates how the second approach can easily be implemented in JavaScript (using caplin.core.Observable).

caplin.namespace("caplinx");

caplin.include("caplin.core.Observable");

caplinx.Trade = function()
{
   this.m_bExecuted = false;
   this.m_oObservable = new caplin.core.Observable();
};

caplinx.Trade.prototype.addTradeListener = function(oListener)
{
   this.m_oObservable.addObserver(oListener);

   // **********************************************
   // if we have already executed the trade notify the newly added
   // listener immediately
   if (this.m_bExecuted)
   {
      oListener.executed();
   }
   // **********************************************
};

caplinx.Trade.prototype.execute = function()
{
   // code to handle trade execution ...

   this.m_bExecuted = true;

   // notify all listeners that the trade has executed
   this.m_oObservable.notifyObservers("executed");
}

The technique suggested here is not necessarily appropriate for all Observer design pattern implementations, however I have found it useful to consider when I am writing a new class that will be observed by others.