So how can we get around the encoding issues and end up with testable locale files?

We have quite a broad base of different operating systems in use at Caplin, most of which are fine. Even Windows XP works well enough after installing some of the optional language support.

Then there are the editors. In addition to various text editors, Eclipse is the IDE of choice at Caplin, and it has a well hidden setting to override the default file encoding (Preferences -> General -> Workspace -> Text file encoding) and work with UTF-8 everywhere. This can also be done on a per-project basis if necessary.

That should be enough to get started. We already had a default locale file, so I used OmegaT to machine translate it into bad Chinese and saved the result. Unfortunately, the new file is full of unicode escape sequences – strings in the form “\uXXXX\uYYYY\uZZZZ”. That can be used but is no good for testing, so I went in search of a simple tool to unescape the text in the file and output the result in UTF-8.

It was surprisingly hard to find a suitable tool online. So much so that I ended up writing my own. First I tried Java because it should work everywhere and has extensive unicode support. After fighting with it for a little while, and only getting encoding problems in the output and no error messages at all, I decided to use something else.

I wrote a quick script in Python to do the conversion and spent some time analysing the errors that occurred when trying to write the output to the console.

After some hair-pulling, I realised that it was trying to output text in the encoding of the calling process, a DOS window (cp850). The DOS, or more properly cmd environment simply doesn’t support asian scripts. It barely even supports English, come to that.

Thankfully, the solution was quite simple. All I had to do was open a file with the desired encoding and write directly to that. The resulting python script to unescape unicode strings is on GitHub.

Embarking on developing a web application can be bewildering: there’s no shortage of JavaScript toolkits of all different shapes and sizes but there’s no guiding light.  Microsoft and Adobe, by contrast, offer a much greater degree of comfort with their methodologies, libraries and tool support – but only if you are willing to restrict your application to running on their respective platforms.

There are some great JavaScript tools out there, however, and this article attempts to navigate through the maze of toolkits to assist making technology choices when building a web application.

User Experience is Paramount

The GUI is the most prominent part of the application, and is usually the most expensive to develop.  An important part of the UX rests on your choice of widgets and how customisable they are.  You should choose widgets that will give you the best possible starting point; if you can’t find what you want, you will have to consider composing your own.

Most widget toolkits are general purpose, but more sophisticated widgets are emerging.  ExtJS, jQuery and Dojo provide widgets ranging from grids to email suites.  Caplin specifically targets widgets for financial sector clients that are oriented to financial domain specific objects and delivering real-time prices to the screen.

Widgets are typically customised using templates, configuration and programatically.  Templating is by far the quickest approach, and this has been well proven with Silverlight’s XAML.  HTML also works very well as a view templating language, and HTML5’s data attributes can be used to further this approach.  Caplin has developed some easy-to-use frameworks that use HTML/CSS templating to build new widgets quickly.

UX does not stop at widget selection though: how they interact is also crucial.  User workflow across all the widgets in the application needs to be implemented with due respect to the overall application structure.

Structural Patterns

Most GUIs conform to some kind of structural pattern.   Passive View, Presenter First and Presentation Model (aka MVVM) are all good design approaches.  There’s little merit in debating which precise pattern is “best” – if it were that simple then Martin Fowler would have told us the answer by now.  The most important thing is that you should be able to build your widgets with the appropriate degree of composability, isolation and testability.

All MVC-derivatives leverage the observer pattern, and it’s crucial to choose an implementation to support the right level of decoupling (such as pub/sub).  For example, a web application typically comprises a layout manager which houses its widgets and manages screen real estate, and you will need to consider how your component-level widgets will communicate with application-level code.

You may decide to use additional utilities to implement structural patterns in your application, but you need to proceed with caution.

Toolkit Selection

JavaScript toolkits come in two categories: widgets and utilities.  Utilities such as prototype.js and backbone.js can serve as the basis for building your own widgets, but that starting point doesn’t suit everyone.

The lack of JavaScript standards means that, invariably, widget toolkits come with their own JavaScript extensions that are both integral to the widgets themselves and also feature in the API.  Combining widgets from more than one toolkit requires special care, and each toolkit brings its own idiosyncratic syntax.  For example, jQuery has its own $() selector syntax, and underscore.js which (as suggested by its name) entirely revolves around the ‘_’ symbol.  This can be quite distracting when trying to standardise development across an enterprise.

For each toolkit you adopt, ensure that the benefits outweigh the incremental burden that you are taking on, and ensure that you are happy to embrace its idioms and syntax.

At Caplin we’ve made enthusiastic use of toolkits such as ExtJS, jQuery and Emprise charting, but have found it necessary to build our own frameworks for layout management, real-time grids and templating/data-binding.  The last one of these has proved so useful that we are planning to open source this in the near future.

This article addresses the issue of how to identify, with confidence, whether an API is backwardly compatible or not. As I mentioned previously there are times when an API is just too incorrect to be kept, however the decision for a breaking change must be a deliberate one – not an accidental one that is typically only discovered when the users of the API find that their existing code no longer works.

Preventing Breaking Changes

It is easy to tell developers that they should keep an API backwardly compatible, however even with the best of intentions it is possible to introduce subtle changes to the behaviour, particularly if you are refactoring collaborating classes that may not directly be part of the public interface. Every developer should be aware of the importance of keeping the API backwardly compatible, however this cannot be relied upon. After all, every developer should be striving to keep their code bug free, but we still write tests to verify this.

The solution is to detect any API changes before it is released, preferably in an automated fashion, so that it can be fixed. The problem is how to know whether an API has actually changed.

Compilation

A starting point for a compiled language, such as Java, is whether some code that was written against the old API still compiles against the new one. Any interfaces that have changed, including those with changes to method signatures and so on, will be flagged up immediately by the compiler. However this is no guarantee that the behaviour of the API hasn’t changed, and this simple verification isn’t available for non-compiled languages like JavaScript.

Unit Tests

The next obvious tool to detect a change are unit tests. The problem with this is that the unit tests will also be updated when the underlying code is modified. The fact that a unit test for a public API is having to be changed should start alarm bells ringing for the developer making the change; are the changes that they are making backwardly compatible or not? Again, however, the problem with this is that sometimes the changes can be subtle and the breaking change may be missed, particularly if the changes are made to a collaborating class rather that the one that is directly part of the public API.

Acceptance Tests

Acceptance tests can be used to detect breaking changes in a similar way to the unit tests. My problem with this approach is that they tend to be less granular and therefore susceptible to missing subtle breaking changes. Ideally I want my tests for API changes to have excellent coverage and to test all possible code paths, which is usually something that isn’t done by acceptance tests.

API Tests

Taking a step back, it does look like unit tests seem a good match for solving this particular problem. The only issue is that the unit tests will be modified when the code is changed, which may mask any breaking changes that have unwittingly been introduced. However we do have a set of unit tests perfectly suited to this which haven’t been modified, those from our previous release. We know that they proved the API for that release worked, so if the same tests pass against the new code it stands to reason that we should be backwardly compatible.

Creating API Tests

At Caplin we have a set of API tests that are a specialised group of unit tests aimed at verifying that all aspects and behaviours of the public API have not changed. They are usually a subset of the unit tests written for a particular library.

Only Running the API Tests

The first problem is specifying which tests are those for the API and which are standard unit tests. This is important since the implementation of collaborating classes can be changed without impacting the public API. As a result the old unit tests for refactored collaborating classes may now fail, however these are unimportant if they have no impact on the API.

Within our JavaScript unit tests we have added the ability to flag a particular test as an API test. We can specify whether we want to run only the API tests or all the tests, including the API tests.

Alternate approaches would be to create different test suites for the API and standard tests, or perhaps to use a custom annotation like @ApiTest rather than @Test in JUnit 4.

Dangers With Mocking

One danger with testing APIs is the use of mock objects. Mock objects combined with dependency injection provide a powerful way to test the logic of a particular piece of code as a single unit, without having to rely on the real logic in the collaborating objects. Instead the mock object acts as a replacement for the real collaborating object, verifying that it is invoked in the correct way and returns the appropriate values.

The danger is that changes to the collaborating object, such as the addition of a statement that throws a runtime exception if an argument is null, will not be emulated by the mock object. These problems are not insurmountable, however you need to be aware of the potential risks. In the case above there is a reasonable argument that there should have been an API test written for the collaborating class that demonstrated that the arguments could have been null previously which would now fail.

Getting the API Tests

At Caplin we upload the API tests to our Maven repository along with the other build artifacts for our libraries. For example when we release version 1.0 of a library, the API tests for 1.0 are uploaded to Maven. When we are building version 1.1 of the library we download the API tests for 1.0 to verify that the API has not changed.

Alternatives to getting the tests from Maven are to use another dependency management system, to pull the tests from a branch or label from source control, or even copying them from a network share.

Updating API Tests

It is possible that a public API hasn’t changed, however the API test for it fails. The reason for this could be that the interaction between the API method and its collaborating objects, particularly if they have been mocked out, have changed, and the test expectations are no longer valid. As a result the code for the API tests will need to be updated.

This change needs to be managed via source control, with a new version of the API tests pushed into Maven, or wherever they are being stored.

Backward Compatibility With Older Versions

At the moment we are only checking backward compatibility against the previous version of the API. I believe that if an API is compatible with the previous one, and that previous one was compatible with the one before, and so on, then it stands to reason that the current API should be compatible with the oldest one.

Conclusions

We are still in the early days of trying out this approach, however the early signs have been encouraging.

Ultimately a large amount of the responsibility still lies with developers to ensure that their code changes to a public API maintain backwards compatibility. The API tests are a safety net that gives confidence that it really is backwardly compatible and provide an early warning of any breaking changes, allowing them to be fixed before a release is made.

Any breaking changes within an API must be a conscious decision, not an inadvertent one.

The purpose of the Observer design pattern is to allow one object to notify other objects of events – usually to do with changes to its state. One common problem I have encountered however is what to do when an Observer  is first registered, since the object that it is observing may already be in a particular state.

There are two obvious approaches to this problem.

1. The object registering the Observer must check what the state of the object it wants to Observe is after the registration and handle the implications of this (e.g. enabling/disabling parts of the GUI). The main issue I have with this is that it can mean that I end up with two code paths, one to do something before the Observer has been registered and one for the event notifications. In the case where there are lots of different types of Observers for the same object this will introduce a lot of extra code paths – and eventually someone is bound to forget to handle the initial state and thus introduce a bug. In a multi-threaded environment this may become more complicated, particularly if you want to ensure that you only receive an event once.
2. When the Observer is registered it is notified of the current state of the object it is observing. Although this is not a standard part of the design pattern, I have found it useful since you only have one code path within the Observer for handling the events.

The following example demonstrates how the second approach can easily be implemented in JavaScript (using caplin.core.Observable).

caplin.namespace("caplinx");

caplin.include("caplin.core.Observable");

caplinx.Trade = function()
{
   this.m_bExecuted = false;
   this.m_oObservable = new caplin.core.Observable();
};

caplinx.Trade.prototype.addTradeListener = function(oListener)
{
   this.m_oObservable.addObserver(oListener);

   // **********************************************
   // if we have already executed the trade notify the newly added
   // listener immediately
   if (this.m_bExecuted)
   {
      oListener.executed();
   }
   // **********************************************
};

caplinx.Trade.prototype.execute = function()
{
   // code to handle trade execution ...

   this.m_bExecuted = true;

   // notify all listeners that the trade has executed
   this.m_oObservable.notifyObservers("executed");
}

The technique suggested here is not necessarily appropriate for all Observer design pattern implementations, however I have found it useful to consider when I am writing a new class that will be observed by others.