Hack Day Report: Building Custom ExtJS Components for Trading

Hack Day had rolled around once again in the office, and this time our idea was simple but ambitious – could we knock together a tool in 24 hours that would enable developers with no web development experience to create a fully featured web trading application?

Our Plan

Caplin hack days involve teams of two or three, so our crack team (Lawrence, Adam and I) hatched a plan based on three observations:

• Caplin Trader is designed to be modular. It basically consists of a layout manager which handles the position and size of each component, plus any number of components (or “widgets”) such as grids and charts. Some components are provided out of the box by Caplin, but there is also a Component API for writing custom widgets and including them in a Caplin Trader application.
• Applications created with the Ext JS framework (now known as Sencha) have a very similar architecture; they have a layout manager, library of existing components and a Component API for creating custom Ext components.
• Ext have a tool called Ext Designer (since renamed Sencha Architect) which is a visual application builder. It provides a GUI for developers to drag and arrange Ext components on to a canvas and configure them with properties, then export the creation as a working web application.

Pulling these observations together gave us a rough idea of what we needed to do – create a wrapper that converts Caplin components into custom Ext components, then use the Ext Designer to build a working application out of Caplin components without writing a single line of code.

What’s the point?

The Caplin components are highly performant and specifically designed to work with high volume financial data. As you scroll our grid component, for example, it dynamically loads from the server only the data relating to rows currently in view. It also reuses a small number of DOM elements rather than naively creating a table row for every entry in the underlying data structure, ensuring that it won’t kill your browser if you load a grid with tens of thousands of rows.
 

We wanted to find out if our components are truly modular – do they really work without the Caplin Trader layout manager and framework around them? How easy is it to embed a Caplin grid on a plain web page? And if they really are stand-alone, does this allow us to co-opt existing visual application building tools like Ext Designer to build applications from Caplin components?

A good start

The first thing we did was have a look at the API documentation for the Ext Component interface. This proved to be encouraging because it resembles the Caplin Component interface very closely, right down to some of the methods having identical names and functions.

To summarise the purpose of a Component interface, it provides the methods that your custom widget must implement in order to respond to lifecycle management events such as creation, resizing and destruction. These lifecycle event methods are called by the layout manager – Ext has a layout manager built in, Caplin Trader uses a layout manager called Webcentric.

The similarity between the interfaces meant that it seemed easy enough to create a wrapper that would turn any Caplin component into a valid Ext component, simply by wiring up the equivalent lifecycle methods. For example when the Ext layout manager calls setSize(width, height) our wrapper would call onResize(width, height) on the underlying Caplin component. We wouldn’t need to change the code of the Caplin component at all in order to make it behave exactly as it would in a Webcentric layout. Confidence in the team at this point was high!

Getting to know Ext Designer

Next, we downloaded Ext Designer and began playing with it. Building basic Ext applications was easy enough – there is a toolbox on the left populated with Ext components, a workspace in the centre and a property window on the right for configuring the components that you drag onto the workspace.
Under the hood we found that Ext Designer outputs two JavaScript files for every component you include in the application – a file for implementing business logic and a non-editable file containing view information such as size and position. The reason the view file should not be edited is that it gets overwritten every time you save the project in Ext Designer. This approach of splitting the logic from the view is a good one, and will be familiar to anyone who has created GUI applications using Visual Studio.

At this point our plan was to create our custom Ext component that wraps a Caplin component and import it into Ext Designer so that it would appear in the toolbox. However a cursory poke around didn’t reveal any obvious way to do this. It seemed like the only way to do the job would be to drag a standard Ext panel into the application, save the project, and then manually paste our wrapper code into the generated logic file. This was obviously far from ideal. Confidence at this point was waning slightly, although the prospect of an imminent pizza delivery kept everybody cheerful.

This was the first lesson we learned from the hack day: if the success of failure of the project depends entirely on an external tool then you should be very sure that the tool provides what you need. A minimal amount of research before the hack day (which our team neglected to do, bravely but stupidly deciding it would constitute “cheating”) would have revealed that Ext Designer doesn’t actually support custom components in the way we assumed.

Dependency hell

Next, we set about trying to write our custom component. The immediate problem was how to load the relevant JavaScript and CSS files, since this is normally handled by the Caplin Trader framework. Caplin Trader includes a servlet that scans the dependencies of the application, bundles up and minifies all of the relevant source files into a single file, and includes it in the page. As we tried various approaches to manually include all of the applicable source files in the correct order we felt the absence of this servlet very keenly.

Since time was getting on and building an elegant dependency manager/file loader (such as RequireJS) was outside the scope of our project, we eventually had to admit defeat and solve the problem with a brute force approach. Simply put, we intercepted the JavaScript and CSS bundles that are sent to the browser when you login to a standard Caplin Trader installation and included them in our custom component lock, stock and barrel. Loading up the code for the entire application in order to display a grid obviously defeats the point of having a modular architecture.

However with enough time we were confident we could solve the dependency management problem, so we used this workaround in the interests of making progress. With the workaround in place we were finally able to load a working Caplin grid and arrange it in a layout with Ext Designer. Progress!

The hours we spent on this problem did teach us that file loading is currently tightly coupled to the Caplin Trader framework. We’re already working to fix this with the new bladed structure of components used in Caplin Trader from version 2.0 onwards.

Putting it all together

After working around a couple of other minor but sneaky problems – such as the Grid component listening to the window.onLoad() event and not working if it is instantiated after the page has loaded – we were in good shape. The final problem was to find a way to configure the Caplin component using Ext Designer. For example, what data from the server should the grid request? Which columns should the grid display?

Caplin components are required to implement a method called createFromSerializedState(). The layout manager passes this method a block of XML configuration, and the component is responsible for parsing the XML and setting itself up accordingly. We needed to find a way to enter this XML configuration in Ext Designer and make it available to the file containing the logic, in order to set up the underlying Caplin component.
The solution was to use one of the fields in the “Component Config” to store our XML.

The Component Config pane allows you to set properties on your component in Ext Designer which can be read programatically by the class containing the code for the component. We found a property conveniently called “xml” and used it to store the block of Caplin-specific configuration. With that piece of the puzzle in place we were now able to remove the hard-coded configuration and instantiate any grid by altering the XML in Ext Designer and reloading the application.

Our last task was to arrange a couple of the now-working grids into a rudimentary Ext application that we presented the following afternoon to a rapturous reception (at least that’s how I remember it).

Lessons learned

A modular architecture is the future for Caplin Trader, and this exercise was an interesting way to find out whether the components we have been creating for the last few years really are as stand-alone as we think they are. Sometimes the only way to test this sort of thing is to rip out a piece of your application, throw it into an unfamiliar environment (like the Ext framework) and see if it sinks or floats. The results from a day of hacking were pretty encouraging, and with a little more work this approach should yield good results.

Are you a Caplin Trader user? Can you see yourself using a graphical tool like Ext Designer to build your applications in the future? Let us know!