Scaling JavaScript

When your JavaScript codebase consists of a couple of functions to do image rollover and form validation, then javascript seems like an extremely easy language.  It has relatively little bureaucracy for getting simple things done, an extremely useful literal notation for maps and arrays, and string manipulation is simple and can’t crash your machine when you get it wrong (I’m looking at you, C).

What you probably didn’t bargain for though is that with first-class functions, closures and a prototype inheritance system it’s one of the more expressive and flexible languages out there.  Expressiveness and flexibility sound like good things but for many tasks actually lead to problems.
For programming-in-the-large, Javascript actually represents an unusual challenge.  While it’s been around as a language for a long time, consensus on the right way to use it to build large applications is only beginning to build now.

Historically, languages that are preferred for large codebases tend not to worry too much about expressiveness and flexibility and instead prioritise stability and explicitness and providing means for writers of libraries and APIs to restrict the uses that others will make of their code.  People often incorrectly associate Java (almost the quintessential nailed down language) and Javascript. Douglas Crockford once said of JavaScript that it was actually Lisp in C’s clothing.  

Now quickly, answer this question:  If you were planning to write a codebase that you expected to reach more than 1 million lines of code (the codebase for our web application CaplinTrader has around 1.5 million lines of code if you include all the test code), would have hundreds of developers working on it, and would ultimately ship as examples, toolkits and library APIs for customer developers to customise would you write it in Java or Lisp?

Now, I’m actually a big fan of Lisp, so if you answered Lisp then good for you and Paul Graham, but I think you know that you are in the minority.
The problem is that when your language is flexible enough to implement multiple different OO concepts, or has features that allow you to program in radically different paradigms then questions about code arise that simply don’t come up in more restrictive languages.  Nobody who writes Java has had to worry about the right way to implement Object Orientation, or when to write higher order functions as opposed to abstracting using inheritance, or when to use the dynamicism of the language instead of a clunky visitor design pattern.

The web is full of people who will offer you their own answers to these questions, and an active community is a massive advantage to a language.  There is just one problem with this when it comes to Javascript;  lots of the information out there on the web is just wrong, and even many of the well respected libraries will breakdown fast when you’re trying to build something that can run multiple days without a page reload and with a massive amount of data arriving in realtime.  Other advice will lead you to create a codebase that is unmaintainable once you start getting up into the hundreds of thousands of lines of code.

I intend to write more in the same vein to expand on the complexities of Javascript for building large scale applications, as will a number of other Caplin blog contributors.

To start off the theme, I’d like to invite you to view part I of Adam Shone’s JavaScript is Hard series, where he writes about his recent experiences at an advanced JavaScript workshop where he was surprised to learn all that he didn’t know about JavaScript. Read his first post and hopefully you will find a thing or two that you didn’t know either.