Secrets of Automatic Code Translation

I’ve recently spent a few dev days at Caplin investigating the possibility of generating code in a number of languages from one source language.

The business costs of maintaining multiple code bases (and their associated documentation) are obvious, but at the same time it’s a business requirement that we are able to provide our APIs to customers working with a variety of languages and environments.  The worst case is having multiple different APIs with different capabilities and bugs.  A middle road is to have the code written in one language as the master copy, and any time it changes to make the changes manually in the other languages, but the ideal would be if the other languages could be generated from the master copy.
Understandably suggesting such an arrangement often meets with skepticism.  Most people’s experience of source code that has been generated by a machine is that it is unreliable and brittle, working only in the relatively narrow range of circumstances that the original author foresaw and sometimes causing difficult to find problems.

Nevertheless, some people seem to have made it work for them.  Joel Spolsky has Wasabi a language that compiles to PHP4 and PHP5 and to VBScript.  Google compile Java to Javascript with GWT and manage to do this with a very high degree of accuracy even for code written by people who don’t know the target language or the details of their translation step.

I think that I can make it work for my use case too.

My Use Case

  1. The master code will be written with the limitations of the translator in mind – there is no requirement that the translator can perfectly convert arbitrary code written by anyone.
  2. The translator will be developed in parallel to support required conversions and iron out bugs.
  3. If some manual tweaks to the generated code are required, this will be acceptable (although I hope we can avoid this) as long as making the manual tweaks is less effort than doing the translation manually.

What I consider the secret sauce however is something that probably occured to you as soon as I talked about ‘unreliable and brittle’:  unit tests.
My plan is that unit tests will be written in the master language and then translated by the translator into the target languages too.  That way the original code and the generated code will be exercised, and given an appropriate level of testing we will have a high level of confidence that not only was the original code correct, but that the translation step did not introduce any new bugs.

Progress

How far have I got?  Well the first step was to decide the master language.  The languages that we want to support are Java, C#, Javascript, Actionscript and possibly C.  Given that in a statically typed language we have information that isn’t needed in dynamic languages, it seemed sensible to choose a statically typed language as the master.  One possibility would be to create a whole new language for this purpose.  I’ve decided not to go down that route, as it would mean developers would have to learn yet another language, and give up the rich development tools that have grown up for some of these languages.  I also took into account the availablity of parsers. Given these considerations I chose Java as my master language, and Javascript as my first translated language.

I used Spoon to parse the Java because it gave me a rich AST.  My code has already successfully converted a class from Java to Javascript, along with its unit tests from JUnit4 to JSUnit. Running the tests the first time showed me that my converter wasn’t correctly dealing with precedence, but when I had both sets of tests green I was confident that the conversion had been done correctly.

Here’s an example Java class followed by the javascript code created by my translator, and below that is a JUnit4 test class that the translator converted to JSUnit code (we have a custom JSUnit runner).

package demo;
import com.caplin.js.lang.Dictionary;
public class Example extends SuperClz {
	private static Dictionary<String&gt; MESSAGES = new Dictionary<String&gt;();
	static {
		MESSAGES.put("english", "Hello!");
		MESSAGES.put("german", "Guten Tag!");
		MESSAGES.put("french", "Bonjour!");
	}
	public Example(String lang) {
		super();
		message = MESSAGES.get(lang);
		if (message == null) {
			throw new IllegalArgumentException("Unknown language "+lang);
		}
	}
	private String message;
	public String greet() {
		return message;
	}
	public void debug() {
		for (String key : MESSAGES) {
			System.err.println(key +" = "+MESSAGES.get(key));
			if (MESSAGES.get(key).equals(message)) {
				System.err.println("** this is my language");
			}
		}
	}
	public static void main(String[] args) {
		Example e = new Example("german");
		System.err.println(e.greet());
		e.debug();
	}
}

becomes

caplin.namespace("demo");
caplin.include("demo.SuperClz", true);
demo.Example = function(/*String*/ lang) {
	// Super
	demo.SuperClz.call(this);
	// Field Definitions
	/*String*/ this.message;
	// Constructor body
	this.message = demo.Example.MESSAGES[lang];
	if (this.message == null) {
		throw new Error("Unknown language " + lang);
	}
};
caplin.extends(demo.Example, demo.SuperClz);
/*Dictionary*/ demo.Example.MESSAGES = {} ;
demo.Example.prototype.greet = function() {
	return this.message;
};
demo.Example.prototype.debug = function() {
	for (var key in demo.Example.MESSAGES) {
		console.log(key + " = " + demo.Example.MESSAGES[key]);
		if (demo.Example.MESSAGES[key] == this.message) {
			console.log("** this is my language");
		}
	}
};
demo.Example.main = function(/*String[]*/ args) {
	var /*demo.Example*/ e = new demo.Example("german");
	console.log(e.greet());
	e.debug();
};
// Static blocks
demo.Example.MESSAGES["english"] = "Hello!";
demo.Example.MESSAGES["german"] = "Guten Tag!";
demo.Example.MESSAGES["french"] = "Bonjour!";

package test;
import static junit.framework.Assert.*;
import org.junit.Before;
import org.junit.Test;
import com.adam.SlidingWindow;
public class TestSlidingWindow {
	SlidingWindow<Integer&gt; window;
	@Before
	public void setup() {
		window = new SlidingWindow<Integer&gt;(3);
	}
	@Test
	public void can_be_filled() {
		assertEquals(null, window.add(1));
		assertEquals(1, window.getLength());
		assertEquals(null, window.add(2));
		assertEquals(2, window.getLength());
		assertEquals(null, window.add(3));
		assertEquals(3, window.getLength());
		assertEquals(new Integer(1), window.add(4));
		assertEquals(3, window.getLength());
	}
	@Test
	public void test_unfilled_iterator() {
		window.add(1);
		window.add(2);
		int index = 1;
		for (Integer i : window) {
			assertEquals(new Integer(index), i);
			index++;
		}
		assertEquals(3, index);
	}
	@Test
	public void test_filled_iterator() {
		window.add(1);
		window.add(2);
		window.add(3);
		window.add(4);
		window.add(5);
		int index = 3;
		for (Integer i : window) {
			assertEquals(new Integer(index), i);
			index++;
		}
		assertEquals(6, index);
	}
}

becomes (note that we have a customised JSUnit runner)

caplin.namespace("test");
caplin.include("com.adam.SlidingWindow");
test.TestSlidingWindow = function() {
	// Field Definitions
	/*com.adam.SlidingWindow*/ this.window;
};
test.TestSlidingWindow.prototype.can_be_filled = function() {
	assertEquals(null, this.window.add(1));
	assertEquals(1, this.window.getLength());
	assertEquals(null, this.window.add(2));
	assertEquals(2, this.window.getLength());
	assertEquals(null, this.window.add(3));
	assertEquals(3, this.window.getLength());
	assertEquals(1, this.window.add(4));
	assertEquals(3, this.window.getLength());
};
test.TestSlidingWindow.prototype.setup = function() {
	this.window = new com.adam.SlidingWindow(3);
};
test.TestSlidingWindow.prototype.test_filled_iterator = function() {
	this.window.add(1);
	this.window.add(2);
	this.window.add(3);
	this.window.add(4);
	this.window.add(5);
	var /*int*/ index = 3;
	var _itr = this.window.iterator();
	while (_itr.hasNext()) {
		var /*Integer*/ i = _itr.next();
		assertEquals(index, i);
		index++;
	}
	assertEquals(6, index);
};
test.TestSlidingWindow.prototype.test_unfilled_iterator = function() {
	this.window.add(1);
	this.window.add(2);
	var /*int*/ index = 1;
	var _itr = this.window.iterator();
	while (_itr.hasNext()) {
		var /*Integer*/ i = _itr.next();
		assertEquals(index, i);
		index++;
	}
	assertEquals(3, index);
};
Test.setUp = function() {
	this.testCode = new test.TestSlidingWindow();
	this.testCode.setup();
};
Test.tearDown = function() {};
Test.can_be_filled = function() {this.testCode.can_be_filled();};
Test.test_filled_iterator = function() {this.testCode.test_filled_iterator();};
Test.test_unfilled_iterator = function() {this.testCode.test_unfilled_iterator();};

Leave a Reply

Your e-mail address will not be published. Required fields are marked *