In Caplin we constantly work on improving our products, feature as well as technology wise. In order to evaluate new things, we are allowed to take one dev-day every two weeks to look into technologies we like or think might be of use to the company. A while back we added a new module to Transformer which needed to have a C as well as a Java API. Of course we followed our standard approach of writing the module in C and then wrapping it with JNI. It’s proven to work, it’s reasonably fast and it’s easy to write – oh, wait … easy to write? Let’s take a closer look at that.
Now, let’s assume we want to expose a simple function that generates a random int value in C and use that function in Java. The standard way of doing this would be to use JNI, as shown in the following example. However, there’s another Library called Java Native access or JNA, that is meant to make a developers life easier*.
Using JNI, we have to define the native function in Java, call javah -jni to generate the C headers and then implement the generated function declarations. The following simplified code snippets illustrate the required code.
Java:
public class JavaNativeInterface {
static {
//randomjni is the name of compiled c library
System.loadLibrary("randomjni");
}
//based on this native method javah
//jni generates a c function declaration in JavaNativeInterface.h
public native int randomNumber(int bounds);
}
C:
#include
#include "JavaNativeInterface.h" //including the javah generated header file
//implement the generated function declaration
JNIEXPORT jint JNICALL Java_JavaNativeInterface_randomNumber(JNIEnv *env, jobject obj, jint bound) {
srand(time(NULL));
return rand() % bound;
}
That doesn’t look too bad or messy, does it? The reason for that is, that we aren’t using anything that’s dynamically allocated or any datatypes other than native ones. The java type jint for example directly maps to int (that can vary depending on the platform), therefore we can use it like any other int in C.
Alright – enough about JNI, let’s look at the same feature in JNA. In order to get JNA working, we need to download the jna jar file and add use it to compile and run our Java program, then implement some of the interfaces provided by JNA and write some C code. And here’s the code.
Java:
import com.sun.jna.Library;
import com.sun.jna.Native;
interface MyCLibrary extends Library {
MyCLibrary INSTANCE = (MyCLibrary) Native.loadLibrary("randomjna", MyCLibrary.class);
int randomNumber(int bound);
}
public class JavaNativeAccess {
public static void main(String[] args) {
try {
System.out.println("Random number from C is: " + MyCLibrary.INSTANCE.randomNumber(150));
} catch (UnsatisfiedLinkError e) {
System.out.println("Exception" + e);
}
}
}
C:
int randomNumber(int bound) {
srand(time(NULL));
return rand() % bound;
}
Okay, what’s changed here? As you can see, pretty much everything. JNA dynamically loads** the libary – in our example it’s called randomjna – and automatically looks up the function randomNumber defined in the MyCLibrary class. One of its main benefits is that we can use an existing C library without writing a JNI wrapper in C. Isn’t that awesome? Well, that’s what I thought and I decided to forget about JNI as it’s bad and evil and do everything in JNA. And then I started to think – can it really be that easy? Is JNA really the solution to all my problems?
In order to answer that question, I’d like to look at another slightly more complicated example. The use-case is to list all files in a directory and pass an array of FileInfo objects to Java. The FileInfo object contains a String – the filename, and an int – the filesize in byte. Let’s look at JNI first:
public class JavaNativeInterface {
static {
System.loadLibrary("randomjni");
}
public static class FileListItem {
public FileListItem(String fileName, long fileSize) {
this.fileName = fileName;
this.fileSize = fileSize;
}
String fileName;
long fileSize;
}
public native FileListItem[] listFiles(String directory);
public void printFileList(String directory) {
FileListItem[] files = listFiles(directory);
for( int i = 0; i < files.length; i++ ) { System.out.println(files[i].fileName + " --> " + files[i].fileSize);
}
}
public static void main(String[] args) {
JavaNativeInterface obj = new JavaNativeInterface();
obj.printFileList(".");
}
}
Alright, that was simple. Create a class that represents the items, expect the method to return an array of instances of this class, iterate over it and voila. It’s all standard Java with standard Java paradigms, we don’t care about the garbage we create and all we know about the native C code is that there’s a function that takes a String and returns a list of FileListItem objects. Sweet. Now – what about the C code?
JNIEXPORT jobjectArray JNICALL Java_JavaNativeInterface_listFiles(JNIEnv *env, jobject obj, jstring jDirectory) {
DIR *dp;
struct dirent *ep;
struct stat buf;
FILE *fp;
int count = 0;
int i = 0;
const char *directory = (*env)->GetStringUTFChars(env, jDirectory, NULL);
jclass j_fileinfo_class;
jmethodID j_fileinfo_constructor;
jobject *object_array = NULL;
jobjectArray j_fileinfo_array = NULL;
if ( (j_fileinfo_class = (*env)->FindClass(env, "JavaNativeInterface$FileListItem")) == NULL) {
printf("Could not load class FileListItem\n");
return NULL;
}
if ( (j_fileinfo_constructor = (*env)->GetMethodID(env, j_fileinfo_class, "", "(Ljava/lang/String;J)V")) == NULL) {
printf("Could not find constructor for FileListItem\n");
return NULL;
}
dp = opendir(directory);
if (dp != NULL) {
while (ep = readdir(dp)) {
if ( ep->d_type != DT_REG )
continue;
fp = fopen(ep->d_name, "r");
if (fp != NULL ) {
if (fstat(fileno(fp), &buf) == 0) {
jobject j_fileinfo;
object_array = realloc(object_array, ++count * sizeof(jobject));
jstring jFileName = (*env)->NewStringUTF(env, ep->d_name);
j_fileinfo = (*env)->NewObject(env, j_fileinfo_class, j_fileinfo_constructor, jFileName, (jlong)buf.st_size);
object_array[count-1] = j_fileinfo;
(*env)->ReleaseStringUTFChars(env, jFileName, NULL);
}
fclose(fp);
}
}
closedir(dp);
j_fileinfo_array = (*env)->NewObjectArray(env, count, j_fileinfo_class, NULL);
for (i = 0; i < count; i++) { (*env)->SetObjectArrayElement(env, j_fileinfo_array, i, object_array[i]);
(*env)->DeleteLocalRef(env, object_array[i]);
}
free(object_array);
}
(*env)->DeleteLocalRef(env, j_fileinfo_class);
return j_fileinfo_array;
}
Bloody hell, what’s going on there? I won’t go into too much detail about how JNI works, please read the JNI guide for the basics, but the thing I want to talk about is the “why”. Why does it look the way it looks?
Java is an interpreted language whose byte code runs on the JVM, whereas the native C code doesn’t. Therefore we have to find a way of translating our C datatypes to datatypes the JVM understand and manage our references to them to allow the JVM to collect garbage.We basically have to make sure that we don’t hold any references to jobjects when we leave the native function.***
Well, that was bad, wasn’t it? – now let’s check out the same thing in JNA.
import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.Structure;
import com.sun.jna.ptr.IntByReference;
import java.util.Arrays;
import java.util.List;
interface MyCLibrary extends Library {
public static class MyStruct extends Structure implements Structure.ByReference {
public String fileName;
public long fileSize;
@Override
protected List getFieldOrder() {
return Arrays.asList("fileName", "fileSize");
}
}
MyCLibrary INSTANCE = (MyCLibrary) Native.loadLibrary("randomjna", MyCLibrary.class);
MyStruct list_files(String directory, IntByReference numElements);
void free_struct(MyStruct.ByReference struct, int numElements);
}
public class JavaNativeAccess {
public static void main(String[] args) {
try {
IntByReference nElements = new IntByReference();
MyCLibrary.MyStruct struct = MyCLibrary.INSTANCE.list_files(".", nElements);
MyCLibrary.MyStruct[] structArr = (MyCLibrary.MyStruct[]) struct.toArray(nElements.getValue());
for (int i = 0; i < nElements.getValue(); i++) { System.out.println(structArr[i].fileName + " --> " + structArr[i].fileSize);
}
MyCLibrary.INSTANCE.free_struct(struct, nElements.getValue());
} catch (UnsatisfiedLinkError e) {
System.out.println("Exception" + e);
}
}
}
Alright – what is going on here, and is that even Java? Of course it is, but it’s Java in, in my opinion, a very ugly way. Our Java Code needs to know that there is a struct in C, it needs to know what members the struct has (memory layout) and it needs to know, that the C library allocates heap memory dynamically and that the structure needs to be freed up once we’ve finished using it. Also, an array in C is just a pointer to the first element and the length information isn’t know at compile time nor stored anywhere – unless done manually, therefore we need to inform our java method about the the size of the array. Now, in the main method, once we’ve got a pointer to the first element of the array and know it’s size, we can transform it into an array of MyStruct and then finally use it in a way we’d expect to use an array in Java.
Okay, enough about that. Let’s look at some C code again:
struct my_struct {
char *filename;
long filesize;
};
struct my_struct* list_files(char *directory, int *elements) {
DIR *dp;
struct dirent *ep;
struct stat buf;
FILE *fp;
struct my_struct *files = NULL;
*elements = 0;
dp = opendir(directory);
if (dp != NULL) {
while (ep = readdir(dp)) {
if ( ep->d_type != DT_REG )
continue;
fp = fopen(ep->d_name, "r");
if (fp != NULL ) {
if (fstat(fileno(fp), &buf) == 0) {
files = realloc(files, ++(*elements)* sizeof(struct my_struct));
files[*elements-1].filename = strdup(ep->d_name);
files[*elements-1].filesize = buf.st_size;
}
fclose(fp);
}
}
closedir(dp);
}
return files;
}
void free_struct(struct my_struct *structure, int num_elements) {
int i;
for (i = 0; i < num_elements; i++) {
free(structure[i].filename);
}
free(structure);
}
Oh, wow, the C code actually looks quite nice. No management of Java objects, no need to manually manage references to Java objects and therefore no need to worry about Java GC – all we have to write is standard C code. As you can see, the point I made earlier in this blog, that we can use C libraries without modifying them/wrapping them in C, still holds true. Most, if not all C libraries expose a function equivalent to free_struct to cleanup internal memory and in that case, all our code can be written in Java, if we make sure that we free up all memory that gets dynamically allocated in C once we don’t need it any more.
Conclusion
After looking into JNA for a bit more than a day I wouldn’t say that I know everything about it, nor that the examples I illustrated are perfect or can’t be simplified. However, the conclusion I came to, is that even though JNI is bad, JNA is ugly.
Both technologies have their advantages and use cases. And even though I personally would use JNA for simple utility libraries that only work with stack allocated data, I would still use JNI for everything that’s more complex. Mainly because as a C developer I’m used to writing long winded code that is hard to understand and that needs to free up every bit of memory it allocates. Therefore JNI nicely fits in to the way C code is written, and the Java code can be written almost without any knowledge of native code.
JNA however is, in my opinion, ugly when it comes to complex libraries dynamically allocated data is being passed around. Even though we don’t need to touch our C libraries, which may reduce the complexity of a project, the Java code doesn’t look like java code and whoever writes the JNA wrapper has to know about dynamically allocated memory and worry about object lifetime and memory cleanup. This isn’t really in line with the way Java code is written nor how most Java developers think.
The whole example projects can be found on Caplin GitHub. Supported platforms are OSX and Linux.
Footnotes:
* Please consult the official JNA documentation for more information about how it works
** JNA uses libffi under the hood to dynamically look up and load functions in a shared library. The shared library needs to be -fPIC compiled
*** There are use cases where we don’t have to manually free local references in C, as they get deleted once the native function returns, but that doesn’t hold true for all cases and it doesn’t help my illustration.