Hackday – gRPC

Our streaming server Liberator serves financial data to browsers with low latency and high resilience.  We’ve used it for trading and pricing all kinds of assets in web based trading systems, and particularly for FX.

However, this genericity means that we need to be careful about defining the contracts between what the various backends send and what the front ends expect.  For FX we solve this with a dictionary of message types defined in our FX Integration API.

We’ve been iterating on and enhancing Liberator for 20 years, and it is battle tested and featureful.  However, this results in a substantial amount of specialized code for us to maintain.  We wanted to see what we could learn from other approaches, and especially if there might be ways we could reduce the amount of custom code we maintain.

In particular, we are interested in systems that meet the following criteria:

  1. are widely adopted
  2. are accessible from many different language ecosystems
  3. have a clear way of expressing contracts between the backend and the frontend
  4. support streaming data (e.g. prices)
  5. work in the browser

We came up with a shortlist of approaches that meet these criteria, but one of the front runners was gRPC

gRPC

A particularly appealing aspect of gRPC is the proto file definition of services.  Here’s a snippet: 

service PriceService {
 rpc RequestPrice (PriceStreamRequest) returns (stream PriceStreamResponse) {}
 rpc RequestSalesPrice (SalesPriceStreamRequest) returns (stream PriceStreamResponse) {}
}

message PriceStreamRequest {
 StreamType streamType = 1;
 LegsType legsType = 2;
 string baseProduct = 3;
 string quoteProduct = 4;
 repeated Leg leg = 5;

 message Leg {
   oneof settlement {
     Tenor tenor = 1;
     LocalDate settlementDate = 2;
   }
   TwoWayAmount amount = 3;
 }
}

Here we’ve got a nice description of the service interface, which tells us it returns a stream, and it also allows us to specify exactly what our messages look like.

You can already see that gRPC is ticking a lot of our boxes:

  1. It’s widely adopted – it has more than 4000 questions on Stack Overflow, is maintained by a large organisation on GitHub with more than 30k stars, and is used by Google, Netflix, Cisco and others.
  2. It’s accessible from every programming language our customers might be interested in – from the proto definition file, you can auto-generate clients and server code in at least 11 different programming languages.
  3. The proto definition file with services allows you to clearly express the interfaces between the client and the server. We actually have an internal format we use for creating the FX Integration API that is somewhat like this, but it’d be nice to have a format that is recognised outside of our Engineering team.
  4. As you can see in the service definition the calls can return streams of messages, not just single instances.

That leaves the question of how well it works in the browser.

gRPC in the Browser

The ecosystem feels quite fragmented when it comes to browser usage.  There are a number of projects (protobuf.js looks like a promising partial implementation), but it seems that the typical ‘to the browser’ use case is for it to be used in request/response style APIs. Support for streaming in the browser is much less mature.

In order to test this practically, for the hackday we used the protoc tool with grpc-web plugin to generate JavaScript files for use in the browser.  There were  a few choices at this point: there’s another implementation from Improbable, and there are other APIs that look promising but aren’t fully featured but we kept within the official approach for this hackday.

Actually running the protoc tool was surprisingly fiddly with it frequently getting upset about exactly where the proto files lived.  Eventually I managed to work out the right set of paths and directory structures to get it to work properly.

In node.js though, you can use the @grpc/protoloader module to do the same thing at run time. 

const path = require('path')
const grpc = require('@grpc/grpc-js');
const protoLoader = require('@grpc/proto-loader');

const PROTO_PATH = path.join(__dirname, "..", "protos", "src", "main", "proto")
const protoOptions = {
   includeDirs: [PROTO_PATH],
   keepCase: true,
   longs: String,
   enums: String,
   defaults: true,
   oneofs: true
}
const pricesPackageDfn = protoLoader.loadSync(path.join(PROTO_PATH, "com.caplin.proto.schema", "price.proto"), protoOptions)
const prices = grpc.loadPackageDefinition(pricesPackageDfn)

const priceService = new prices.PriceService('grpc-server:10000', grpc.credentials.createInsecure())

const espStream = priceService.RequestPrice({
   streamType: prices.StreamType.ESP,
   legsType: "OUTRIGHT",
   baseProduct: "EUR",
   quoteProduct: "USD",
   leg: [{
           settlementDate: {year: 2021, month: 6, day: 16},
           amount: {product: "USD", value: "1000"}
   }]
})
espStream.on('data',  price => {
   console.log('received price', price)
}).on('error', err => {
   console.error('got error', err)
})

We ran against an example server written in Kotlin, and successfully received price updates.

So far so good.  gRPC is a protocol that is heavily based on HTTP2.  However, perhaps surprisingly, it’s not currently possible to create a client running in a browser that can talk directly to a standard gRPC endpoint, despite modern browsers supporting HTTP2.  A big part of the problem is that browsers try to hide the intricacies of the protocol they use to communicate with the server from the client code, and so at the moment it’s not possible for code running in a browser to have the level of control over the HTTP2 connection it needs to work correctly as a gRPC client.

In order to avoid this problem, the current recommendation is to stick an instance of Envoy Proxy in front of the gRPC servers. So our next task was to spin up and configure an Envoy Proxy container and get that running.


Finally, we could take the JavaScript code by grpc-web plugged into the protoc tool and load it into a browser and make and receive a price request.  The generated code seemed to need a getters/setters style interface which isn’t idiomatic for js code, but we got there in the end.

We wanted to see the code actually running in our FX applications, so I created a hack to redirect requests for prices in FX Pro, so that instead of going over our StreamLink connection to the Liberator, they were diverted to gRPC going via the Envoy Proxy instance.

Loading up the app, we discover that only the first 5 pricing tiles load.  This is because each gRPC request looks like a separate request to the server, and because we’re returning streams, those connections don’t close, and browsers still have very tight restrictions on connections per server.  StreamLink doesn’t suffer from these restrictions particularly because we multiplex all requests over at most two different simultaneous connections to the Liberator, and usually we are able to make a websocket connection anyway, which browsers treat differently.

HTTP2 actually allows browsers to multiplex different logical connections over a single physical connection too, and when that is happening, the browsers relax the connection limit (often to around 100).  In our case, the browser was making an HTTP1.1 connection to the envoy proxy so it couldn’t take advantage of this feature.  In order to allow all our pricing tiles to load, we needed to enable HTTPS (mkcert was helpful here), and change the envoy.yml configuration to allow HTTP2 connections.  Finally, the browser loaded using HTTP2 and all price streams were available.

Conclusion

This was an interesting exploration of using gRPC for our use case, but there were certainly challenges along the way:

  1. The ecosystem, despite being large, still seems surprisingly immature.  The tooling tends to have sharp edges, there are multiple libraries to choose from which are often not idiomatic or complete. @mitchellh’s tweet seems pretty accurate: ‘the ecosystem is of questionable quality almost across the board’.
  2. Browsers do not give you programmatic access to whether your connection is HTTP1.1 or HTTP2, and HTTP1.1 without multiplexing would be unusable for price streams.  Either manual multiplexing would be needed or the proxy would need to be configured to reject HTTP1.1 connections, making it work only on browsers released in the last 5 years.
  3. There are a number of features that Liberator and StreamLink support that are extremely useful in financial services that are not supported out of the box by gRPC.  For example,
    1. when serving the same price stream to multiple clients, fan out is done in Liberator, so no networking or load is placed on the pricing server
    2. messages sent to clients from Liberator are only the deltas from the messages that the clients have already received, reducing network traffic
    3. new clients / reconnecting clients get told about the latest state of their subscriptions, even if no new update has occurred
    4. StreamLink can fallback to using different messaging approaches and support even very old clients
    5. configurable throttling and conflation allows messages to only be sent at an appropriate rate for the client
    6. Liberator supports the notion of ‘containers’, which are windows into potentially very large collections of updating items.  The items themselves can update, and the contents of the containers can update and move around.

All of these facilities could be supported over gRPC, but they would require twisting the protocol so that the service definitions in the proto file would no longer look like neat definitions of the business logic of getting and receiving prices but would start to look much lower level to do with sending and receiving diffs of arbitrary data.  Even multiplexing could be done too at the gRPC layer, with a similar cost.  While possible, these would massively diminish the value of our third requirement – to have a clear definition of the contract between frontend and backend.

Some significant benefit for these use cases could be achieved with a custom proxy in place of the Envoy proxy, communicating usually over websockets and coping with multiplexing and fallback.  However, a big part of why it would be interesting for us to use gRPC is to reduce the amount of custom code we maintain, and that custom proxy would be a significant investment.

gRPC remains an interesting approach to us, but there are still many challenges to be explored and overcome.

Leave a Reply

Your e-mail address will not be published. Required fields are marked *