Writing an RPC library in Node.js
Understand Remote Procedure Calls better by implementing one yourself.
In case you didn’t know, GraphQL and REST aren’t the only client-server interaction patterns in history. Yes, they are the most popular ones at the moment and with good reason! But at the beginning (especially then) one of the first ways developers started envisioning shared functionality between client and servers was using what they called Remote Procedure Call. In other words, they wanted to use remote code locally, and in the process hiding the complexities of the remote call.
That being said, a remote call can’t ever be exactly like a local one because of all the things that can go wrong with the client-server connection. There is latency, there are connection problems and of course, if you’re working with Node.js, there is asynchronous behavior.
So the best way I know how to understand a concept in software development is by implementing it myself without looking at other dev’s versions. That way I get to reason through the different processes and try to understand how they need to work together. And in this article, I’ll show you what I had to do in order to implement a quick RPC library, hopefully, the process will share some insight and even show you that not everything needs to be GraphQL and REST, there is still room for other mechanics.
Tip: Easily reuse small modules across your JS projects
Use Bit to easily develop, isolate and publish small modules in your different projects. Don’t install entire libraries and don’t copy-paste code, when you can build faster with small reusable modules. Take a look.
What are we implementing again?
Now that I got that intro out of the way, let me explain what exactly am I going to be implementing here because there are many flavors of RPC.
I’ll be implementing a JSON-WSP compatible RPC library which I so aptly decided to name jWhisper (sorry about the name, I really tried to come up with cooler one, but clearly, I failed at it). What that means is:
- The data transfer format to be used between client and server will be, of course, JSON.
- Unlike JSON-RPC which is simply a way to transfer data between both entities using JSON, JSON-WSP also adds a way to define a schema (something similar to what an WSDL file does for a SOAP service) using JSON.
- The communication protocol to be used here is going to be HTTP. Yes, we’re not going to be using Sockets as one might assume for this type of interaction. This makes things easier (at least to me).
And that is it, those are the restrictions we’ll need to work with, as well as a pre-defined format for the schema (which makes sense if you think about it since it tries to be a standard for systems to implement and be able to interact with others platforms).
The formats we’ll need to follow are the following.
Formatting the schema of your shared service
By the way, a shared service is what I call the classes you’re trying to share with your clients. In other words, you’ll be grouping a set of functions into a class (i.e its methods) and the ones you choose to specify as part of the schema will be shared with any connecting client.
The format for this schema is the following:
Now, wait a second, I know that’s not exactly the schema described in the Wikipedia page for JSON-WSP, and that on purpose. The library is taking care of writing the complete version of it. The above code is a simplification specifying only what you need: in other words, it’s a description of your service.
Creating an RPC server
Here is a full example of how you’d setup a JSON-WSP server with jWhisper:
Notice how the code defines a class, called (so eloquently) Test which has two methods ( add and mult ) although through the schema defined below , it only shares one. The server is then instantiated with a set of configuration parameters (i.e the service name, the list of shared services and types), finally the server is started with the start method.
Internally, the start method defines a simple HTTP server, and once the server is ready, it’ll setup the shared services by going through your defined schema. The way I’ve implemented this is by making the server an event emitter and having it emit a new event every time a request is received. For each class shared, I’m wrapping their behavior into an individual instance of a Service class. Each instance will subscribe to the name of the method they handle, and once that event is triggered, a generic method will be called.
The setupServices method is part of the server, and there you can see how it simply iterates over all the services (i.e the classes you’re sharing) creating new instances of the Service class.
The most interesting part of this process is this internal class with only one method:
The key of the above code resides on the callback defined for the on method. It takes care of understanding which method you’re calling, capturing the sent parameters, executing the actual method and grabbing the returned value in order to send it back to the client. Notice how this particular callback is using Rest parameters. This particular feature came in very handy, because the number of arguments the actual method receives is dynamic, yet the first parameter for the callback needs to always be the same (the response object) to send the data back to the client.
That last part is done through the parseMsg method which captures the HTTP request, makes sure it’s of the right type (a POST message sent to the right endpoint) and simply grabs the body of the request, parses it into an actual JSON and then uses its properties to emit the correct event.
Let’s now take a quick look at the other side of the equation: the client.
Defining an RPC client
This is my favorite part of the entire client-server interaction in RPC and that’s because the client object is dynamically extended by gaining methods based on the schema returned by the server.
The way you’ll be using the library to define the client is like this:
In other words:
- Instantiate the client
- Wait for the ‘ready’ event to be triggered
- Once that’s ready, simply use it, remembering that all new methods are, indeed, asynchronous by nature.
The example above uses the await keyword , so it needs to be inside an async function. You could very well use a Promise based approach.
The client’s class code is quite straight forward, the only relevant part of that code, is the method that takes care of parsing the schema returned by the server and dynamically adds the methods to the object. Here is that code:
Essentially the code grabs the keys of the methods property and loops through them. For every method name, the code will create a new method and add it to the host object (the client itself). This new function contains a request object (which essentially is the request that needs to be sent to the server) and then returns a new promise. This newly created promise contains, in turn, the request done to the server, grabbing its response and essentially parsing it and resolving the promise with it.
This flow solves everything your client object needs to do, and the key here is to understand how the newly created methods contain the same logic: send an HTTP POST request to the server, where the actual method’s logic is executed.
Potential improvements
That is it for the review of this build, if you’d like to look at the full code it is available over here at Github. That being said, some quick improvements that you might want / need to add if you’re looking to implement such a library for your own project:
- Adding attachment support on the server side. If you read the definition of JSON-WSP carefully, you probably noticed how there is actually a whole section dedicated to the attachment support. Due to lack of time, I did not implement it.
- Checking for method name collision on the client. Right now, you’re able to export any type of method names on the server side, and once you import them from the client, those names could overwrite the client’s object function. So you’d either want to be careful with the method names exported, or resolve collisions properly. A good idea for this could be adding these methods on a brand new object and returning it as part of the
readyevent. But I’ll leave that to you and your better judgement.
Conclusion
I hope you enjoyed the blow by blow of this project, I basically built it as a fun side project after noticing there was no JSON-WSP compatible library published in npm’s registry (amazing, considering the amount of packages published there).
If either this article or the actual library end up being useful to anyone, then that’s an amazing win already!
Leave a comment below if you’d like to share your thoughts on the build or go to Github and publish a PR if you got improvements to contribute!
Otherwise, I’ll see you on the next one!
