In the world of microservices and clients separated from the services, integration is becoming crucial part of developing applications. Almost every application consumes some API. Sometimes that API is so complex, that we, as front-end developers, have to wait for it days or weeks, which can lead us to situations like this one…
Late integration
Yet another busy Thursday. You and your friend Bart are working on the sprint goal – user registration form. You are developing GUI, Bart is responsible for the API. Interface is ready, you have some final touches to do, but API is still in progress…
– Bart, how the API will look like?
– Oh, you know, issue “post” request on slash-register.
– And…? What do you expect in the body?
– Id and password fields, of course.
– Ok. And response?
– I will reply with status code 200 or 400 if something goes wrong.
– OK.
– And remember about error structure, you know, it is a list of error messages…
– Yes, yes, I remember.
So far so good. Service for that functionality should be simple and easy.
One day later…
Friday is a “demo day”! Client is ready, meeting is set up, everyone is waiting for the presentation of the key feature – registration. You are opening shiny new registration form, typing “demo” in id field, password “test123” and submitting the form… Waiting…Waiting…
– Uhm… There should be some kind of spinner maybe..?
Well, yes indeed. But ok – registration completed after few seconds, green lamp, everyone is happy. And client is asking:
– What about error handling?
– Oh, error should display above the form, when credentials are wrong. I can show you…
Typing “demo” again and “wrong password” in password field, waiting, waiting and… Blank screen? That’s weird…
– Should we present some more specific message for the user? – client is curious.
What happened? You are quickly inspecting “Network” tab in DevTools and… Oops! Network error! That crappy wifi network… You were disconnected and got 404 error. User Story is not delivered, story points are lost – no spinner, no error handling, week of work and those unhappy faces in front of you…
Retrospective!
– Maybe you and Bart should agree upon some shape of the API and write documentation before implementation?
– Well… no, no one is reading documentation.
– Ok, so we can ask Patrick to test integration points more carefully.
– Yes, but you know, we’ve finished that user story yesterday and there wasn’t enough time to test it properly.
– Ok then…
– Maybe we can implement API mock? Some kind of dummy server with fake responses working on developer’s machines? It could resolve issues with missing spinner and unhandled errors. I could spot that mistakes before the demo and simulate all kind of responses for my app.
API mocks for the rescue
That’s real life situation. Integration is the key. Or better – fast integration is the key. Fast integration means shorter feedback loop and short loop means more time to resolve issues (if you don’t know why feedback loop is so important, please read The importance of agile feedback loop). But sometimes there is no way to establish API upfront because it is too complicated or your backend developer is too busy working on critical bugs.
In this case you have two choices:
- develop GUI by assuming some shape of the response and miss that spinner would be helpful or specific kind of error would crash the application, or:
- use API mocks and forget about guessing. Just simulate every possible situation yourself.
What is API mock exactly?
API mock is an application dedicated to serve fake responses for real endpoints. Usually it should be sufficient enough to prepare and handle all kinds of data in any client application. Mock could be written in any technology, but most likely it will be created in JS, because front-end developers need that tool to get things done. The mock should be considered as part of the project, but it doesn’t matter, whether is it kept in project’s repo or in separated one. What matters is that it should cover entire API and simulate different kinds of responses.
This sounds like a lot of additional work to do, but it is not.
When to start mocking?
You should start every “user story” that requires integration by writing a mock and testing it. It is really simple and fast process (we will cover that later) and should take up to one hour to cover even the most complex endpoints. If you are working with legacy code and don’t have any API stub – don’t worry. You can easily add one within few hours. You can add mocks only for new endpoints, and you will still benefit from that.
Approaches
As always, there are several ways to setup own mock server:
- simple static json file – very easy to implement, but limited. Only GET request can be mocked, so it should be treated as a last resort.
- ready to use solutions, like http://www.mockapi.io, http://www.mock-server.com/ or https://www.getpostman.com/docs/v6/postman/mock_servers/intro_to_mock_servers, https://github.com/typicode/json-server and dozens of packages in npm registry
- own http server – the most flexible, customizable and prefferred solution.
Tools
You can use any tool to develop API mock, but there are a few on the market that are particularly useful in node.js ecosystem (yes, we will be talking about API mocks in JS context):
- http server – we can, of course, write server using built in node API, but let’s face it – it wouldn’t be so easy to maintain and develop. There are better choices: restify or express. They both have similar API, but for the purpose of this post I will use restify.
- data faking library – again, you could prepare list of fake names and addresses by yourself, but there is a better option – faker.js. This great library covers a lot of data sets in many languages. It can generate fake names, streets, cities, numbers and much more.
Let’s start
First, initialize a new npm project, install restify (we will use version 7.1.1) and faker (version 4.1.0).
npm install --save-dev restify faker
Start your server:
node index.js
That’s it. Your server is up and listening on port 4000. How simple is that?That’s it. Your server is up and listening on port 4000. How simple is that?
How to handle all kind of responses?
In our little story above there was a problem with unexpected error. Lesson learned – we have to test application with all possible responses. Well, almost all… In real life we can limit ourselves to handling:
- positive responses
- expected application errors, usually with code 400 and some specified body structure (if we have one)
- most common errors like 401, 403, 404, 500, 503
But how can we do that with a mock? It is as easy as:
Do you need different responses per request? Here you go:
And remember – by default you should serve positive response, because there is nothing more annoying than going through multi step wizard just to see an error from the last endpoint.
Moreover, make use of factory functions/classes/builders. Don’t copy and paste code, because you don’t want to change name of the property in hundreds of entities. Treat the mock as your production code and apply DRY rule to it.
And remember – by default you should serve positive response, because there is nothing more annoying than going through multi step wizard just to see an error from the last endpoint.
Moreover, make use of factory functions/classes/builders. Don’t copy and paste code, because you don’t want to change name of the property in hundreds of entities. Treat the mock as your production code and apply DRY rule to it.
How to simulate network delays?
Very important aspect of API mock is network delay. Usually user has to wait for response from the server, therefore we should provide some kind of spinner or message to indicate, that something is going on behind the scenes. Simple solution is the best – use setTimeout() with randomly chosen time to reply to the client. How to do that with restify? Let’s revisit previous handler:
Now we can test it with cURL or any HTTP client like Postman:
How to keep mock up to date with real implementation?
The hardest part. There is no easy way to ensure 100% consistency between real API and your mock. The mock is a separate application, so we have to create naive copy of real API and keep it in sync with the backend manually. There is of course room for bugs, but still this is much better than no API mock at all.
In small teams it shouldn’t be a problem – you have daily standups, plannings and so on to be informed, that something has changed. In bigger teams you should prepare some kind of policy – maybe backend developers should inform about every important change in API shape? Maybe you have living docs and someone (or better – some script) will check its compatibility with mock? Or maybe backend developers would like to create mock before they start real implementation (I’m kidding, they won’t)…
Pros
- testability – with mock you can test your application and make sure, that it plays nicely with every expected and unexpected response,
- ease – API mock is really easy to write and maintain. It should be as dumb as it can be. The best logic is no logic – only simple if/switch statements,
- self documented – believe me, when you have mock, sometimes it is much faster to find out the shape of the expected response by looking at the mock than at the service (mostly because the mock has simple structure and you can easily spot file responsible for specific endpoint),
- no delays – you can start working on new functionality on the very first day of the sprint, just agree on the shape of the endpoint, mock it and use it,
- useful for reproduction of bugs – some bugs are strictly related to backend responses. We can’t simply anticipate all cases, but with mock we could setup problematic response and test the behavior of our app,
- doesn’t require internet connection – we could develop with no network at all, because all data is available locally through the mock,
- flexibility – with mocks you are not limited by any external conditions – you can simulate long delays, timeouts, server errors, large responses – everything is up to you
Cons
- consistency with real API – it could be a pain, especially in large teams
- time consuming – nothing is for free, you have to spend some time to create the mock, but this is minor issue considering all pros
Summary
Even in small teams of developers with great communication skills you will face issues with integration. This is one of the hardest part of developing software. Especially when you are working in microservices architecture, where almost every service has some kind of API to expose.
API mock is one of the easiest way to mitigate that risk. It requires almost no knowledge from experienced developers. It is easy to write and pretty easy to maintain. Furthermore, it shorts feedback loop and you should always strive to get feedback as quick as possible.
If you don’t have API mock yet, you should create one in very next sprint, no matter the size of the app you are developing right now. Benefits will come almost instantly.
