Breaking monolith by creating Web APIs — Lessons learnt

In this article, we will explore Web API integration as a way to communicate between Microservices. We will see the best practices and common pitfalls in this pattern. We should aim to avoid using Web API calls as much as possible when interacting with other systems to achieve more decoupling. However, when it is not possible, then the below learnings might be helpful while developing new Web Apis (especially while breaking monolith).

Key Take-Aways

1. Plan, Design and Develop: Web APIs are easy to build, but easy to misuse. Carefully plan and build a Web API. Have proper documentation and process around it. Once created, it is very challenging to modify the endpoint.
2. Loose Coupling: REST services should remain unaware that they are participating in a “global“ transaction. Avoid transactions over HTTP.
3. Resources over-utilization: When calling Web API, be aware of various resources that are part of the process. For e.g. Memory, CPU, Socket consumption. Have experienced SocketExceptions in Production due to many Web API calls in the batch run. This will cause the next calls to fail because there are no more available sockets left to process the communication.
4. Caching: Consider caching Web API response that is read frequently but updated infrequently (e.g system info and configurations, settings, etc).
5. Make it more resilient: Check for critical Web API requests which when failed, then halts the whole flow. For e.g User Logon, Get settings, configurations, etc. If they are failed for the first time, then with retry logic, they can pass in a second retry. Maybe the Bearer auth token is expired the first time, so send in a new token in the second attempt. This will improve the overall success rate.
6. Throw meaningful exception: Tell the client what to do in the error response. Use HTTP Status code wisely. (Especially helpful when implementing retry logic. Trigger retry logic based on certain HTTP Status Code returned from the failed response)
7. Special characters in Route: It can cause 404 errors which can be difficult to troubleshoot. Encoding such of such query params on the client-side and decoding them back in the Web API is one way to handle such scenarios.
8. Avoid Transaction Scope: Avoid calling Web API inside a transactional scope. The primary reason is that Web Apis does not participate in the transaction out of the box. They are stateless. Another reason is that Web API call over the network will prolong transaction duration which should be avoided.
9. Load Test: Do load testing for the value chains which involves Web API interactions.
10. API Throttling: It offers an extra layer of protection of backend resources. With throttling, a single client cannot choke Web API applications. It will also enhance the performance and improve the overall user experience.
11. HTTP Client: Use HttpClient wisely. Decide if you want to cache it and reuse it for m next calls or n minutes. (Caching HttpClient doesn’t obey DNS changes). Keep connection timeout as low as possible to avoid Socket overuse. Requests burst in can end up in no more available network socket exception and may disrupt other applications hosted on the same server.