Ensuring code quality for API’s with Request Manager API Pattern

What problem does Request Manager API Pattern solve? Coding guidelines and code reviews are good ways to ensure code quality and enforce structure. But they also have their limitations. When time is in short demand and the lead developer/architect have no chance of keeping up with the new code being developed, the architecture, structure and thereby quality of the code-base declines. This makes it more prone to errors, which in turn are harder to fix. Furthermore random code structure also makes it hard for new/other developers to comprehend code in new business areas and understand the basics. They do not understand the business area and they do not understand the structure and therefore have no point of reference.

Therefore setting and keeping code structure and guidelines is a very smart move. This Request Manager API Pattern helps with exactly this.

But how do we go about doing this in a way that is sustainable even in a hectic project with a tight deadline(As they all are). We of course make good use of patterns. Well enter generics and base classes. With clever use of those and a little help from general structuring of code, we can actually make sure that the code fulfills the needs we have – no matter who implements it – and further more make it easier to review and understand for new/other developers.

The story and origination of this pattern

So while I will lay claims to having come up with this pattern for the project in which I used it, I won’t claim that I am the only one to come up with something like it or that everything came out of thin air in my head. Part of this pattern is actually taken from another pattern a very good architect and old mentor of mine used to poll a database that acted as "worker queue" for an old airline system. That pattern planted the seed of using generics and abstract methods to "force" a good implementation on the implementers. But the distinct implementation and use of these tools in this pattern came from my head and the problem I was asked to solve.

The project was a rather big one with 10-12 developers when we were most. A combination of permanent employees and freelancers (whih I also was) and also spread out across different countries. What quickly became clear to me was that we had to bring some structure to both the layers of code but also actual implementations, to not drown in code reviews and differnt implementations.

The biggest road block was that everyone was already started on their implementation of the API they had the responsibility for. So asking a lot of people to re-implement and change their code was a very difficult decision to make. But I am very glad that I did and as you can read at the end of this it also payed of in an unexpected and fun way.

I will introduce the Request Manager API Pattern with a simple scenario and try my best to explain it with both text and code. The entire repository is free and hosted on BitBucket so you can clone it and play aroud with it. I will provide the link at the end.

Setting the scene

For this example we have a scenario where we are building a SaaS product. This specific product has several business specific API endpoints that each has a number of operations/methods.

Across all methods/operations we have common things that needs to be validated, primarily to do with authorization in one way or another. These are things that all implementers(developers) need to take into account and therefore a prime candidate for general implementation.

Furthermore, apart from the general validation of general properties, it is required that all operations/methods validate the incoming request with their own business specific validation for the operation/method.

The actual validation of this, we cannot generalize, as it is request-specific. But we can “force” the developer to actually validate an incoming request before working on it. And further more also make the developers implement handling of invalid and potentially “warning” requests.

Creating request and response base classes

To do so, we will make good use of inheritance and generics. First we need base classes for requests and responses.

In the example included I have actually made two levels of base classes, as I have RequestBase and ResponseBase that are at the top of the hierarchy and then in this case; ExternalRequestBase and ExternalResponseBase just below.

This opens up the possibility of using the same pattern for internal request/response objects or whatever else you would want to distinguish on. But they could absolutely be omitted.

Having base classes for request/response objects is needed in order to have a generic processor class that can handle processing of all classes that inherits from those.

Creating validation interfaces

I our example we needed to have some general validation of all incoming requests and then some custom validation for each specific operation.

In other words an interface for a generic validator, that needs to take a request-class as input and then return a validation result. Using this generic approach enables us to use the same structure for the general validation as for the specific validation.

And the return value interface:

The ValidationResultType here is just an enum with Success, Warning, Error options.

The IInputValidationResult can be implemented by any specific business area, if extra return values are needed, but we also implement a default that can be used by everyone out of the box.

One of my main issues with explaining the use of this pattern to other developers probably revolves around this area, where you can create your own implementation of the IInputValidationResult and in turn have your own properties available inside the HandleValid, HandleInvalid and HandWarningRequest as we will get to.

But this can be used to pass information from the validator to the specific request manager. We used it if we had heavy validation-logic that ran through structures with information we might as well save for the actual processing inside the request manager.

Creating main generic request manager

So speaking of which, let us look at the IRequestManager interface, as we have all the building blocks in place:

This is a very basic interface and all it does is expose a "ProcessRequest" method that takes a "Req" of type RequestBase and returns a "Resp" which is of type ResponseBase. The reason for having an interface here is that we can inject different IRequestManagers and always just call "ProcessRequest".

But this is only part of the magic, the rest happens inside the RequestManagerBase class, which is the biggest part of this pattern:

The RequestManagerBase class is abstract as it is not supposed to be instantiated, but only used as base class for actual implementations. It inherits the IRequestManager interface which we created earlier. In the constructor it takes an IRequestValidator which is used as the baseValidator, meaning it will be used to validate the common properties across all requests.

From the IRequestManager interface it gets its only public method, being ProcessRequest. This is where the main logic of this request manager API pattern resides. The logic is explained in comments inside the code itself, but basically it first validates the properties from the BaseRequest and if valid then it asks the implementing class through the abstract method GetInputRequestValidator for the validator that should be used to validate the incoming request. Once this is returned it will then call this validator and ask it to validate the request. Depending on what the validator returns it will then call either of the abstract methods HandleValidRequest, HandleWarningRequest or HandleInvalidRequest.

Finally we have the four abstract methods described in description above. By way of being abstract we force the implementing class to implement these methods. We will look at an example of what this looks like once you create a class that inherits from the RequestManagerBase class. But this concludes the RequestManagerBase class.

Implementing specific request manager

I won’t go into detail with the specific implementations of the request and response classes as that it too simple to show. But you can of course clone the repository (link is at the end of the post) and see for yourself.

But I will just show the simple implementation of a IRequestValidator that is being used in the unit tests of the project.

As you can see this implementation is very basic and oly to show the functionality. What is important is that depending on InputValidationResult either HandleValidRequest, HandleInvalidRequest or HandleWarningRequest will be called on the implementing request manager.

So let us look at the request manager from the unit testing:

As you can see in the first line, we inherit from RequestManagerBase and in doing so also specifies which actual request and response classes to be used and also which validator. By doing so all the protected override methods will automatically be generated and are ready to be filled with logic required for handling the specific requests.

This saves a lot of time for developers and it avoids everyone having to make branching logic around validation in their own API’s we have everything centralized and can hence ensure that validation is done and developers can fill the methods with logic depending on whether the request was valid, invalid or warning.

Praise from security

The full power of using this API request manager pattern can of course be a little hard to see with such simple examples. But I can finish of with a little anecdote from the project where I actually introduced this pattern. I think it initially took a little time for all to see the advantages of this pattern. It does require some extra work around creation of classes – but not too much in my opinion.

But one of the real eye openings for the other developers and a big victory for my part, was when we had to have our project security approved. Everyone feared this as it usually took very long and we were under a lot of time pressure. As part of this security assesment there was a dreaded code review to be performed which almost always caused a lot of hassle and additional changes.

The funny thing happened that, after I walked the security approver through the code base, which I had taken responsibility for structuring apart from also implementing the API request manager pattern, he actually ended up not just approving our solution but actually praising this exact pattern highly. He could clearly see the advantages of ensuring that all requests were validated by "forcing" developers to think about all the scenarios and also making it a lot easier for code reviews to spot potential loop-holes.

And that is how the department I was atteached as a freelancer ended up being applauded and given praise from the security department.

This API pattern essentially boils down to this:

Easy to do the right thing – hard to do the wrong thing

We make it easy for developers to do the right thing and hard to do the wrong thing. And in my opinion we do so without taking initiative and freedom away from the developer.

Potential improvements

This pattern is of course by no means perfect and can probably be enhanced in a number of ways. I have just listed a few things that have come to mind while watching this pattern in use with a medium project with 5-10 developers attached.

  • RequestManagerBase could take a special IRequestBaseValidator to further ensure that the BaseRequest is validated by the right validator. With current implementation it would be possible to use a "wrong" validator for BaseRequest – but it would be easy to spot on code review at least.
  • Originally I had planned for HandleValidRequest method to only take the request as input, HandleInvalidRequest to only take validation result and HandleWarningRequest was the only one who took both request and validation result as parameters. This looked a lot cleaner, but over time it just hindered the scenarios where you needed for instance a lot of work in the validator which could be re-used in the HandleValidRequest method – then suddenly we had no way of passing that. Hence I ended up giving all methods both parameters.

This concludes my introduction to the Request Manager API Pattern. I hope you find it understandable and potentialy useful. I have included link to the public repository on BitBucket, so have fun with it.

You can clone use and play around with the code from here: Bitbucket.org/jmreiche/requestmanager

Feel free to reach out to me with questions or suggestions for improvements.