Using the never type in typescript

on
0

Introduction to OpenAPI: Empowering Your REST API

on

In the ever-expanding landscape of web services and applications, a well-designed and thoroughly documented REST API is a cornerstone for success. Whether you're building an API to serve internal needs or opening your services to the wider world, clarity, consistency, and accessibility are needed for your API to be a great API. OpenAPI is a very common tool to make your API clear, consistent and accessible.

OpenAPI benefits

  1. Documentation: With OpenAPI, you can effortlessly generate interactive and user-friendly documentation for your API. This documentation serves as a vital resource for both your development team and external developers who wish to integrate with your API.
  2. Code Generation: OpenAPI allows for the automatic generation of client libraries and SDKs in various programming languages. This accelerates the development process for those who want to interact with your API.
  3. Testing and Validation: OpenAPI tools often include built-in validation features, helping to ensure that API requests and responses conform to the specified schema, reducing errors and improving reliability.
  4. Collaboration: By using OpenAPI, different teams, such as front-end and back-end developers, testers, and documentation writers, can collaborate more effectively. The shared OpenAPI specification serves as a central reference, ensuring everyone is on the same page.
  5. Ecosystem Support: OpenAPI enjoys a rich ecosystem of tools and integrations, enabling you to leverage a wide array of resources to enhance and manage your API.
  6. Community and Industry Adoption: OpenAPI is widely adopted in the industry, making it a well-recognized standard. This standardization enhances consistency and understanding, making your API more accessible and user-friendly.

An OpenAPI example

Imagine we have a Laravel API with a hello world API call. The controller would be something like this:

class ExampleController extends Controller
{
    public function helloWorld(string $name)
    {
    	return response()->json(['message' => 'Hello ' . $name]);
    }
}
Then we can write an OpenAPI spec like this to document our API: 

openapi: 3.0.1
info:
  title: 'The name of my API with a version number'
  version: 1.0.0
paths:
  '/hello/{name}':
    get:
      description: 'Gives a hello world message'
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                required:
                  - message
                type: object
                properties:
                  message:
                    type: string
                    example: "Hello World"
      parameters:
        -
          name: name
          in: path
          required: true
          schema:
            minLength: 1
            type: string

servers:
  -
    url: 'https://api.example.com/'
    description: 'production server'
  -
    url: 'http://localhost/api/'
    description: 'local development'

As you can see it's a long document that defines the entire API call. The {name} placeholder in the url is defined as a string with minimum length 1. We tell the this API call will return an object with a message property of type string that will always be there (that's why it says required: ['message']). We have 2 servers that we can test our API on. Because we probably get the same object back or have cyclic object references it's better to reference a set of defined objects instead. We will get an OpenAPI spec like this:


openapi: 3.0.1
info:
  title: 'The name of my API with a version number'
  version: 1.0.0
paths:
  '/hello/{name}':
    get:
      description: 'Gives a hello world message'
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HelloWorld'
      parameters:
        -
          name: name
          in: path
          required: true
          schema:
            minLength: 1
            type: string

servers:
  -
    url: 'https://api.example.com/'
    description: 'production server'
  -
    url: 'http://localhost/api/'
    description: 'local development'
components:
  schemas:
    HelloWorld:
      required:
        - message
      type: object
      properties:
        message:
          type: string
          example: "Hello World"

Right now it seems a bit like overkill because the controller is only a few lines and this specification is 40 lines of YAML, but it becomes more clear the moment we put this openapi spec on this website: https://editor.swagger.io/ (do not worry, it does only client-side communication)

Wow! That looks a lot nicer and makes your API calls much more visible. This API viewer is made with a library called swagger ui that reads the OpenAPI file and displays the full API. It also has a 'try it out' button to test the API calls. If you do not like swagger UI, you can also display your Rest API with redoc for example (see screenshot) or import it in Postman. You can find a long extensive list of OpenAPI tools on https://openapi.tools/. Think mock servers, documentation, generated API clients, etc.


If you want a full spec of what can be described you can look at the website of OpenAPI. As you can see it's very thorough and extensive, but the more you specify, the better any tooling can handle your API. Doing this manually for a full API is very time consuming and error-prone for changes. So what tooling can we use to speed up creating an OpenAPI spec?

Use annotations or attributes

One of the oldest helping tools is the composer package: zircote/swagger-php. This library adds specific phpdoc docblocks or PHP attributes to add information:

use OpenApi\Attributes as OA;

#[OA\Schema]
class Message {
    public string $message;
}

class ExampleController extends Controller
{
    #[OA\Get(path: '/hello/{name}')]
    #[OA\Response(response: 200, description: 'Return message', @OA\JsonContent(ref="#/components/schemas/Message"))]
    #[OA\Response(response: 401, description: 'Not allowed')]
    public function helloWorld(string $name)
    {
    	return response()->json(['message' => 'Hello ' . $name]);
    }
}

This looks nice, but the big problems is that you not only need to have some documentation knowledge op OpenAPI, you also need to have knowledge of the attributes of zircote/swagger-php that can be used and how they are being converted into OpenAPI specs. It still writes just the documentation and does not test if the API call and the OpenAPI spec match. The only benefit is that it removes documentation if you remove the API call or field property.

It's also possible that your code is harder to read if your controller action has like 8 attributes written on it, since these attributes can confuse or distract developers from the actual code.

Validating API calls

The OpenAPI spec can also be used to validate API calls. There is also a composer package for this: 
This package can validate the request and can also validate if the response is valid. For validating the response it still needs to know what was the request. This package uses PSR Requests, so you require to convert the framework request into a PSR-7 request. In Laravel we could use the Symfony PSR bridge directly to convert to a PSR7 request and/or response since the validator can only validate PSR7 requests and responses. We could also use it in an integration test that way:

use Illuminate\Http\Request;
use Illuminate\Http\Response;
use League\OpenAPIValidation\PSR7\OperationAddress;
use League\OpenAPIValidation\PSR7\ValidatorBuilder;
use Nyholm\Psr7\Factory\Psr17Factory;
use Symfony\Bridge\PsrHttpMessage\Factory\PsrHttpFactory;

class HelloWorldApiCallController extends WebTestCase
{
    public function testHelloWorld()
    {
        $testResponse = $this->get('/hello/world');
        $testResponse->assertJsonFragment(['message' => 'Hello world']);
        $this->assertApiCallIsDefinedCorrectly(app('request'), $testResponse->response);
    }
    
    private function toPsrRequest(Request $request): ServerRequestInterface
    {
        $psr17Factory = new Psr17Factory();
        $psrHttpFactory = new PsrHttpFactory($psr17Factory, $psr17Factory, $psr17Factory, $psr17Factory);
        return $psrHttpFactory->createServerRequest($request);
    }
    
    private function toPsrResponse(Response $response): ResponseInterface
    {
        $psr17Factory = new Psr17Factory();
        $psrHttpFactory = new PsrHttpFactory($psr17Factory, $psr17Factory, $psr17Factory, $psr17Factory);
        return $psrHttpFactory->createResponse($request);
    }
    
    private function assertApiCallIsDefinedCorrectly(Request $request, Response $response)
    {
    	$psrRequest = $this->toPsrRequest($request);
        $psrResponse = $this->toPsrResponse($response);
        $yamlFile = __DIR__ . '/../fixtures/openapi-spec.yaml';
        
        $responseValidator = (new ValidatorBuilder)->fromYamlFile($yamlFile)->getResponseValidator();
        $operation = new OperationAddress($psrRequest->getUri()->getPath(), strtolower($psrRequest->getMethod()));
        $responseValidator->validate($operation, $psrResponse);
        
        $requestValidator = (new ValidatorBuilder)->fromYamlFile($yamlFile)->getServerRequestValidator();
        $requestValidator->validate($psrRequest);
        
    }
}

You could move these private methods to a trait, so more tests can use it. If you make more integration tests, for example a test that you should see a 401 if you are not logged in, then you will get a validation error that the expected error was not defined, so the more tests you have the more accurate your OpenAPI spec should become. Still it's manual labour, so let's see we can automate it.

Auto-generate complete spec: API Platform

As for generating an entire OpenAPI spec and let the programmer never have to think about it, we could use tools that generate the entire specification and Rest API. The biggest competitor here is the API Platform. Even though the core of the API Platform is built framework agnostic, it leans so much into Symfony, that it can only be used in a Symfony application.

The API Platform is an amazing tool to start thinking API-first and my Apie library was heavily inspired by it. The OpenAPI spec generated by the API Platform is very limited and inaccurate. The internal code quality is also very bad where you can only extend it by decorating the OpenapiFactory class.

There are a few quirks with the generated specs that I advise other developers not to lean too much on the generated spec:
  • I tested the generated spec on mock servers and found out that since API platform does not mark fields as required for return values that mock servers will randomly pick fields or not.
  • Fields marked as required for POST, PUT and PATCH are read from validation for the Symfony Validator component. It ignores constructor arguments completely.
  • Constructor arguments not mapped to a property are completely missing.
  • Enums are only working in some cases (when mapped as a doctrine column). In some cases it will be displays as an object with name and value property.
  • No support for union and intersection typehints.
  • No way to tell API platform how to map your custom value objects.
In a nutshell: API Platform works best if you create your API resources as anemic Doctrine entities (see my article about the value of value objects) or don't care about it's accuracy.  And of course: they keep improving it, so maybe the above list will be fixed in the future.

Apie: generate the OpenAPI spec and validate it internally

With the mistakes in the API Platform, I decided I should do better than what the API platform had to offer. I don't thinkthe API Platform was doing a bad job, it's just a very hard challenge to get an accurate OpenAPI spec automatically.
So how did I approach making it generate an OpenAPI spec:
  1. I made an apie/schema-provider package to create the schema and used a chain of responsibility to make it extendible (see my article about making libraries extendible)
  2. I took a TDD approach first and tried to make it get correct JSON schema's for any primitive and the Apie defined objects (you find this in my first article about the start of my Apie project). I even made use cases for union and intersection typehints.
  3. The next step is let it generate a complete OpenAPI spec using apie/schema-provider for the typehints. I needed a ComponentsBuilder because of dealing with circular type definitions and to make sure almost all definitions are references to the components section in the OpenAPI spec.
  4. The next step was writing integration tests for API calls. This overlaps with what I did in the article I wrote about handling framework agnostic integration tests.
  5. So the next step I did was adding the validation of my API calls in the integration tests. The result: 70 failing integration tests.....
So I fixed the tests with some small refactoring. It was quite interesting and it also gives it a little bit more thought how you should use OpenAPI too.
So what did I find after adding validation:
  • I have an integration test that submits the string "12" for a integer typehint to test if it casts to the integer 12. However the OpenAPI said that you should send the integer 12. I made the test ignore the request validation, so it's an undocumented 'feature' of the API.
  • For every class it could create up to 3 component references in OpenAPI: one for POST, one for GET and one for PUT/PATCH. However you can not have nullable references, so I had to modify it to 6 possible variations of an object.
  • I misread the documentation of oneOf: I thought it's okay if it matches more than one, but in reality it should only be allowed to match one, not two. This proved some difficulty for polymorphic objects and union types. Some typehints can still give trouble and I have no solution for it. For example function setField(int|float $number) will give problems with validating.
While it took longer than expected I'm quite happy with being able to test and validate that the OpenAPI spec that is being generated is valid. I've worked for years with OpenAPI and I absolutely can't live without it when writing a proper Rest API that does not suck so much.



Comments

Post a Comment