Written by: Dylan Goldsborough

While we love using Java in Picnic, we also adore Python. Over the years, our tech landscape has been enriched with numerous Python services, for instance to distribute product information across our backend ecosystem. We started small, adopting aiohttp as our HTTP server framework of choice. For many years, our Python services flourished as part of the bigger Picnic machine.

Growth

Recently, our ambition grew. We wanted more Python microservices, and fast!

On the one hand, we were accelerating our Data Science capabilities. Machine learning models are hugely important in our supply chain. Developing a solid model is one thing, but if forecasts cannot make their way to systems that can utilize them their impact cannot be realized. As a result we were striving to accelerate the process of creating Python microservices to host these models.

On the other hand, we had a need to make the creation of services more accessible. Working in tandem with our software engineers there are analysts who drive our operations, designing our operational flows and analyzing them to ever improve them. These teams are generating a lot of valuable proof of concept projects written primarily in Python. To support these teams we wanted to unlock the simple creation of services for people outside of the tech team.

This brings us to an important aspect of aiohttp: while it is a small and fast framework it is also verbose. In Picnic we are heavy users of Swagger, and generating and maintaining the Swagger documentation was proving quite the task. (De-)serialization of requests and responses was also adding to the verbosity, in addition to the complexity of gathering inputs from the API path, query parameters and request body.

As such, we set off on a quest to find a new framework. We were looking for an async framework with comparable performance to aiohttp, Swagger integrations, and the ability to remove some of the serialization burden.

Enter FastAPI on our tech radar! FastAPI is a high performance web framework with a focus on ease of implementation and conciseness. It fits all our requirements and more.

Working with FastAPI

Migrating aiohttp to FastAPI was a pleasant surprise in terms of effort, in part due to the aforementioned focus on conciseness. Let us say that we have a POST request that uses some URI path variables and receives a request body. In aiohttp it would look something like

from aiohttp import webfrom model.foo_model import FooModel
from service.foo_service import FooServiceclass RequestHandler(web.View):
    async def post(self):
        self.service = self.request.app[FooService]
        self.api_version = request.match_info.get("api_version", 1)
        self.name = request.match_info.get("name", None)
        try:
            body = await self.request.json()
            foo_model = FooModel(
                name=self.name, version=self.api_version, **body
            )
        except ValueError as ex:
            raise web.HTTPBadRequest(
                reason=f"Failed to process request body."
            ) from ex
        await self.service.persist(foo_model)

to set up a simple handler. Note that here we persist the service instance in the application state, and that we pull this instance out when handling our request. In FastAPI we can write

from model.foo_model import FooModel
from service.foo_service import FooService@router.post(
    "{api_version}/foo/{name}",
    status_code=status.HTTP_204_NO_CONTENT,
)
async def handle_request(
    api_version: int,
    name: str,
    foo_model: FooModel,
    service: FooService = Depends(FooService),
):
    await service.persist(foo_model)

to do the equivalent. Here our deserialization is done for us, as the desired type is given in the function signature. In addition, FastAPI is smart enough to recognize that some of the input parameters are URI parameters, so the implementation does not need to specify that api_version and name should be pulled from the path whereas foo_model should be deserialized from the request body.

It does not end here: FastAPI will take the defined routes and build and host a Swagger page for us! We can add extra documentation to this Swagger page from our code by tagging routers and adding descriptions to fields by wrapping their default value in a fastapi.Query. FastAPI also adds definitions of your DTOs to the Swagger page, and creates list fields for enums.

Another aspect we appreciate in FastAPI is their support for dependency injection. Services can be used when handling a request by using the fastapi.Depends object supplied with the class name. Dependency injection can also be applied in a service with some extra work: we define a DependsOnClient that queries from stateful clients in the application state that you have set up during your application startup. When receiving a request the dependencies are resolved and your service instance is instantiated.

class FooClient:
    ...class FooService:
    def __init__(
        self, client: Fooclient = DependsOnClient(FooClient)
    ):
        ...@router.delete("/{name}", status_code=status.HTTP_200_OK)
async def delete_view(
    api_version: int,
    name: str,
    service: SyncService = Depends(FooService),
):
    await service.delete_view_definition(name, api_version)

Caveats

In our migration towards FastAPI we did hit a few snags. While the pydantic-powered serialization is powerful it does come at a performance cost. In one of our services we serve large JSON-responses. We defined the response class as JSONResponse, and subsequently saw a major drop in performance compared to our previous implementation that uses json.dumps(). Using ORJSONResponse solves this performance issue, as it uses the fast orjson library instead of pydantic in serializing the response.

Our second surprise came in the form of how cancelled requests are handled. In aiohttp, if the requester prematurely disconnects while waiting for the response to a request the task handling the request is cancelled. In FastAPI this is not the case: the request is finished, and only when trying to communicate back the response an exception is raised. For us, this meant that a cancelled POST request we did not expect to have changed the state of a service did have an effect. This is in accordance with the ASGI framework, but did come as a surprise.

Finally, we found ourselves limited due to the dependency injection framework being tied into the processing of a request. In Picnic we do service-to-service communication through a message broker. As a result, some services implement both a REST and AMQP interface. To process requests coming in through the broker we had to instantiate a long-lived service instance to wire to the incoming message handler, which breaks the FastAPI dependency injection pattern.

Results

Caveats aside, we are now happily migrated to FastAPI. We are benefitting from the faster setup and lower maintenance around Swagger documentation. We see that setting up a new service takes less time. Performance wise we saw no significant difference, which was in line with our goals and expectations.