Skip to content

Healthchecks#

A common pattern for healthchecks is a low-cost HTTP endpoint, FastStream implements this feature and allows you to create liveness and readiness probes using ASGI feature.

Liveness Probe#

Liveness probes determine when to restart a container. For example: liveness probes could catch a deadlock when an application is running but unable to make progress.

Readiness Probe#

Readiness probes determine when a container is ready to start accepting traffic. This is useful when waiting for an application to perform time-consuming initial tasks, such as establishing network connections, loading files, and warming caches.

Let's implement readiness and liveness probes#

Structure of our app: 

dummy
├── docker-compose.yml
├── Dockerfile
├── main.py

This example shows how to implement liveness and readiness probes in FastStream. Readiness probe checks connection to Redis, RabbitMQ and Postgres. Liveness probe checks that app just works.

main.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
import asyncio
import logging
from typing import Awaitable, Callable, Any

import asyncpg
import redis.asyncio as redis
import uvicorn
from faststream import FastStream
from faststream.asgi import AsgiResponse, get
from faststream.rabbit import RabbitBroker


@get
async def liveness(scope: dict[str, Any]) -> AsgiResponse:
    return AsgiResponse(b"", status_code=204)


def readiness(
    broker: RabbitBroker,
    redis_connection: redis.Redis,
    postgres_connection: asyncpg.Pool,
) -> Callable[[dict[str, Any]], Awaitable[AsgiResponse]]:
    healthy_response = AsgiResponse(b"", 204)
    unhealthy_response = AsgiResponse(b"", 500)

    @get
    async def func(scope: dict[str, Any]) -> AsgiResponse:
        try:
            await redis_connection.ping()
        except (redis.ConnectionError, Exception):
            logging.exception("Redis not ready")
            return unhealthy_response

        try:
            await broker.ping(timeout=5.0)
        except Exception:
            logging.exception("RabbitMQ not ready")
            return unhealthy_response

        try:
            await postgres_connection.fetchval("SELECT 1")
        except (asyncpg.exceptions.PostgresConnectionError, Exception):
            logging.exception("Postgres not ready")
            return unhealthy_response

        return healthy_response

    return func


async def main() -> None:
    redis_connection = redis.Redis(host="redis", port=6379)

    postgres_connection = await asyncpg.create_pool(
        "postgresql://user:password@postgres/postgres"
    )

    broker = RabbitBroker("amqp://guest:guest@rabbitmq:5672/")
    app = FastStream(broker)

    asgi_routes = [
        ("/internal/alive", liveness),
        ("/internal/ready", readiness(broker, redis_connection, postgres_connection)),
    ]

    uvicorn_config = uvicorn.Config(
        app.as_asgi(asgi_routes),
        host="0.0.0.0",
        port=8000,
    )
    server = uvicorn.Server(uvicorn_config)
    await server.serve()


if __name__ == "__main__":
    asyncio.run(main())

Pack to Dockerfile#

And let's build a simple Dockerfile for our app:

Dockerfile
1
2
3
4
5
6
7
FROM python:3.12-slim

RUN apt-get update && apt-get install -y curl

WORKDIR /app
RUN pip install faststream[rabbit] uvicorn redis asyncpg
COPY main.py /app

Docker-compose with healtcheck#

Docker-compose doesn't allow you to realize the full power of trials, but it's enough to increase the stability of your application.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
services:
  rabbitmq:
    image: rabbitmq:3.8-management
    container_name: rabbitmq-local
    environment:
      RABBITMQ_DEFAULT_USER: guest
      RABBITMQ_DEFAULT_PASS: guest
    ports:
      - "15672:15672"
    healthcheck:
      test: [ "CMD", "rabbitmq-diagnostics", "-q", "ping"]

  redis:
    image: redis:7.2
    container_name: redis-local
    healthcheck:
      test: [ "CMD", "redis-cli", "--raw", "incr", "ping" ]

  postgres:
    image: postgres:14
    container_name: postgres-local
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
    healthcheck:
      test: [ "CMD", "pg_isready", "-U", "user" ]

  app:
    build: .
    container_name: app-local
    command: bash -c "python main.py"
    ports:
      - "8000:8000"
    volumes:
      - ./main.py:/app/main.py
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/internal/ready"]
      interval: 60s
      timeout: 5s
      retries: 5
      start_period: 5s
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
      rabbitmq:
        condition: service_healthy

Kubernetes deployment#

But if you use k8s you can use the full power of this feature because you can use live and ready probes together. This is an example of deployment with liveness and readiness probes:

faststream-deployment.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
apiVersion: apps/v1
kind: Deployment
metadata:
  name: dummy-app
  labels:
    app: dummy-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dummy-app
  template:
    metadata:
      labels:
        app: dummy-app
    spec:
      containers:
        - name: dummy-app
          image: "dummy-app:latest"
          command: ["/bin/sh", "-c", 'python3 main.py']
          ports:
            - containerPort: 8000
          livenessProbe:
            httpGet:
              path: /internal/alive
              port: 8000
            initialDelaySeconds: 5
            periodSeconds: 20
          readinessProbe:
            httpGet:
              path: /internal/ready
              port: 8000
            periodSeconds: 60