Ingress in Kubernetes exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. By setting rules, it routes requests to appropriate services (precisely requests are sent to individual Pods by Ingress Controller). Rules can be set up dynamically and I find it’s more efficient compared to traditional reverse proxy.

Traefik is a modern HTTP reverse proxy and load balancer and it can be used as a Kubernetes Ingress Controller. Moreover it supports other providers, which are existing infrastructure components such as orchestrators, container engines, cloud providers, or key-value stores. To name a few, Docker, Kubernetes, AWS ECS, AWS DynamoDB and Consul are supported providers. With Traefik, it is possible to configure routing dynamically. Another interesting feature is Forward Authentication where authentication can be handled by an external service. In this post, it’ll be demonstrated how path-based routing can be set up by Traefik with Docker. Also a centralized authentication will be illustrated with the Forward Authentication feature of Traefik.

How Traefik works

Below shows an illustration of internal architecture of Traefik.

The Traefik website explains workflow of requests as following.

  • Incoming requests end on entrypoints, as the name suggests, they are the network entry points into Traefik (listening port, SSL, traffic redirection…).
  • Traffic is then forwarded to a matching frontend. A frontend defines routes from entrypoints to backends. Routes are created using requests fields (Host, Path, Headers…) and can match or not a request.
  • The frontend will then send the request to a backend. A backend can be composed by one or more servers, and by a load-balancing strategy.
  • Finally, the server will forward the request to the corresponding microservice in the private network.

In this example, a HTTP entrypoint is setup on port 80. Requests through it are forwarded to 2 web services by the following frontend rules.

  • Host is k8s-traefik.info and path is /pybackend
  • Host is k8s-traefik.info and path is /rbackend

As the paths of the rules suggest, requests to /pybackend are sent to a backend service, created with FastAPI. If the other rule is met, requests are sent to the Rserve backend service. Note that only requests from authenticated users are fowarded to relevant backends and it is configured in frontend rules as well. Below shows how authentication is handled.

Traefik setup

Here is the traefik service defined in the compose file of this example - the full version can be found here.

 1version: "3.7"
 2services:
 3  traefik:
 4    image: "traefik:v1.7.19"
 5    networks:
 6      - traefik-net
 7    command: >
 8      --docker
 9      --docker.domain=k8s-traefik.info
10      --docker.exposedByDefault=false
11      --docker.network=traefik-net
12      --defaultentrypoints=http
13      --entrypoints="Name:http Address::80"
14      --api.dashboard      
15    ports:
16      - 80:80
17      - 8080:8080
18    labels:
19      - "traefik.frontend.rule=Host:k8s-traefik.info"
20      - "traefik.port=8080"
21    volumes:
22      - /var/run/docker.sock:/var/run/docker.sock
23...
24networks:
25  traefik-net:
26    name: traefik-network

In commands, the Docker provider is enabled (--docker) with a custom domain name (k8s-traefik.info). A dedicated network is created and it is used for this and the other services (trafic-net). A single HTTP entrypoint is enabled as the default entrypoint. Finally monitoring dashboard is enabled (--api.dashboard). In lables, it is set to be served via the custom domain (hostname) - port 80 is for individual services while 8080 is for the monitoring UI.

It is necessary to have a custom hostname when setting up rules that include multiple hosts or enabling a HTTPS entrypoint. Although neither is discussed in this post, a custom domain (k8s-traefik.info), which is accessible only in local environment, is added - another post may come later. The location of hosts file is

  • Windows - %WINDIR%\System32\drivers\etc\hosts or C:\Windows\System32\drivers\etc\hosts
  • Linux - /etc/hosts

And the following entry is added.

1# using a virtual machine
2<VM-IP-ADDRESS>    k8s-traefik.info
3# or in the same machine
40.0.0.0            k8s-traefik.info

In order to show how routes are configured dynamically, only the Traefik service is started as following.

1docker-compose up -d traefik

When visiting the monitoring UI via http://k8s-traefik.info:8080/dashboard, it’s shown that no frontend and backend exists in the docker provider tab.

Services

The authentication service is just checking if there’s an authorization header and the JWT value is foobar. If so, it returns 200 response so that requests can be forward to relevant backends. The source is shown below.

 1import os
 2from typing import Dict, List
 3from fastapi import FastAPI, Depends, HTTPException
 4from pydantic import BaseModel
 5from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
 6from starlette.requests import Request
 7from starlette.status import HTTP_401_UNAUTHORIZED
 8
 9app = FastAPI(title="Forward Auth API", docs_url=None, redoc_url=None)
10
11## authentication
12class JWTBearer(HTTPBearer):
13    def __init__(self, auto_error: bool = True):
14        super().__init__(scheme_name="Novice JWT Bearer", auto_error=auto_error)
15
16    async def __call__(self, request: Request) -> None:
17        credentials: HTTPAuthorizationCredentials = await super().__call__(request)
18
19        if credentials.credentials != "foobar":
20            raise HTTPException(HTTP_401_UNAUTHORIZED, detail="Invalid Token")
21
22
23## response models
24class StatusResp(BaseModel):
25    status: str
26
27
28## service methods
29@app.get("/auth", response_model=StatusResp, dependencies=[Depends(JWTBearer())])
30async def forward_auth():
31    return {"status": "ok"}

The service is defined in the compose file as following.

 1...
 2  forward-auth:
 3    image: kapps/trafik-demo:pybackend
 4    networks:
 5      - traefik-net
 6    depends_on:
 7      - traefik
 8    command: >
 9      forward_auth:app
10      --host=0.0.0.0
11      --port=8000
12      --reload      
13...

The Python service has 3 endpoints. The app’s title and path value are returned when requests are made to / and /{p} - a variable path value. Those to /admission calls the Rserve service and relays results from it - see the Rseve service section for the request payload. Note that an authorization header is not necessary between services.

 1import os
 2import httpx
 3from fastapi import FastAPI
 4from pydantic import BaseModel, Schema
 5from typing import Optional
 6
 7APP_PREFIX = os.environ["APP_PREFIX"]
 8
 9app = FastAPI(title="{0} API".format(APP_PREFIX), docs_url=None, redoc_url=None)
10
11## response models
12class NameResp(BaseModel):
13    title: str
14
15
16class PathResp(BaseModel):
17    title: str
18    path: str
19
20
21class AdmissionReq(BaseModel):
22    gre: int = Schema(None, ge=0, le=800)
23    gpa: float = Schema(None, ge=0.0, le=4.0)
24    rank: str = Schema(None)
25
26
27class AdmissionResp(BaseModel):
28    result: bool
29
30
31## service methods
32@app.get("/", response_model=NameResp)
33async def whoami():
34    return {"title": app.title}
35
36
37@app.post("/admission")
38async def admission(*, req: Optional[AdmissionReq]):
39    host = os.getenv("RSERVE_HOST", "localhost")
40    port = os.getenv("RSERVE_PORT", "8000")
41    async with httpx.AsyncClient() as client:
42        dat = req.json() if req else None
43        r = await client.post("http://{0}:{1}/{2}".format(host, port, "admission"), data=dat)
44        return r.json()
45
46
47@app.get("/{p}", response_model=PathResp)
48async def whichpath(p: str):
49    print(p)
50    return {"title": app.title, "path": p}

The Python service is configured with lables. Traefik is enabled and the same docker network is used. In frontend rules,

  • requests are set to be forwarded if host is k8s-traefik.info and path is /pybackend - PathPrefixStrip is to allow the path and its subpaths.
  • authentication service is called and its address is http://forward-auth:8080/auth.
  • authorization header is set to be copied to request - it’s for adding a custom header to a request and this label is mistakenly added.

If the frontend rules pass, requests are sent to pybackend backend on port 8000.

 1...
 2  pybackend:
 3    image: kapps/trafik-demo:pybackend
 4    networks:
 5      - traefik-net
 6    depends_on:
 7      - traefik
 8      - forward-auth
 9      - rbackend
10    command: >
11      main:app
12      --host=0.0.0.0
13      --port=8000
14      --reload      
15    expose:
16      - 8000
17    labels:
18      - "traefik.enable=true"
19      - "traefik.docker.network=traefik-net"
20      - "traefik.frontend.rule=Host:k8s-traefik.info;PathPrefixStrip:/pybackend"
21      - "traefik.frontend.auth.forward.address=http://forward-auth:8000/auth"
22      - "traefik.frontend.auth.forward.authResponseHeaders=Authorization"
23      - "traefik.backend=pybackend"
24      - "traefik.port=8000"
25    environment:
26      APP_PREFIX: "Python Backend"
27      RSERVE_HOST: "rbackend"
28      RSERVE_PORT: "8000"
29...

R Service

whoami() that returns the service name is executed when a request is made to the base path (/) - see here for details. To /admission, an admission result of a graduate school is returned by fitting a simple logistic regression. The result is based on 3 fields - GRE (Graduate Record Exam scores), GPA (grade point average) and prestige of the undergraduate institution. It’s from UCLA Institute for Digital Research & Education. If a field is missing, the mean or majority level is selected.

 1DAT <- read.csv('./binary.csv')
 2DAT$rank <- factor(DAT$rank)
 3
 4value_if_null <- function(v, DAT) {
 5  if (class(DAT[[v]]) == 'factor') {
 6    tt <- table(DAT[[v]])
 7    names(tt[tt==max(tt)])
 8  } else {
 9    mean(DAT[[v]])
10  }
11}
12
13set_newdata <- function(args_called) {
14  args_init <- list(gre=NULL, gpa=NULL, rank=NULL)
15  newdata <- lapply(names(args_init), function(n) {
16    if (is.null(args_called[[n]])) {
17      args_init[[n]] <- value_if_null(n, DAT)
18    } else {
19      args_init[[n]] <- args_called[[n]]
20    }
21  })
22  names(newdata) <- names(args_init)
23  lapply(names(newdata), function(n) {
24    flog.info(sprintf("%s - %s", n, newdata[[n]]))
25  })
26  newdata <- as.data.frame(newdata)
27  newdata$rank <- factor(newdata$rank, levels = levels(DAT$rank))
28  newdata
29}
30
31admission <- function(gre=NULL, gpa=NULL, rank=NULL, ...) {
32  newdata <- set_newdata(args_called = as.list(sys.call()))
33  logit <- glm(admit ~ gre + gpa + rank, data = DAT, family = "binomial")
34  resp <- predict(logit, newdata=newdata, type="response")
35  flog.info(sprintf("resp - %s", resp))
36  list(result = resp > 0.5)
37}
38
39whoami <- function() {
40    list(title=sprintf("%s API", Sys.getenv("APP_PREFIX", "RSERVE")))
41}

The Rserve service is configured with lables as well.

 1...
 2  rbackend:
 3    image: kapps/trafik-demo:rbackend
 4    networks:
 5      - traefik-net
 6    depends_on:
 7      - traefik
 8      - forward-auth
 9    command: >
10      --slave
11      --RS-conf /home/app/rserve.conf
12      --RS-source /home/app/rserve-src.R      
13    expose:
14      - 8000
15    labels:
16      - "traefik.enable=true"
17      - "traefik.docker.network=traefik-net"
18      - "traefik.frontend.rule=Host:k8s-traefik.info;PathPrefixStrip:/rbackend"
19      - "traefik.frontend.auth.forward.address=http://forward-auth:8000/auth"
20      - "traefik.frontend.auth.forward.authResponseHeaders=Authorization"
21      - "traefik.backend=rbackend"
22      - "traefik.port=8000"
23    environment:
24      APP_PREFIX: "R Backend"
25...

In order to check dynamic routes configuration, the Python service is started as following. Note that, as it depends on the authentication and Rserve service, these are started as well.

1docker-compose up -d pybackend

Once those services are started, the frontends/backends of the Python and Rserve services appear in the monitoring UI.

Below shows some request examples.

 1#### Authentication failure responses from authentication server
 2http http://k8s-traefik.info/pybackend
 3# HTTP/1.1 403 Forbidden
 4# ...
 5# {
 6#     "detail": "Not authenticated"
 7# }
 8
 9http http://k8s-traefik.info/pybackend "Authorization: Bearer foo"
10# HTTP/1.1 401 Unauthorized
11# ...
12# {
13#     "detail": "Invalid Token"
14# }
15
16#### Successful responses from Python service
17http http://k8s-traefik.info/pybackend "Authorization: Bearer foobar"
18# {
19#     "title": "Python Backend API"
20# }
21
22http http://k8s-traefik.info/pybackend/foobar "Authorization: Bearer foobar"
23# {
24#     "path": "foobar",
25#     "title": "Python Backend API"
26# }
27
28#### Succesesful responses from Rserve service
29http http://k8s-traefik.info/rbackend "Authorization: Bearer foobar"
30# {
31#     "title": "R Backend API"
32# }
33
34#### Successful responses from requests to /admission
35echo '{"gre": 600, "rank": "1"}' \
36  | http POST http://k8s-traefik.info/rbackend/admission "Authorization: Bearer foobar"
37# {
38#     "result": true
39# }
40
41echo '{"gre": 600, "rank": "1"}' \
42  | http POST http://k8s-traefik.info/pybackend/admission "Authorization: Bearer foobar"
43# {
44#     "result": true
45# }

The HEALTH tab of the monitoring UI shows some request metrics. After running the following for a while, the page is updated as shown below.

1while true; do echo '{"gre": 600, "rank": "1"}' \
2  | http POST http://k8s-traefik.info/pybackend/admission "Authorization: Bearer foobar"; sleep 1; done