HTTP Compression

here is an "usual" request flow:

lets start from the back

we have some service and it returns 200kb of data

it does not matter how fast our service is, even if it will return all data from memory, there will be some time taken to transmit bytes over the wire

for a reference we have something like this:

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.MapGet("/kb/{kb:int}", (int kb) => new String('a', kb * 1024));
app.Run();

so we have some endpoint to perform tests

at beginning we are going to check if thats work at al:

curl localhost:5000/kb/2

it should return us a bunch of a letters

now lets check the response size:

curl -s -o /dev/null localhost:5000/kb/2 -w '%{size_download}\n'

this one will print 2048, exactly the number of bytes we have asked for

now lets check the same thing but also measure total time:

curl -s -o /dev/null localhost:5000/kb/2 -w 'bytes: %{size_download} took: %{time_total}\n'

in my case it returned:

bytes: 2048 took: 0.009286

so we can say that localy we have downloaded 2kb of data in 9ms

next we are going to introduce compression

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddResponseCompression(); // POI
var app = builder.Build();
app.UseResponseCompression(); // POI
app.MapGet("/kb/{kb:int}", (int kb) => new String('a', kb * 1024));
app.Run();

next measurement, but now we are going to request 200kb of data:

curl -s -o /dev/null localhost:5000/kb/200 -w 'bytes: %{size_download} took: %{time_total}\n'

and we receive:

bytes: 204800 took: 0.176370

what is interesting, concretely in this case it took 176ms, localy

now lets try the same, but with compression:

curl -H 'Accept-Encoding: gzip' -s -o /dev/null localhost:5000/kb/200 -w 'bytes: %{size_download} took: %{time_total}\n'

and we got

bytes: 935 took: 0.075951

we can not say that it is a dealbreaker, but traffic reduced houndrets of times, and that affected not only overall response time which took x2 times less, but also it will affect traffic consts

How does it work

If client can work with compressed responses, he will add Accept-Encoding header to requests with comma separated list of supported compression algorithms.

The server will add Content-Encoding header to response notifying client about choosen compressim algorithm

Actualy it works the same way as "content negotiation" but for compression

In practice it looks like this:

If we are sending requests without asking for compression we will receive raw response (response does not contain Content-Encoding header and response body is raw/readable)

curl -s -i localhost:5000/kb/1

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Date: Sun, 16 Oct 2022 12:14:57 GMT
Server: Kestrel
Transfer-Encoding: chunked

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...

If in request we will say that we may work with zip by adding Accept-Encoding header with gzip value

curl -i localhost:5000/kb/1 -H 'Accept-Encoding: gzip'

we will receive something like that:

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Date: Sun, 16 Oct 2022 12:17:21 GMT
Server: Kestrel
Content-Encoding: gzip
Transfer-Encoding: chunked
Vary: Accept-Encoding

Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.

From one side, we can tell that response is compressed because there is Content-Encoding: gzip header, but what is event more interesting, curl warns us that response seems to be some binary data and it can not be handled.

To "fix" this, we can add --compressed flag to cul command

curl -i localhost:5000/kb/1 -H 'Accept-Encoding: gzip' --compressed

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Date: Sun, 16 Oct 2022 12:19:57 GMT
Server: Kestrel
Content-Encoding: gzip
Transfer-Encoding: chunked
Vary: Accept-Encoding

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa....

Novadays brotly is more prefered compression algorithms:

curl -i localhost:5000/kb/1 -H 'Accept-Encoding: br' --compressed

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Date: Sun, 16 Oct 2022 12:24:57 GMT
Server: Kestrel
Content-Encoding: br
Transfer-Encoding: chunked
Vary: Accept-Encoding

curl: (61) Unrecognized content encoding type. libcurl understands deflate, gzip content encodings.

From a response we can tell that server successfully processed request and responded with response compressed with brotly, but curl can not handle it

Offtopic: how can we check if server uses compression?

using Microsoft.AspNetCore.HttpLogging;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddResponseCompression();
builder.Services.AddHttpLogging(options => options.LoggingFields = HttpLoggingFields.All); // POI
var app = builder.Build();
app.UseHttpLogging(); // POI // FIXME: I should go after compression, left here by intent to observe compressed response bodies
app.UseResponseCompression();
app.MapGet("/kb/{kb:int}", (int kb) => new String('a', kb * 1024));
app.Run();

By adding http logging we will see somethin like:

info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[1]
      Request:
      Protocol: HTTP/1.1
      Method: GET
      Scheme: http
      PathBase:
      Path: /kb/1
      Accept: */*
      Host: localhost:5000
      User-Agent: curl/7.79.1
      Accept-Encoding: gzip
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
      Executing endpoint 'HTTP: GET /kb/{kb:int}'
info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[2]
      Response:
      StatusCode: 200
      Content-Type: text/plain; charset=utf-8
      Content-Encoding: [Redacted]
      Vary: [Redacted]
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
      Executed endpoint 'HTTP: GET /kb/{kb:int}'
info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[4]
      ResponseBody: JL�!0#5����U|

Note: by intent i have added middleware in a wrong order, because it is realy obvious if compression is used or not by observing all that crappy ��

Why do it need all that theoretic part?

The key thing here is that it is not possible to enable compression only by doing something on server or client sides, both need to be prepared for this.

In the simples case from client side we need something like:

builder.Services
    .AddHttpClient("Api", client =>
    {
        client.Timeout = TimeSpan.FromSeconds(2);
        client.DefaultRequestHeaders.AcceptEncoding.Add(new System.Net.Http.Headers.StringWithQualityHeaderValue("gzip"));
        client.DefaultRequestHeaders.AcceptEncoding.Add(new System.Net.Http.Headers.StringWithQualityHeaderValue("deflate"));
    })
    .ConfigurePrimaryHttpMessageHandler(_ => new HttpClientHandler
    {
        AutomaticDecompression = DecompressionMethods.All
    });

Here we are enabling decompression (do not know why, but from my experiments it seems that it is not enabled by default) and also we are adding Accept-Encoding to all outgoing requests

nginx

To see what nginx can give us lets prepare following setup

Program.cs

using Microsoft.AspNetCore.HttpLogging;
using Microsoft.AspNetCore.ResponseCompression;
using Microsoft.Net.Http.Headers;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHttpLogging(options =>
{
    options.RequestHeaders.Add("X-Forwarded-For");
    options.RequestHeaders.Add("True-Client-IP");
    options.RequestHeaders.Add("CF-RAY");
    // X-Request-ID
    // X-Real-IP
    // X-Forwarded-For
    // X-Forwarded-Host
    // X-Forwarded-Port
    // X-Forwarded-Proto
    // X-Forwarded-Scheme
    // X-Scheme
    // X-Original-Forwarded-For
    // CF-RAY
    // CF-Visitor
    // CF-Connecting-IP
    // True-Client-IP
    // CF-IPCountry
    // CDN-Loop

    options.ResponseHeaders.Add(HeaderNames.ContentEncoding);
    options.ResponseHeaders.Add(HeaderNames.Vary);

    options.LoggingFields = HttpLoggingFields.All;
});
if (!string.IsNullOrEmpty(Environment.GetEnvironmentVariable("ENABLE_COMPRESSION"))) {
    builder.Services.AddResponseCompression(options => {
        options.EnableForHttps = true;
        options.Providers.Add<BrotliCompressionProvider>();
        options.Providers.Add<GzipCompressionProvider>();
        options.MimeTypes = ResponseCompressionDefaults.MimeTypes.Concat(new[] { "image/svg+xml" });
    });
}

var app = builder.Build();

app.UseHttpLogging();
if (!string.IsNullOrEmpty(Environment.GetEnvironmentVariable("ENABLE_COMPRESSION"))) {
    app.UseResponseCompression();
}

app.MapGet("/", () => "Hello World!");

/*
curl -s -i -o /dev/null localhost:5000/kb/2 -w '%{size_download}\n' # 2048
curl -H 'Accept-Encoding: gzip' -s -i -o /dev/null localhost:5000/kb/2 -w '%{size_download}\n' # 48


# without compressions we received 200kb in 35ms
curl -s -i -o /dev/null localhost:5000/kb/200 -w 'Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} Bytes: %{size_download}\n'
# with compression we received 0.9kb in 11ms
curl -H 'Accept-Encoding: gzip' -s -i -o /dev/null localhost:5000/kb/200 -w 'Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} Bytes: %{size_download}\n'
*/
app.MapGet("/kb/{kb:int}", (int kb) => new String('a', kb * 1024));

app.Run();


/*
curl -s -i -o /dev/null localhost:5000/kb/200 -w 'took: %{time_total} bytes: %{size_download}\n'
curl -H 'Accept-Encoding: gzip' -s -i -o /dev/null localhost:5000/kb/200 -w 'took: %{time_total} bytes: %{size_download}\n'

*/

in this demo app, depending on ENABLE_COMPRESSION we will enable or disable compression support

Dockerfile

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS builder
WORKDIR /app

COPY demo.csproj .
RUN dotnet restore

COPY . .
RUN dotnet publish -c Release -o publish

FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS runner
WORKDIR /app
COPY --from=builder /app/publish .
ENV ASPNETCORE_ENVIRONMENT Production
EXPOSE 80
ENTRYPOINT ["dotnet", "demo.dll"]

# export DOCKER_DEFAULT_PLATFORM=linux/amd64
# docker build -t demo .
# docker run -it --rm -p 8080:80 demo

docker-compose.yml

version: "3.9"
services:
  raw:
    container_name: raw
    hostname: raw
    build: .
    ports:
      - "8081:80"
  zip:
    container_name: zip
    hostname: zip
    build: .
    ports:
      - "8082:80"
    environment:
      - ENABLE_COMPRESSION=true
  
  # no compression
  nginx-defaults-to-raw:
    image: nginx
    ports:
      - "8083:80"
    volumes:
      - ./nginx-defaults-to-raw.config:/etc/nginx/conf.d/default.conf
  
  # compressed response proxied if "acccep-encoding: gzip" passed
  nginx-defaults-to-zip:
    image: nginx
    ports:
      - "8084:80"
    volumes:
      - ./nginx-defaults-to-zip.config:/etc/nginx/conf.d/default.conf

  nginx-proxy_set_header-to-zip:
    image: nginx
    ports:
      - "8085:80"
    volumes:
      - ./nginx-proxy_set_header-to-zip.config:/etc/nginx/conf.d/default.conf

  nginx-gzip-to-raw:
    image: nginx
    ports:
      - "8086:80"
    volumes:
      - ./nginx-gzip-to-raw.config:/etc/nginx/conf.d/default.conf

and nginx configs

nginx-defaults-to-raw.config

server {
  location / {
    proxy_pass   http://raw:80;
  }
}

nginx-defaults-to-zip.config

server {
  location / {
    proxy_pass   http://zip:80;
  }
}

nginx-gzip-to-raw.config

server {
  gzip on;

  location / {
    proxy_pass   http://raw:80;
  }
}

nginx-proxy_set_header-to-zip.config

server {
  location / {
    proxy_set_header Accept-Encoding "gzip";
    gunzip on;
    proxy_pass   http://zip:80;
  }
}

now if we start all this docker compose up we can check follwoing cases

curl localhost:8081/kb/2

it works

curl -H 'Accept-Encoding: gzip' -s -i -o /dev/null localhost:8081/kb/2 -w '%{size_download}\n' # 2048
curl -s -i -o /dev/null localhost:8081/kb/2 -w '%{size_download}\n' # 2048

in both cases we receive 2kb because 8081 has disabled compression

curl -s -i -o /dev/null localhost:8082/kb/2 -w '%{size_download}\n' # 2048
curl -H 'Accept-Encoding: gzip' -s -i -o /dev/null localhost:8082/kb/2 -w '%{size_download}\n' # 48

in contrast 8082 has enabled compression, thats why we receiving only 48 bytes instead of 2048

curl -H 'Accept-Encoding: gzip' -s -i -o /dev/null localhost:8083/kb/2 -w '%{size_download}\n' # 2048

nginx by default does not compress responses, aka - expectation was that even so we are talking to a service without compression, nginx will compress response

curl -H 'Accept-Encoding: gzip' -s -i -o /dev/null localhost:8084/kb/2 -w '%{size_download}\n' # 48

but if compression is configured on a downstream service nginx will proxy everything as is if accept encoding passed

curl -H 'Accept-Encoding: gzip' -s -i -o /dev/null localhost:8085/kb/2 -w '%{size_download}\n' # 48
curl -s -i -o /dev/null localhost:8085/kb/2 -w '%{size_download}\n' # 48

with proxy_set_header no matter if we asking for compressed response or not - we will receive it, may be problematic for clients that can not work with compressed responses

but with gunzip on; nginx starts respecting client request and takes care about compression

curl -H 'Accept-Encoding: gzip' -s -i -o /dev/null localhost:8086/kb/2 -w '%{size_download}\n' # 48
curl -s -i -o /dev/null localhost:8086/kb/2 -w '%{size_download}\n' # 2048

now we are zipping responses from raw service on nginx

kubernetes ingress

short note for kubernetes ingress, in its config map we may want to add

use-gzip: "true"

documentation

apollo garphql gateway

note that apollo gateway by default does send all requests to downstream services with compression