Skip to content

Add Hummingbird: Swift HTTP framework on SwiftNIO (first Swift entry!)#23

Open
BennyFranciscus wants to merge 4 commits intoMDA2AV:mainfrom
BennyFranciscus:add-hummingbird
Open

Add Hummingbird: Swift HTTP framework on SwiftNIO (first Swift entry!)#23
BennyFranciscus wants to merge 4 commits intoMDA2AV:mainfrom
BennyFranciscus:add-hummingbird

Conversation

@BennyFranciscus
Copy link
Collaborator

Hummingbird — Swift HTTP Framework

Adds Hummingbird (~1,700 ⭐) to HttpArena — this is the first Swift framework in the benchmark suite!

What is Hummingbird?

Hummingbird is a lightweight, flexible HTTP server framework built on SwiftNIO. It's designed with minimal dependencies and uses Swift's structured concurrency (async/await) throughout. Part of the Swift Server Work Group incubation ecosystem.

Why it's interesting for benchmarks

  • SwiftNIO under the hood — non-blocking event-driven I/O, similar in philosophy to Netty
  • Minimal abstraction overhead — designed to be lean, unlike heavier Swift frameworks like Vapor
  • Swift's performance story — compiled language with ARC (no GC pauses), interesting to see how it competes with Rust/C++/Go
  • Modern async/await — uses Swift's structured concurrency, not callback-based

Implementation details

  • All standard HttpArena endpoints implemented: /pipeline, /baseline11, /baseline2, /json, /compression, /db, /upload, /static/{filename}
  • Compression via hummingbird-compression middleware
  • SQLite via C API directly (no ORM overhead)
  • Pre-cached JSON responses for /json and /compression endpoints
  • Static files loaded into memory at startup
  • Compiled with -O and cross-module optimization

Tests enabled

baseline, noisy, pipelined, limited-conn, json, upload, compression, mixed


cc @adam-fowler @Joannis — thought it'd be cool to see how Hummingbird stacks up in HttpArena! Would love to hear if there are any Swift-specific optimizations we should try.

swift-async-algorithms (transitive dep via Hummingbird 2.x) uses
#isolation macro which requires Swift 6.0+. The build fails with
'non-built-in macro cannot be used as default argument' on 5.10.
@BennyFranciscus
Copy link
Collaborator Author

Build fix: swift-async-algorithms (transitive dep pulled in by Hummingbird 2.x) uses the #isolation macro which requires Swift 6.0. The Dockerfile was using swift:5.10-jammy.

Bumped to swift:6.0-jammy. Should compile cleanly now.

@BennyFranciscus
Copy link
Collaborator Author

The SQLite3 overlay module (import SQLite3) only exists on macOS/Darwin. On Linux, Swift doesn't ship it.

Added a CSQLite system library target with a modulemap that wraps <sqlite3.h>. The code already had a #if canImport(CSQLite) guard so it picks it up automatically.

Should compile on Linux now.

@github-actions
Copy link

Benchmark Results

Framework: hummingbird | Profile: all profiles

hummingbird / baseline / 512c (p=1, r=0, cpu=unlimited)
  Best: 430079 req/s (CPU: 9419.8%, Mem: 115.4MiB) ===

hummingbird / baseline / 4096c (p=1, r=0, cpu=unlimited)
  Best: 434538 req/s (CPU: 8816.8%, Mem: 151.3MiB) ===

hummingbird / baseline / 16384c (p=1, r=0, cpu=unlimited)
  Best: 427856 req/s (CPU: 8407.8%, Mem: 219.7MiB) ===

hummingbird / pipelined / 512c (p=16, r=0, cpu=unlimited)
  Best: 691628 req/s (CPU: 8852.7%, Mem: 90.8MiB) ===

hummingbird / pipelined / 4096c (p=16, r=0, cpu=unlimited)
  Best: 699806 req/s (CPU: 8276.7%, Mem: 192.8MiB) ===

hummingbird / pipelined / 16384c (p=16, r=0, cpu=unlimited)
  Best: 727335 req/s (CPU: 8365.8%, Mem: 213.6MiB) ===

hummingbird / limited-conn / 512c (p=1, r=10, cpu=unlimited)
  Best: 116715 req/s (CPU: 2702.3%, Mem: 52.0MiB) ===

hummingbird / limited-conn / 4096c (p=1, r=10, cpu=unlimited)
  Best: 116872 req/s (CPU: 2660.8%, Mem: 54.2MiB) ===

hummingbird / json / 4096c (p=1, r=0, cpu=unlimited)
  Best: 432891 req/s (CPU: 8563.8%, Mem: 195.0MiB) ===

hummingbird / json / 16384c (p=1, r=0, cpu=unlimited)
  Best: 428706 req/s (CPU: 8269.6%, Mem: 302.9MiB) ===

hummingbird / upload / 64c (p=1, r=0, cpu=unlimited)
  Best: 342 req/s (CPU: 1616.3%, Mem: 2.3GiB) ===

hummingbird / upload / 256c (p=1, r=0, cpu=unlimited)
  Best: 324 req/s (CPU: 2505.3%, Mem: 7.0GiB) ===

hummingbird / upload / 512c (p=1, r=0, cpu=unlimited)
  Best: 327 req/s (CPU: 2251.4%, Mem: 10.0GiB) ===

hummingbird / compression / 4096c (p=1, r=0, cpu=unlimited)
  Best: 2545 req/s (CPU: 12492.5%, Mem: 1005.0MiB) ===

hummingbird / compression / 16384c (p=1, r=0, cpu=unlimited)
  Best: 2502 req/s (CPU: 11903.6%, Mem: 1009.0MiB) ===

hummingbird / noisy / 512c (p=1, r=0, cpu=unlimited)
  Best: 309329 req/s (CPU: 9322.3%, Mem: 78.3MiB) ===

hummingbird / noisy / 4096c (p=1, r=0, cpu=unlimited)
  Best: 334941 req/s (CPU: 9318.4%, Mem: 153.7MiB) ===

hummingbird / noisy / 16384c (p=1, r=0, cpu=unlimited)
  Best: 320477 req/s (CPU: 8443.3%, Mem: 167.2MiB) ===

hummingbird / mixed / 4096c (p=1, r=5, cpu=unlimited)
  Best: 9053 req/s (CPU: 12021.1%, Mem: 1.1GiB) ===

hummingbird / mixed / 16384c (p=1, r=5, cpu=unlimited)
  Best: 8944 req/s (CPU: 10155.1%, Mem: 1.1GiB) ===
Full log
  Per-template-ok: 884475,585298,0,0,0

  WARNING: 585044/2054817 responses (28.5%) had unexpected status (expected 2xx)
  CPU: 7745.9% | Mem: 257.5MiB

=== Best: 320477 req/s (CPU: 8443.3%, Mem: 167.2MiB) ===
  Input BW: 32.40MB/s (avg template: 106 bytes)
[dry-run] Results not saved (use --save to persist)
httparena-bench-hummingbird
httparena-bench-hummingbird

==============================================
=== hummingbird / mixed / 4096c (p=1, r=5, cpu=unlimited) ===
==============================================
b4966af060669d1acb5437ed5ee5c74a89cd69dae9b10163fc9d6da3fdcfa03b
[wait] Waiting for server...
[ready] Server is up

[run 1/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   47.66ms   15.30ms   133.90ms   209.80ms   626.30ms

  55503 requests in 5.00s, 44441 responses
  Throughput: 8.88K req/s
  Bandwidth:  379.60MB/s
  Status codes: 2xx=44441, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 44441 / 44441 responses (100.0%)
  Reconnects: 10812
  Errors: connect 0, read 142, timeout 0
  Per-template: 4355,4429,4410,4535,4504,4351,4408,4593,4600,4256
  Per-template-ok: 4355,4429,4410,4535,4504,4351,4408,4593,4600,4256
  CPU: 11387.1% | Mem: 1007.0MiB

[run 2/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   54.08ms   15.50ms   141.10ms   316.80ms    1.79s

  55398 requests in 5.00s, 45201 responses
  Throughput: 9.04K req/s
  Bandwidth:  386.10MB/s
  Status codes: 2xx=45201, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 45201 / 45201 responses (100.0%)
  Reconnects: 10983
  Errors: connect 0, read 47, timeout 0
  Per-template: 4571,4466,4387,4439,4570,4395,4589,4767,4618,4399
  Per-template-ok: 4571,4466,4387,4439,4570,4395,4589,4767,4618,4399
  CPU: 12140.5% | Mem: 1.1GiB

[run 3/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   57.38ms   15.10ms   139.70ms   313.90ms    3.34s

  56461 requests in 5.00s, 45265 responses
  Throughput: 9.05K req/s
  Bandwidth:  395.48MB/s
  Status codes: 2xx=45265, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 45265 / 45265 responses (100.0%)
  Reconnects: 11113
  Errors: connect 0, read 164, timeout 0
  Per-template: 4555,4489,4529,4528,4524,4302,4413,4663,4759,4503
  Per-template-ok: 4555,4489,4529,4528,4524,4302,4413,4663,4759,4503
  CPU: 12021.1% | Mem: 1.1GiB

=== Best: 9053 req/s (CPU: 12021.1%, Mem: 1.1GiB) ===
  Input BW: 905.87MB/s (avg template: 104924 bytes)
[dry-run] Results not saved (use --save to persist)
httparena-bench-hummingbird
httparena-bench-hummingbird

==============================================
=== hummingbird / mixed / 16384c (p=1, r=5, cpu=unlimited) ===
==============================================
944e44f8971f6aff2da134f72563bc752b6254227a347d5ff7c0c11ace39c1ff
[wait] Waiting for server...
[ready] Server is up

[run 1/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   72.22ms   20.10ms   153.50ms   655.70ms    1.85s

  53550 requests in 5.00s, 44722 responses
  Throughput: 8.94K req/s
  Bandwidth:  362.41MB/s
  Status codes: 2xx=44722, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 44722 / 44722 responses (100.0%)
  Reconnects: 10462
  Errors: connect 0, read 40, timeout 0
  Per-template: 4781,4743,4759,4720,4708,4141,4069,4357,4259,4185
  Per-template-ok: 4781,4743,4759,4720,4708,4141,4069,4357,4259,4185
  CPU: 10155.1% | Mem: 1.1GiB

[run 2/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   81.70ms   21.50ms   167.90ms   937.10ms    1.87s

  57655 requests in 5.00s, 43434 responses
  Throughput: 8.68K req/s
  Bandwidth:  376.49MB/s
  Status codes: 2xx=43434, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 43434 / 43434 responses (100.0%)
  Reconnects: 10862
  Errors: connect 0, read 509, timeout 0
  Per-template: 4536,4575,4586,4647,4504,4081,3932,3768,4567,4238
  Per-template-ok: 4536,4575,4586,4647,4504,4081,3932,3768,4567,4238
  CPU: 11490.2% | Mem: 1.1GiB

[run 3/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   157.05ms   62.90ms   347.40ms    1.36s    1.93s

  53750 requests in 5.00s, 41602 responses
  Throughput: 8.31K req/s
  Bandwidth:  372.72MB/s
  Status codes: 2xx=41602, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 41602 / 41602 responses (100.0%)
  Reconnects: 10196
  Errors: connect 0, read 180, timeout 0
  Per-template: 4708,4437,4258,4348,4224,3640,3616,3621,4478,4272
  Per-template-ok: 4708,4437,4258,4348,4224,3640,3616,3621,4478,4272
  CPU: 10234.0% | Mem: 1.5GiB

=== Best: 8944 req/s (CPU: 10155.1%, Mem: 1.1GiB) ===
  Input BW: 894.97MB/s (avg template: 104924 bytes)
[dry-run] Results not saved (use --save to persist)
httparena-bench-hummingbird
httparena-bench-hummingbird
[skip] hummingbird does not subscribe to baseline-h2
[skip] hummingbird does not subscribe to static-h2
[skip] hummingbird does not subscribe to baseline-h3
[skip] hummingbird does not subscribe to static-h3
[skip] hummingbird does not subscribe to unary-grpc
[skip] hummingbird does not subscribe to unary-grpc-tls
[skip] hummingbird does not subscribe to echo-ws
[restore] Restoring CPU governor to powersave...

@adam-fowler
Copy link

The SQLite3 overlay module (import SQLite3) only exists on macOS/Darwin. On Linux, Swift doesn't ship it.

Added a CSQLite system library target with a modulemap that wraps <sqlite3.h>. The code already had a #if canImport(CSQLite) guard so it picks it up automatically.

Should compile on Linux now.

You should be able to just install sqlite3-dev in the Dockerfile for SQLite headers. You shouldn't need to mess with additional shims.

Also you are best using latest Swift 6.2. I haven't looked at the code as I am away for the next week or so but will have a look when I get back.

Per adam-fowler's review: use latest Swift 6.2 image.
Also bumped swift-tools-version from 5.9 to 6.0.
@BennyFranciscus
Copy link
Collaborator Author

Thanks for the review @adam-fowler! 🙏

Bumped to Swift 6.2 — good call, no reason to stay on 6.0.

On the CSQLite shim: I actually do have libsqlite3-dev installed in the Dockerfile already — the issue is that on Linux, Swift doesn't expose a SQLite3 module even with the dev headers present (that module only exists on Darwin via the SDK overlay). The CSQLite systemLibrary target is the standard SPM pattern to wrap the C headers into a Swift-importable module — it's just a modulemap + one header file, so pretty minimal.

The source code already handles both with:

#if canImport(CSQLite)
import CSQLite
#elseif canImport(SQLite3)
import SQLite3
#endif

So it works on both macOS (via the SDK overlay) and Linux (via the systemLibrary). If there's a cleaner way in Swift 6.2 I'm not aware of, happy to update!

Enjoy the week off — no rush at all on the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants