red.systems / blog / zig-server

Writing an HTTP/1.1 server in Zig from scratch

· 10 min read · Zig Systems Networking

Here is a reality check: you don't always need a heavy framework. Sometimes, you just need a machine, an OS, and a lot of patience.

This is the story of how I replaced a framework-backed server with 600 lines of Zig, raw POSIX sockets, and Linux epoll. I wanted to see what the syscall boundary actually looks like, where memory gets tricky, and how a backend can run with a binary size of around 300 KB.

Not even full documentation. Just learnings, late-night debugging, and raw code.

Sockets, Endianness, and File Descriptors

I learned all about file descriptors here. Basically, in Linux, everything is a file. I’m still wrapping my head around the philosophical implications, but practically, it just means that if I create a file descriptor, it acts like an object I can manipulate.

To create a TCP server, you need posix.AF.INET for IPv4 and posix.SOCK.STREAM for a TCP connection. TCP is a reliable "must send all data" protocol. Contrast that with UDP, which just blasts data out as fast as it can and hopes for the best (honestly, I kind of want to build a project around UDP now).

Then there's nativeToBig. This is where you deal with little-endian and big-endian formatting. It basically means that when we send data to a network, we must reverse the byte order because, apparently, early programmers were indecisive.

After that, it's the standard networking lifecycle:

  • Bind the IP address and port.
  • Listen to it.
  • Accept the connection (which spawns a new client_fd).
  • Receive (recv) the data.

We read the incoming data until we hit \r\n\r\n—the universal signal that the HTTP request header is done.

The HTTP Parser & File Serving

Writing an HTTP parser is incredibly simple. It’s literally just string matching the methods, paths, and HTTP versions. If something is misaligned, throw an error.

Serving files introduced me to MIME types. Browsers are dumb; you have to explicitly tell them what media type you are returning. If I return an HTML file but set the header to text/plain, the browser just spits my raw code onto the screen. It needs text/html to actually render it.

The logic: read the file, create a response, attach the content-type, and send it back. You can also just concatenate public/ to the requested path to easily serve your public folder.

Concurrency: Threads vs. Epoll

Multi-connections using threading: Pretty simple in theory. For every connection, you spawn a thread. Since a single thread has blocking I/O, if I don't close the client FD, it remains open and blocks other clients. The "one connection = one thread" model solves this.

But it’s highly inefficient. A standard thread eats around 8MB of memory. In high concurrency, that scales terribly.

Multi-connections using epoll: Complicated, but worth it. You create an epoll FD, then manage a watchlist using epoll_ctl (ctl_add to add FDs, ctl_del to remove them).

When data arrives, epoll wakes up and processes it. This means the thread only actually works when there is something to do. epoll_wait is basically the server sleeping until an event triggers an alarm.

For every client, we need metadata to remember who they are. I used a connection struct for this, stored in a hashmap where the key is the client_fd and the value is the struct itself.

Structured Routing & Zero-Copy

You need a routing table. After parsing the header, we check if the method and path exist in our API definitions (for dynamic routes that need actual functions). If there's no match, we fallback to serving static files from the public folder.

At first, to serve a file, I was copying the file into a buffer and sending it to POSIX.

Old Flow: Driver -> Kernel -> User Space -> Kernel -> Driver.

Then I learned about the zero-copy function: sendfile. With this, the kernel handles the copying directly.

New Flow: Driver -> Kernel -> Kernel -> Driver.

Huge performance win.

Security and Networking Tweaks

  • Path Traversal: I realized you could literally just send a request for ../.env and my server would hand over my secrets. Frameworks hide this from you by sanitizing paths out of the box. My fix? If the path contains .., instantly return a Bad Request.
  • Keep-Alive Connections: Initially, my event handler closed the TCP socket after every single request. If a browser needed 3 files, it would do the full TCP handshake 3 separate times. Keep-alive keeps the socket open, processes subsequent requests, and saves a massive amount of overhead.
  • Cache Control: Simple enough. Cache the static assets that persist (CSS, main JS), and set "no-cache" for API routes that constantly change (like /health).

Frontend

Don't talk to me. I hate frontend.

The Reality of Systems Programming

Implementing connection timeouts was a humbling experience. I added a last_active variable that updates after every request. In the event loop, epoll_wait checks every 5 seconds. I used an iterator to go through my hashmap of clients to clear out inactive ones.

My first mistake: I was destroying connections while iterating through them. Check entry -> Time > 30s? -> Destroy -> Continue. Turns out, modifying a hashmap while iterating over it triggers undefined behavior.

My fix: Allocate a 64-element array. Iterate first to find the connections that need to timeout, store them in the array, then iterate through the array to close them.

My second mistake: I didn't put a bounds checker on that 64-element array. It was a stack buffer overflow waiting to happen. It ran fine locally, so I didn't check it right away.

Deployment

I wanted this running 24/7, so I rented a Microsoft Azure VM (b2ats v2: 1GB RAM, 2 vCPUs) in the Southeast Asia region for lower latency.

The setup:

  • Security: Configured SSH keys for access and locked down the firewall to only accept ports 22 (SSH), 80 (HTTP), and 443 (HTTPS).
  • Domain: Grabbed a domain on Namecheap and forwarded it to the Azure public IP.
  • Reverse Proxy: Set up Caddy in front of my Zig server. Caddy handles the TLS handshake, decrypts packets, validates the HTTP, and passes the raw HTTP to my local port.
  • Persistence: Used systemctl to pass control of the server binary to the OS, ensuring it stays alive even if I close my terminal.
The Full Request Flow: 
Access Domain -> DNS Lookup -> Azure IP -> Firewall Check -> Caddy (TLS/Validation) -> Raw HTTP to Zig port -> Systemd maintains process -> Epoll wakes up -> File Served.

The Results

wrk Benchmarks (1 Thread 200 Connections):

  • 13.10k avg requests/sec
  • 15.49ms average latency
  • 391129 requests in 30.04s
  • 0 failed requests

wrk Benchmarks (10 Thrads 200 Connections)

  • 759.66 requests/sec
  • 33.52ms average latency
  • 220075 requests in 30.08s
  • 0 failed requests

Memory Footprint:

  • RSS of the Zig server is 0.3MB.

What's Next?

  • Adding Gzip compression
  • Adding HEAD method support
  • Graceful shutdown on SIGTERM