URL Shortener
Built and load-tested event by event, up to 10% of Bitly’s traffic. October 2026.
I built a URL shortener on one server and ran nine scripted events against it: more traffic, new requirements, and failures. When an event broke the design, I changed it only enough to pass. The final version handles about 877 million redirects a month at a 14.8 ms p99, with no errors, for €9.56 a month.
TypeScript · Node.js · SQLite · nginx · Litestream · Linux · Bash · k6 · Python
Code and raw resultsScoreboard
What a URL shortener does
You paste a long link and get a short one, like bit.ly/FXsk. Anyone who opens the short link is sent to the long one. Short links are used where long ones don’t fit or are hard to type: social posts, text messages, printed material and QR codes. The service also counts clicks, which is often the reason a company uses one.
It is a common system design question because it is easy to state but touches most of the core topics: generating unique IDs, a read-heavy workload, popular keys, counting writes cheaply, durability, and recovering from failure.
Requirements
Functional, with the event that introduces each one:
- Create a short link for any http or https URL up to 2,048 characters; the code is at most 7 letters and digits. (E01)
- Redirect a short link to its URL; unknown codes get “not found”. (E01)
- Let a creator choose a custom name, with exactly one owner per name. (E03)
- Show clicks per link per day, at most 60 s behind and within 1%. (E05)
- Let an operator take a link down; it must stop redirecting within 60 s, browsers included. (E09)
Non-functional:
- Redirect p99 under 100 ms, create p99 under 300 ms, errors under 0.1%.100 ms is about where a response stops feeling instant (Nielsen). p99, not the average: at 1,000 clicks a second the slowest 1% is still 10 people every second (Dean and Barroso, The Tail at Scale).
- Power cut: no acknowledged link lost, redirects back within 5 minutes.
- Machine lost: back within 1 hour, losing at most the last 5 minutes of new links. In interview terms, an RTO of 1 hour and an RPO of 5 minutes.
The numbers
Scale is measured against Bitly, which reports over 100 billion clicks and 2 billion new links a year.bitly.com/pages/about, checked 1 October 2026. The target here is 10% of that.
| Quantity | Value | How |
|---|---|---|
| Bitly, average | 3,200/s | 100 B clicks ÷ seconds in a year |
| Reads per write | 50 : 1 | 100 B clicks ÷ 2 B links |
| Peak factor | 3 | assumed: the busiest time is 3× the daily average |
| Peak redirects | 1,000/s | target: 10% of Bitly’s peak |
| Peak creates | 20/s | 1,000 ÷ 50 |
| Clicks a month | 877 M | average (peak ÷ 3) × 2,629,800 s |
| Links a month | 17.5 M | same formula, from 20/s |
| Links stored | 100 M | about six months of creation |
| Database size | 12 GB | measured, at 100 M links |
| Bytes per link | ≈ 120 | 12 GB ÷ 100 M, with index |
The system is sized for the peak, because that is when it breaks. Monthly totals use the average: 1,000 ÷ 3 ≈ 333 redirects a second, over the 2,629,800 seconds in a month.
Two traffic details matter more than they look. Clicks follow a Zipf distribution, so a few links get most of them. And every click opens a new HTTPS connection, because short links are mostly clicked once by different people. That second detail turned out to drive most of the CPU cost.
How long the codes need to be
Each character of a code is a letter or a digit: 26 + 26 + 10 = 62 options. So a code of n characters has 62n possible values. Dividing by the 17.5 million links created a month shows how long each length lasts:
| Characters | Possible codes | Runs out after |
|---|---|---|
| 4 | 14.8 million | under a month |
| 5 | 916 million | about 4 years |
| 6 | 56.8 billion | about 270 years |
The requirement allows 7 characters, so codes never run out. Because codes count up from 1, they also stay as short as possible: link 10,000,000 is FXsk.
API and data model
POST /linkswith{"url": …, "name": …}→201and the short link;409if the name is taken.GET /<code>→301to the URL, withCache-Control: no-store;404unknown;410taken down.GET /links/<code>/stats→ clicks per day.POST /links/<code>/takedownwith an operator secret.
Three tables: links(id, url, created_at, taken_down_at), names(name, url, created_at, taken_down_at), and clicks(code, day, count). A generated code is the link’s row number written in base 62. Custom names must contain a hyphen or be longer than 7 characters, so they can never collide with a generated code.
Event by event
The nine events were fixed before anything was built and never changed afterwards. Each one runs against the current version. If it passes, nothing changes. If it fails, the smallest fix that passes becomes the next version, and every earlier event is rerun on it. The diagram beside each event shows the version that passed it, with what that version added in red.
v0
E01Launch
- Event
- 1 redirect/s, 1 create a minute, 1,000 links.
- Version
- v0: one Node.js program with SQLite in one file, serving HTTPS itself, on a 2-vCPU server. Codes are a counting-up number in base 62.
- Result
- Pass.
v0 · unchanged
E021% of Bitly
- Event
- 100 redirects/s, 2 creates/s, 10 million links (1.2 GB).
- Version
- v0, unchanged.
- Result
- Pass, redirect p99 17.7 ms. No cache, queue or load balancer needed.
v1 · SQLite: names table added
v2 · added: nginx for TLS
E03Custom names
- Event
- Under E02 load, 50 clients claim the same name at the same moment, repeated with 1,000 names. Exactly one may win each time.
- Version
- v0.
- Result
- Fail. v0 ignored names, so two creators both “won” the same one.
- Fix
- v1: a
namestable. One process checks and saves each claim, so claims are handled one at a time. Correct in every round, but redirect p99 jumped to 179.5 ms: bursts of 50 new TLS handshakes queued on Node’s single thread. - Fix
- v2: nginx in front to terminate TLS on both cores; Node gets plain HTTP on localhost.
- First try: nginx reused idle connections that Node had already closed, causing 502s. Setting nginx’s upstream
keepalive_timeoutto 4 s, below Node’s, fixed it. - Result
- Pass, p99 69.2 ms.
v3 · added: 4 vCPUs
E04A viral link
- Event
- On top of E02 load, one link ramps to 2,000 redirects/s and holds for 10 minutes.
- Version
- v2.
- Result
- Fail. Out of CPU. About 2,100 new TLS connections a second is the cost, not the database.
- Fix
- v3: the same software on 4 vCPUs.
- Result
- Pass, p99 14.6 ms.
v4 · added: click counts in memory, saved once a second
E05Click counts
- Event
- Rerun E04 with counting on. Counts per link per day, at most 60 s behind and within 1%.
- Version
- v3 has no counts. The first try saved every click before redirecting.
- Result
- Fail. It collapsed at about 166 saves a second, with Node’s thread waiting on the disk and 91% errors.
- Fix
- v4: count in memory and save all counts once a second in one transaction: one disk wait per second instead of one per click. Redirects carry
Cache-Control: no-store, so browsers can’t skip the server on repeat clicks. - Result
- Pass. 1,355,748 clicks counted, exactly; p99 14–15 ms.
v5 · SQLite: synchronous = EXTRA
E06Power cut
- Event
- Under E02 load, the server is powered off without warning and started again.
- Version
- v4.
- Result
- Fail. SQLite’s default setting rolled back a link that had already been confirmed, and its code was then given to a different link.
- Fix
- v5:
PRAGMA synchronous = EXTRA, so a confirmed save survives power loss. - Result
- Pass. No confirmed link lost; back in 60–67 s.
v6 · SQLite in WAL mode, copied to a volume by Litestream; health check added
E07Machine lost
- Event
- Under E02 load, the server and its disk are deleted for good. Back within 1 hour, losing at most 5 minutes of new links.
- Version
- v5.
- Result
- Fail. The only copy of the data was on the deleted disk.
- Fix
- v6: WAL mode, and Litestream copies every change to a separate volume that outlives the server. A new link is confirmed only once it is in the copy. A monitor checks
/health, and one script rebuilds the server on the same IP address. - Two tries failed first. Copying in the background let a restore reissue a lost link’s code. Then a rebuild stopped because Hetzner had no server of that type free, so the script now retries for 20 minutes before falling back to another type.
- Result
- Pass. Back 3.4–5.9 minutes after the loss, nothing confirmed lost.
v7 · Node.js: counter jumps a million after a restore
E0810% of Bitly
- Event
- 1,000 redirects/s, 20 creates/s, 100 million links (12 GB, more than the server’s 8 GB of memory).
- Version
- v6.
- Result
- Fail. Create p99 reached 753–848 ms, because every create waited for the copy while Litestream rewrote it.
- Fix
- v7: generated links are confirmed on local disk again. After any restore, the counter jumps a million ahead, so a code lost with the machine is never reused. Custom names still wait for the copy.
- First try froze the app for 30.5 s: after a fresh full copy, Litestream’s checkpoint scanned the whole 1.7 GB copy while holding the write lock. A one-row write every second keeps that scan small.
- Result
- Pass. Redirect p99 14.5 ms, create p99 22.3 ms.
v8 · added: takedowns
E09Takedowns
- Event
- Under E08 load, 100 links are taken down. None may redirect more than 60 s later.
- Version
- v7 had no way to remove a link.
- Fix
- v8:
POST /links/<code>/takedownwith an operator secret; the link answers410 Gonefrom the next click. - Result
- Pass. Zero redirects after any of 300 takedowns across three runs. Redirect p99 14.8 ms, create p99 26.2 ms.
What was measured, and what it caught
Every run recorded what users feel and what each resource was doing, once a second. A break was diagnosed only when a measurement named the resource that ran out.
What users feel
| Measured | What it shows | What it caught here |
|---|---|---|
| p50, p95 and p99 latency, per 10-second window | How slow the slowest requests are, and when | The 30.5 s freeze in E08, which one number per run would have hidden. On v1, p99 was over 100 ms in all 30 windows of E03 |
| Latency split into TCP connect, TLS handshake and server response | Where the time goes | The slow hosts: opening the connection took an extra 50–60 ms while the server answered in about 3 ms |
| Error rate | Failed or timed-out requests | The 502s from nginx reusing connections Node had closed (v2, first try) |
Which resource ran out
| Measured | What it shows | What it caught here |
|---|---|---|
| CPU per process | Which program uses the CPU | In E03, Node never used a whole core in any second (peak 74%), yet one redirect in eight took over 100 ms: 50 handshakes of about 1.8 ms each queued within a fraction of a second. Per-second averages hide bursts. At E08: nginx about 1 core, Litestream up to 1.5, Node about a quarter |
| iowait | CPU idle because it is waiting for the disk | Saving every click: iowait 15% with the CPU mostly idle, so the disk was the limit (E05, first try) |
| CPU steal | The host giving CPU time to other customers’ machines | 0% in E03, which ruled out a noisy neighbour: the problem was in the design |
| Memory and page cache | Whether the data fits in memory | 12 GB database on an 8 GB server: programs used about 1.5 GB, and the rest cached the database (E08) |
| Connections waiting or turned away | Requests queued because no program can accept them | nginx ran out of connection slots when Node stalled on the disk (E05, first try) |
| Write-lock probe | How long the database write lock is held | Litestream’s checkpoint holding the lock for 30.5 s (E08) |
The final design
- One server with 4 vCPUs and 8 GB of memory.
- nginx terminates TLS and passes requests to one Node.js program on the same machine.
- SQLite in one file: WAL mode,
synchronous = EXTRA. - Click counts kept in memory and saved once a second.
- Litestream copies every change to a separate volume; a script rebuilds on a new server with the same IP.
- An external monitor checks
/healthevery 30 s.
Monthly cost€4-vCPU server8.49IPv4 address0.5010 GB volume for the copy0.57Monitoring0.00Total9.56Hetzner prices, excluding VAT, checked 3 and 5 October 2026.At the largest run, 1,000 redirects a second with peak traffic assumed to be three times the average:
1,000 redirects/s ÷ 3 × 2,629,800 s/month ≈ 877 million redirects/month
What runs out next is CPU. In the largest run it averaged about 50%: nginx used about one core for TLS, and Litestream up to one and a half while rewriting its copy.
Caveats
- One event at a time. Combinations, like a viral spike during a power cut, were never tested.
- Failures at a smaller scale. The power cut and the machine loss ran at 1% of Bitly’s traffic with 10 million links. At 100 million links, restoring alone takes about 3 minutes.
- Measured next door. The load generator sat in the same data centre, so the times don’t include visitors’ own internet connection.
- One machine, one data centre. The copy lives in the same location, so losing the whole data centre isn’t covered.
- No domain. Links used the server’s IP address and a self-made certificate. A real service needs a domain and a proper certificate.
- An assumed peak factor. The monthly figure assumes peak traffic is three times the average. Only the 1,000 redirects/s peak was measured, and no capacity above it is claimed.
- Slow hosts. On some Hetzner hosts about 1.5% of new TCP connections took an extra 50–60 ms to open, putting redirect p99 near 100–118 ms in two of three machine-loss runs. The server’s own response stayed about 3 ms.
What this taught me about system design
- Start simple and add parts on evidence. One process and one SQLite file served 100 million links at 1,000 redirects a second with a p99 around 15 ms.
- Find the real bottleneck before adding parts. It was TLS handshakes from new connections, not the database. nginx was added for TLS on more cores, not for load balancing.
- Batch frequent small writes. Saving per click collapsed at 166 a second; saving once a second counted over a million clicks exactly at 2,000 a second.
- Know what “saved” means. A database default can roll back a write it already confirmed. Decide what an acknowledgement promises, then test it by pulling the plug.
- A counter is a fine ID generator. It is unique by construction. Its one risk, reusing codes after a restore, is closed by jumping ahead. No separate key service needed.
- Every cache has a cost. A cache or CDN would have broken click counting and made takedowns slow. Without one, takedowns work from the next click.
- Set recovery targets first. An RTO of 1 hour and an RPO of 5 minutes were met with a continuous copy and a rebuild script, for 57 cents a month.
- Watch p99, not averages. The 30-second freeze and the slow connections only showed up in the tail.
What usual designs add, and when you’d need it
The usual interview answer for a URL shortener includes the components below. None were needed for these requirements. Each one would be needed if the requirements changed.
| Usual component | What it’s for | Not needed here because | You’d need it if |
|---|---|---|---|
| Load balancer and several app servers | More CPU than one machine; no single point of failure | One server averaged about 50% CPU at 1,000 redirects/s, and a few minutes of downtime was allowed (RTO 1 hour) | Traffic beyond what was tested (Bitly’s full scale was not), or minutes of downtime are not acceptable |
| Cache (Redis) | The database is too slow or too far away for popular links | SQLite ran in the same process and the operating system kept popular links in memory: p99 about 15 ms with a 12 GB database. Without a cache, takedowns applied at once | The database is on another machine, so every lookup crosses the network, or several app servers share it |
| Sharded or NoSQL database | Data or writes too large for one machine | 100 million links fit in 12 GB, and 20 creates/s is light for one writer | Far more links or writes than tested, or writes from several regions |
| Key-generation service, or random codes | Unique codes from many servers without coordination | One process hands out counting-up numbers, so codes are unique by construction | Several servers create links, or codes must be hard to guess: counting-up codes can be enumerated (FXsk, FXsl, …) |
| Queue or stream for clicks (Kafka) | Detailed analytics, several consumers, decoupling | Only daily counts per link were needed; an in-memory tally saved once a second was exact | Per-click analytics (referrer, location, device), several teams reading clicks, or counts that must survive a crash exactly |
| CDN or edge caching | Low latency for users around the world | One region was tested, and cached redirects would skip click counting and delay takedowns | Latency targets on several continents |
| Standby server or several regions | Fast failover; surviving a data-centre outage | The 1-hour RTO was met by a rebuild in minutes | An RTO of seconds, or surviving the loss of a whole data centre |