← Hussain Abuwala

URL Shortener

Built and load-tested event by event, up to 10% of Bitly’s traffic. October 2026.

I built a URL shortener on one server and ran nine scripted events against it: more traffic, new requirements, and failures. When an event broke the design, I changed it only enough to pass. The final version handles about 877 million redirects a month at a 14.8 ms p99, with no errors, for €9.56 a month.

TypeScript · Node.js · SQLite · nginx · Litestream · Linux · Bash · k6 · Python

The full build and every test run, 1 h 39 min. In the video, versions are called stages. Loads YouTube when you press play.

What a URL shortener does

You paste a long link and get a short one, like bit.ly/FXsk. Anyone who opens the short link is sent to the long one. Short links are used where long ones don’t fit or are hard to type: social posts, text messages, printed material and QR codes. The service also counts clicks, which is often the reason a company uses one.

It is a common system design question because it is easy to state but touches most of the core topics: generating unique IDs, a read-heavy workload, popular keys, counting writes cheaply, durability, and recovering from failure.

Requirements

Functional, with the event that introduces each one:

Non-functional:

The numbers

Scale is measured against Bitly, which reports over 100 billion clicks and 2 billion new links a year.bitly.com/pages/about, checked 1 October 2026. The target here is 10% of that.

QuantityValueHow
Bitly, average3,200/s100 B clicks ÷ seconds in a year
Reads per write50 : 1100 B clicks ÷ 2 B links
Peak factor3assumed: the busiest time is 3× the daily average
Peak redirects1,000/starget: 10% of Bitly’s peak
Peak creates20/s1,000 ÷ 50
Clicks a month877 Maverage (peak ÷ 3) × 2,629,800 s
Links a month17.5 Msame formula, from 20/s
Links stored100 Mabout six months of creation
Database size12 GBmeasured, at 100 M links
Bytes per link≈ 12012 GB ÷ 100 M, with index

The system is sized for the peak, because that is when it breaks. Monthly totals use the average: 1,000 ÷ 3 ≈ 333 redirects a second, over the 2,629,800 seconds in a month.

Two traffic details matter more than they look. Clicks follow a Zipf distribution, so a few links get most of them. And every click opens a new HTTPS connection, because short links are mostly clicked once by different people. That second detail turned out to drive most of the CPU cost.

How long the codes need to be

Each character of a code is a letter or a digit: 26 + 26 + 10 = 62 options. So a code of n characters has 62n possible values. Dividing by the 17.5 million links created a month shows how long each length lasts:

CharactersPossible codesRuns out after
414.8 millionunder a month
5916 millionabout 4 years
656.8 billionabout 270 years

The requirement allows 7 characters, so codes never run out. Because codes count up from 1, they also stay as short as possible: link 10,000,000 is FXsk.

API and data model

Three tables: links(id, url, created_at, taken_down_at), names(name, url, created_at, taken_down_at), and clicks(code, day, count). A generated code is the link’s row number written in base 62. Custom names must contain a hyphen or be longer than 7 characters, so they can never collide with a generated code.

Event by event

The nine events were fixed before anything was built and never changed afterwards. Each one runs against the current version. If it passes, nothing changes. If it fails, the smallest fix that passes becomes the next version, and every earlier event is rerun on it. The diagram beside each event shows the version that passed it, with what that version added in red.

v0

E01Launch

Event
1 redirect/s, 1 create a minute, 1,000 links.
Version
v0: one Node.js program with SQLite in one file, serving HTTPS itself, on a 2-vCPU server. Codes are a counting-up number in base 62.
Result
Pass.

v0 · unchanged

E021% of Bitly

Event
100 redirects/s, 2 creates/s, 10 million links (1.2 GB).
Version
v0, unchanged.
Result
Pass, redirect p99 17.7 ms. No cache, queue or load balancer needed.

v1 · SQLite: names table added

v2 · added: nginx for TLS

E03Custom names

Event
Under E02 load, 50 clients claim the same name at the same moment, repeated with 1,000 names. Exactly one may win each time.
Version
v0.
Result
Fail. v0 ignored names, so two creators both “won” the same one.
Fix
v1: a names table. One process checks and saves each claim, so claims are handled one at a time. Correct in every round, but redirect p99 jumped to 179.5 ms: bursts of 50 new TLS handshakes queued on Node’s single thread.
Fix
v2: nginx in front to terminate TLS on both cores; Node gets plain HTTP on localhost.
First try: nginx reused idle connections that Node had already closed, causing 502s. Setting nginx’s upstream keepalive_timeout to 4 s, below Node’s, fixed it.
Result
Pass, p99 69.2 ms.

v3 · added: 4 vCPUs

E04A viral link

Event
On top of E02 load, one link ramps to 2,000 redirects/s and holds for 10 minutes.
Version
v2.
Result
Fail. Out of CPU. About 2,100 new TLS connections a second is the cost, not the database.
Fix
v3: the same software on 4 vCPUs.
Result
Pass, p99 14.6 ms.

v4 · added: click counts in memory, saved once a second

E05Click counts

Event
Rerun E04 with counting on. Counts per link per day, at most 60 s behind and within 1%.
Version
v3 has no counts. The first try saved every click before redirecting.
Result
Fail. It collapsed at about 166 saves a second, with Node’s thread waiting on the disk and 91% errors.
Fix
v4: count in memory and save all counts once a second in one transaction: one disk wait per second instead of one per click. Redirects carry Cache-Control: no-store, so browsers can’t skip the server on repeat clicks.
Result
Pass. 1,355,748 clicks counted, exactly; p99 14–15 ms.

v5 · SQLite: synchronous = EXTRA

E06Power cut

Event
Under E02 load, the server is powered off without warning and started again.
Version
v4.
Result
Fail. SQLite’s default setting rolled back a link that had already been confirmed, and its code was then given to a different link.
Fix
v5: PRAGMA synchronous = EXTRA, so a confirmed save survives power loss.
Result
Pass. No confirmed link lost; back in 60–67 s.

v6 · SQLite in WAL mode, copied to a volume by Litestream; health check added

E07Machine lost

Event
Under E02 load, the server and its disk are deleted for good. Back within 1 hour, losing at most 5 minutes of new links.
Version
v5.
Result
Fail. The only copy of the data was on the deleted disk.
Fix
v6: WAL mode, and Litestream copies every change to a separate volume that outlives the server. A new link is confirmed only once it is in the copy. A monitor checks /health, and one script rebuilds the server on the same IP address.
Two tries failed first. Copying in the background let a restore reissue a lost link’s code. Then a rebuild stopped because Hetzner had no server of that type free, so the script now retries for 20 minutes before falling back to another type.
Result
Pass. Back 3.4–5.9 minutes after the loss, nothing confirmed lost.

v7 · Node.js: counter jumps a million after a restore

E0810% of Bitly

Event
1,000 redirects/s, 20 creates/s, 100 million links (12 GB, more than the server’s 8 GB of memory).
Version
v6.
Result
Fail. Create p99 reached 753–848 ms, because every create waited for the copy while Litestream rewrote it.
Fix
v7: generated links are confirmed on local disk again. After any restore, the counter jumps a million ahead, so a code lost with the machine is never reused. Custom names still wait for the copy.
First try froze the app for 30.5 s: after a fresh full copy, Litestream’s checkpoint scanned the whole 1.7 GB copy while holding the write lock. A one-row write every second keeps that scan small.
Result
Pass. Redirect p99 14.5 ms, create p99 22.3 ms.

v8 · added: takedowns

E09Takedowns

Event
Under E08 load, 100 links are taken down. None may redirect more than 60 s later.
Version
v7 had no way to remove a link.
Fix
v8: POST /links/<code>/takedown with an operator secret; the link answers 410 Gone from the next click.
Result
Pass. Zero redirects after any of 300 takedowns across three runs. Redirect p99 14.8 ms, create p99 26.2 ms.

What was measured, and what it caught

Every run recorded what users feel and what each resource was doing, once a second. A break was diagnosed only when a measurement named the resource that ran out.

What users feel

MeasuredWhat it showsWhat it caught here
p50, p95 and p99 latency, per 10-second windowHow slow the slowest requests are, and whenThe 30.5 s freeze in E08, which one number per run would have hidden. On v1, p99 was over 100 ms in all 30 windows of E03
Latency split into TCP connect, TLS handshake and server responseWhere the time goesThe slow hosts: opening the connection took an extra 50–60 ms while the server answered in about 3 ms
Error rateFailed or timed-out requestsThe 502s from nginx reusing connections Node had closed (v2, first try)

Which resource ran out

MeasuredWhat it showsWhat it caught here
CPU per processWhich program uses the CPUIn E03, Node never used a whole core in any second (peak 74%), yet one redirect in eight took over 100 ms: 50 handshakes of about 1.8 ms each queued within a fraction of a second. Per-second averages hide bursts. At E08: nginx about 1 core, Litestream up to 1.5, Node about a quarter
iowaitCPU idle because it is waiting for the diskSaving every click: iowait 15% with the CPU mostly idle, so the disk was the limit (E05, first try)
CPU stealThe host giving CPU time to other customers’ machines0% in E03, which ruled out a noisy neighbour: the problem was in the design
Memory and page cacheWhether the data fits in memory12 GB database on an 8 GB server: programs used about 1.5 GB, and the rest cached the database (E08)
Connections waiting or turned awayRequests queued because no program can accept themnginx ran out of connection slots when Node stalled on the disk (E05, first try)
Write-lock probeHow long the database write lock is heldLitestream’s checkpoint holding the lock for 30.5 s (E08)

The final design

Monthly cost€4-vCPU server8.49IPv4 address0.5010 GB volume for the copy0.57Monitoring0.00Total9.56Hetzner prices, excluding VAT, checked 3 and 5 October 2026.At the largest run, 1,000 redirects a second with peak traffic assumed to be three times the average:

1,000 redirects/s ÷ 3 × 2,629,800 s/month ≈ 877 million redirects/month

What runs out next is CPU. In the largest run it averaged about 50%: nginx used about one core for TLS, and Litestream up to one and a half while rewriting its copy.

Caveats

What this taught me about system design

  1. Start simple and add parts on evidence. One process and one SQLite file served 100 million links at 1,000 redirects a second with a p99 around 15 ms.
  2. Find the real bottleneck before adding parts. It was TLS handshakes from new connections, not the database. nginx was added for TLS on more cores, not for load balancing.
  3. Batch frequent small writes. Saving per click collapsed at 166 a second; saving once a second counted over a million clicks exactly at 2,000 a second.
  4. Know what “saved” means. A database default can roll back a write it already confirmed. Decide what an acknowledgement promises, then test it by pulling the plug.
  5. A counter is a fine ID generator. It is unique by construction. Its one risk, reusing codes after a restore, is closed by jumping ahead. No separate key service needed.
  6. Every cache has a cost. A cache or CDN would have broken click counting and made takedowns slow. Without one, takedowns work from the next click.
  7. Set recovery targets first. An RTO of 1 hour and an RPO of 5 minutes were met with a continuous copy and a rebuild script, for 57 cents a month.
  8. Watch p99, not averages. The 30-second freeze and the slow connections only showed up in the tail.

What usual designs add, and when you’d need it

The usual interview answer for a URL shortener includes the components below. None were needed for these requirements. Each one would be needed if the requirements changed.

Usual componentWhat it’s forNot needed here becauseYou’d need it if
Load balancer and several app serversMore CPU than one machine; no single point of failureOne server averaged about 50% CPU at 1,000 redirects/s, and a few minutes of downtime was allowed (RTO 1 hour)Traffic beyond what was tested (Bitly’s full scale was not), or minutes of downtime are not acceptable
Cache (Redis)The database is too slow or too far away for popular linksSQLite ran in the same process and the operating system kept popular links in memory: p99 about 15 ms with a 12 GB database. Without a cache, takedowns applied at onceThe database is on another machine, so every lookup crosses the network, or several app servers share it
Sharded or NoSQL databaseData or writes too large for one machine100 million links fit in 12 GB, and 20 creates/s is light for one writerFar more links or writes than tested, or writes from several regions
Key-generation service, or random codesUnique codes from many servers without coordinationOne process hands out counting-up numbers, so codes are unique by constructionSeveral servers create links, or codes must be hard to guess: counting-up codes can be enumerated (FXsk, FXsl, …)
Queue or stream for clicks (Kafka)Detailed analytics, several consumers, decouplingOnly daily counts per link were needed; an in-memory tally saved once a second was exactPer-click analytics (referrer, location, device), several teams reading clicks, or counts that must survive a crash exactly
CDN or edge cachingLow latency for users around the worldOne region was tested, and cached redirects would skip click counting and delay takedownsLatency targets on several continents
Standby server or several regionsFast failover; surviving a data-centre outageThe 1-hour RTO was met by a rebuild in minutesAn RTO of seconds, or surviving the loss of a whole data centre