HLDArena
HLD-409·Search & Indexing
Expert · 2,000 pts

Planet-Scale Web Search Inverted Index & Query Aggregator

Design Google / Bing’s real-time web search serving tier combining tiered result caches, term/document-partitioned inverted index leaves, and breaking-news fresh index ingestion.

Max p99
≤ 65 ms
Availability
≥ 99.6%
Budget
≤ $11,800

Functional Requirements

  • FR-1GET /search?q={query_hash} — Serves top-10 ranked web + fresh-news results under 65ms p99.
  • FR-2POST /index/fresh — Commits breaking-news crawled documents into the real-time memory index.
  • FR-3PATCH & DELETE /search/{query_hash} — Updates, revokes, or deletes query_hash entries with bounded TTL or explicit cache/index invalidation.
  • FR-4Must survive 8x slow index-leaf stragglers via hedged/circuit-broken timeouts and multi-tier query caches.

Workload Profile

Global Search Baseline92,000 Query RPS / 7,000 Fresh Index Write RPS
Breaking Global Event Peak240,000 Query RPS / 16,000 Fresh Write RPS
Active Working Set64 GB hot query result cache & fresh posting lists
Fault Scenario8x leaf storage tail-latency brownout in Wave 3

Consistency Model

Near-real-time (< 5s) fresh index visibility; aggressive query result caching absorbs trending queries.

Design Write-up
0 boxes · 0 edges
0 words
Hidden Cases #4–#7 run automatically on Submit
Case 1: Smoke & API ContractInput Workload
Read RPS
2,000 rps
Write RPS
400 rps
Key Distribution
Uniform (Low Skew)
Chaos Fault
None (Healthy Cluster)
Expected Pass CriteriaASSERTIONS
• Tail Latency: p99 ≤ 130 ms
• Availability: ≥ 99.6%
Valid end-to-end Client → Compute → Storage topology