HLD-225·Search & Indexing
Medium · 1,000 ptsCentralized Log Ingestion & Full-Text Search Pipeline
Design a Datadog Logs / ELK-style observability pipeline ingesting structured JSON logs at high write throughput while serving incident search queries.
Max p99
≤ 85 ms
Availability
≥ 99.4%
Budget
≤ $5,750
Functional Requirements
- FR-1POST /v1/logs/ingest — Buffers high-throughput service log batches keyed by tenant_service.
- FR-2GET /v1/logs/search?service={tenant_service} — Queries recent indexed logs for incident triage.
- FR-3PATCH & DELETE /v1/logs/ingest/{tenant_service} — Updates, revokes, or deletes tenant_service entries with bounded TTL or explicit cache/index invalidation.
- FR-4Must survive indexing storage gray latency spikes during production outage log storms.
Workload Profile
Steady Fleet Logs14,000 Query RPS / 16,000 Log Batch Write RPS
Incident Log Storm36,000 Query RPS / 42,000 Log Write RPS
Active Working Set28 GB hot 15-minute inverted log segments
Traffic DistributionWrite-heavy burst from crashing tenant_service clusters
Consistency Model
Near-real-time log search visibility (< 5s segment refresh); zero dropped accepted log batches.
Solutions Locked
Sign in and either achieve AC or complete 3 official submissions on this question to inspect community diagrams and writeups.
Loading solutions...
Submission History (0)Select to replay
No submissions yet
Click Submit in the top bar to evaluate against all 7 judge cases and save each run here.
Design Write-up
0 boxes · 0 edges
0 words