HLD-314·Messaging & Fan-out
Hard · 1,500 ptsChatGPT-Scale Conversational Assistant, Context & SSE Gateway
Design a ChatGPT / Claude conversational backend managing multi-turn conversation trees, sliding context-window caches, token quota checks, and SSE response streaming.
Max p99
≤ 70 ms
Availability
≥ 99.5%
Budget
≤ $8,700
Functional Requirements
- FR-1POST /chat/{conversation_id}/completions — Appends user prompt, loads context window, and streams tokens.
- FR-2GET /chat/{conversation_id}/history — Fetches multi-turn dialogue branch with read-after-write consistency.
- FR-3PATCH & DELETE /chat/{conversation_id}/completions — Updates, cancels, or deletes conversation_id records and immediately invalidates or updates cached state (read-after-write).
- FR-4Must isolate 6x downstream storage/model gray latency spikes with circuit breakers and semantic caching.
Workload Profile
Workday AI Traffic44,000 Context Read RPS / 9,000 Turn Write RPS
Model Launch Surge122,000 Read RPS / 24,000 Turn Write RPS
Active Working Set38 GB active conversation context windows & prompt cache
Consistency GateRead-after-write turn history so follow-up prompts never miss prior turn
Consistency Model
Strict read-after-write on conversation_id history so multi-turn follow-ups always include the latest assistant turn.
Solutions Locked
Sign in and either achieve AC or complete 3 official submissions on this question to inspect community diagrams and writeups.
Loading solutions...
Submission History (0)Select to replay
No submissions yet
Click Submit in the top bar to evaluate against all 7 judge cases and save each run here.
Design Write-up
0 boxes · 0 edges
0 words