HLD-325·Caching & Consistency
Hard · 1,500 ptsInline Low-Latency LLM Prompt-Injection & Guardrails Gateway
Design an inline AI safety proxy sitting in front of enterprise LLM endpoints to enforce PII redaction, prompt-injection blocklists, and tenant token-spend caps under 35ms p99.
Max p99
≤ 35 ms
Availability
≥ 99.5%
Budget
≤ $8,600
Functional Requirements
- FR-1POST /guardrails/{tenant_id}/inspect — Evaluates tenant policy rules, PII patterns, and rolling spend.
- FR-2PUT /guardrails/{tenant_id}/policy — Updates blocked topics or emergency tenant kill-switches.
- FR-3PATCH & DELETE /guardrails/{tenant_id}/inspect — Updates, cancels, or deletes tenant_id records and immediately invalidates or updates cached state (read-after-write).
- FR-4Policy kill-switches must apply immediately (read-after-write) and survive 6x audit-store brownouts.
Workload Profile
Steady AI API Calls56,000 Inspect Read RPS / 9,500 Audit/Spend Write RPS
Prompt-Injection Attack Wave144,000 Read RPS / 26,000 Write RPS
Active Working Set28 GB compiled tenant policy ASTs, signature caches & spend counters
Latency GateInline proxy budget p99 <= 35 ms
Consistency Model
Read-after-write on tenant safety policy updates and token budget circuit-breakers.
Solutions Locked
Sign in and either achieve AC or complete 3 official submissions on this question to inspect community diagrams and writeups.
Loading solutions...
Submission History (0)Select to replay
No submissions yet
Click Submit in the top bar to evaluate against all 7 judge cases and save each run here.
Design Write-up
0 boxes · 0 edges
0 words