Global technology company
Secure RAG knowledge base with governance
An LLM interface over sensitive internal documentation, with data kept in-account, access controlled by SSO, and cost governed per user.
- +35%
- self-service resolution rate, after 3 months
- AWS Bedrock, LiteLLM, Enterprise SSO, EU data residency
- stack
Problem
- A multi-team enterprise needed to query large volumes of sensitive internal documentation through an LLM interface.
- They could not risk exposing data to external providers, losing identity-level access controls, or letting cost and provider routing escape governance oversight.
Approach
- Production UI on AWS, integrated with enterprise SSO so user and role context propagates through every request.
- AWS Bedrock Knowledge Bases as the retrieval layer over sensitive documents, with data kept in-account and inside EU regions.
- A LiteLLM proxy for multi-provider routing, per-user and per-group quotas, centralised cost telemetry, and prompt and access logging aligned to the client's data-sensitivity framework.
Outcome
- Roughly 35% increase in self-service resolution rate for internal queries after three months, lowering load on subject-matter experts and support teams.
- Live for hundreds of internal users across multiple business functions.
- Per-request cost attribution, provider-level fallback, and SSO-enforced access controls in production.
Have a system that needs to work in production?
Tell us what you are building, or what has stopped working. Thirty minutes, no pitch deck.