Building RAG for a single tenant is straightforward: one knowledge base, one set of permissions, one retrieval pipeline. Building RAG for multiple tenants — each with their own documents, their own access controls, and potentially their own model configuration — is a fundamentally different architecture challenge. Multi-tenant RAG is the core of AI SaaS products, and getting the architecture right determines whether you can scale.
The Multi-Tenant Architecture
Multi-tenant RAG has three architectural options: (1) Shared database, filtered by tenant_id: all tenants share the same vector database, with a tenant_id column on every chunk. Retrieval queries filter by tenant_id. Simple to implement, but tenant isolation depends on the filter being correctly applied. (2) Schema-per-tenant: each tenant gets their own database schema. Better isolation, but migrations become complex. (3) Database-per-tenant: maximum isolation, but highest infrastructure cost. For most AI SaaS products, option 1 (shared database with RLS) is the right default.
Per-Tenant Knowledge Base Management
Each tenant has their own knowledge base: documents, embeddings, and index. The ingestion pipeline must: identify the tenant for each document, store documents in the tenant's namespace, embed and index within the tenant's scope, and handle tenant-specific ingestion rules (some tenants may have additional processing requirements). The retrieval pipeline must: scope searches to the tenant's knowledge base, enforce tenant-level access controls, and handle tenant-specific retrieval configuration (different chunk sizes, different embedding models, different retrieval thresholds).
Note
Per-tenant configuration is a competitive advantage: different tenants can use different chunking strategies, different embedding models, and different retrieval thresholds based on their specific needs.
Resource Isolation and Fairness
In a shared RAG infrastructure, one tenant's heavy usage can degrade performance for other tenants. Implement resource isolation: per-tenant rate limits (max queries per minute), per-tenant storage quotas (max document chunks per tenant), per-tenant compute limits (max concurrent retrievals), and fair scheduling (distribute retrieval capacity evenly). These limits prevent noisy neighbour problems and ensure consistent performance across tenants.
Conclusion
Multi-tenant RAG is the core of AI SaaS products. The shared database with RLS approach provides the best balance of simplicity, isolation, and cost. Per-tenant configuration is a competitive advantage. Resource isolation prevents noisy neighbour problems.
Key Takeaways
- Shared database with RLS is the best default for multi-tenant RAG
- Each tenant needs their own namespace for documents, embeddings, and index
- Per-tenant configuration (chunking, embedding, retrieval) is a competitive advantage
- Resource isolation: per-tenant rate limits, storage quotas, compute limits, fair scheduling
- Multi-tenant RAG is the core of AI SaaS — get the architecture right for scale