Direct answer
Frame it as a data-location question, not a technology one. If the corpus contains contracts, pricing, personnel records or customer data, where those documents physically reside is a commercial decision with legal weight — which usually points to self-hosting. If the content is public marketing material or product documentation, SaaS is faster and cheaper. The decision rule: self-host the documents you would not email to a vendor without a contract, and consider SaaS for the rest.
The decision rule
Ask one question of each document set: would I be comfortable sending this to an outside supplier as an email attachment?
- No — contracts, quotations, pricing, personnel material, customer data. These belong in a self-hosted deployment.
- Yes — public product information, published policies, marketing content. SaaS is fine and faster to stand up.
Most businesses turn out to need a split: the sensitive corpus self-hosted, the public corpus wherever it is easiest.
What self-hosting actually costs
Self-hosting is not only a preference — it carries real costs, and being honest about them prevents a bad decision in either direction:
- Setup. A containerised deployment on a machine you control, with the retrieval engine, the document pipeline and the interface.
- Operation. Backups, updates, and somebody able to restart a service.
- Model access. A self-hosted knowledge base can still call a hosted model for the answering step, which is a separate decision from where the documents sit.
The important distinction: self-hosting the documents does not require self-hosting the language model. Data location and inference location are two choices, not one.
What SaaS actually costs
SaaS is faster to start and usually cheaper to run, and the trade-off is control: your documents sit on someone else's infrastructure, under their terms, and your ability to retrieve them intact if you leave depends on their export.
For a business whose corpus is genuinely public, that is a good trade. For one whose corpus is the business, it is usually not.
A practical middle path
Run the sensitive corpus on a machine you control and keep the tooling simple enough that one person can operate it. Container-based deployment means the service can be moved or rebuilt without redesigning it, which is what makes the choice reversible.
Reversibility matters more than the initial decision. Choose the option you can leave in a year without losing your documents.
Key facts
| Real question | Where do the documents physically sit, not which technology is better |
| Decision rule | Self-host documents you would not email to a vendor without a contract |
| Common outcome | Sensitive corpus self-hosted, public corpus on SaaS |
| Separate decisions | Document location and model inference location are independent |
| What to optimise for | Reversibility — can you leave in a year without losing your content |
Sources
- Container-based self-hosted retrieval deployment practice
- Data location considerations for commercially sensitive documents
Follow-up questions
Is self-hosting technically hard?
With container-based tooling it is a deployment task rather than a development project. The ongoing requirement is operational — backups, updates, and someone who can restart it — rather than engineering.
Can a self-hosted system use a cloud AI model?
Yes, and this is often the right balance: documents stay on your machine, and only the query and the retrieved passage are sent to the model for the answering step. Confirm what the arrangement actually transmits before relying on it.
What if I start on SaaS and want to move?
Make sure you can export your content in a usable form before you commit. The exit path is the part of a SaaS decision that is hardest to change later.