What a private RAG service is

Retrieval-augmented generation — RAG — is the technique of answering a question by first FINDING the relevant passages in your own material and then asking a language model to answer using them. The model is not trained on your documents and does not memorise them. It is handed the passages at the moment of the question, the way you would hand a colleague the two pages that matter before asking what they say.

That distinction is the whole point. A model answering from memory produces something plausible and unverifiable. A RAG system answers from passages it can show you, so every claim has a source you can open and read. When it has nothing relevant, it should say so rather than fill the gap.

A private RAG service is that pipeline run so that your material never becomes anybody else's asset. In practice that means three specific things, and a service that does not do all three is not private in any sense worth paying for: your documents are not used to train a model, your queries are not pooled with other customers', and the model calls are made on a key you control so the bill and the data path are both yours.

Indexing. A document is read — a PDF's text layer where it has one, OCR where it does not, a vision model for a photograph, speech-to-text for audio and video. What comes out is text plus the structure around it: which page a line was on, which table a number sat in, when the source was published.

Chunking. The text is cut into passages small enough to be precise and large enough to make sense on their own. Cut too finely and a passage loses the sentence that explains it; too coarsely and an answer drags in three paragraphs of irrelevance for one useful line.

Embedding. Each passage becomes a vector — a list of numbers positioning it by meaning rather than by the words it happens to use. This is why a question about “termination notice” finds a clause that says “ending this agreement”.

Retrieval. The question is embedded the same way and compared against every stored passage. The strongest matches go into the prompt. Good systems also apply a floor, so a question the library knows nothing about retrieves nothing rather than the least-bad passages available.

Generation and citation. The model writes an answer from the passages it was given, marking which passage supports which claim. The citation is the product: without it you have a chatbot with a confident manner and no way to check it.

Bring your own key. If the service calls a model on ITS key, your documents pass through its account and its bill scales with your usage. If the calls are made on YOUR provider key, the data path is one you already have a contract for, and the cost is visible to you at the provider rather than marked up.

Per-account isolation, enforced in the query. Multi-tenant systems usually separate customers with a filter. The question to ask is whether that filter is applied in one place the code cannot route around, and what happens to a request that arrives with no account attached — the safe answer is that it retrieves nothing.

Deletion that is measured, not asserted. “We delete your data” is a sentence. What matters is whether every store is reached — documents, passages, vectors, generated files, caches, per-chat databases — and whether anything checks afterwards. A deletion that reaches nine of ten stores leaves material that is still retrievable and still citable.

Self-hosting as an actual option. The strongest privacy guarantee is that the software runs on your own server and the vendor cannot reach it at all. A private RAG service that cannot be self-hosted is asking you to take its word.

Self-hosting removes the vendor from the data path entirely, which for regulated material is often the only permissible arrangement. It costs you a server, updates, backups and somebody to notice when something breaks.

A private RAG SaaS removes that operational burden and reintroduces a third party. The question is not which is safer in the abstract, it is which specific guarantees survive: whose key pays for the model call, whether your documents are isolated per account, whether you can export everything and leave, and whether the vendor can read your material at all.

The honest middle is software you can run either way, with the same code, so the decision is yours to revisit. PrivateRAG is that: hosted here for a monthly fee, or run on your own machine with a Docker image and no licence server.

The obvious design — one shared index everyone can query — fails the first time somebody uploads a contract that half the team should not read. The useful shape separates what is genuinely shared from what is personal, and makes the boundary visible in the answer.

In PrivateRAG the uploaded knowledge bank is the shared thing, and it is shared by an explicit grant: an owner offers it, the other person accepts or declines, and either can withdraw. What is NOT shared is a person's conversations, the pages the assistant read while answering their questions, and their database connections — those are properties of a person's work, not of the library.

The part people miss is provenance in the answer itself. When a citation can come from your own document or from somebody else's shared one, the card has to say which, or a reader assumes everything they see is theirs. Sharing an API key is the same shape: it is a second, separate grant, because being on somebody's share list is not the same as being allowed to spend their money.

Ask whose key pays for the model call, and what happens to the answer if you revoke it. Ask what a citation links to — a page number, or the actual passage. Ask what the system does when your library contains nothing relevant; the right answer is that it says so.

Ask how a document is read when it has no text layer, because most real archives contain scans. Ask what happens to tables: a total computed from prose by a language model is a guess, while the same total computed with SQL over extracted rows cannot silently drop a row.

Ask what deletion reaches and whether anything verifies it. Ask whether you can export your material in a format you can read without them. And ask for the price of the thing that actually varies — model usage — rather than only the subscription.

PrivateRAG is a private RAG service for documents: it indexes PDFs, Word, Excel, PowerPoint, plain text and images, reads scans with OCR, describes pictures with a vision model, transcribes audio and video with Whisper, queries tables it extracts with SQL rather than guessing at arithmetic, and answers with citations you can open. It runs on your own provider key — OpenAI, Claude, Gemini, Qwen, Mistral, Kimi, or any OpenAI-compatible endpoint — and the same image can be self-hosted.

What it is not: it is not an enterprise search platform with dozens of SaaS connectors, it has no SSO today, and it is a small product built by one person rather than a company with a support desk. If you need Confluence and Salesforce connectors and a signed enterprise agreement, the platforms listed in any buyer's guide will serve you better. If you want your documents answered accurately, with citations, on a key you control, at a price that does not scale with your questions, this is aimed at you.

A system that answers questions from your own documents by retrieving the relevant passages and giving them to a language model, arranged so your material is not used for training, not pooled with other customers, and the model calls run on a key you control.

It depends on three specifics rather than on the label: whether the model calls are made on your own provider key, whether each account's material is isolated in the query itself, and whether the software can be self-hosted so the vendor is out of the data path entirely.

Fine-tuning changes the model's weights with your data, so the material becomes part of the model and cannot be cited or removed cleanly. RAG leaves the model alone and hands it the relevant passages at question time, which is why an answer can carry a citation you can open and a document can be deleted.

Yes, if the design separates the shared library from personal work. In PrivateRAG the uploaded knowledge bank is shared by an explicit grant the recipient accepts, while conversations, the pages the assistant read for you, and database connections stay private to each account.

Yes, and that is deliberate. Model calls run on your key, so the data path is one you already have a contract for and the usage cost is visible to you at the provider rather than marked up. Any OpenAI-compatible endpoint works, including self-hosted ones.

Yes. A PDF with a text layer is read locally in milliseconds; a scan goes to OCR, which returns text with coordinates so a value can be shown ringed on the page it came from. Photographs are described by a vision model, and audio and video are transcribed with Whisper.

Yes. It ships as a Docker image with SQLite for storage, and there is no licence server to call home to. The hosted plans exist so you do not have to run it, not because the software requires us.