The problem: the knowledge is there, but nobody can find it
Contracts, briefs, protocols, manuals, minutes, internal regulations. Any business with a few years of history builds up thousands of documents and, when someone needs an answer, they search by hand or ask the colleague who knows. The temptation is to upload everything to a cloud ChatGPT, but those are exactly the documents that shouldn’t leave: personal data protected by the GDPR, client information under professional secrecy, industrial know-how or case files. And if you then offer that assistant to your clients, the EU AI Act (Art. 50) also requires you to tell them they are talking to an AI.
A private RAG solves both problems: the AI answers from your documents, cites the source and everything lives on your server. In practice, it is a private company ChatGPT that knows your archive.
How we do it, step by step
- Source inventory. Which folders, which document management system, which formats and who can see what. If you are in Barcelona, we do this at your office.
- Reference questions. Together we define 30 to 50 real questions with their correct answers. They are the yardstick for the whole project.
- Ingestion. We extract the text (including scanned documents), split it into meaningful chunks and record which document and page each one comes from.
- Indexing and tuning. We test combinations of chunking, embeddings and reranker until the reference questions come out right.
- Interface and permissions. The search is built into the company’s private chat, with access groups.
- Handover with metrics. We show you the accuracy rate, the remaining errors and how to reduce them.
The technology, in plain words
- Embeddings: a small model turns each chunk into a numerical “fingerprint” that captures its meaning. That way it finds “termination” even if you ask about “breaking the contract”.
- Qdrant: the database where those fingerprints are stored, inside your server.
- Reranker: a second model that sorts the results by relevance before passing them to the AI.
- The language model: runs locally with vLLM or Ollama and writes the answer, citing the sources.
- The interface: Open WebUI or a custom page, connected through the LiteLLM gateway, which logs who asks what.
If you don’t have a server yet, we start with local AI for business and the RAG is built on top.
When we DON’T recommend it
If you have a few dozen documents, a good folder structure and your system’s built-in search are enough. If your documents already live in a tool with a good search of its own, you may only need to connect it. If they are public (for example, legislation published in Spain’s Official State Gazette, the BOE), a cloud service can handle it with no risk to your data. And if the documentation is disorganised, out of date or contradictory, the AI will inherit that mess: sometimes the first step is to put things in order, and we’ll tell you so.
We use it every day
Our own agents, in our Barcelona lab, store and retrieve information with Qdrant, embeddings and a reranker running on our GPUs, without touching the cloud. The agent that manages a family estate in the Penedès also keeps its memory in Qdrant. It is the same component we install in your business, proven in daily use and not just in a demo.