Building a custom AI chatbot with RAG for Swiss SMEs
A custom chatbot is not a fine-tuned model. It is a small, private retrieval system with a language model bolted on top, engineered so your data never leaves Swiss or EU infrastructure.
Why RAG, and not fine-tuning
Most SMEs asking for a 'custom AI chatbot' assume the answer is a model trained on their documents. It almost never is. Fine-tuning changes how a model writes; it does not reliably change what it knows, and it silently bakes your data into a model that is expensive to update and impossible to audit line by line.
Retrieval-Augmented Generation solves the actual problem. Your documents stay in a database you control. At query time, the relevant passages are retrieved and passed to a general-purpose language model as context, together with a strict instruction to answer only from that context. The chatbot answers with citations; you can trace every response back to a source paragraph, delete a document and have it vanish from answers immediately, and swap the underlying model without retraining anything.
The five pieces of a real RAG system
A production chatbot is five components, not one. Skip any of them and you get a demo that hallucinates.
First, an ingestion pipeline that reads your sources, PDFs, wiki pages, ticket exports, contracts, and normalises them into clean text with stable identifiers. Second, a chunker that splits that text into passages small enough for a model to reason over but large enough to carry meaning, with the source URL and page number attached. Third, an embedding model that turns each chunk into a vector, stored in a vector database (pgvector, Qdrant, Weaviate). Fourth, a retriever that, for each user question, finds the top-k most relevant chunks and, ideally, re-ranks them with a stronger model. Fifth, an answering step where a language model receives the question, the retrieved chunks, and a system prompt that forbids answering outside them.
Everything else, chat memory, multilingual routing, safety filters, analytics, sits around those five.
Training an AI chatbot on your own knowledge base
The phrase 'train an AI chatbot with a custom knowledge base' is misleading. You are not training a model. You are curating a corpus, embedding it, and pointing a retriever at it.
Start small and honest. Pick a bounded domain, internal HR policy, product documentation, a specific service line, where the ground truth actually lives in written form. Ingest that first. Measure retrieval quality before you touch the generation prompt: for a set of real questions, does the top-3 retrieval actually contain the answer? If it does not, no amount of prompt engineering will save the chatbot; fix chunking and embeddings first.
Once retrieval is solid, tune the answering prompt to refuse politely when the context does not contain the answer. A chatbot that says 'I do not have that information' is infinitely more useful than one that invents it.
The Swiss constraint: keeping data private end to end
For a Swiss SME, the interesting engineering is not the model choice. It is the data path. nFADP and GDPR both care about where personal data travels, how long it is retained, and who can see it, and default vendor stacks route everything through US inference APIs on undisclosed terms.
A compliant RAG deployment keeps documents and embeddings inside a Swiss or EU database you control. It uses inference providers with EU or Swiss data residency, zero-retention agreements, and no training on submitted data. It logs every prompt and response for audit, encrypts embeddings at rest, and enforces per-user access so a salesperson cannot retrieve HR documents.
None of this is exotic. It is a matter of choosing the boring option at every layer and writing it down in a data flow diagram before the first document is ingested.
Common failure modes, and how to avoid them
Four failure modes account for most of the RAG projects we see rescued.
Retrieval is tuned on synthetic questions instead of real ones. The system looks great in the demo and misses on day one in production. Always evaluate against a set of questions the actual users have asked.
Chunks are too big or too small. Passages that span three topics dilute retrieval; single-sentence chunks lose context. Start with roughly 400-800 tokens with overlap, then adjust with measurement.
The prompt lets the model answer from prior knowledge. A serious system instruction explicitly forbids using anything outside the provided context and requires citations by source ID.
No one owns the corpus. Documents go stale, get duplicated, or contradict each other, and the chatbot cheerfully surfaces the wrong version. A production RAG needs a named human owner and a monthly review, exactly like any other reference system.
The bottom line
A custom AI chatbot built on RAG is a small, well-engineered retrieval system with a language model at the end. The interesting work is in the corpus, the chunking, the retriever, and the data path, not the model.
If you are considering a private assistant on your own documents, the fastest useful next step is a thirty-minute call. We tell you honestly whether your corpus is ready, what a compliant Swiss deployment would look like, and what to fix first if the answer is 'not yet'.