By Sebastiaan Hoogeveen | 23 August 2026 | reading time 2 minutes
For many companies, the AI train is racing ahead. Many companies are choosing to work with Copilot, for example, to support staff with AI. At the same time, many companies prohibit their employees from sharing sensitive information with public LLMs, as this information could otherwise end up in the hands of an external party.
RAG (Retrieval-Augmented Generation)
But how, then, can AI be applied safely within a company? Ideally, employees should be able to retrieve company information quickly and reliably, without sensitive data leaving the company. This is not usually solved by training a model on internal documents (which is costly and does not provide reliable, verifiable answers). Instead, RAG (Retrieval-Augmented Generation) is used: internal documents are made searchable, and when a question is asked, the system retrieves the relevant passages to provide context for a language model.
Search locally, formulate externally: the hybrid solution
A local, self-hosted open-source language model can be used for this search (meaning the sensitive data never leaves your own environment). For the final ‘understanding’ and formulation of the answer, an external API (such as GPT or Claude) can still be used, provided that only non-traceable or already approved context fragments are sent. This creates a hybrid approach that saves costs compared to a fully self-hosted system, whilst still allowing you to retain control over which data is shared externally. This translates into gains in terms of digital sovereignty.
When choosing an open-source language model, it is important to check how it has been trained, how transparent the developers are regarding training data and licensing, and how you intend to systematically evaluate the quality of the results.
Want to find out more? Get in touch with us!
