Local AI: language models on your own servers
Not every piece of information should go to a cloud provider: personnel files, engineering data, client records or health data are better kept in your own network. Open models such as Llama and Mistral for text and Whisper for speech can run on your own hardware. Sortia Labs plans, installs and operates such local AI systems for companies in Munich and across Germany.
- Local AI
- Local LLM
- On-premise AI
- Llama and Mistral
- AI without the cloud
What we build for your local AI
The right hardware
The model size your tasks need determines graphics card, memory and therefore cost. We work this out in advance with your examples, so you buy neither too small nor needlessly large.
Models in your own data centre
Installation and operation of open models such as Llama and Mistral for text and Whisper for speech, in Docker containers on your server or in a private cloud, behind an interface in the style of the OpenAI API.
Your documents, searchable
A local chatbot with RAG answers questions from your documents, with the vector database (pgvector or Qdrant) on the same server. Documents, questions and answers stay on your own hardware.
Hybrid setups
The local model handles confidential material, a cloud model with an EU data centre handles general tasks. A routing layer decides by fixed rules which request goes where and logs the decision.
Operation and updates
Monitoring of load and response times, updates to new model versions after a test with your examples, and an access log. If you like, we train your IT team to take over operations.
The road to your own AI
- 1
Tasks and data
We clarify which tasks the AI should take over and which data must not leave the building. That shows whether a local model is needed or an EU cloud is enough.
- 2
Model test
Several open models compete on your real examples. You see what quality a model of a given size delivers before any hardware is bought.
- 3
Build and connect
Server, containers, interface, access rights and the connection to your applications. On existing hardware or in your current Kubernetes cluster if you prefer.
Software we build and run ourselves.
Questions about Local AI
Are local models as good as GPT or Claude?
For well-defined tasks such as classifying, extracting, summarising or searching your own documents, open models are often good enough. For very complex tasks the large cloud models are often ahead. That is why we test with your examples and combine both where it helps.
What hardware do we need for a local language model?
That depends on model size, number of users and the response time you want. Smaller models run on a server with a current graphics card, larger ones need more GPU memory or several cards. After the model test you get a concrete recommendation that fits your load.
Is local AI automatically better for data protection?
It removes a big item from the list: there is no external AI provider receiving your data. Access rights, logging, retention periods and hardening the server remain tasks that we define and implement with you, with GDPR in mind.
Can we start in the cloud and move local later?
Yes, if the application is built with a routing layer from the start; then the model can be swapped without rewriting the application. A sensible path can be to build the prototype with a cloud model in the EU and move confidential parts to your own hardware later.
Read on.
Which data should stay in your building?
Tell us which tasks the AI should take over and what is confidential. We tell you whether a local model, an EU cloud or a mix fits.