The two options
A hosted model runs on the provider's servers. You send text, it sends an answer back, and you pay for what you use. A local model runs on a computer you control, using software such as llama.cpp or Ollama, and nothing is sent anywhere.
Choose hosted when
- The task needs the best available quality, such as drafting long texts or working across many documents.
- The data is not sensitive, or the provider's terms for handling it are acceptable to you.
- Use is occasional, so paying per use costs less than owning capable hardware.
Choose local when
- The data must not leave your premises: client files, health or financial records, internal documents.
- The task is narrow: sorting, extracting fields, summarising your own documents.
- It has to work offline, or with a fixed cost instead of a usage bill.
What local costs you
Local models are smaller than the largest hosted ones, so expect weaker results on open-ended writing and complex reasoning. They need a reasonably powerful computer, above all enough memory, and they are slower. Test both on a sample of your real work before deciding: for some tasks the difference is small, for others it is large.
Using both
A common split is to let a local model handle the steps that touch sensitive data, such as reading documents and pulling out fields, and to send only the non-sensitive result to a hosted model for drafting. Either way, keep a person reviewing the result; our guide on human-in-the-loop AI covers how.
Shift Brief uses a local model to select the evidence for a brief, and Forensic Lens runs a model inside a browser extension.
Before you decide
Write down which data the tool will see, how sensitive it is, and roughly how much text it will process in a month. Those three answers point to local, hosted or a mix. Our page on AI integration with human controls describes how we build either kind.