This is the companion article to Closed Network Dispatch 04. Everything mentioned here (runtimes, model hubs, trusted quantizers and the full OpSec checklist) is collected on the hub page: Local LLM Downloads →
Think about the last thing you typed into a chatbot.
I don't mean the recipe or the email you wanted tidied up. I mean the question about the rash, the contract you didn't understand, or the message to your ex that you asked the machine to make sound less angry before you sent it.
People tell AI things they don't tell their doctor. Every one of those prompts left their device, crossed the internet, landed on a server they don't control, and got written down.
In Episode 61 I mentioned in passing that you can run one of these models on your own computer, with no account, no cloud and no log on anyone else's server. A lot of you wrote in asking how, so this is the how. It's also the why, which I think matters more than it first sounds.
Every prompt is a log entry
When you use a hosted chatbot, your words are processed on the company's machines. What happens to them after that is governed by a privacy policy you probably didn't read, one that can change, held by a company whose servers sit within reach of a court.
We've already seen a court reach in.
In May 2025, in the New York Times' copyright lawsuit against OpenAI, a federal magistrate judge ordered OpenAI to preserve ChatGPT output logs, including conversations users had deleted. One user went to court to argue that his private and medical chats shouldn't be swept up. The judge turned him down, saying the order was about preservation, not disclosure, and that the court is not a law enforcement agency. For anyone who'd assumed "delete" meant delete, it turned out to mean delete unless a judge says otherwise.
That blanket preservation order was lifted in October 2025, with exceptions. Logs already saved stayed saved, and accounts flagged by the Times still had to be preserved. Then on December 9, 2025, the same magistrate ordered OpenAI to produce 20 million de-identified ChatGPT logs to the news plaintiffs and refused to pause the order while OpenAI objected. On January 5, 2026, District Judge Sidney Stein affirmed it.
To be fair, OpenAI fought that order, so on this point the company was on its users' side.
That's what worries me, though. The decision wasn't OpenAI's to make. Once your words are on someone else's server, the company's good intentions are only one factor. A judge is another, and so are a subpoena, a breach or an acquisition.
Sam Altman said much the same thing himself in July 2025, on Theo Von's podcast. "If you talk to a therapist or a lawyer or a doctor about those problems, there's legal privilege for it," he said. "And we haven't figured that out yet for when you talk to ChatGPT." As TechCrunch reported it, the practical upshot is that in a lawsuit, OpenAI could be legally required to produce those conversations.
A chatbot has no doctor-patient privilege and no attorney-client privilege. What you type into it is just data.
The leaks you don't see coming
Courts aren't the only risk.
In the summer of 2025, people discovered that shared ChatGPT conversations were turning up in Google search results. The share dialog had an opt-in checkbox that made a chat discoverable by search engines. People ticked it without fully understanding what it did, and personal conversations ended up indexed on the open web. OpenAI called it a "short-lived experiment" and removed it within about a day, while warning that some chats might stay visible for a while because of search engine caching.
Nobody was hacked and nothing was breached. A product team shipped a feature, and private conversations became public.
That's the threat model I'd like you to keep in mind. With a cloud AI service you aren't only trusting the company's security. You're also trusting every product decision it makes in future, every policy update, and every terms-of-service email you'll archive without reading.
A model running on your own machine has no product roadmap and no privacy policy to update, and nobody can subpoena its logs except by coming to you.
What "local" actually means
A large language model is, in the end, a file, and usually a big one of somewhere between two and forty gigabytes. That file holds the "weights," billions of numbers that encode what the model learned. Meta, Google, Mistral, Alibaba and others publish these files openly as open-weight models.
If you have the file and a program that can run it, you have a chatbot on your desk. Unplug the network cable and it keeps working.
I'll be honest about the limits, because this site doesn't do hype:
- A model that fits on your laptop won't match the biggest frontier models. Cloud models are still ahead on deep research, long and complex coding, and the hardest reasoning tasks.
- Local models make things up too. They're no more reliable than cloud models, so check anything that matters.
- For most everyday work, they're plenty. Summarizing documents, drafting emails, explaining concepts, rewording things and answering the embarrassing question are all well within a good small model's reach.
And for anything sensitive, a model that's good enough and private beats one that's brilliant and logged.
What hardware do you need?
These figures are rules of thumb, not specs:
| Memory (RAM) | What you can run | Experience |
|---|---|---|
| 8 GB | Small models, around 3–4 billion parameters | Useful but limited |
| 16 GB | 7–14 billion parameters | The sweet spot. Local AI starts to feel genuinely capable |
| 32 GB+ | Larger models | Approaching the cloud models for many tasks |
Apple Silicon Macs (M1 or newer) are especially well suited. Their processor and graphics share one pool of memory, which suits these models unusually well. On Windows, a dedicated NVIDIA graphics card makes a big difference, and LM Studio recommends at least 4 GB of graphics memory.
Leave room on your drive as well. Each model takes anywhere from 2 to 40 GB, and you'll probably end up with more than one.
Getting it running
You need one of these two tools, not both.
LM Studio is the one to start with if you've never opened a terminal. It's an ordinary desktop app: you search for a model, click download, and start chatting in a window that looks like the chatbots you already use.
Ollama works more under the hood. It has become the standard engine for running local models, and many other apps connect to it. It now also has a simple chat window of its own on Mac and Windows.
Both are free, and both are listed in section 04 of the hub page.
macOS
You'll need an Apple Silicon Mac (M1 or later) running macOS 14 Sonoma or newer. LM Studio doesn't support Intel Macs.
- LM Studio: download it, drag it to Applications, and open it.
- Ollama: download it from ollama.com, drag it to Applications, and open it. It runs in the background from your menu bar. If you use Homebrew, run:
brew install ollama
Windows
You'll need Windows 10 or 11, and 16 GB of RAM is recommended. LM Studio also supports Snapdragon-based ARM laptops.
- LM Studio: run the installer like any other program.
- Ollama: download and run
OllamaSetup.exe. Ollama runs as a background service with a system tray icon. - NVIDIA owners: update your graphics drivers first. Nothing else makes a bigger difference to speed.
Linux
- LM Studio: it ships as an AppImage. Download it, mark it executable, and run it:
chmod +x LM-Studio-*.AppImage
./LM-Studio-*.AppImage
- Ollama: there's an official one-line install script. You know who reads this site, though, and piping a script from the internet straight into your shell should make you flinch. Download it, read it, and then run it:
curl -fsSL https://ollama.com/install.sh -o ollama-install.sh
less ollama-install.sh
sh ollama-install.sh
The script is short. It installs Ollama and sets it up as a systemd service.
Your first conversation
With Ollama installed on any platform, you start a chat with two words and a model name:
ollama run <model-name>
That downloads the model and drops you straight into a chat, where you can type a question and press Enter. In LM Studio, you search for the model, download it, and open a new chat.
Which model should you pick first? On an 8 GB machine, choose a small model of around 3–4 billion parameters. On 16 GB, aim for something around 8 billion. The main families to look for are Llama, Gemma, Qwen and Mistral. New releases come out every month, so I keep current picks on the hub page rather than freezing them in this article.
Using the hub page without getting burned
"Download a forty-gigabyte file from the internet" is exactly the kind of instruction that should make a privacy-minded person nervous. That's why the Local LLM Downloads hub exists.
The page is a directory. Closed Network hosts nothing itself. It points you to where the models actually live and is organized by how careful you want to be:
- Primary hubs. Hugging Face is the default, joined by ModelScope (where many Qwen releases appear first), Kaggle and NVIDIA's catalog.
- One-command runtimes. Alongside Ollama and LM Studio, it lists Jan and GPT4All (both offline-first desktop apps), plus container-based options for homelab users.
- Trusted quantizers. Most people don't run a model at its original full size. They run a quantized copy, compressed to fit in ordinary amounts of memory at a small cost in quality. Third parties make these copies, so who made yours matters, and the hub lists the ones with a track record.
- Torrents and P2P. These are for models that have been pulled from the main hubs, or for when you'd rather not have an account recording every download.
The OpSec checklist
The most important section of the hub is the checklist. These are the points I'd want you to remember:
- If you torrent, every peer in the swarm can see your IP address. Use a VPN you trust, or download from a server.
- Verify the SHA-256 hash against the original publisher's. A magnet link proves you have the same file as everyone else. It doesn't prove the file is what it claims to be.
- Prefer
.safetensorsand.gguffiles. Older.binand.ptfiles use Python's pickle format, and a pickled file can execute code on your machine as soon as you load it. A model file can be malware. Ollama and LM Studio already use GGUF, but this matters as soon as you go off the beaten path. - Read the license. "Open weights" doesn't always mean you can use the model however you like.
- Check who uploaded it. A brand-new account offering a popular model with no history is a red flag.
- Prove it works offline. After the download finishes, turn off your Wi-Fi and ask the model something. If it answers, you know your conversation isn't going anywhere. If you want it enforced rather than just tested, firewall the runtime.
The fine print: two ways to undo all of this
"Local app" doesn't always mean "local model"
Ollama now also offers cloud models, tagged :cloud (for example, gemma4:cloud). These run on Ollama's servers, not yours, and need you to sign in. Ollama says it doesn't train on those prompts, but the whole point of this exercise is that your prompts never leave your machine.
To switch cloud features off completely, add this to ~/.ollama/server.json:
{
"disable_ollama_cloud": true
}
Or set the environment variable OLLAMA_NO_CLOUD=1. Restart Ollama afterwards, and its logs should show Ollama cloud disabled: true. This also turns off Ollama's web search feature.
Don't open it to the world
By default, Ollama listens only on 127.0.0.1:11434, which means only your own computer can talk to it. That's the setting you want.
Some tutorials tell you to set OLLAMA_HOST=0.0.0.0 so you can reach your model from a phone or another machine. That opens Ollama to your whole network, and one bad router rule later, to the internet. Researchers keep finding Ollama servers sitting wide open online for exactly this reason. If you want remote access to your local AI, put it behind something like Tailscale or WireGuard rather than opening a port.
Where I come down
You don't have to give up cloud AI, and I won't pretend I have. But you should get to choose what goes to the cloud. Most people currently can't, because they have no alternative installed.
So install the alternative. It takes about fifteen minutes: LM Studio if you want the easy route, Ollama if you want the engine.
Then set yourself one rule. Anything you wouldn't want read aloud in a courtroom (health, money, legal matters, relationships, work you're under NDA for) goes to the model on your own machine. Everything else can go wherever you like.
There's one more reason, and it connects to something we discussed on the show recently. If the internet goes down, whether for an afternoon or a week, a local model still works. It's a reference librarian that doesn't need a connection. That's a resilience argument rather than a privacy one, but it reaches the same conclusion.
Own the stack.
Start here → closednetwork.io/local-llm-downloads
— Simon
Sources & further reading
- Local LLM Downloads: Closed Network Hub (runtimes, model hubs, quantizers and the full OpSec checklist)
- New judge's ruling makes OpenAI keeping a record of all your ChatGPT chats one step closer to reality from TechRadar (June 2025 preservation ruling)
- OpenAI no longer has to preserve all of its ChatGPT data, with some exceptions from Engadget (October 2025)
- Order denying OpenAI's stay of the 20M-log production, Dec. 9, 2025 from S.D.N.Y. via CourtListener (primary document)
- Judge Stein order upholding ChatGPT logs discovery, Jan. 5, 2026 from ChatGPT Is Eating the World (links the order)
- Sam Altman warns there's no legal confidentiality when using ChatGPT as a therapist from TechCrunch
- OpenAI kills "short-lived experiment" where ChatGPT chats could be found on Google from Malwarebytes
- LM Studio system requirements from LM Studio docs
- Ollama cloud models and Ollama FAQ from Ollama docs
- Ollama's new app for macOS and Windows from Gigazine