Overview
I built a completely air-gapped environment for running a large language model locally, so organizations can use AI on confidential material without any of it leaving their control. The motivation is simple: a law firm, hospital, or investment desk can't paste privileged documents into ChatGPT, because the moment data touches a third-party server it's an attorney-client, HIPAA, or insider-trading problem. This setup gives them the productivity without the exposure.
Everything runs offline on isolated hardware — document analysis, summarization, research — with zero outbound network traffic by design, not by policy.
The Architecture
I layered the controls so that a failure in one doesn't undo the whole thing.
Network isolation
Network interfaces disabled at the hardware level, kernel firewall rules dropping all outbound traffic, and no Wi-Fi, Bluetooth, or cellular present at all. I verified it the only way that counts — packet capture confirming zero packets ever leave. Even if the application layer were fully compromised, there's no path out.
Local inference
An open-weight model (Llama 3 / Mistral class) runs entirely in local memory via Ollama / llama.cpp, GPU-accelerated, with no cloud API calls. Model weights live on the encrypted disk, and system prompts are tuned for domain work — contract review, literature triage, financial analysis.
Encryption at rest
LUKS full-disk encryption with a passphrase-gated boot, plus encrypted swap so nothing sensitive lands in memory dumps. If the machine is stolen or seized, the data isn't readable.
Application sandboxing
The inference process is sandboxed with restricted filesystem access and resource limits, so a vulnerability in the AI tooling can't be used to roam the system or reach documents it has no business touching.
Stack
Platform
Dedicated GPU workstation (NVIDIA RTX) with network hardware removed
Arch Linux, hardened kernel, drop-all firewall
Ollama / llama.cpp for local inference (Llama 3, Mistral)
LUKS full-disk encryption with hardware-backed authentication
What I Took From It
The interesting part wasn't getting a model to run — it was proving a negative. "It can't leak" is only credible if you can demonstrate it, so the verification work (packet capture, filesystem inspection, process monitoring) was as much the deliverable as the system itself. The same pattern fits any regulated setting where the data genuinely cannot move: legal due diligence, clinical research on PHI, proprietary R&D, anything under attorney-client privilege.
Evidence & availability
Built and run on my own hardware, so there's nothing client-confidential here — which means I can show the working parts. Configuration, scripts, and a full walkthrough are available on request.