About this video
The era of renting intelligence from Silicon Valley is over. This video provides a comprehensive guide for CEOs and founders on setting up local AI agents on a Mac, moving away from the SaaS subscription model toward personal software ownership. We cover everything from hardware specifications to the nuances of local model serving. Key Takeaways: - Hardware Priority: Why RAM is more important than CPU for local AI tasks. - Model Selection: Navigating Hugging Face and understanding MLX quantisation. - The Stack: How to use Ollama, LM Studio, and OMLX to serve models. - Agent Harnesses: Setting up Open Code and Claude Code for local workflows. - Performance Reality: Managing thermals and understanding the speed trade-offs of local vs cloud.
Terminal cheat sheet
- cd
- Change directory — move into a folder, or use ../ to go back up a level.
cd folder-name · cd ../ - ls
- List the files in the current directory.
ls - mkdir
- Create a folder named foldername in the current directory.
mkdir foldername - touch
- Create a new, empty file.
touch filename.md - !
- Copy the last command you ran.
!
OpenCode config~/.config/opencode/opencode.json
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"omlx": {
"npm": "@ai-sdk/openai-compatible",
"name": "oMLX (local)",
"options": {
"baseURL": "http://127.0.0.1:8000/v1",
"apiKey": "1234"
},
"models": {
"mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit": { "name": "Qwen 3.6 OptiQ 35B 4bit" },
"mlx-community/Qwen3.6-35B-A3B-4bit": { "name": "Qwen 3.6 35B 4bit" },
"mlx-community/Qwen2.5-Coder-7B-Instruct-4bit": { "name": "Qwen 2.5 Coder 7B Instruct 4bit" },
"mlx-community/Qwen2.5-Coder-32B-Instruct-8bit": { "name": "Qwen 2.5 Coder 32B Instruct 8bit" },
"mlx-community/Qwen2.5-Coder-32B-Instruct-4bit": { "name": "Qwen 2.5 Coder 32B Instruct 4bit" },
"mlx-community--gemma-4-12B-it-8bit": { "name": "Gemma 4 12B" },
"mlx-community/Ornith-1.0-35B-4bit": { "name": "Ornith 1.0 35B" },
"mlx-community/Ornith-1.0-9B-4bit": { "name": "Ornith 1.0 9B" },
"mlx-community/Qwen3.5-35B-A3B-4bit": { "name": "Qwen 3.5 35B" },
"mlx-community/gpt-oss-20b-MXFP4-Q8": { "name": "GPT-OSS 20B MXFP4 Q8" }
}
}
}
}Pi config~/.pi/agent/models.json
{
"providers": {
"omlx": {
"baseUrl": "http://127.0.0.1:8000/v1",
"api": "openai-completions",
"apiKey": "1234",
"models": [
{ "id": "mlx-community/Qwen3.6-35B-A3B-8bit", "name": "Qwen 3.6 35B A3B 8bit" },
{ "id": "mlx-community/Qwen2.5-Coder-7B-Instruct-4bit", "name": "Qwen 2.5 Coder 7B Instruct 4bit" },
{ "id": "mlx-community/Qwen2.5-Coder-32B-Instruct-4bit", "name": "Qwen 2.5 Coder 32B Instruct 4bit" },
{ "id": "mlx-community/Ornith-1.0-35B-4bit", "name": "Ornith 1.0 35B 4bit" }
]
}
}
}openclaw config~/.openclaw/openclaw.json
{
models: {
providers: {
lmstudio: {
baseUrl: "http://localhost:1234/v1",
apiKey: "${LM_API_TOKEN}",
api: "openai-completions",
models: [
{
id: "qwen/qwen3-coder-next",
name: "Qwen 3 Coder Next",
reasoning: false,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 128000,
maxTokens: 8192,
},
],
},
},
},
}The Era of Renting Intelligence is Over: A Guide to Local AI Agents on Mac
The era of renting intelligence from Silicon Valley is over. For years, founders and CEOs have been comfortable paying the 'AI tax'—subscriptions to OpenAI, Anthropic, and various SaaS middlemen—to access large language models. But as hardware evolves, the narrative is shifting. We are moving towards a world of personal software where you own the model, the data, and the compute.
Running local AI agents on a Mac is no longer a hobbyist's pipe dream; it is a strategic advantage. This guide covers everything from hardware selection to the software harnesses that will replace your cloud dependencies.
The Hardware: RAM is the New Gold
If you are building a local AI workstation, the rules of the game have changed. In the past, CPU clock speeds were the primary metric. Today, it is all about Unified Memory (RAM).
The more RAM you have, the larger and more sophisticated the models you can run. While a base M5 chip is capable, the real work starts at 48GB of RAM. If you are serious about agentic workflows—AI that actually performs tasks and writes code—you should be aiming for 96GB to 128GB. For those looking to compete with frontier-level models, a Mac Studio with 512GB or even a terabyte of RAM is the ultimate goal.
Understanding the Landscape: Models and Quantization
You do not need to run the 'full fat' version of every model. Through a process called quantization, we can shrink models down to 4-bit or 8-bit versions that retain most of their intelligence while fitting into your Mac's memory.
We generally look for models in the MLX format, which is specifically optimised for Apple Silicon. Whether you choose a 'Dense' model (all parameters active) or a 'Mixture of Experts' (MoE) model (only a subset of parameters active per request), the goal is to find the balance between speed and reasoning that suits your specific business tasks.
The Software Stack: Serving Your Own AI
To run these models, you need a way to 'serve' them locally. While tools like Ollama and LM Studio are popular for beginners, OMLX offers advanced features like SSD KV caching. This allows your machine to store the 'history' of your conversations on your disk, speeding up subsequent requests.
Once a model is served, you can point your 'harnesses' (the UI you actually interact with) to your local endpoint. Tools like Open Code or Claude Code allow you to interact with your files and data without a single byte ever leaving your local network.
Why This Matters for Founders
Privacy is the obvious benefit, but the 'SaaS is dead' narrative goes deeper. When you run local agents, you are not limited by rate limits, subscription tiers, or the changing whims of a cloud provider's terms of service. You are building a private, permanent intellectual asset for your company.
The setup might take an afternoon, but the independence it provides lasts forever. It is time to stop renting your brain and start owning your infrastructure.
Transcript▾
### The Local AI Setup Guide for Mac
**Introduction** A lot of my videos jump into working with local agents, but I wanted to provide a baseline setup guide for running local agents on a Mac. This covers everything from buying the hardware to choosing and running your first agent.
**Hardware Selection: Desktop vs Laptop** - **Form Factor:** Desktops (Mac Studio/Mini) offer better thermals and more RAM than laptops. - **The RAM Rule:** More RAM is always better. - **Minimums:** 16GB–25GB is for basic chat; 48GB is the starting point for real work; 96GB–128GB is ideal for coding and agentic workflows. - **The Dream:** 512GB to 1TB of RAM on a Mac Studio allows you to compete with frontier cloud models.
**Chip Generations** - **M4/M5:** These are where Apple got serious about AI. M5 chips are excellent for local agents. - **M3 Ultra:** The bare minimum in the Ultra series for high-end performance. - **Bandwidth:** Higher-tier chips (Max/Ultra) offer better memory bandwidth, which is crucial for model speed.
**Understanding Models** - **Hugging Face:** The primary repository for open-source models. - **Quantization:** Reducing model size (e.g., 4-bit or 8-bit) to fit in memory with minimal intelligence loss. - **Dense vs. MoE:** Dense models load all parameters; Mixture of Experts (MoE) models use active subsets for efficiency. - **Context (KV Cache):** The 'memory' of your conversation. Beware of 'context rot' as history grows.
**Serving Tools** - **Ollama:** Great for beginners; supports cloud shortcuts. - **LM Studio:** A user-friendly wrapper for Hugging Face models. - **OMLX:** Recommended for advanced users due to SSD KV caching capabilities.
**Setting Up the Harness** - **Open Code / Claude Code:** Terminal-based or UI-based tools that connect to your local model. - **Configuration:** Most tools require a simple JSON setup to point to your local API endpoint. - **Terminal Basics:** Using tools like Warp and basic commands (cd, ls, mkdir) to manage your local agent environment.
**Performance Factors** - **Pre-fill Speed:** Local models have a slow initial response time on Mac. - **Thermals:** High-intensity AI work will max out your GPU and generate significant heat. - **Multitasking:** Screen recording or other heavy apps will slow down your local AI.