Run Open Models on Your Mac With One Local Server

You've probably got a Mac with more unified memory than you know what to do with, and you've probably also noticed that running open models locally usually means stringing together a Python environment, a model loader, and a client that speaks a slightly different API than the last one. MLX-Serve takes a different approach: one native app, one local server, and a set of APIs your existing tools already know how to talk to.

What It Does

MLX-Serve is a local server for Apple Silicon that runs open models for chat, coding agents, images, video, music, speech, and 3D. It's a native Mac app plus a command-line server, and it speaks the OpenAI, Anthropic, and Ollama APIs. That last part matters more than it sounds—it means you point any client that already supports one of those APIs at http://localhost:11234 and it just works. There's also a built-in web console if you open that address in a browser.

It runs entirely offline, needs no Python and no account, and is MIT licensed. The app requires macOS 26.2 or later on Apple Silicon, suggests a model that fits your machine, and updates itself. Under the hood it's built on MLX, Apple's array framework, though the README keeps the focus on what you can actually do with it rather than the internals.

Why It's Cool

• The API compatibility is the real draw. Supporting OpenAI, Anthropic, and Ollama formats means you don't have to rewrite your tooling. If you've been using Ollama, the CLI will feel familiar—mlx-serve run, pull, and serve are right there. If you're on Claude Code, mlx-serve launch claude starts it against your Mac with the model's actual context window configured.

• It goes well beyond text. The README lists FLUX.2, Qwen-Image, Krea, and MOSS-Image for images; LTX-Video and MiniMax-H3 for video; ACE-Step and YuE2 for music; Qwen3-TTS with voice cloning for speech; and Hunyuan3D for 3D. All of that is reachable from the app or over HTTP. Speech-to-text covers 30 languages with word timings and speaker identification, and the menu bar can record a call and write up notes.

• Bigger models than your RAM. This is the clever bit: GLM-5.3-Flash, MiMo-V2.6-Flash, and DeepSeek-V4.1-Flash can run on a 128 GB Mac by reading their experts from the SSD. If one Mac isn't enough, two Macs on a Thunderbolt 5 cable can run a single model together, and Macs on your network can use each other's models. That's a genuinely practical answer to the "my model doesn't fit" problem.

• The assistant angle. You can chat with your documents, talk to it hands-free, give agents a sandboxed shell and MCP tools, or schedule tasks in plain English. It's the kind of thing that usually requires stitching three services together.

• It's not Mac-only. Every release ships Linux downloads for NVIDIA GPUs—x64 for RTX 20 through 50 cards, arm64 for DGX Spark, GH200, and Jetson Thor. Windows with an NVIDIA card gets a build script. There's even an iPhone app with the same engine.

How to Try It

The fastest path is the app:

1. Download the DMG from the releases page, drag it to Applications, and open it. It'll suggest a model that fits your Mac.

If you prefer Homebrew:

brew tap ddalcu/mlx-serve https://github.com/ddalcu/mlx-serve
brew trust ddalcu/mlx-serve
brew install --cask mlx-serve   # the app
brew install mlx-serve          # or only the command-line server

And if you're coming from Ollama, the terminal flow will be immediately familiar:

mlx-serve run gemma4          # downloads Gemma 4 E4B, serves it and chats right here
mlx-serve pull gemma4:26b     # download only, resumable
mlx-serve serve               # serve everything you downloaded; models load when asked for

The server listens on http://localhost:11234. Point your client at it, or open it in a browser for the web console. Full details are in the repository, with additional docs for CLI usage, integrations, the API, and an FAQ.

Final Thoughts

MLX-Serve is aimed squarely at people who want local models without the usual setup tax—Mac users who already have the hardware and just need something that serves it properly, and developers who want their existing OpenAI- or Anthropic-compatible tools to keep working. The API compatibility and the SSD-offload trick for large models are the standout features, and the breadth of media support is a nice bonus if you need it. The macOS 26.2 requirement is worth checking before you commit, and Linux support is there but clearly secondary. If you've been putting off running models locally because the tooling felt like a project in itself, this is a reasonable place to start.