Turn a Photo Into a 3D Mesh Without Sending Your Data Anywhere

You've got a photo of an object and you want a 3D model of it. The obvious move is to feed it into some cloud service—except now you're uploading your data to someone else's server, waiting in a queue, and hoping the free tier doesn't run out. What if the whole pipeline just ran on the GPU sitting in your machine? That's the pitch behind Modly, an open source desktop app that does AI-powered image-to-3D mesh generation entirely locally.

What It Does

Modly is a desktop application for Windows, Linux, and Apple Silicon macOS that turns a photo into a 3D model using open source AI models running on your own GPU. It's built as an Electron-style app (the setup uses npm, a Node toolchain, and a Python backend under api/) with the actual inference handled by a separate Python environment. You install JS dependencies, spin up a Python virtualenv, and run it in development with npm run dev.

The architecture is deliberately modular. Rather than baking in a single model, Modly uses an extension system: each model is its own GitHub repository containing a manifest.json and whatever runtime files its type requires. There are official extensions for Hunyuan3D 2 Mini (in standard, Turbo, and Fast variants), TripoSG, and Trellis2 GGUF. You install them from the app's Models page by pasting in an HTTPS URL, then download the model weights or a variant. Process extensions are ready once installation and setup finish.

Why It's Cool

• Everything stays local. This is the whole point. Your photos and generated meshes never leave your machine, which matters if you're working with anything you'd rather not hand to a third party. The tradeoff is that you're supplying the compute—but if you already own a decent GPU, that's compute you've paid for.

• AMD cards aren't an afterthought. Support for AMD GPUs runs through ROCm on Linux and Windows, and the README notes a Radeon card is detected automatically, with extensions steered to ROCm PyTorch wheels—no separate ROCm install required. That's a pleasant surprise given how often this class of tooling is CUDA-only.

• The extension system keeps it flexible. Because models live in separate repos, the core app doesn't have to ship with every model baked in. You pick the one that fits your hardware and quality needs—a Fast variant when you want speed, a heavier model when you want detail. The README also mentions support for model nodes whose weights are split across multiple Hugging Face repositories via model_sources, with validation and sequential downloads handled in one action.

• It respects your workspace. Workflow wiring is validated before a run, and invalid graphs stay put with inline or toast warnings instead of silently dropping your current mesh view. Imported meshes can be smoothed and decimated in-app, with optimized results written back into the workspace. Small details, but the kind that tell you someone actually used this thing.

• A live RAM indicator in the top bar. Trivial, maybe, but when you're running model inference locally, knowing your memory situation at a glance is genuinely useful.

How to Try It

The easiest path is the Releases page, where you'll find installers for Windows, Linux, and Apple Silicon macOS. If you'd rather run from source, clone the repo and use the launch script:

# Windows
launch.bat

# Linux / macOS
./launch.sh

For development, set up both halves of the stack:

# 1. Install JS dependencies
npm install

# 2. Set up the Python backend
cd api
python -m venv .venv
.venv\Scripts\activate     # Windows
source .venv/bin/activate  # Linux / macOS
pip install -r requirements.txt

# 3. Run in development
npm run dev

To check your setup, the README suggests npm test, a TypeScript type check with tsc --noEmit, and npm run build. Once you're in, head to the Models page, click Install from GitHub, paste an extension URL, and download your model. Package Apple Silicon builds with npm run package:mac. Full details are in the repository.

Final Thoughts

Modly is for people who already have a capable GPU and would rather run image-to-3D generation on their own terms than rent it by the request. It's not trying to be a polished commercial product—it's a working local pipeline with a sensible extension model and honest platform notes. If you're on Apple Silicon, Linux, or Windows with either an NVIDIA or AMD card, it's worth a look. The local-first approach is the right instinct, and the extension system means the app can grow as better open models show up.

Follow @githubprojects for more developer tools and open source projects.