Marmot: SQLite Replication Made Simple
If you've ever built an app with SQLite, you know its strengths: it's fast, reliable, and embedded. But you also know its classic weakness: syncing data between devices or to a central server can be a real headache. What if you could keep SQLite's simplicity but add seamless, automatic synchronization?
That's exactly what Marmot does. It's a tool that brings effortless replication to SQLite databases, letting you focus on your app logic while it handles keeping data in sync across nodes.
What It Does
Marmot is a distributed SQLite replication layer. In simpler terms, it watches your SQLite database for changes (inserts, updates, deletes) and automatically replicates those changes to other SQLite databases in its network. It works by capturing a snapshot of your database and then continuously streaming changes as a log, which other nodes can consume to stay up-to-date.
It's designed to be simple to embed and operates without external dependencies—it's just a Go library. You can run multiple nodes that follow each other, creating flexible topologies like star or chain setups for backup, aggregation, or multi-device sync.
Why It's Cool
The clever part is how non-intrusive it is. Marmot doesn't require you to change your application's database access patterns. It works as a sidecar or embedded process, tapping into SQLite's snapshot and WAL (Write-Ahead Log) to capture changes. This means your app keeps talking to SQLite as usual.
Some standout features:
- No Server Required: Nodes connect directly to each other (peer-to-peer), though you can use a NATS server as an optional message bus for complex network setups.
- Incremental Snapshots: After the initial sync, only changes are transferred, which is efficient and fast.
- Flexible Topologies: You can set up nodes to follow one or many sources, enabling hub-and-spoke models for data aggregation or simple chain replication for backups.
- Embeddable: It's a Go package, so you can integrate it directly into your application, keeping your stack simple.
Think about use cases like syncing data from edge devices to a central server, keeping multiple instances of a desktop app updated, or creating a simple distributed data store without the complexity of a full database cluster.
How to Try It
The quickest way to see Marmot in action is to check out the repository and run the example. You'll need Go installed.
# Clone the repository
git clone https://github.com/marmotdata/marmot
cd marmot
# Run the basic example
go run ./examples/simple
This example starts a couple of nodes and shows replication in action. For a more detailed setup, the README has instructions on building a binary, creating snapshots, and setting up follower nodes. You can also explore using the optional NATS integration for production environments where direct P2P connections aren't possible.
Final Thoughts
Marmot feels like a pragmatic solution to a specific, common problem. It doesn't try to turn SQLite into PostgreSQL. Instead, it adds just one missing piece—synchronization—in a way that stays true to SQLite's ethos of simplicity.
If you're building an application that needs offline-first capabilities or needs to sync data across several instances, Marmot is definitely worth a look. It removes a huge chunk of custom sync code you'd otherwise have to write and maintain. It's a classic example of a focused tool doing one job very well.
Check out the project on GitHub to dive into the details and see if it fits your next project.
@githubprojects