rqlite 9.0: Real-time Change Data Capture for Distributed SQLite

rqlite is a lightweight, user-friendly, open-source, distributed relational database. It’s written in Go, employs Raft for distributed consensus, and uses SQLite as its storage engine.

rqlite 9.0 introduces Change Data Capture (CDC), enabling you to stream the changes occurring to the underlying SQLite database to other systems. With CDC, rqlite is no longer just a distributed database—it’s a live event source. So let’s take a look at what the latest release allows us to do, and some of the design challenges that needed solving — especially building Change Data Capture on SQLite.

Continue reading “rqlite 9.0: Real-time Change Data Capture for Distributed SQLite”

On the way to rqlite 9.0

rqlite is a lightweight, open-source, distributed relational database written in Go, which uses SQLite as its storage engine.

Development for the 9.0 release has started, and should be an important release. The main goals for 9.0 include:

  • Change Data Capture (CDC). CDC will allow you to stream changes from rqlite as they happen. This long-requested feature will make it easy to integrate rqlite with caches, pub-sub systems, and more.
  • API improvements. This will include better error codes, while maintaining current behaviour.

Upgrading from the 8.x release series should be seamless.

Consistency Over Availability: How rqlite handles the CAP Theorem

rqlite is a lightweight, user-friendly, open-source, distributed relational database. It’s written in Go and uses SQLite as its storage engine.

When it comes to distributed systems the CAP theorem is an essential concept. It states that it’s impossible for a distributed database to simultaneously provide Consistency, Availability, and Partition tolerance. The challenge is in the face of a network partition, a database can only be available or consistent, but not both.

Let’s take a look at the CAP theorem and see how rqlite fits into this fundamental trade-off.

Continue reading “Consistency Over Availability: How rqlite handles the CAP Theorem”

rqlite 8.38.0 to 8.42.0: Leader Stepdown, Google Cloud Backups, and More

rqlite is a lightweight, user-friendly, open-source, distributed relational database. It’s written in Go and uses SQLite as its storage engine. Recent versions—8.38.0 through 8.42.0—introduce several notable improvements, including Google Cloud Storage support, new HTTP APIs for cluster control, and improvements to Backup support.

Continue reading “rqlite 8.38.0 to 8.42.0: Leader Stepdown, Google Cloud Backups, and More”

The Developer’s Manifesto for the Age of LLMs

We built the old software systems by hand. Line by line. Module by module. That era is ending.

Something profound is happening. For the first time in the history of software, we are not the only entities capable of understanding and generating code. And yet the role of the open-source developer has never mattered more.

This post is a statement of what we must preserve, what we must change, and how we must lead—now that we write software alongside intelligent systems.

Continue reading “The Developer’s Manifesto for the Age of LLMs”

GPT is writing the GitHub issues, Copilot is fixing them

rqlite  is a lightweight, open-source, distributed relational database written in Go. It uses SQLite as its storage engine and Raft for consensus.

Last week, I wrote about my experience with language models, how they evolved from simple Python helpers to full-blown design partners and even a sort of AI Product Manager for rqlite, advising on features like Change-Data-Capture. I talked about assigning entire GitHub issues to agents, treating them like junior engineers who just wouldn’t quit until the PR was merged.

It was impressive, a fundamental shift in how I thought about shipping code. But that was last week. This week, the workflow has taken another, mind-blowing, leap.

Continue reading “GPT is writing the GitHub issues, Copilot is fixing them”

rqlite development: the Agents are here

rqlite  is a lightweight, open-source, distributed relational database written in Go. It uses SQLite as its storage engine and Raft for consensus.

I’ve been developing rqlite for about ten years, and until recently, the workflow hadn’t changed much. I write the code. Others contribute here and there. GitHub issues are filed, triaged, and resolved. The usual.

Then language models showed up, and things started to shift.

Continue reading “rqlite development: the Agents are here”

rqlite turns 10: Observations from a decade building Distributed Systems

rqlite is a lightweight, open-source, distributed relational database written in Go, which uses SQLite as its storage engine and Raft for consensus.

Ten years ago, I tagged the first release of rqlite—a project I started to explore distributed consensus, to learn Go, and to build something useful. I had no roadmap, no plans for scale, and certainly no idea it would still be active a decade later.

Continue reading “rqlite turns 10: Observations from a decade building Distributed Systems”

rqlite 8.37.0: Automatic reloading of X.509 Certificates

rqlite is a lightweight, user-friendly, open-source, distributed relational database. It’s written in Go and uses SQLite as its storage engine.

Version 8.37.0 adds automatic reloading of X.509 certificates and keys. rqlite now monitors the relevant files and reloads them without requiring a restart—removing the need for manual intervention during certificate rotation.

With the CA/Browser Forum reducing TLS certificate lifetimes to 47 days, automated reloads will help operators stay compliant with less effort.

Gemini-driven code reviews in GitHub

I recently installed the Gemini code review tool for GitHub. It’s free, and installation was easy. Today Gemini found its first real issue with my code. While the bug was very minor, it’s helpful to even have these minor bugs addressed during development.

Otherwise the review feedback provided by Gemini so far has been fine, but nothing spectacular. The rqlite code base is highly structured with clearly-defined interfaces and unit tests. This probably reduces the chance for major improvement.

New Java client for rqlite

rqlite is a lightweight, open-source distributed relational database implemented in Go, and which uses SQLite as its storage engine.

I have developed a new Java client for rqlite. It provides minimal abstraction and interacts directly with the rqlite HTTP API. Of particular note is the method of its creation: I fed the latest Go client source code into ChatGPT (o3-mini-high), which then produced a corresponding Java implementation. The initial output required only minor adjustments and now appears to be fully functional.

Who needs a transpiler when you’ve got GenAI?

New rqlite Go client

rqlite is a lightweight, open-source distributed relational database implemented in Go, and which uses SQLite as its storage engine.

I’ve decided to code a new, high-quality HTTP client library for rqlite. My hope is that this library will be useful on its own or as a foundation for higher-level libraries. Once written I may then have GPT generate versions in other languages from the Go source.

How rqlite is tested

rqlite is a lightweight, open-source, distributed relational database written in Go, and built on SQLite and Raft. With its origins dating back to 2014, its design has always prioritized reliability, and quality. The robustness of rqlite is also a testament to its disciplined testing strategy: after more than 10 years of development and deployments, users have reported fewer than 10 instances of panics in production.

Testing a distributed system like rqlite is no small feat. It requires careful consideration of various layers: from individual components to the entire system in operation. Let’s explore how rqlite is tested, following its philosophy of maintaining quality without unnecessary complexity.

Continue reading “How rqlite is tested”

rqlite 8.34.0 to 8.36.0: New Features and Enhancements

rqlite is a lightweight, user-friendly, open-source, distributed relational database. It’s written in Go and uses SQLite as its storage engine. rqlite has received several important updates in recent weeks. These updates, spanning versions 8.34.0 through 8.36.0, introduce new features and make significant improvements to the database’s implementation and usability.

Continue reading “rqlite 8.34.0 to 8.36.0: New Features and Enhancements”

New Configuration Guide for rqlite

rqlite is a lightweight, user-friendly, open-source, distributed relational database. It’s written in Go and uses SQLite as its storage engine.

I’ve added a new Configuration Guide for rqlite, which has been regularly requested. rqlite has many options and although the defaults work for the vast majority of use cases, occasionally rqlite requires some operational tweaks.

Faster Reads, Same Guarantees: Linearizable Consistency in rqlite 8.32

rqlite is a lightweight, open-source, distributed relational database. It’s written in Go, built on the Raft consensus algorithm, and uses SQLite as its storage engine. With release 8.32, rqlite now supports fast, reliable, Linearizable reads—delivering strong consistency guarantees without the performance costs of the past.

Distributed systems present many challenges, but there is only one really hard problem: ensuring a consistent state across nodes, even in the face of network partitions, machine failures, or unexpected bugs. And a closely related requirement is ensuring up-to-date data reads — both requirements are the essential challenges that every distributed database must address.

Continue reading “Faster Reads, Same Guarantees: Linearizable Consistency in rqlite 8.32”

Concurrency Control in Go: Inside rqlite’s Custom Synchronization Primitives

rqlite is a lightweight, open-source, distributed relational database written in Go, which uses SQLite as its storage engine.

Concurrency and synchronization are at the heart of any database — and distributed systems too. Today, I’d like to share the set of synchronization primitives and race-free data structures I’ve developed for rqlite, and explain how I use them to program key aspects of the database. They might even be useful to others confronted with similar requirements.

Continue reading “Concurrency Control in Go: Inside rqlite’s Custom Synchronization Primitives”