rqlite is an open-source distributed relational database, with SQLite as its storage engine. v3.4.1 is out. This release includes some minor fixes as well as source code improvements (linting).
You can download the release from GitHub.
Philip O'Toole
Software engineering, distributed systems, databases, and the teams that build them
Philip O'Toole
Software engineering, distributed systems, databases, and the teams that build them
Ekanite v1.1.0 released
A new version of Ekanite, the syslog server with built-in search, has been released. v1.1.0 includes an important bug fix related to TCP connection handling, as well as some other minor fixes and improvements.
You can download v1.1.0 from the GitHub releases page.
The evolution of the rqlite source code
rqlite is an open-source distributed relational database, with SQLite as its storage engine.
Check out this cool video, showing the evolution of the source code.
rqlite v3.4.0 released
rqlite is an open-source distributed relational database, with SQLite as its storage engine. v3.4.0 is out, which includes a minor change to the HTTP responses.
You can download the release from GitHub.
Be a Conviction Engineer
I’m a conviction engineer.
To me it’s the only way to be an effective engineer, software developer, and technical leader. You’ve simply got to truly believe in the value of what you do, and you’ve got to believe in doing it the right way.
rqlite v3.3.0 released
rqlite is an open-source distributed relational database, with SQLite as its storage engine. v3.3.0 has been released and includes new functionality to control the Raft consensus subsystem, along with improvements to the CLI.
You can download the release from GitHub.
rqlite v3.2.1 released
rqlite is an open-source distributed relational database, with SQLite as its storage engine. v3.2.1 has been released and includes new functionality for cluster management, along with new documentation on running clusters. v3.2.0 also includes some bug fixes.
You can download the release from GitHub.
How to organise a Go project
Some fellow developers, using Go for the first time, recently asked me how to organise a Go project and for some high-level guidance on programming using the language.
I thought the most effective way to answer this question was to build a simple Go HTTP service, that provides a key-value store. It also includes a README, outlining my most important guidelines for Go programming. You can check it out here.
rqlite v3: Globally replicating SQLite
rqlite is an open-source distributed relational database, which uses SQLite as its storage engine. rqlite is written in Go and uses Raft to achieve consensus across a set of SQLite databases. It gracefully handles leader election, and can tolerate machine failure.
With the v3 release series, rqlite can now replicate SQLite databases on a global scale, with very little effort. Let’s see it in action using the AWS EC2 cloud.
rqlite v3.0.1 released with leader redirection
rqlite is an open-source distributed relational database, with SQLite as its storage engine. v3.0.1 has been released and it is a significant upgrade relative to the 2.0 series. The 3.0 series allows more sophisticated clusters to be built and simplifies rqlite client coding requirements.
Continue reading “rqlite v3.0.1 released with leader redirection”
New Python clients for rqlite
Thanks to Zac Medico, there are new Python clients available for rqlite — a pure Python client, and a SQLAlchemy mapper.
Check them out on GitHub.
rqlite moved to new repository
To help with the growth of rqlite, it’s been moved to a new repository, under a dedicated organization. Github made this transfer very easy.
I decided on a new logo too.
rqlite v2.2.1 released with HTTPS, Basic Auth, and user permissions
rqlite is a replicated relational database built on SQLite, with distributed consensus provided by the Raft consensus protocol. It gracefully handles leader election, and can tolerate machine failure.
Written in Go, v2.2.1 is out now.
Continue reading “rqlite v2.2.1 released with HTTPS, Basic Auth, and user permissions”
rqlite at the San Francisco Go Meetup
I made a presentation on rqlite tonight at the San Francisco Go Meetup. It was an enjoyable evening, and I had a chance to discuss why I built rqlite, how it works, and where it might go in the future.
What I learned from programming databases
Programming a database is fascinating work. I’ve been deeply involved with developing open source databases for the past two years and programming a database is possibly the most instructive project one can ever complete as a software developer.
What’s really striking however, is how much my attitude towards databases has changed over the past 6 years. From a state of disinterest, I’ve come to think of these systems as a pinnacle of software engineering.
Continue reading “What I learned from programming databases”
Pre-built binaries now available for rqlite and Ekanite
rqlite 2.0 released with configurable read-consistency
rqlite – replicated SQLite with new Raft consensus and API
rqlite provides robust replication for SQLite databases using the Raft consensus protocol. Coded in Go it ensures that all changes made to the leader SQLite database are replicated to all other nodes in the cluster, providing fault-tolerance and reliability.
It’s been 18 months since development of rqlite first started and it’s time for version 2.
Continue reading “rqlite – replicated SQLite with new Raft consensus and API”
rqlite and Hashicorp Raft Consensus
I’ve started replacing go-raft within rqlite with the implementation from Hashicorp. go-raft is no longer maintained, and I’ve good experience with the Hashicorp code, due to my work with InfluxDB and hraftd.
I’m also going to change the API, so it’s more useful. The existing implementation and API has been tagged as v1.0, so it’s still available.
You can follow the work on this branch, and I hope to merge it to master in the near future.
New browser interface for Ekanite
Ekanite is an open-source Syslog server with built in log search. Thanks to some nice work by Fabian Zaremba, Ekanite now supports searching your logs via a browser.
If you’d like to understand more about the design and development of Ekanite, check out this series of posts.
Gresham’s law and Slack
“Bad money drives out good.”
When is the last time you spoke with your fellow developer? I mean actually spoke? Or was it just over Slack?
I like really Slack, Flowdock, Hipchat and the like. I couldn’t do my job without them. But as with Gresham’s law, bad communication is driving out good.
Continue reading “Gresham’s law and Slack”
Revisiting syslog-gollector
It’s been 18 months since the first commit to my first significant Go project — syslog-gollector. After an initial burst of activity to create a functional Syslog Collector that streamed to Apache Kafka, the source code hadn’t been updated much since. But today I received a report that it no longer built, so I spent some time porting the code to the latest Shopify Sarama framework.
It was amusing to see how naive much of my early Go code was.
Running services is fun
While running services is hard, it’s also fun. It confronts you with a whole set of engineering challenges because you’re building — and running — large scale systems.
So after 18 great months working on the InfluxDB source, I’m returning to Services and have started a new role with Percolate, as Director of Engineering on their Data Platform team.
InfluxDB and the Raft consensus protocol
Designing a search system for log data — part 3
This is the last part of a 3-part series “Designing and building a search system for log data”. Be sure to check out part 1 and part 2.
In the last post we examined the design and implementation of Ekanite, a system for indexing log data, and making that data available for search in near-real-time. Is this final post let’s see Ekanite in action.
Continue reading “Designing a search system for log data — part 3”