Category Uncategorized

Building a distributed key-value store using Raft

Hashicorp provide a nice implementation of the Raft consensus protocol, and it’s at the heart of InfluxDB (amongst other systems). I wanted to experiment with a simple system built using this particular Raft implementation, so was inspired by raftd to…

Coding like it’s 1999

“Run into an obstacle in what you’re working on? Hmm, I wonder what’s new online. Better check.” If you haven’t already, you should start reading Paul Graham’s essays. In one on philosophy, Graham believes that many of the answers provided…

Who watches the watchers?

I’ve written my first post for the InfluxDB blog. In it I discuss the new statistics and monitoring system built into InfluxDB, starting with the 0.9.4 release. Functionality like this is critical when it comes to running a distributed database…

400 days of Go

It’s been 418 days since my first Github commit of Go code. In that time I’ve written a Syslog-to-Kafka producer, a Raft-based distributed SQLite database, a near real-time log search system, and become a core developer of InfluxDB.

Leadership without Management

I came across another great video about software engineering management, this time by Bryan Cantrill. It’s a really great talk, and discussed in-depth — with plenty of humour thrown in — the importance of Mission to high-performing software developers.

Running services is hard

I’ve recently been thinking about why running Services is particularly hard. By Services I mean Software-as-a-Service platforms. During the years, I’ve written software for many different systems — embedded software, web services, databases, and distributed systems, but being involved with…

Gophercon 2015

This past week I attended Gophercon 2015, in Denver, CO. It was also a chance to get together with the rest of the InfluxDB team. And because the Go community is still relatively young and small, it was a great…

InfluxDB 0.9.0 released

The first version of the 0.9.0 series of InfluxDB has been released. It’s alpha-quality software but all of us on the InfluxDB team are very excited to see the software reach this stage. You can read more about the release…

Reviewing Elasticsearch Cookbook

I recently acted as one of the official technical reviewers for ElasticSearch Cookbook – Second Edition by Alberto Paro. Published by Packt Publishing, the book contains a large number of “recipes” for elasticsearch.

Code reviews still rule

Recently at InfluxDB we discussed how code reviews fit in during the various stages of development. It’s great to see the team reach consensus about how we should develop software. It made me think more deeply about why I remain…

History of Software Engineering

I recently came across a talk on YouTube titled History of Software Engineering, given by Paolo Perrotta. Normally I find online videos to have a low information-to-time ratio, but this one was excellent. It’s not too long, with plenty of…

Book Review: Cassandra High Availability

Packt recently asked me to review their new publication Cassandra High Availability, written by Robbie Strickland. I’ve worked with Cassandra in the past — early designs of Loggly‘s 2nd generation Log analytics platform used Cassandra as its authoritative store for…

Software Development for Infrastructure

Bjarne Stroustrup has another very interesting paper on his website.  Titled Software Development for Infrastructure, it discusses some key ideas for building software that has “…more stringent correctness, reliability, efficiency, and maintainability requirements than non-essential applications.”  It is not a…

Eudyptula Challenge

The Eudyptula Challenge is a series of programming tasks, with the goal of getting one up-to-speed on Linux kernel programming. When I first heard about it, it immediately intrigued me. I’ve written a few production Linux kernel modules in my…

Drop, Throttle, or Buffer

Real-time — or near real-time — data pipelines are all the rage these days.  I’ve built one myself, and they are becoming key components of many SaaS platforms. SaaS Analytics, Operations, and Business Intelligence systems often involve moving large amounts…

Measure Everything

Tomorrow I join the team at InfluxDB, something I’m really excited about. I’m really looking forward to coding in Go full-time — it’s a language with real promise, a nice clean tool chain, and a very active community.

Replicating SQLite using Raft Consensus

SQLite is a “self-contained, serverless, zero-configuration, transactional SQL database engine”.  However, it doesn’t come with replication built in, so if you want to store mission-critical data in it, you better back it up. The usual approach is to continually copy…