Software engineering, distributed systems, databases, and the teams that build them
Software engineering, distributed systems, databases, and the teams that build them
I’ve started replacing go-raft within rqlite with the implementation from Hashicorp. go-raft is no longer maintained, and I’ve good experience with the Hashicorp code, due to my work with InfluxDB and hraftd. I’m also going to change the API, so…
Ekanite is an open-source Syslog server with built in log search. Thanks to some nice work by Fabian Zaremba, Ekanite now supports searching your logs via a browser. If you’d like to understand more about the design and development of…
It’s been 18 months since the first commit to my first significant Go project — syslog-gollector. After an initial burst of activity to create a functional Syslog Collector that streamed to Apache Kafka, the source code hadn’t been updated much…
This is the last part of a 3-part series “Designing and building a search system for log data”. Be sure to check out part 1 and part 2. In the last post we examined the design and implementation of Ekanite,…
This is the second part of a 3-part series “Designing and building a search system for log data”. Be sure to check out part 1. Part 3 follows this post. In the previous post I outlined some of the high-level…
This is the first part of a 3-part series “Designing and building a search system for log data”. Part 2 is here, and part 3 is here. For the past few years, I’ve been building indexing and search systems, for…
When you’d like to contribute to an open-source project it can be difficult to know where to start. Check out my latest post for the InfluxDB blog, explaining how we on the Core team have curated a set of issues,…
Another post for the InfluxDB blog — on testing the storage engines within InfluxDB. You can check it out here.
Hashicorp provide a nice implementation of the Raft consensus protocol, and it’s at the heart of InfluxDB (amongst other systems). I wanted to experiment with a simple system built using this particular Raft implementation, so was inspired by raftd to…
“Run into an obstacle in what you’re working on? Hmm, I wonder what’s new online. Better check.” If you haven’t already, you should start reading Paul Graham’s essays. In one on philosophy, Graham believes that many of the answers provided…
I always use the names of economists for my machines’ hostnames. keynes, friedman, marx, fisher, ricardo. So every so often the strange economics of open-source software hits me.
I’ve written my first post for the InfluxDB blog. In it I discuss the new statistics and monitoring system built into InfluxDB, starting with the 0.9.4 release. Functionality like this is critical when it comes to running a distributed database…
It’s been 418 days since my first Github commit of Go code. In that time I’ve written a Syslog-to-Kafka producer, a Raft-based distributed SQLite database, a near real-time log search system, and become a core developer of InfluxDB.
I came across another great video about software engineering management, this time by Bryan Cantrill. It’s a really great talk, and discussed in-depth — with plenty of humour thrown in — the importance of Mission to high-performing software developers.
This past week I attended Gophercon 2015, in Denver, CO. It was also a chance to get together with the rest of the InfluxDB team. And because the Go community is still relatively young and small, it was a great…
Well, almost nothing. Obviously it’s got something to do with computers since developers spend so much of their time in front of one. But software development is actually all about people. And successful software development even more so.
Search is everywhere. Once you’ve built search systems, you see its potential application in many places. So when I came across bleve, an open-source search library written in Go, I was interested in learning more about its feature set and…
Recently at InfluxDB we discussed how code reviews fit in during the various stages of development. It’s great to see the team reach consensus about how we should develop software. It made me think more deeply about why I remain…
I recently came across a talk on YouTube titled History of Software Engineering, given by Paolo Perrotta. Normally I find online videos to have a low information-to-time ratio, but this one was excellent. It’s not too long, with plenty of…
Bjarne Stroustrup has another very interesting paper on his website. Titled Software Development for Infrastructure, it discusses some key ideas for building software that has “…more stringent correctness, reliability, efficiency, and maintainability requirements than non-essential applications.” It is not a…
The Eudyptula Challenge is a series of programming tasks, with the goal of getting one up-to-speed on Linux kernel programming. When I first heard about it, it immediately intrigued me. I’ve written a few production Linux kernel modules in my…
Something just doesn’t feel right about node.js. After coding in it for almost a year, it’s been fun, but I’ve decided it’s just a waypoint to somewhere better.
Bjarne Stroustrup has great paper on his website titled Evolving a language in and for the real world: C++ 1991-2006. It provides fascinating insights on the development of the language, the challenges involved, and discusses interesting design ideas. If you…
SQLite is a “self-contained, serverless, zero-configuration, transactional SQL database engine”. However, it doesn’t come with replication built in, so if you want to store mission-critical data in it, you better back it up. The usual approach is to continually copy…
So far coding in Go has been fun. It comes with nice functionality that lets you know that the Go team really have been writing system software (useful stuff like this, and this). And then I read about the Go…