Running A Full Bitcoin Node

One of the things I set up and got working recently was operating a full node on the Bitcoin network. There were two motivations for me for doing this.

The first is that I was generally interested in learning more about Bitcoin and scripting the way that I interact with Bitcoin. Running a full node and using the official bitcoin-cli client is a good way to do both of these things; it's also beneficial to the Bitcoin community as a whole.

The second is that after the recent near death of my phone I realized that I'm better with backing up and scripting tasks on servers than I am at doing the same with my phone. Previously I would write down my Mycelium backup codes on a post-it note and then lose the post-it note. By contrast writing an automated cron job that makes a copy of my wallet, encrypts it, and then puts it in a private S3 bucket is easy, cheap, and has good security. There are some good arguments that using a physical copy of the wallet can be more secure, but in practice the way I organize my life I think that th solution of dumping and encrypting a wallet on a Unix server using cron is easier. I also think that it would be harder for an attacker to get a login on one of the Unix servers I administer (which only allow non-root key-based SSH logins with a single key that I am good at securing) than it would if they had my phone (which has encryption at multiple layers, but all based around short numeric PINs that are easy for me to remember).[1]

Additionally, it seems plausible that it would be easier (and funner) for me to write and automate things for managing multiple wallets and the money in them myself rather than relying on the solution the Mycelium developers come up with. As an example, recently I wanted to experiment with JoinMarket. Doing this on a server is easy, doing this with a phone-based wallet would not be.

I looked into a couple of options for running a full node:

I determined that I didn't want to use my laptop mostly because of the overhead of running bitcoind. It uses more disk space, RAM, and network bandwidth than I was happy with for something that will always be on on my laptop. Particularly on the last point, I'd only want to run bitcoind when I was on a good network link and that would mean trying to write complicated scripts to govern if bitcoind should be running or not based on the perceived quality of my internet link.

The mini-ITX server in my apartment thing seemed cool at first, but once I specced it out it didn't really make sense. I wanted to build a fanless system capable of running RAID 1 (arguably this is overkill, but whatever) with sufficient storage. I specced something out with this motherboard using a RAID 1 with mSATA connectors, and after buying the motherboard, CPU, RAM, power supply, disks, and enclosure it worked out to something like $800. Arguably such a system is overkill (e.g. if backups are being made to S3 then RAID is not necessary, and w/o RAID a cheaper motherboard would be used) but that was the system that was compelling to me.

By contrast I looked at using a g1-small machine type on GCE. For the same cost as the mini-ITX server I could run an instance type like this with storage and whatnot for something like two years. Plus with the GCE instance it would be easier to do things like OS or hardware upgrades (boot a new machine, copy the wallet over, power off the old VM). Based on my personal experiences running home servers and running machines on GCE, I felt like the GCE solution would be as reliable or more reliable than something connected to the internet/power in my apartment unit.

If you're really obsessed with security the AWS/GCE solution is probably a nonstarter, since it means you have your wallet on a machine not in your physical posession, which could be subpoenaed by law enforcement, etc. For me, however, the security model of GCE is sufficient with balancing my convenience and my actual real world security profile.

After I decided to go with the GCE route, I spun up a few g1-small images and played around with them until I got to a happy state. I tried using Debian Jessie, Fedora, and Ubuntu 14.04 LTS. The Ubuntu LTS is by far the easiest to get working because you can install bitcoind via a PPA. You can compile bitcoind on Jessie or Fedora easily enough, but it's a fair amount of work and among other things involves building and linking against a specific version non-packaged version of BerkeleyDB 4.8. I also found that compiling bitcoind uses a lot of memory due to the style of C++ coding they're using, and I was worried that perhaps one day you'll need even more memory to compile and link bitcoind which would necessitate an instance type upgrade for something that is a relatively infrequent task. Plus, if I was running a bitcoind I built from scratch it would be my job to keep it up to date, whereas installing packages out of a PPA from someone I trust who maintains it is a lot easier. So as much as I love dislike Ubuntu, I ended going with Ubuntu LTS.

After getting the image running and installing bitcoind, I found that it takes a few days to sync the block chain using this instance type. The limiting factor in syncing the block chain on this instance type is the CPU. The g1-small instance type only has 1 CPU which is anemic and can't even close to keep up with the bandwidth on the instance. When I have to spin up another instance for upgrades I will probably figure out how to copy over the block chain to the new instance, because this would surely be much more efficient than resyncing the entire block chain from scratch.

Once the bitcoin block chain is synced, the bitcoind process uses about 300 MB of RAM when running and uses a small amount of CPU (usually < 1%, but it will burst much higher from time to time.) My 15 minute load average seems to be about 0.10 or less which includes all of the other tasks on the system.

I haven't written any interesting scripts yet, but doing so seems to be really easy. The bitcoin-cli command prints all of its output to stdout as JSON, so consuming its output from Python should be trivial.

One final thing to note is that all of these solutions have non-trivial cost and complexity. Using a lightweight wallet on your laptop or phone is much cheaper and easier than running a full node. What I've described here is more akin to something out of a cypherpunk's wet dream than it is a practical solution for people who just want to partake in the Bitcoin ecosystem.

[1] As I've alluded in other blog posts, I have a pretty whacky non-standard SSH system in place that I've built using envoy. I will blog about this sometime since I think it solves some interesting problems.