Multiprocess Bitcoin

Google Chrome and Mozilla Firefox have popularized using multiple processes to achieve isolation among browser components. This is done to increase browser security, as memory corruption problems are isolated in a way that is not possible with a single-process multiple-threads design. On Linux, these coordinating worker processes are further contained using seccomp filters and/or Linux namespaces. These sandboxing techniques mitigate the risk of bugs that can lead to code execution, since an attacker who finds an arbitrary code execution vulnerability is confined to the process' sandbox.

Bitcoin Core should use this technique to protect the wallet database and private keys. First we propose introducing a new bitcoin-agent process, responsible for managing in-memory decrypted private keys and transaction signing. Later a new bitcoin-wallet process can be added that manages the wallet database. To simplify process management, these can be managed as regular non-daemon child processes of bitcoind or bitcoin-qt.

Seccomp and setns are non-portable, and can only be used in Linux builds. However, macOS, BSD, and Windows will still see some benefit of this design due to the fact that processes have separate memory spaces.

Note: For the purposes of exposition, the remainder this document will just refer to bitcoind and not bitcoin-qt. However, the exact same sandboxing technique can and would be used unmodified for bitcoin-qt.

Phase 1: bitcoin-agent

The first phase of the plan is to create a new executable protecting a Bitcoin wallet's most sensitive data, the decrypted private keys. The new process will be called bitcoin-agent, and is analogous to ssh-agent or gpg-agent. The bitcoin-agent process will be responsible for holding decrypted private keys in memory and signing transactions. The decrypted keys can have an expiry in memory, as is the case with encrypted wallet functionality in Bitcoin Core today.

The bitcoin-agent process will be created early in the lifetime of bitcoind. The main process will create a pipe using pipe(2) and then fork/exec the bitcoin-agent process. The bitcoin-agent process then creates a set of seccomp filters that disallow nearly all system calls (more details on this below). The first time the bitcoind process needs to access the wallet, it will send the encrypted wallet seed to the bitcoin-agent process over the pipe. Subsequently, bitcoind will handle request for walletpassphrase RPCs by sending the passphrase and timeout to the agent process over the pipe. If the agent successfully decrypts the seed, it will mlock(2) the decrypted data and persist it in memory until the timeout elapses. The agent also notifies bitcoind whether it was able to unlock the wallet, so that bitcoind can respond to the RPC. The walletlock RPC will work analogously. When bitcoind receives RPC commands that require it to sign transactions, it forwards the signing request to the agent process.

In summary, there are three commands handled by bitcoin-agent:

• Lock wallet
• Unlock wallet
• Sign message

The communication protocol between bitcoind and the agent can either be a custom ad hoc protocol, or a simple framed protocol using protobufs.

To avoid creating zombie processes when bitcoind dies unexpectedly (e.g. OOM killed by the kernel), the bitcoin-agent process will exit if it detects the pipe to bitcoind is closed. The kernel guarantees this will happen when the parent is killed. Process reaper policies can also be used on Linux as an extra safeguard.

Security Considerations

The agent process can be restricted via seccomp to the following system calls:

• read(2) and write(2) calls that apply to the pipe fd only
• select(2) to implement timeouts, and detect when the pipe is closed
• mlock(2) and mprotect(2) to lock keys into read-only memory
• mmap(2) for memory allocation[^1]
• Safe system calls needed for things like logging (e.g. gettimeofday(2))

[^1]: This may not be necessary, if it turns out that it's possible to write the agent to not require dynamic memory allocation.

The agent process would be disallowed from things like the following:

• Reading/writing arbitrary file descriptors other than its pipe, e.g. regular files
• Accessing the wallet.dat file in any way
• Opening regular files
• Establishing new network connections
• Forking or execing other processes.

The agent process can also enter an isolated namespace using setns(2). This is analogous to how Linux containers like Docker isolate processes. There are other ways the process can also be locked down that would not normally make sense for bitcoind. For example, the agent can call ulimit(3) to prevent the kernel from creating core dumps if bitcoin-agent crashes.

Benefit to Bitcoin

By creating a bitcoin-agent process, certain flaws that would normally allow private keys to be leaked will be mitigated. In particular, things like buffer overflows that can lead to code execution in bitcoind will not cause private keys to be compromised.

Since the bitcoin-agent process can be made extremely minimal, it will have a reduced surface area for bugs. For instance, it does not need to load BerkeleyDB/LevelDB code, nor does it need a JSON encoder/decode. It would need to a load a vastly smaller set of shared libraries (possibly just crypto libraries).

Note that this design is not sufficient by itself to prevent attackers from stealing funds. An attacker who gains code execution in bitcoind could still create and sign transactions that transfer Bitcoin to a wallet controlled by the attacker.

Phase 2: Introduce bitcoin-wallet Process

The second phase is to introduce another new process, bitcoin-wallet. The wallet process would manage access to the wallet.dat database. Signing and management of decrypted key data would still be done by the bitcoin-agent process, as before.

The main motivation for introducing the bitcoin-wallet process is to lock down access to the wallet.dat file. As before, early in its lifetime bitcoind will fork/exec the bitcoin-wallet process, and communication between bitcoind and bitcoin-wallet will happen over file descriptors created using pipe(2). When bitcoind receives any wallet related RPC command, it will parse the request and then send the decoded RPC to the bitcoin-wallet process over the pipe, perhaps using a simple protocol based on protocol buffers.

The novel part of this design is that after the bitcoin-wallet process is created, the bitcoind process can limit filesystem access using a seccomp policy. This policy can be constructed such that bitcoind can't access the wallet.dat file.

Security Considerations

The bitcoin-wallet process will have a seccomp policy that restricts it to:

• Perform regular I/O system calls (e.g. read/write/seek) on the wallet.dat file
• Communicate over its pipe
• Allocate/deallocate memory.

The bitcoin-wallet process would be disallowed from things like the following:

• Read/writing other parts of the filesystem, including other parts of the Bitcoin data directory[^2]
• Establishing new network connections
• Forking or execing other processes.

[^2]: The backupwallet RPC requires arbitrary filesystem access. Therefore a new flag will need for users who still rely on this command. The flag would prevent bitcoin-wallet from inhibiting its own filesystem access. Users would be encouraged to use the dumpwallet RPC instead, as it does not need to write to the filesystem.

On Linux systems the bitcoind process will use a seccomp policy that prevents it from accessing the wallet.dat file after forking the wallet process. This means that an RCE in bitcoind will not be able to do leak the contents of the wallet.dat file, or upload it to a server controlled by the attacker.

With this change bitcoind will no longer have to link against libdb4, since all of the BerkeleyDB code would be moved into the bitcoin-wallet process.

Benefit to Bitcoin

By moving wallet functionality into a new process, the main bitcoind process can restrict its own access to the wallet.dat file. This means that an RCE in bitcoind would not let an attacker access the wallet.dat file.

An attacker who is able to exploit an RCE in bitcoind would still be able to steal funds if the wallet passphrase is unlocked. This is because the attacker would be able to call dumpwallet or an RPC method like sendtoaddress. While this is a big hole, it's still an improvement over the current design of bitcoind. An attacker today who finds an RCE can get upload a copy of the encrypted wallet seed to the internet, where the attacker can try to brute force it offline. The seccomp policy described here would prevent this attack.

As a secondary benefit, this change also mitigates the impact that BerkeleyDB errors have on Bitcoin. For instance, there have been reports that corrupted wallet files can cause bitcoind to segfault. This is a bug that should be fixed regardless, but in the multiprocess architecture such a bug would crash the bitcoin-wallet process but not the bitcoind process.

Future Directions

One interesting aspect of the multiprocess design is that the bitcoin-agent and bitcoin-wallet processes could become pluggable in the future. This allows interesting possibilities for vendors of third party wallet software. For instance, Trezor could add functionality where the hardware wallet assumes the jobs of bitcoin-wallet (and bitcoin-agent). This would then allow Trezor users to keep their private keys and wallet data on a secure device, but have the wallet connected to a secure full node.

There are some less sophisticated sandboxing techniques that can be used on BSD/macOS and Windows, e.g. chroot(2) jails on BSD/macOS. It's worth exploring what techniques browsers use to implement sandboxes on these operating systems.