btcsuite / btcwallet

A secure bitcoin wallet daemon written in Go (golang)

btcwallet

btcwallet is a daemon handling bitcoin wallet functionality for a single user. It acts as both an RPC client to btcd and an RPC server for wallet clients and legacy RPC applications.

Public and private keys are derived using the hierarchical deterministic format described by BIP0032. Unencrypted private keys are not supported and are never written to disk. btcwallet uses the m/44'/<coin type>'/<account>'/<branch>/<address index> HD path for all derived addresses, as described by BIP0044.

Due to the sensitive nature of public data in a BIP0032 wallet, btcwallet provides the option of encrypting not just private keys, but public data as well. This is intended to thwart privacy risks where a wallet file is compromised without exposing all current and future addresses (public keys) managed by the wallet. While access to this information would not allow an attacker to spend or steal coins, it does mean they could track all transactions involving your addresses and therefore know your exact balance. In a future release, public data encryption will extend to transactions as well.

btcwallet is not an SPV client and requires connecting to a local or remote btcd instance for asynchronous blockchain queries and notifications over websockets. Full btcd installation instructions can be found here. An alternative SPV mode that is compatible with btcd and Bitcoin Core is planned for a future release.

Wallet clients can use one of two RPC servers:

1. A legacy JSON-RPC server mostly compatible with Bitcoin Core

   The JSON-RPC server exists to ease the migration of wallet applications from Core, but complete compatibility is not guaranteed. Some portions of the API (and especially accounts) have to work differently due to other design decisions (mostly due to BIP0044). However, if you find a compatibility issue and feel that it could be reasonably supported, please report an issue. This server is enabled by default.

2. An experimental gRPC server

   The gRPC server uses a new API built for btcwallet, but the API is not stabilized and the server is feature gated behind a config option (--experimentalrpclisten). If you don't mind applications breaking due to API changes, don't want to deal with issues of the legacy API, or need notifications for changes to the wallet, this is the RPC server to use. The gRPC server is documented here.

Requirements

Go 1.12 or newer.

Installation and updating

Windows - MSIs Available

Install the latest MSIs available here:

https://github.com/btcsuite/btcd/releases

https://github.com/btcsuite/btcwallet/releases

Windows/Linux/BSD/POSIX - Build from source

• Install Go according to the installation instructions here: http://golang.org/doc/install

• Ensure Go was installed properly and is a supported version:

$ go version
$ go env GOROOT GOPATH

NOTE: The GOROOT and GOPATH above must not be the same path. It is recommended that GOPATH is set to a directory in your home directory such as ~/goprojects to avoid write permission issues. It is also recommended to add $GOPATH/bin to your PATH at this point.

• Run the following commands to obtain btcwallet, all dependencies, and install it:

$ cd $GOPATH/src/github.com/btcsuite/btcwallet
$ GO111MODULE=on go install -v . ./cmd/...

• btcwallet (and utilities) will now be installed in $GOPATH/bin. If you did not already add the bin directory to your system path during Go installation, we recommend you do so now.

Updating

Windows

Install a newer MSI

Linux/BSD/MacOSX/POSIX - Build from Source

• Run the following commands to update btcd, all dependencies, and install it:

$ cd $GOPATH/src/github.com/btcsuite/btcwallet
$ git pull
$ GO111MODULE=on go install -v . ./cmd/...

Getting Started

The following instructions detail how to get started with btcwallet connecting to a localhost btcd. Commands should be run in cmd.exe or PowerShell on Windows, or any terminal emulator on *nix.

• Run the following command to start btcd:

btcd -u rpcuser -P rpcpass

• Run the following command to create a wallet:

btcwallet -u rpcuser -P rpcpass --create

• Run the following command to start btcwallet:

btcwallet -u rpcuser -P rpcpass

If everything appears to be working, it is recommended at this point to copy the sample btcd and btcwallet configurations and update with your RPC username and password.

PowerShell (Installed from MSI):

PS> cp "$env:ProgramFiles\Btcd Suite\Btcd\sample-btcd.conf" $env:LOCALAPPDATA\Btcd\btcd.conf
PS> cp "$env:ProgramFiles\Btcd Suite\Btcwallet\sample-btcwallet.conf" $env:LOCALAPPDATA\Btcwallet\btcwallet.conf
PS> $editor $env:LOCALAPPDATA\Btcd\btcd.conf
PS> $editor $env:LOCALAPPDATA\Btcwallet\btcwallet.conf

PowerShell (Installed from source):

PS> cp $env:GOPATH\src\github.com\btcsuite\btcd\sample-btcd.conf $env:LOCALAPPDATA\Btcd\btcd.conf
PS> cp $env:GOPATH\src\github.com\btcsuite\btcwallet\sample-btcwallet.conf $env:LOCALAPPDATA\Btcwallet\btcwallet.conf
PS> $editor $env:LOCALAPPDATA\Btcd\btcd.conf
PS> $editor $env:LOCALAPPDATA\Btcwallet\btcwallet.conf

Linux/BSD/POSIX (Installed from source):

$ cp $GOPATH/src/github.com/btcsuite/btcd/sample-btcd.conf ~/.btcd/btcd.conf
$ cp $GOPATH/src/github.com/btcsuite/btcwallet/sample-btcwallet.conf ~/.btcwallet/btcwallet.conf
$ $EDITOR ~/.btcd/btcd.conf
$ $EDITOR ~/.btcwallet/btcwallet.conf

Issue Tracker

The integrated github issue tracker is used for this project.

GPG Verification Key

All official release tags are signed by Conformal so users can ensure the code has not been tampered with and is coming from the btcsuite developers. To verify the signature perform the following:

• Download the public key from the Conformal website at https://opensource.conformal.com/GIT-GPG-KEY-conformal.txt

• Import the public key into your GPG keyring:

  gpg --import GIT-GPG-KEY-conformal.txt

• Verify the release tag with the following command where TAG_NAME is a placeholder for the specific tag:

  git tag -v TAG_NAME

License

btcwallet is licensed under the liberal ISC License.

btcwallet's Issues

writeDirtyToDisk is slow

writeDirtyToDisk is extremely slow. Look at improving the performance.

rpc: implement gettxoutsetinfo

implement gettxoutsetinfo

rpc: implement getreceivedbyaccount

implement getreceivedbyaccount

rpc feature: implement 'listtransactions <account>'

just a placeholder to state interest in this feature above the rest of the missing rpc's as discussed on irc

support wallet backup

add an api to backup the wallet.

implement rpc command getunconfirmedbalance

implement getunconfirmedbalance which does what it says.

Implement UTXO rescanning

Currently, and funds sent to a btcwallet addresses while wallet is not connected to a running btcd instance will miss the notification as the new block is processed. Rescanning must be used to catch up to a more recent btcd and save unspent transaction outputs from missed block notifications.

These are a couple things that must be done in btcwallet first before a proper rescan is implemented. Each address saved in the wallet has fields for when it was first created and last used. Additionally, btcwallet must save the last seen block to disk, so on a later open, all rescanning before that block can be avoided.

This change can also hugely benefit from btcd keeping a seperate store of all UTXOs at the current block height. If so, this smaller data set (bitcoind uses similar, and is currently ~200-300MB total) can be checked for any unspent transactions outputs to a btcwallet address, instead of querying the database for each transaction in each block that was missed. Before a proper redesign of rescan is implemented, this btcd design should be debated and used if chosen.

Request transactions for generated change addresses.

Currently, change addresses do not receive any notifications of transactions sent to them, causing the change UTXO to never move from unconfirmed to the confirmed balance.

rpc: implement gettxout

implement gettxout

Add extension to query transactions to or from select addresses.

In the case where a frontend must know details of transactions to or from either a single or group of addresses, rather than caring about details for all addresses in a wallet, with the standard API it must use a listtransactions request and filter out transactions that don't matter to it. An extension request should be added to handle this case. The extension should be generic enough to query btcwallet for all or a range of transactions to or from one or more addresses. Write an extension request which meets these requirements

multiple btcd server support

It would be nice to have multiple btcd server support for, at minimum, failover. You would need to specify multiple connection properties for each server, (ip, port, user, pass, certificate). Upon disconnect, btcwallet would cycle through the list of servers until it has connected again.

support dumpprivkey

support dumpprivkey for paper backups and importing into bitcoind if need be.

Fix logging output.

Logging output does not currently match btcd, causing issues with timestamps not being printed in a human readable format, and debugging output being printed when info should be the default level.

This code can be mostly copied as is from btcd.

create wallet prompt is not persistent

using btcgui to connect to a new btcwallet, it correctly prompts for a passphrase to create a new wallet.

However, if you kill btcgui and reconnect before doing so, it doesn't prompt again until you restart btcwallet.

Support pay-to-script-hash notifications

Currently the registered notifications only support pay-to-pubkey-hash. Support for detecting pay-to-script-hash still needs to be added.

Accounts/wallets sub-directories scaling issue

I know we are very early into the development and I apologize if this premature.

btcwallet creates a new wallet for each account into a new directory. The number of sub-directories is limited in *nix file systems (seems like under 50k). Quite a few services have over 50k accounts, so it's not very difficult to hit this limit.

Am I thinking about this the wrong way and is it advisable simply to use a single account/wallet for everything and handle accounts in the application layer?

Missing or mishandling received transaction notifications.

Myself and others have been able to reproduce an issue where wallet funds are not correctly received and a manual rescan has to be performed.

This needs to be fixed before next release.

support import private key

support import private key

rpc: implement listaddressgroupings

implement listaddressgroupings

GetReceivedByAddressCmd

GetReceivedByAddressCmd would be very helpful in wallet (I know all the standard commands will eventually be implemented, but I'd vote for this being one of the next ones to go in).

rpc: implement listlockunspent

implement listlockunspent

rpc: implement addmultisigaddress

add support for addmultisigaddress

rpc: implement importwallet

implement importwallet

doesn't support unix homedir expansion

Before:

$ btcwallet --cafile=~/.btcd/rpc.cert -d=debug
08:25:43 2013-11-21 [ERR] BTCW: cannot open CA file: open ~/.btcd/rpc.cert: no such file or directory

After:

$ ./btcwallet --cafile=~/.btcd/rpc.cert
08:26:01 2013-11-21 [INF] BTCW: RPCS: RPC server listening on 127.0.0.1:18332
08:26:01 2013-11-21 [INF] BTCW: RPCS: RPC server listening on [::1]:18332
08:26:02 2013-11-21 [INF] BTCW: Established connection to btcd.

respond to importprivkey before rescan

btcwallet should respond to the importprivkey command before initiating a rescan request to btcd. this would stop the client, such as btcctl, from waiting for rescan to finish.

Be less dependent on sync.Mutex

Currently, the account type relies on sync.Mutex for correct concurrent access to the wallet, utxo, and tx structures. This should be changed to use a Go channel to serialize requests to access and modify the structures.

rpc: implement keypoolrefill

implement keypoolrefill

Wallet sync fails upon createencryptedwallet

Fired {"id":"createencryptedwallet", "method":"createencryptedwallet", "params": ["testwallet", "bananas", "testpassphrase"] }, but btcwallet keeps giving the following output:

[ERR] BTCW: cannot sync dirty wallet: open /Users/alexanderhanhikoski/Library/Application Support/Btcwallet/btcwallet-testwallet/wallet.bin-1384986032: no such file or directory

Once I created btcwallet-testwallet directory, it was able to sync.

Starting btcwallet with default instructions fails

Installed btcd and btcwallet according to the instuctions. Trying to run btcwallet results in:

18:38:14 2013-11-20 [ERR] BTCW: cannot open wallet: wallet file does not exist
18:38:14 2013-11-20 [ERR] BTCW: cannot open CA file: open /Users/alexanderhanhikoski/Library/Application Support/Btcwallet/cert.pem: no such file or directory

/Library/Application Support/Btcwallet/btcwallet/ is empty. Is btcwallet failing to create a default empty wallet or should this be done explicitly by the user? How?

rpc: implement backupwallet

implement backupwallet

Add getblocktemplate support

We have had a few requests for miner support. Please add support for getblocktemplate.

rpc: implement createmultisig

implement createmultisig

rpc: implement getreceivedbyaddress

implement getreceivedbyaddress

Implement better private key import RPC extension.

The standard RPC call for importing private keys, importprivkey, has a number of issues. Namely:

1. The command does not allow specifying any hints as to when an address first appeard, either by date or blockchain height. This can result in unnecessarily long rescans.
2. As a rescan is triggered for each import, and importprivkey has no way of importing multiple private keys at once, this ends up triggering multiple rescans.

An extension RPC command should be added which resolves these issues.

Add temporary handlers for unimplemented standard bitcoind rpc calls.

All standard rpc calls need a handler, even if all it does is reply back with "this is currently unimplemented". Otherwise, unrecognized commands will be passed to btcd and btcd will respond with "please ask wallet", which wallet will return to the caller.

New addresses must immediately appear in frontends.

Currently, newly-requested addresses (such as through a getnewaddress request) and addresses created for transaction change do not appear immediately in frontends. Make this a notification so all frontends are aware of any new addresses.

rpc: implement dumpwallet

implement dumpwallet

rpc: implement gettransaction

implement gettransaction

visually diffrentiate networks

btcwallet doesn't make it obvious which network it is running on. bitcoin-qt shows [testnet] in the title bar and uses a different color icon. It would be good if btcgui also did some really obvious things to make this clear.

btcwallet should write emtpy tx.bin and utxo.bin files

On initial startup (with a new, empty wallet), btcwallet gives the following errors:
10:41:48 2014-01-08 [ERR] BTCW: cannot open tx file: open /home/john/.btcwallet/mainnet/tx.bin: no such file or directory
10:41:48 2014-01-08 [ERR] BTCW: cannot open utxo file: open /home/john/.btcwallet/mainnet/utxo.bin: no such file or directory

Those files should not be there initially, so rather than an error, btcwallet should just write the empty files.

Add getwork support

We have had a few requests for miner support. Please add support for getwork.

importprivkey doesn't reject duplicates

importprivkey doesn't reject duplicates, so btcwallet could have multiple entries for the same address.

Only provide frontends with data they ask for (at least as an option)

Currently, when you connect to btcwallet with a websocket as a frontend, it sends the info btcgui wants (account info) and then keeps sending blockheight as that changes. It would be nice if the frontend actually had some say in the info it recieved rather than always being spammed with things it did not ask for.

rpc: implement getrawchangeaddress

implement getrawchangeaddress

Support pay-to-pubkey notifications

Currently the registered notifications only support pay-to-pubkey-hash. Support for detecting compressed and uncompressed keys still needs to be added.

Comment

In wallet.go is this comment:

    // This is a perfect example of YOLO crypto.  Armory claims this XORing
    // with the SHA256 hash of the pubkey is done to add extra entropy (why
    // you'd want to add entropy to a deterministic function, I don't know),
    // even though the pubkey is generated directly from the privkey.  In
    // terms of security or privacy, this is a complete waste of CPU cycles,
    // but we do the same because we want to keep compatibility with
    // Armory's chained address generation.

I think without the entropy you would be able to look at public keys on the blockchain and calculate the next deterministic public key, thus allows anyone to figure out that a chain of addresses (as long as they were used at least once, thereby revealing the public keys) belong to the same wallet. This is a privacy concern.

Keypool must automatically refill if empty

Currently, if the unused addresses in the initial keypool created with the wallet are all used up, the keypool is not refilled, and no new addresses are created.

Repetitive error after launching btcdsuite

After launching btcdsuite, the following error messages appear in the btcws console window on Win7x64:

02:33:28 2013-12-05 [ERR] BTCW: cannot open utxo file: open C:\Users\L\
AppData\Roaming\btcwallet\btcwallet\utxo.bin: The system cannot find the file sp
ecified.
02:33:29 2013-12-05 [INF] BTCW: Established connection to btcd.
02:33:37 2013-12-05 [ERR] BTCW: cannot sync dirty wallet: rename C:\Users\Lr\AppData\Roaming\btcwallet\btcwallet\wallet.bin-1386232417 C:\Users\L\AppData\Roaming\btcwallet\btcwallet\wallet.bin: Cannot create a file when tha
t file already exists.
02:33:48 2013-12-05 [ERR] BTCW: cannot sync dirty wallet: rename C:\Users\L\AppData\Roaming\btcwallet\btcwallet\wallet.bin-1386232428 C:\Users\L\AppData\Roaming\btcwallet\btcwallet\wallet.bin: Cannot create a file when tha
t file already exists.
02:33:58 2013-12-05 [ERR] BTCW: cannot sync dirty wallet: rename C:\Users\L\AppData\Roaming\btcwallet\btcwallet\wallet.bin-1386232438 C:\Users\L\AppData\Roaming\btcwallet\btcwallet\wallet.bin: Cannot create a file when tha
t file already exists.
02:34:08 2013-12-05 [ERR] BTCW: cannot sync dirty wallet: rename C:\Users\L\AppData\Roaming\btcwallet\btcwallet\wallet.bin-1386232448 C:\Users\L\AppData\Roaming\btcwallet\btcwallet\wallet.bin: Cannot create a file when tha
t file already exists.

And continues to appear until the program is terminated.

Implement better sendmany RPC extension.

The sendmany RPC command is great because it is flexible enough to create a single tx with multiple outputs, however the resulting tx may include a greater fee than the user is willing to pay. An extension RPC command should be added which takes the same parameters as the sendmany command, but instead of sending the raw tx to btcd to relay to the network, the rawtx and included fee should be returned to the caller. This will allow a frontend to present a user with a message of whether the included fee is acceptable or not, and if so, the returned raw tx may be sent.

!hardcode localhost

currently the port but not the host can be configured

I understand this will get looked at after the tls bits are in place to secure connections to other hosts