graymauser / btcposbal2csv

List all bitcoin addresses with positive balance.

Dump Bitcoin addresses with positive balance

Simple utility to list all bitcoin addresses with positive balance. It works by analysing the current unspent transaction output set (UTXO), aggregating outputs to same addresses together and write them to csv file.

Prequisities:

python 2.7
pip

To install:

run pip install -r requirements.txt

or install following packages with pip manualy

for linux：

• plyvel
• base58
• sqlite3

for windows：

• plyvel-win32
• base58
• pysqlite3

Usage

To use this script, you will need copy of chainstate database as created by bitcoin core client. I've not tried different clients.

To get current addresses with positive balance, let the full node client sync with the network. Stop the bitcoin-core client before running this utility. If you not stop the client, the database might get corrupted.
Then run this program with path to chainstate directory (usualy $HOME/.bitcoin/chainstate).

Show help

python btcposbal2csv.py -h

Example:

The following will read from /home/USER/.bitcoin/chainstate, and write result to /home/USER/addresses_with_balance.csv.

python btcposbal2csv.py /home/USER/.bitcoin/chainstate /home/USER/addresses_with_balance.csv

Notice

• The output may not be complete as there are some transactions which are not understood by the decoding lib, or that which do not have "address" at all. Such transactions are not processed. Number of them and the total ammount in such transactions is displayed after the analysis.
• The output csv file only reflects the chainstate leveldb at your disk. So it will always be few blocks behind the network as you need to stop the bitcoin-core client.

Converting to RIPEMD160

Per request, I'm adding script which is able to convert BTC address to RIPEMD160 representation. BTC address must be in fist column, RIPEMD160 is added to csv. Output goes to stdout.

Example:

python convert2ripemd160.py /home/USER/addresses_with_balance.csv

Acknowledgement

This utility builds on very nice bitcoin_tools lib, which does the UTXO parsing.

Support

If you like this utility, please consider supporting the bitcoin_tools library which does all the heavy lifting.

If this particular functionality made your life easier you can support coffee consumption in BTC 1FxC1mgJkad63beJcECfZMRaFSf4PBLr2f.

FAQ

• Python 3 does not work. Yes, the the code is for python 2 - as stated in the requirements.
• Can this be used for coin XYZ? Currently only BTC is supported. The direct BTC derivatives (LTC, BCH, and others) should be fairly straitforward to add, other not so much.
• Currently this library is not actively developed. It is sufficient for my usecase. If you want some feature added, fork it, open PR, etc. Should some tips come in my way I would be much more inclined to develop this project more actively.

btcposbal2csv's Issues

Chainstate end 2021

I got parsed transaction 78299200 but in CSV still 1.5gb
When convert to pickle it only got about 2xxxxxxx only which not completed.
Understand Excell only go for 1048576 only.
How to solve this? Need separate to many CSV then convert to pickle?
When I run convert to pickes I got about 128pickle but when run Plutus it read total database only 2xxxxxxx only which way out of total 78299200 parsed.
Who can help on this ..thank you.tq

FOR PYTHON 3?

can you update it for python3 some lib are obsolete

Hello graymauser, can you tell me what tool can I use to convert the generated csv file into a pickle file? Thank you graymauser

Hello graymauser, can you tell me what tool can I use to convert the generated csv file into a pickle file? Thank you graymauser

Support for bc1 addresses (Bech32)

It seems that the converter doesn't support the new BTC address format that starts with bc1 and is usually longer in length.

example:

37XuVSEpWW4trkfmvWzegTHQt7BdktSKUs (older format)
bc1qgdjqv0av3q56jvd82tkdjpy7gdp9ut8tlqmgrpmv24sq90ecnvqqjwvw97 (new format)

This is the error when it tries to convert it:

Traceback (most recent call last):
  File "D:\btcposbal2csv\convert2ripemd160.py", line 41, in <module>
    process(args.csvin)
  File "D:\btcposbal2csv\convert2ripemd160.py", line 19, in process
    ripemd_bin = tocondensed(row.split(',')[0])
  File "D:\btcposbal2csv\convert2ripemd160.py", line 7, in tocondensed
    return base58.b58decode(add_or_pk)[1:-4]
  File "C:\Users\Lenovo\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\base58\__init__.py", line 96, in b58decode
    acc = b58decode_int(v, alphabet=alphabet)
  File "C:\Users\Lenovo\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\base58\__init__.py", line 79, in b58decode_int
    decimal = decimal * 58 + alphabet.index(char)
ValueError: subsection not found

Doesn't work with Ubuntu20.04 and Python3

kevin@Kevin-PC:~/btcposbal2csv$ python3 btcposbal2csv.py /windir/g/.bitcoin/chainstate/ /windir/g/bitcoin_balance.csv
reading chainstate database
inmem
Traceback (most recent call last):
  File "btcposbal2csv.py", line 201, in <module>
    for address, sat_val, block_height in add_iter:
  File "btcposbal2csv.py", line 98, in in_mem
    for add, val, height in parse_ldb(
  File "/home/kevin/btcposbal2csv/utils.py", line 317, in parse_ldb
    o_key = db.get((unhexlify("0e00") + "obfuscate_key"))
TypeError: can't concat str to bytes

Extract with or without balance

Is there a way/option to extract wallets without balance? (balance = zero satoshi)
Or all... with and without balance...

This does not write to the csv file?

Problem "PATH_TO_CHAINSTATE_DIR, OUTFILE"

Hi
I run on window
I don't know what happen with below? path problem? Thanks.

D:\btcposbal2csv-master>btcposbal2csv.py chainstate addresses_with_balance.csv
usage: btcposbal2csv.py [-h] [--bitcoin_version BITCOIN_VERSION] [--keep_sqlite PATH_TO_SQLITE_FILE)] [--lowmem]
[--P2PKH bool] [--P2SH bool] [--P2PK bool] [--sort ASC/DESC]
PATH_TO_CHAINSTATE_DIR OUTFILE
btcposbal2csv.py: error: the following arguments are required: PATH_TO_CHAINSTATE_DIR, OUTFILE

version error

i find there many small error now ,Is it because it doesn't apply to the current chainstare version？

Error: unable to decode 831061 transactions

Hi,
I'm using bitcoind v0.18.0 and I have the node up2date.

When I run the btcposbal2csv.py, it process up to transaction #58027800 before die with the error "unable to decode 831061 transactions".

(btcbal) [bitcoin@btcd btcposbal2csv]$ python btcposbal2csv.py /home/bitcoin/.bitcoin/chainstate/ addrs2.txt
reading chainstate database
inmem
 parsed transactions: 58027800
unable to decode 831061 transactions
totaling 24371245714994 satoshi
writen to addrs2.txt

I have enough RAM (the process does not die with an out of memory error). At the moment of the unexpected exit, is just using around 6gb of memory.

Do anyone have any idea of what can I check?
Thanks

how to delete this?need help

1FtZwEZKknoquUb6DyQHFZ6g6oomXJYEcb,257219114.44438908,740429
1AQthEXNcQWSfGkUiNAUi6Jj4NYuYLcqdw,5477.777777777777,572839
14frN8WdboAvn29bgED9K5zsCTYThRtzDK,7170.222222222223,586780

The address seems incomplete

I took a few random checks

Block 9901 of 1ly5zpj84hr5wlqsbqk6jb7jsqzkkkcnxb isn't one of them

Can I get a more complete address?

current balance 0 BTC

possible to modify this code for write all btc address with or without balance but with transaction ?

Is the generated file limited to 1.5G？

Is the generated file limited to 1.5G？

Can't install plyvel

What should I do?

btcposbal2csv.py doesn't extract segwit

btcposbal2csv.py doesn't extract segwit addresses, for example:

Public Address bc1 P2WPKH: bc1qetrqrygl7t0fn6hrdmuug3rrp8vjmum8fd8rsk
Public Address bc1 P2WSH: bc1q26swzzjmqgwvcjxplktpyj9xpkzpv8kgs7t545ku6s7v6fl0xt3slnx8k9

Litecoin support

what are the Vs for litecoin so i can edit ? im trying to make it work as arg --network litecoin

if out['out_type'] == 0:
                if out['out_type'] not in types:
                    continue
                add = hash_160_to_btc_address(out['data'], 0)
                yield add, out['amount'], value['height']
            elif out['out_type'] == 1:
                if out['out_type'] not in types:
                    continue
                add = hash_160_to_btc_address(out['data'], 5)
                yield add, out['amount'], value['height']
            elif out['out_type'] in (2, 3, 4, 5):
                if out['out_type'] not in types:
continue

plyvel._plyvel.IOError: IO error: Invalid argument

I am receiving the below error, please could you assist.

(envname) jordyn@DESKTOP-R6AIPP6:~/btcposbal2csv$ python btcposbal2csv.py /mnt/b/.Bitcoin/chainstate /home/jordyn/addresses_with_balance.csv
reading chainstate database
inmem
Traceback (most recent call last):
File "btcposbal2csv.py", line 201, in
for address, sat_val, block_height in add_iter:
File "btcposbal2csv.py", line 101, in in_mem
types=get_types(in_args)):
File "/home/jordyn/btcposbal2csv/utils.py", line 314, in parse_ldb
db = plyvel.DB(fin_name, compression=None) # Change with path to chainstate
File "plyvel/_plyvel.pyx", line 247, in plyvel._plyvel.DB.init
File "plyvel/_plyvel.pyx", line 88, in plyvel._plyvel.raise_for_status
plyvel._plyvel.IOError: IO error: /mnt/b/.Bitcoin/chainstate: Invalid argument

TypeError: can't concat str to bytes

Hello, gets error with Python 3.8.5
python3 btcposbal2csv.py /root/.bitcoin/chainstate /root/addresses_with_balance.csv
reading chainstate database
inmem
Traceback (most recent call last):
File "btcposbal2csv.py", line 201, in
for address, sat_val, block_height in add_iter:
File "btcposbal2csv.py", line 98, in in_mem
for add, val, height in parse_ldb(
File "/root/btcposbal2csv-master/utils.py", line 317, in parse_ldb
o_key = db.get((unhexlify("0e00") + "obfuscate_key"))
TypeError: can't concat str to bytes

Getting Invalid argument error.

I have chainstate data on my windows machine, I am trying to run the script from WSL. But I am getting the following error.

reading chainstate database
inmem
Traceback (most recent call last):
File "btcposbal2csv.py", line 201, in
for address, sat_val, block_height in add_iter:
File "btcposbal2csv.py", line 101, in in_mem
types=get_types(in_args)):
File "/mnt/e/Finrocks/btc/btcposbal2csv/utils.py", line 323, in parse_ldb
db = plyvel.DB(fin_name, compression=None) # Change with path to chainstate
File "plyvel/_plyvel.pyx", line 247, in plyvel._plyvel.DB.init
File "plyvel/_plyvel.pyx", line 88, in plyvel._plyvel.raise_for_status
plyvel._plyvel.IOError: IO error: /mnt/f/Bitcoin/chainstate/: Invalid argument

Email

How can I contact you on an email?

Can't install plyvel

What should I do?

may i get the addresses file

hey
I'm not a professional programmer. I tried to install hashlib, plyvel and SQLite3 with python3 pip, but they couldn't succeed. Can I get all the address files directly?
thank guitar

How much RAM program needs to fully parse current size of blockchain?

I've tried at run it with 4GB of ram, and still got out of memory exception.

Can't get past install errors

I tried several times on Windows before jumping ship and spinning up a VPS with Ubuntu 18.04 and Python 2.7
When I run pip install -r requirements.txt you can see in the screenshot it kicks some errors and I don't know how to fix this or what it wants....

If I try to install the required modules manually I get the same error.

Any help is highy appreciated!

convert2ripemd160.py return error "'NoneType' object is not iterable"

Hi!

Thank you for sharing your script.

I have tried to run convert2ripemd160.py on a .csv file containing 1 address per line (starting by "1" or "bc1") but it is returning an error (see below). The error appears when it hits adresses starting with "bc1" and if I remove them all in the .csv file there is no errors.

On Debian 10 with Python 3.7.3:
Traceback (most recent call last):
File "convert2ripemd160.py", line 46, in
process(args.csvin)
File "convert2ripemd160.py", line 22, in process
ripemd_encoded = binascii.hexlify(bytearray(script_int))
TypeError: 'NoneType' object is not iterable

On Windows with Python 3.9.2:
Traceback (most recent call last):
File "C:\Users\Dolaor\Desktop\btcposbal2csv\convert2ripemd160.py", line 46, in
process(args.csvin)
File "C:\Users\Dolaor\Desktop\btcposbal2csv\convert2ripemd160.py", line 22, in process
ripemd_encoded = binascii.hexlify(bytearray(script_int))
TypeError: cannot convert 'NoneType' object to bytearray

Thanks for your help.

Can't concat str to bytes

D:\btcposbal2csv>python btcposbal2csv.py H:\Bitcoin\chainstate addresses_with_balance.csv
reading chainstate database
inmem
Traceback (most recent call last):
File "btcposbal2csv.py", line 201, in
for address, sat_val, block_height in add_iter:
File "btcposbal2csv.py", line 98, in in_mem
for add, val, height in parse_ldb(
File "D:\btcposbal2csv\utils.py", line 317, in parse_ldb
o_key = db.get((unhexlify("0e00") + "obfuscate_key"))
TypeError: can't concat str to bytes

Running on Windows 11 64bit
Bitcoin Core 22.0

Possible to make it dump txs hashes ?

Thank you for this great code, i was wondering is it possible to make it dump transactions hashes ?

Search for positive balance on compressed addresses

Is this possible?

Python 3 support

Not an issue as such, more of a request. Is there any chance of getting this to work with Python 3?

Unciphered - Randstorm - BitcoinJS SecureRandom

Hi Folks,

Given the recent publication

Which goes on to reveal that "Bitcoin private keys should be generated with 256-bits of entropy; unfortunately, affected keys generated with vulnerable BitcoinJS (or dependent projects) often used less entropy than required.... reduces the amount of necessary work anywhere from 32 to 64-bits." due to several potential problems encountered with Random Number Generation in browser-based software used at the time. And that these vulnerable wallets were created between 2011 and 2014.

What modifications would be required to the code to only produce a database of non-zero, dormant wallets generated within a targeted time period - e.g 2011-2012 or 2011-2014?

Appreciate your thoughts, thnaks

Support for LTC and DOGE?

Is it possible to get this working for other chainstates? I tried with DOGE and LTC and it did not work. Thanks!