https://github.com/AustEcon/bitsv/blob/master/bitsv/transaction.py#L200

The problem is when the balance(A) is 1000 and you want to send 600 to someone(B), and the calculated fee is 300, then the code will generate the tx like this:
Input A: 1000
Output B:600, A100
(implicit) Fee: 300

Then bitindex will generate a tx for you and broadcast it, but this tx will not be accepted by any miner, because a uxto with a value less than dust is not accepted.

One possible solution is to move all 400 to the miner.
Input A: 1000
Output B:600
(implicit) Fee: 400

This only applies to the case when the remnant < dust(546), if the remnant is > dust, there will be no effect.

Of course, we could adjust this in the future once the new agreement reached for most of the SV miner.
https://www.reddit.com/r/bitcoincashSV/comments/abj9nb/bitcoin_needs_to_completely_remove_the_dust/

How about your guys opinion? @AustEcon @joshua-s @teran-mckinney

• Add Hierarchical Deterministic key support for priv, xpub --> generation of electrum sv and handcash wallet public / private key pairs (derivation path m/0')
• Create a separate HandCash api powered by bitsv in a separate repo.
• Create functions for posting of images / media etc. to blockchain (e.g. bitpaste compatible way)
• Add in BitIndex xpub related queries
• BitIndex3 added by Joshua-S
• Fix tests
• Fix testnet
• Tidy up final re-branding and documentation
• Add "confirmations" field to unspents so 0-conf balance versus 1-conf etc. balance can be calculated (may need to contact BitIndex)

Edit 12/04/19 Please comment below with a wishlist of features you want added and I will endeavour to provide! Or recruit help! Donations gratefully received!

Or if any of the above features are important to you... let me know so I can gauge if my time will actually be well spent there.

Seems like Bitindex3 may not be working. bchsvexplorer.com has Cloudflare and isn't configured to not block Tor.

Plans / Thoughts:

One consideration with this NetworkAPI setup for redundancy is that there have been instances of seemingly random sync issues that in my estimation arise from querying for unspents from one api service whilst using a different api for broadcast. For this reason I am of half a mind to encourage users to go with one querying / broadcast api OR THE OTHER but never intermixing them at the same time. E.g. possible plan could be to have a "BitIndexAPI" api and a WhatsonchainAPI" and to make it so that if any aspect of one of them fails, it updates a global variable to now prefer the OTHER API(s) for ALL services and to only try again with the API that failed as a last resort for the remainder of the session (a ranking system). This should prevent future syncing issues whilst offering redundancy. The other challenge is that different APIs offer a different set of features so rolling it all into one "NetworkAPI" means that one must restrict the function set to what is common to all. More specialised API queries unique to a particular API would need to be accessed via "bitsv.network.services.WhatsonchainAPI...." or "bitsv.network.services.BitIndex... maybe could expose this later to the PrivateKey class somehow --> my_key.BitIndex... or my_key.Whatsonchain... for convenience...

(Possible future if I ever get the time)
I have had requests for RegTest support (@joshua-s). I have had some helpful suggestions from @BitcoinSofia on twitter) ... at present I am waiting on either Whatsonchain or BitIndex to release an open source version of their APIs (which is rightfully their decision and they may not decide to do this).

But if they do, that could allow a separate repository for regression testing. Would need

1. bitcoind.exe instance on localhost in "RegTest" mode (easy to do - done in fact)
2. Various APIs used by this library to run on localhost (e.g. BitIndex, Whatsonchain, ??
   even some of _unwriter's stuff like BitDB variants / Planaria in the future even)
3. A configurable script that starts up and connects together (1) and (2) with a localhost API endpoint for bitsv testing modules to target with a heavy duty testing suite. Could make the script automatically produce one new block per 5-10 seconds or something if you wanted to configure it all for a live / interactive session with a python interpreter open.

Later could include bit:// b:// c:// d:// protocol related stuff... for travis CI automated testing for every commit

Let me know if there are any flaws / suggestions around this kind of approach and / or if you like the idea and want to help out with getting it going... don't wait around on me! (It's unlikely that I will have time to implement this kind of RegTest idea for quite a long time - if at all).

PS: If someone was able to build this kind of virtual localhost testing environment, it would not just be specific to bitsv, it would be an invaluable platform for any client side software development / wallets that use these API endpoints.

Looks like we renamed get_unspent() to get_unspents() recently. Would be nice to have a wrapper function with the same name for backwards compatibility.

Currently the way to 'sendall' to an address is to do:
my_key.send([], leftover='ADDRESS_TO SENDALL_TO')

Which basically passes in an empty list for outputs and sends all of the remaining change (everything) to the desired address. This is not intuitive to figure this out though and could be described as a bit of a "hack".

I therefore propose adding a sendall() to PrivateKey class so that syntax can be:
my_key.sendall('ADDRESS_TO SENDALL_TO')

As Mattercloud is now a paid service and our tests are failing, I think I'll make a simple change that disables use of mattercloud's api in the absence of an environment variable:
MATTERCLOUD_API_KEY

We could add that to our Travis CI later on if we want to I believe.

It would be up to application developers to set this environment variable prior to launch which would activate mattercloud's api and use it as the highest priority API.

Any objections to this methodology?
Sound alright @MatterCloud?

bitcoinsv-rates.com is no longer resolving. Not having any luck getting rates with bitsv at the moment.

Does anyone know of the owners of that site? Are there any other rates APIs we can use?

This is the error message I get if I try to use whatsonchain to broadcast via their GUI - so I think that means their node still has the default 100kb limit in place for datacarriersize=100000

But even beyond that... if I broadcast via mattercloud, it is relayed okay (Atilla kindly raised the limit to 3MB now)(7025785c8955db545c605693557bcd7ac951fe5c18f15b51fc2bff84ce69ba1f)... but will not appear on whatsonchain while unconfirmed. Nor will it show after 1 confirmation... maybe it will show later on (after a lone wolf miner includes it...) - I don't know.

So I am just putting this here to document the current state of things on the 'data carrier transaction' front.

It would make a big difference in particular for this bitsv library to have a larger cap because mattercloud is throttling number of requests per second and so bcat type data uploads become unfeasible without a paid subscription / api key. (So the first thing I think we can do to address this is to add whatsonchain to our list of apis - which I bet doesn't throttle to the same extent)

OP_PUSHDATA within the executed script would go up to 100MB I believe. But there is not currently a widely adopted standard for 3rd parties to parse this etc.... or how exactly to imbed the payload... and I don't much feel like being on the 'bleeding edge' of it and having to change later.

> k.get_balance()
'7216'
> to=[('1GrFgWWadFMyB3a2aCp27vLD33i3oCSsEK',1000, 'satoshi')]
> k.get_unspents()
[Unspent(amount=7216, confirmations=148, script='76a914bad4871d5bdeabb087f23a04f8342c31248c23ed88ac', txid='77b999b91631ae580dcd73046b228c9769eff54465faed7d0f8fd1c1446c680c', txindex=0)]
> k.send(to)
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.6/site-packages/bitsv/wallet.py", line 295, in send
outputs, fee=fee, leftover=leftover, combine=combine, message=message, unspents=unspents
File "/usr/local/lib/python3.6/site-packages/bitsv/wallet.py", line 253, in create_transaction
custom_pushdata=custom_pushdata
File "/usr/local/lib/python3.6/site-packages/bitsv/transaction.py", line 147, in sanitize_tx_data
raise ValueError('Transactions must have at least one unspent.')
ValueError: Transactions must have at least one unspent.

This address has balance, the uxto tx has > 1 confirmation, but it never successfully send the bsv to other address.
Is there someone else has a similar experience like this?

https://github.com/psf/black

What do you think about doing this for bitsv? Obviously should be a single commit with nothing else going on that commit.

Reminder

The my_key.send([], message=data, custom_pushdata=True) works by allowing the user to pre-generate metadata with the functions contained within op_return.py (which generates a bytearray with the correct op_pushdata op codes for as many elements as there are). But it is the send() that adds the OP_RETURN to the beginning of these bytes. This has caused some users issues when they pregenerate these bytes using other libraries (which have already added the OP_RETURN.

They have gotten it to work by simply removing this first OP_RETURN byte to let bitsv add it but this is a hack and I think it is great to see people using bitsv in combination with other libraries and would like to support it.

I wonder if we cannot simply make the op_return.py functions for generating the bytes fall into line with how these other libraries are doing things.

In hindsight, it actually makes more sense to do it this way so that people can pre-generate any kind of crazy, complicated locking scripts and append them into a transaction as they see fit.

I don't know the best way to implement this though - and haven't given it much thought...

• I wonder if "custom_pushdata" could be changed to something more like "custom_output=True" and maybe change message --> custom_output(s) (accepting a list of them would future-proof us for a future with multiple custom outputs) ...

We could keep the message and custom_pushdata kwargs for backwards compatibility but deprecate these after a few releases maybe?

Open to suggestions...

I think I can make the send_op_return() work nicely without anyone even noticing that it's changed under the hood... but the send() is like the "low level" function that does everything if you really need it to and so would be nice to have it as intuitive as possible...

I think it will not be too difficult to fix up whatsonchain such that it returns the number of confirmations and can therefore be compatible with bitsv's Unspents dataclass.

As per previous discussion...#39 (comment)

I think 2 function calls should be acceptable for this purpose...
One to get current block height and another for the utxos --> calculate number of confirmations.

Once this is fixed we can finally add whatsonchain to main / test and stn as a second-line API for redundancy. Which I see as quite important.

.address seems to return in the cashaddr format. Is that correct?

Integrating this in walkingliberty.

Even a basic call like this fails.

https://api.bitindex.network/api/v3/main/addr/17msuwJxJoQKfH7197VvLRA3xzVDGNZqUT

Currently both of those options are constants (SEQUENCE and LOCK_TIME).

Made a testnet transaction confirmed by other api's but not being confirmed by whatsonchain api.

The SV community is hoping to use OP_RETURN as the most important OP to move all the valuable data into the block.

Now, most of the major miner has increased the restriction. Most of the major wallet service provider also increased the number as well.

We believe at least 99k is a reasonable value for now.

https://github.com/moneybutton/bsv/blob/50ca3fa2032478895f222cd6cc235df5b9387728/test/script/script.js#L338-L349

When I attempt to use send_op_return() on a PrivateKeyTestnet key, I get an AttributeError.

Hi. as I read in bitsv documentation I need to call the prepare transaction when I'm online to prepare a transaction and then I can sign this transaction offline. logically the prepare_tranaction function shouldn't need the key object to work. but when I run the tx_data = PrivateKey.prepare_transaction(address=sender.address, outputs=[(receiver.address, amountToSpend , 'bsv')]) I get this error :
TypeError: prepare_transaction() missing 1 required positional argument: 'self'

I checked the wallet.py file of bitsv and recognized that prepare_transaction isn't a @classmethod while it should be!

Reminder

seems like whatsonchain is down.
For example when I try to get balance request to: https://api.whatsonchain.com/v1/bsv/test/chain/info
takes too long, eventually gets a timeout error.

None of the polyglot uploads are visible on bico.media anymore (https://bico.media/be8b6a79e66934d3419265fbf3295d03e331a4c08098ae7f817a7592ffaedd2b gives "Not found..."), and it seems to be because bitsv doesn't place the OP_RETURN as the first output. Instead it places it last.
The relevent line to fix is

I mentioned the issue in unwriter's slack channel, and another dev said they also assumed the op_return was first in their code, so it seems good practice to put it there. This is of course also a bug in bico.media, which I haven't reported at this time. Not sure how.

Hi,

I have few questions about Hierarchical Deterministic usage:

1. In the example of the readme it shows the following line:
   xprv = bitsv.Bip32utils.get_xprv_bip32_node("xprv9s21ZrQH143K4Un4SHjdvXpzzdQjpm7vVhQ79BMi5V58nptUo4NGqytwH68XAVj5LkDxjSqdVjdDinFCT8WqfBT7zigdtaGcrffTmBdwFH5")
   From where exactly do get the string in the function parameter?

2. When i receive a payment into the addresses supplied by bitsv.Bip32utils.get_addresses_from_xprv() output does the balance of the received funds summarizes into a single parent address?

Assuming 2 is true:

3. Can i send payments from that parent address using the first example in the readme?
   if not then how?

4. When that transaction is made then does multiple transactions of all the HD addresses are created? or only from the parent address?

5. Can the transactions made by the parent address be tracked if someone has a child HD address of it?

Thanks

Got another one. I'm looking into it.

File "/usr/local/lib/python3.5/dist-packages/bitsv/network/rates.py", line 639, in satoshi_to_currency_cached
     num / Decimal(currency_to_satoshi_cached(1, currency))
TypeError: unsupported operand type(s) for /: 'float' and 'Decimal'

Would be nice to cut a new release with the rates API. However, it still won't install on 3.5 with bitcoinX.

Do you think it's doable to move bip32 out to its own repository? Another possibility is fixing 3.5 support in bitcoinX, but it seems to depend on electrumSV which also uses f-strings, so seems like quite the thing to chase down.

I tried moving my stuff over to Debian Buster which has Python 3.7, but that was a whole can of worms. Made some headway, but think I want to wait until the next Saltstack release and ideally a minor release or two on 10.

Hi Aust,

The dust policy has been changed after Node 1.0.5, now the minimum output is 135sat, according to the release note.

I broadcast 135sat and 134sat on WoC.

• 135sat, OK

https://whatsonchain.com/tx/a147ec1a4b5eddfcd333f4f1156e301a43e5c8336d423c45273279e9edf0b220

• 134sat, failed

0100000001477a5b512c95704c0a8d652b2bce4e4b170bbf2a53d59543e22489d640914f8e000000006a473044022007cda4515046d56836dd704ddc5b9ef766a4a648aa09d90916ad6872421e0d8d022072424611f754b362d1af04e60b7302e89c88ff6c14a72654125d52cc6573221541210272da37fa8c7873db2e57ade507525737f8f67ba3e7a61c7667e27a4cd9c743c7feffffff0286000000000000001976a9144b267d063a80f3e4ae65149c7c80107a5d4f3ce688ac42110000000000001976a914570abc2c148eefccd119fbb2eccf82c3e55280e088ac640c0a00

Any schedule to follow this? Thank you!

Todo:

• Update README.md

Anything else that we should do before then?

Shower thought:

I wonder if the network backend should have an abstract base class (interface)... that forces implementation of 5 main methods...

And allow a formal way of extending the PrivateKey class with additional 3rd party methods.

The reason I'm thinking this is that there are a number of different service providers and will probably be more in future. Not to mention things like mAPI (merchant API), ElectrumX, and other chain indexers.

and I simply don't have time to keep up with it all and support all of them.

This is especially true for _unwriter's tools which I want to support but I'm stretched a bit thin to formally include it in an optimal way... perhaps there's a way to cleanly outsource this via a standardized network interface + plugin-like model? (Then _unwriter tool devs can be free to build out their own bitsv extensions as they see fit)

This is just added as a reminder

Hi Aust, per our conversation in Slack, raise this issue as a reminder.

Thanks.

https://github.com/AustEcon/bitsv/blob/master/bitsv/network/services/network.py#L57

Never, ever should a library print to stdout unless asked to. We can change this to write to stderr. We can remove it. Or we can just do logging.warning and let the upstream logging setup handle it. We can also change the default logging level. All of these are fine.

Also, Bitindex is down, again.

ConnectionError: All APIs are unreachable, exception:504 Server Error: Gateway Time-out for url: https://api.bitindex.network/api/v3/main/addrs/utxo

Been down for a few hours now. Do you have external monitoring in place?

Added here as a reminder

Added as a reminder

I think we should do this next release (i.e. after 0.9.0)

Hi @AustEcon, per our discussion in Slack, raising this issue.

Currently, it's not easy to send all the amount of several unspents since the fee is calculated inside the functions. Below code is my way, not clean and elegant, but works...

# Sum the amount of these UTXOs
amount = 0
for unspent in unspent_group:
    amount += unspent.amount

# Calculate transaction fee
fee = 0
try:
    # Send the whole balance directly should be fail since insufficient funds
    priv_key.send(outputs=[(send_to, amount, 'satoshi')], message=message, unspents=unspent_group)
except InsufficientFunds as e:
    match = re.match(r'Balance (.+) is less than (.+)\(including fee\).', str(e))
    if len(match.groups()) == 2:
        fee = int(match.group(2)) - int(match.group(1))

# Resend the transaction including the accurate fee
tx_hash = priv_key.send(outputs=[(send_to, amount - fee, 'satoshi')], message=message, unspents=unspent_group)

Already read the code of estimate_tx_fee and sanitize_tx_data

calculated_fee = estimate_tx_fee(len(unspents), num_outputs, fee, compressed, total_op_return_size)

It seems that I need to recalculate the total OP_RETURN size again by myself at least, also need many related parameters, which means repeated code(not sure).

So I'm wondering whether bitsv could expose some functions that calculate the fee ahead of time or just give out a function like create_transaction and send, which will send all the amount of inputs.

Thanks 😄

domain bitcoinsv-rates.com has expired
api at http://bitcoinsv-rates.com/api/rates/ used by satoshi_to_currency_cached is broken
is there some other api that can be consumed?