block-core / blockcore Goto Github PK

Fee Estimation ported from bitcoin core

Copy over the code for fee estimation from stratis repo

This PR was initially started by @AndrewZvvv
stratisproject/StratisBitcoinFullNode#1486

And continues here and later by @zeptin
stratisproject/StratisBitcoinFullNode#4040

Due to the link between repos now removed the code probably needs to be copied over

Enable TOR functionality into fullnode

If I configure my node to use the TOR network and in my config I set externalip=XXX.onion - I get the following error:

There was a problem initializing the node. Details: 'No such device or address'

Uint256_CreateNew

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18362
Intel Core i9-9980HK CPU 2.40GHz, 1 CPU, 16 logical and 8 physical cores
.NET Core SDK=3.1.201
  [Host]     : .NET Core 3.1.3 (CoreCLR 4.700.20.11803, CoreFX 4.700.20.12001), X64 RyuJIT
  DefaultJob : .NET Core 3.1.3 (CoreCLR 4.700.20.11803, CoreFX 4.700.20.12001), X64 RyuJIT

Uint256_Serialize

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18362
Intel Core i9-9980HK CPU 2.40GHz, 1 CPU, 16 logical and 8 physical cores
.NET Core SDK=3.1.201
  [Host]     : .NET Core 3.1.3 (CoreCLR 4.700.20.11803, CoreFX 4.700.20.12001), X64 RyuJIT
  DefaultJob : .NET Core 3.1.3 (CoreCLR 4.700.20.11803, CoreFX 4.700.20.12001), X64 RyuJIT

Uint256_ToBytes

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18362
Intel Core i9-9980HK CPU 2.40GHz, 1 CPU, 16 logical and 8 physical cores
.NET Core SDK=3.1.201
  [Host]     : .NET Core 3.1.3 (CoreCLR 4.700.20.11803, CoreFX 4.700.20.12001), X64 RyuJIT
  DefaultJob : .NET Core 3.1.3 (CoreCLR 4.700.20.11803, CoreFX 4.700.20.12001), X64 RyuJIT

It seems as though the getnewaddress rpc command only supports legacy addresses. Please add segwit compatibility.

This might not be up to the blockcore, but example on handling auto-register of listeners on reconnect would be useful.

From discord by sondreb:

If the masterfile.json is invalid JSON (written during a crash or something), then at startup this is logged:

[2019-10-22 19:07:24.1226 15] INFO: Stratis.Bitcoin.Features.Dns.DnsSeedServer.Initialize Loading cached DNS masterfile from

And then:

[2019-10-22 19:07:24.1368 7] INFO: Stratis.Bitcoin.Features.Dns.DnsSeedServer.SeedMasterFile Seeding DNS masterfile with SOA and NS resource records

Which it logs again and again and again, without exception logged. So this scenario must be fixed and properly logged as exception.

In order to not need to completely override the MempoolValidator (for downstream projects) the premempool can also go in rule of its own (or alternatively an override)

Needed for BTCPayServer integration

stratisproject/StratisBitcoinFullNode#3306

Add to the following pages:

• send

A problem with running a hot wallet is that you have no control over who can stake using your wallet i.e., as a hot wallet operator I am unable to refuse to receive stake delegations.

This could be used as a vector for denial of service - if a staking address is leaked or provided as part of a staking pool, it is possible to force the stakers wallet to manage an unreasonable amount of UTXOs, it may impact the staking performance and possibly increase orphans. The cost of such attack is negligible costs (only the fees required in the stake delegations) as the coins always remain in possession of the attacker.

It would also be possible for a user to identify another stakers address simply by examining the blockchain and then delegate staking to that address.

One option would be to allow the staker to whitelist only "approved" addresses. Another consideration would be to add a flag that would purposely allow cold staking rather than normal staking.

Add segwit to getnewaddress RPC call

For example MaxTrxFee
https://github.com/obsidianproject/ObsidianX/issues/61

Also some changes we made in ODX that can go in the network class are probably needed to allow more flexibility for the platform (I will compile a list and track it on this PR)

Some projects that have been discussed on discord, developers interested can get help and support from blockcore channels on these topics.

NEXT PROJECT: What should our next project/service/feature be? I'm thinking we need some dapp that brings more devs, more users and more attention.

I can continue working on Indexer, Explorer, Insight until my face turns purple, but it won't help us grow or gain much additional traction.

Few suggestions:

• Chat: Enable users to communicate and all messages must be paid for with a small fee ("Stamp"). There is already work in progress on this for BCH chain, we could join forces, fork, etc. with that or just take inspiration from them.

• Dapp SDK/Platform: Make it easier for anyone to build distributed applications on the Blockcore Platform.

• DEX: Distributed Exchange with atomic swaps between Blockcore chains.

• BTCPay support, making it possible for merchants to easily adopt any Blockcore coin for payment.

• Point-of-Sale: I'm planning on making a basic PoS, which could be easier to do if it initially is built around BTCPay.

• Identities: This is almost done on prototype-level/quality.

• Token Platform: Make it super easy and simple to create tokens, for tickets, proof of ownership, etc.

• API Wallet: This is fairly important, Bitcore provides this with their module node for bitcoin. Bitcore is the hub for many Bitcoin-related software and services. So make a replacement for "Bitcore Wallet Service". Should consider just forking and making the backend compatible, as oppose to develop our own solution. Being compatible with Bitcore, would make adoptions of other third party software built on Bitcore, super easy.

• DID: Decentralized identity, this is not the same as identity described above, this is on-chain and can be used for authentication.

My main personal goal of all this, is to bring on more devs that can help contribute and develop everything we do even further. And use what we make, to help Liberstad (and Liberland + other projects) be successful.

Because a hot address is formatted exactly the same as a normal spending address, it is very easy for a user to become confused and send funds directly to the hot address. Whilst it's possible to retrieve these funds (with a bit of manual effort) ... it would be preferable that the hot address is easily differentiated (different first character or different length) and a UI could stop funds being sent directly to this address.

Should perhaps show the account balance when on sending screen to see how much one can possibly send?

This will support protocol version 70014 and 70015 (compact with segwit)

https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki

Add transaction details page showing:

• Total Amount
• Total fee paid
• Coin ticker
• Block confirmed
• Number of confirmations
• Tx id

Add to the following pages:

• wallet history
• cold staking history

When calling validateaddress with a segwit address I would expect the iswitness result to be true.

{
  "isvalid": true,
  "address": null,
  "scriptPubKey": null,
  "isscript": false,
  "iswitness": false
}

There are no connection between the default config file and what is written there, and the command line argument descriptions.

As suggested by MithrilMan, consider doing something like this and instead use reflection to output to users:

[NodeOption(defaultValue: null, type: typeof(string), description: "sets the default data dir folder of the node", group: "core"]
public string datadir {get; set}

[NodeOption(defaultValue: null, type: typeof(bool), description: "enables or desable ip range filtering", group: "network"]
public bool iprangefiltering {get; set}

Hi stumbled upon this project. I may be interested in contributing especially around improving the functionality of the bitcoin fullnode.

.NET standard seems to lag core in functionality. Sounds as though its planned to merge core and standard in .net 5 so I'm not sure sticking with standard at this point is worthwhile.

Interested in hearing the maintainers thoughts, if they are in agreement I could possibly look at preparing a PR to upgrade the projects to .net core 2.2

Here is draft that can be used to define severity of security bounties

issue no.  severity (0-10)   type              worst case                          note
00               0           performance       lower validation performance        
01               3           memleak           nodes crash                         
02               1           usability         lose track of coins                 
03               6           crash             nodes stuck, need update            
04             0 / 6         freeze            nodes stuck, need update            suspected/theoretical, unverified
05               7+          det. validation   chain split                         not checked if can be standalone
06               8           logic             disable part of validation          
07              10           combo RCE         money stolen inc. some main chain   implication of 05 + 06 to increase severity
08              10           RCE               money stolen inc. some main chain   
09               4           crash             nodes stuck, need update            1 malicious/hacked miner needed
10               4           crash             nodes stuck, need update            1 malicious/hacked miner needed

TBD

This is in order to support protocol version 70013

https://github.com/bitcoin/bips/blob/master/bip-0133.mediawiki

This issue is to tack the PR on ObsidianX (but it will be merged to bock core as well)

https://github.com/obsidianproject/Obsidian-StratisNode/pull/41

This was fixed here and need to be ported to blockcore
stratisproject/StratisBitcoinFullNode#4119

With the following connections: [in:22 out:6], I am seeing the following errors in my logs

Connection from client '[::ffff:xxx.xxx.xxx.xxx]:42694' was rejected and will be closed, reason: Inbound Refused: Max Inbound Connection Threshold Reached, inbounds: 109

Automatically calculate the max amount minus a fee. It should be added to:

• Send
• Coldstake setup
• Coldstake delegate
• Coldstake withdraw

When recovering a wallet using the mnemonic 12 words, only walletName/account0 gets discovered.

walletName/coldStakingColdAddresses and walletName/coldStakingHotAddresses are not discovered, thus showing no balances and to the user it looks as if they were never set-up before.

Enabling Hot-Staking or Cold-Staking on a newly recovered wallet does not discover the previously used address either. Rescan and full datadir wipe does not help.

Looking at the recovered wallet.json file and comparing it against the original file, we can see that there is no data on the Internal Addresses starting "index": 10000000;

The only way to safely back-up your cold-staking balance or hot staking wallet is to regularly make a copy of the *.wallet.json, which really defeats the purpose of having a mnemonic recovery.

This will be a consensus item change for all networks.

This issue will track the discussion around this proposal

As part of the work on ObsidianX (ODX) @aubergemediale has proposed that we remove the Time property form a POS transaction as this is not really needed, a coinstake is attached to a block and the block time must be the same to a coinstake trx.

Time field is a left over of coinage that was intended to have some weight in the UTXO ability to participate in staking but that was ditched in previous algo versions
See the white paper for more info
https://decred.org/research/king2012.pdf
https://blackcoin.org/blackcoin-pos-protocol-v2-whitepaper.pdf

Impact on existing chains - they should mandate a time span padding of zeros to avoid malleability and potentially increment protocol version to know what type of trx to send to peers.

#8

This is one PR that can be ported to blockcore
MetacoSA/NBitcoin#763 (comment)

As well as some changes in the bitcoin serialization code (to use span for example)
but that depends on this PR #11

This discussion can be useful as well
stratisproject/StratisBitcoinFullNode#3336

Additional improvements please share in this issue

stratisproject/StratisBitcoinFullNode@e11931b

peers are always banned even that RejectUntil is true

It would be useful for nodes running on servers with constrained hard disk space to enable pruning.

This is actually already implemented just need to signal when node is pruned

https://github.com/bitcoin/bips/blob/master/bip-0159.mediawiki

This issue is to maintain a collection of links to the proposal by @MithrilMan

Architecture, discussions and issues can be found here
https://github.com/MithrilMan/StratisBitcoinFullNode/tree/research/wallet/src/Stratis.Features.Wallet/Documentation

Note: many of the desired changes where already made in that branch

In order to reduce the nodes memory bloating we are trying to understand where is most of the memory allocated ad where should we focus our efforts.

This is a breakdown by @MithrilMan of the block elements and how much memory each will take

basically an array of blockheaders take:
24 byte for the array header that is composed by
- one pointer to  ObjectHeader (8 byte)
- one pointer to  MethodTable (8 byte)
- one byte to hold the array length

then for each item in the array, we have
- one byte for the pointer to the item
- the size of the header object
so looking at 100 headers instantiated, we see 16024 bytes, -24 of the header, means that every header causes the allocation of 160 bytes (152 is the header size, + 8 for the item pointer)
let me now take a look at the blockheader to see why it takes 152 bytes
protected uint256 hashPrevBlock;
protected uint256 hashMerkleRoot;
protected uint nTime;
protected uint nBits;
protected int nVersion;
protected uint nNonce;
4 int/uint = 16
2 uint256 = 2 * (pointer to uint256  + uint256 size)
uint256 should be hopefully 32 byte if doesn't have other members inside, let see
wait blockheader has other members that are private
nVersion = 4 byte
no no nvm

Easy way to import private key (swipe) should be added to the node and REST API.

This is not import per se, but an swipe. Swipe means the funds available on the private key, is sent to a new unused address in the HD wallet.

Difference explained here: https://99bitcoins.com/bitcoin-wallet/paper/private-key-sweep-import/

This will allow us to release and make the paperwallet available for Blockcore chains.

This is probably a bug, as there should be a new entry for every block reward, and maybe even a "reward": true, value is needed.

Currently, the terminology of balanceChangedHeight is correct, but when you are staking, and when receiving a reward the satoshi balance changes for every reward, but the balanceChangedHeight does not change, and there is not a new entry.

{
  "balancesData": [
    {
      "address": "TDzLEpQ1Q5K5wUihAnNC6AD5aKz2KkdVsb",
      "balanceChanges": [
        {
          "deposited": true,
          "satoshi": 105000000000,
          "balanceChangedHeight": 336318
        }
      ]
    }
  ],
  "consensusTipHeight": 339973,
  "reason": null
}

It should be possible to click a button to mark the transaction to send max amount, without having to manually estimate the fee. Which will be wrong depending on size of UTXOs as input.

In order to more effectively test the Blockcore's new features, let's add a Blockcore reference implementation. Initially at least this should run both POW & POS.

The GetStakingNotExpired API, it doesn't return the expiry date/time ... this would be useful as a future enhancement

The nodes (peers) are just listed, would be nice to have an counter of how many inbound and how many outbound. Perhaps even aggregate on the agents as well?

Implement CI to build blockcore nuget packages

Should be part of the generate sample node.

This issue is to track the refactor of the coin database.

Some history on stratis about this change and more useful links
stratisproject/StratisBitcoinFullNode#481
stratisproject/StratisBitcoinFullNode#2414

@MithrilMan did some reach on the topic as well
https://github.com/MithrilMan/StratisBitcoinFullNode/blob/research/wallet/src/Stratis.Features.Wallet/Documentation/discussion-and-use-cases.md

Bitcoin core links
bitcoin/bitcoin#10195
https://bitcoincore.org/en/releases/0.15.0/

Analysis of the Bitcoin UTXO set
https://eprint.iacr.org/2017/1095.pdf

Step 1:
Move to a UTXO model (instead of trx model) this means coins are indexed based on a UTXO [trxhash-n] the benefits are that when pulling a utxo to be spent we don't have to pull the entire transaction from disk (even if it has utxos we are not interested in) and deserialize the whole payload, this save perf when persisting the trx back to disk (if it had 3 outputs and we spent only 1 the other two get persisted again) it also mean more bloated rewind data.

This is done in #51

Step 2:
Use a different data store that actually deletes entries.
TBD

So it can change based on chain