Comments (4)
D'oh, I should have collected more data before parsing from zero.. :)
Anyway, some data with the serialization via 4ef1b9eb0e:
./src/bitcoin-cli listtransactions_MP "*" 1 0 0 999999
listtransactions_MP(): 135.888 ms, found: 1 txs, 1 txs seen, 127.934 ms populated, 135.888 ms/tx
listtransactions_MP(): 12.887 ms, found: 1 txs, 1 txs seen, 5.104 ms populated, 12.887 ms/tx
listtransactions_MP(): 11.616 ms, found: 1 txs, 1 txs seen, 5.039 ms populated, 11.616 ms/tx
listtransactions_MP(): 13.200 ms, found: 1 txs, 1 txs seen, 4.957 ms populated, 13.200 ms/tx
./src/bitcoin-cli listtransactions_MP "*" 10 0 0 999999
listtransactions_MP(): 365.838 ms, found: 10 txs, 10 txs seen, 359.305 ms populated, 36.584 ms/tx
listtransactions_MP(): 60.197 ms, found: 10 txs, 10 txs seen, 52.126 ms populated, 6.020 ms/tx
listtransactions_MP(): 55.568 ms, found: 10 txs, 10 txs seen, 47.824 ms populated, 5.557 ms/tx
listtransactions_MP(): 59.676 ms, found: 10 txs, 10 txs seen, 51.577 ms populated, 5.968 ms/tx
./src/bitcoin-cli listtransactions_MP "*" 100 0 0 999999
listtransactions_MP(): 2013.492 ms, found: 100 txs, 120 txs seen, 2004.196 ms populated, 20.13492 ms/tx
listtransactions_MP(): 517.316 ms, found: 100 txs, 120 txs seen, 508.285 ms populated, 5.17316 ms/tx
listtransactions_MP(): 507.737 ms, found: 100 txs, 120 txs seen, 498.137 ms populated, 5.07737 ms/tx
./src/bitcoin-cli listtransactions_MP "*" 999999 0 0 999999
listtransactions_MP(): 243649.652 ms, found: 5151 txs, 5343 txs seen, 243482.901 ms populated, 47.301 ms/tx
listtransactions_MP(): 26720.799 ms, found: 5151 txs, 5343 txs seen, 26567.702 ms populated, 5.187 ms/tx
listtransactions_MP(): 27407.749 ms, found: 5151 txs, 5343 txs seen, 27259.639 ms populated, 5.321 ms/tx
It's notable that the tests were done on top of #74, which explains the 4-10x bump for the 2nd+ calls, due to the input transaction cache in ParseTransaction()
.
It can be seen that the average time to list one transaction is roughly 47 ms uncached, and around 5.2 ms after the first call.
The first time it took 4:04 min to list all 5151 Omni wallet transactions (of 5343 wallet transactions in total), and only about 27 seconds to list all transactions, if called a second time.
Once the mainnet parsing finished, I'm going to get some numbers to compare for the JSON serialization.
from omnicore.
Here we go, without byte serialization:
./src/bitcoin-cli listtransactions_MP "*" 1 0 0 999999
listtransactions_MP(): 137.132 ms, found: 1 txs, 1 txs seen, 129.788 ms populated, 137.132 ms/tx
listtransactions_MP(): 139.959 ms, found: 1 txs, 1 txs seen, 98.398 ms populated, 139.959 ms/tx
listtransactions_MP(): 134.041 ms, found: 1 txs, 1 txs seen, 121.070 ms populated, 134.041 ms/tx
listtransactions_MP(): 129.049 ms, found: 1 txs, 1 txs seen, 117.385 ms populated, 129.049 ms/tx
./src/bitcoin-cli listtransactions_MP "*" 10 0 0 999999
listtransactions_MP(): 1822.721 ms, found: 10 txs, 10 txs seen, 1815.296 ms populated, 182.272 ms/tx
listtransactions_MP(): 909.400 ms, found: 10 txs, 10 txs seen, 901.604 ms populated, 90.940 ms/tx
listtransactions_MP(): 860.536 ms, found: 10 txs, 10 txs seen, 853.687 ms populated, 86.054 ms/tx
listtransactions_MP(): 990.114 ms, found: 10 txs, 10 txs seen, 981.632 ms populated, 99.011 ms/tx
./src/bitcoin-cli listtransactions_MP "*" 100 0 0 999999
listtransactions_MP(): 16344.688 ms, found: 100 txs, 120 txs seen, 16335.132 ms populated, 163.447 ms/tx
listtransactions_MP(): 9397.616 ms, found: 100 txs, 120 txs seen, 9388.876 ms populated, 93.976 ms/tx
listtransactions_MP(): 9253.002 ms, found: 100 txs, 120 txs seen, 9239.713 ms populated, 92.530 ms/tx
listtransactions_MP(): 8890.666 ms, found: 100 txs, 120 txs seen, 8881.084 ms populated, 88.907 ms/tx
listtransactions_MP(): 10226.605 ms, found: 100 txs, 120 txs seen, 10217.782 ms populated, 102.266 ms/tx
./src/bitcoin-cli listtransactions_MP "*" 999999 0 0 999999
listtransactions_MP(): 651622.547 ms, found: 5151 txs, 5343 txs seen, 651484.830 ms populated, 126.504 ms/tx
listtransactions_MP(): 457356.142 ms, found: 5151 txs, 5343 txs seen, 457234.876 ms populated, 88.790 ms/tx
So the average time for one uncached transaction is 127 ms, and around 89 ms for subsequent calls.
In total, it took 10:52 min (vs. 4:04 min) to list all 5151 wallet transactions, if called the first time, and 7:37 min (vs. 27 seconds) for cached queries.
That's around 17x slower.
from omnicore.
This is EPIC - nice work mate!!!!!
from omnicore.
Given that these results are better than expected, we should probably convert the other parts as well.
And since we're using the serialization of Bitcoin Core, things could get a lot easier, because values can simply be wrapped into READWRITE(value)
and whole classes are immediately serializable.
Say there is a class to represent recipients of STO transactions:
class CSerializableStoRecipients
{
private:
uint256 hashTxid;
uint256 hashBlock;
std::string strSender;
uint32_t propertyId;
int64_t amountFee;
std::set<std::pair<int64_t, std::string> > recipientsSet;
public:
/** Creates a new recipients object */
CSerializableStoRecipients()
{
// Set null or so
}
ADD_SERIALIZE_METHODS;
template <typename Stream, typename Operation>
inline void SerializationOp(Stream& s, Operation ser_action, int nType, int nVersion) {
READWRITE(hashTxid);
READWRITE(hashBlock);
READWRITE(strSender);
READWRITE(propertyId);
READWRITE(amountFee);
READWRITE(recipientsSet);
}
/** Transaction hash */
uint256 getTxid() const { return hashTxid; }
/** Block hash */
uint256 getBlock() const { return hashBlock; }
/** Sender */
std::string getSender() const { return strSender; }
/** Property identifier */
uint32_t getPropertyId() const { return propertyId; }
/** Amount paid as STO fee */
int64_t getFeeAmount() const { return amountFee; }
/** Recipients */
std::set<std::pair<int64_t, std::string> > getRecipients() const { return recipientsSet; }
/** Sum of all recipient values */
int64_t getTotalAmount() const;
}
Then it could be stored or retrieved as follows:
bool CMPSTOList::StoreRecipients(const CSerializableStoRecipients& recipients)
{
// DB key for entry
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
ssKey << std::make_pair('t', recipients.getTxid());
leveldb::Slice slKey(&ssKey[0], ssKey.size());
// DB value for entry
CDataStream ssValue(SER_DISK, CLIENT_VERSION);
ssValue.reserve(ssValue.GetSerializeSize(recipients));
ssValue << recipients;
leveldb::Slice slValue(&ssValue[0], ssValue.size());
// Persist entry
leveldb::Status status = pdb->Put(writeoptions, slKey, slValue);
return status.ok();
}
bool CMPSTOList::GetRecipients(const uint256& hashTxid, CSerializableStoRecipients& retRecipients) const
{
// DB key for entry
CDataStream ssKey(SER_DISK, CLIENT_VERSION);
ssKey << std::make_pair('t', hashTxid);
leveldb::Slice slKey(&ssKey[0], ssKey.size());
// DB value for entry
std::string strValue;
leveldb::Status status = pdb->Get(readoptions, slKey, &strValue);
if (!status.ok()) {
PrintToLog("%s(): ERROR: %s\n", __func__, status.ToString());
return false;
}
// Deserialize entry
try {
CDataStream ssValue(strValue.data(), strValue.data() + strValue.size(), SER_DISK, CLIENT_VERSION);
ssValue >> retRecipients;
} catch (const std::exception& e) {
PrintToLog("%s(): ERROR: %s\n", __func__, e.what());
return false;
}
return true;
}
No need to manually parse the list of recipients ... :)
from omnicore.
Related Issues (20)
- What happens to tokens sent to a M address non generated by an Omnilite Core wallet? HOT 2
- Does anyone know that bitcoin core and Omni core can be installed together HOT 2
- I have "invalid transaction" in omnicore explorer but the transaction was proceed in the BTC network as normal transaction ...what do i need to do ? HOT 2
- one big risk that tx result from omnicore rpc is different from omniexplorer.info HOT 9
- Synchronization is too slow. I want normal peer information HOT 2
- Bugs n alice
- omni_gettradehistoryforaddress : Add `blocktime` for each match in matches array of JSON response HOT 1
- Ether
- V0.12.0 release is mandatory but provides no estimation or date for activiation HOT 1
- JSON-RPC method to find transactions in initial OMNI (MSC) crowdsale HOT 2
- Sendtomany transaction in omni_listtransactions & omni_listpendingtransactions
- Update Copyright year to 2022 (Omni Core 0.12.0 release)
- RegTest failed when running against omnicore-qt v0.12.0 HOT 1
- Document Gitian (reproducible) build procedure for Omni Core HOT 1
- Running Omni node in prune mode
- Performance improvements
- Lost 11B on trading platform
- bc1 not supported by OMNI, native segwig support needed for omni tokens HOT 1
- it will override all the other settings to false each time of iterator HOT 1
- Too many open files error HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from omnicore.