blocksci_parser - stuck about blocksci

[shihao-guo]

if you use top or htop you will see the cpu and memory use of the process, which gives you some processing indications.

[pranay01]

I see blocksci_parser working in htop. But it takes only 0.7% CPU and 228MB memory. I have a system which has a RAM of 6.8 GB.

When I try to import it in python3.5 , it gives a segmentation fault.
`
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.

import blocksci
chain = blocksci.Blockchain("/mnt/disk1/blocksci_data")
Segmentation fault (core dumped)
`

[hkalodner]

Hi @pranay01,

The segmentation fault is from the python module is likely due to corruption of some sort in the data located at /mnt/disk1/blocksci_data. The code is still fairly research quality so it doesn't handle the unexpected very well 😄

Have you tried the new release of BlockSci that we released yesterday? It should provide feedback through the entire process. If I remember correctly, the previous version didn't provide feedback during the initial block header parsing stage. Now all stages should provide percentage done estimates.

The full parse process takes a little under a day at this time. The many data processing takes about 8 hours and then there is a long index generation process. BlockSci doesn't require complete chain data and the parser in fact has a --max-block parameter that lets you process only up to a given block height.

[will-holley]

@hkalodner - Is the data set you are using in development publically available?

[hkalodner]

I'm not exactly sure what you mean by data set. BlockSci's data is generated directly from raw blockchain data. The readme links to an EC2 AMI which has a BlockSci instance running on it which contains the full Bitcoin blockchain as of a couple days ago. Additionally the data can be generated on any machine running a Bitcoin full node.