btccom / btcpool-abandoned

backend of pool.btc.com

BTCPool for Bitcoin, Ethereum, Decred, Bytom, ...

Branch	Status
master
deveth

BTCPool is backend system of https://pool.btc.com.

This is a version of BTCPool that supports more blockchains. Check the SHA256-only version at master branch.

The pool backend support these blockchains at current:

• SHA256
  • Bitcoin
  • BitcoinCash
  • UnitedBitcoin
• Scrypt
  • Litecoin
• ETHash / Daggerhashimoto
  • Ethereum
  • Ethereum Classic
• EquiHash
  • ZCash
  • Beam

  Tips: ZCash and BEAM use different EquiHash parameters and are not compatible in mining.

• Cuckoo Cycle
  • Grin

  Tips: Grin supports Cuckaroo Cycle 29 and Cuckatoo Cycle 31 at the same time, miners can choose an algorithm to mine.

• Blake-256
  • Decred
• Tensority
  • Bytom
• Eaglesong
  • Ckb (finished but the test is not enough)

  Tips: only Tested with bminer and nbminer. need to be updated after the new version of ckb is released.

• Others
  • Siacoin (not finished and need test)

It also support these merged mining blockchains of SHA256 blockchains:

• SHA256 merged mining
  • Namecoin
  • RSK
  • ElastOS
  • Other blockchains that compatible with Bitcoin merged mining specification

If you want merged mine more than one chains that compatible with Bitcoin merged mining specification, use merged mining proxy.

Note: The project is still a testbed and work in progress, all things could be changed.

See Also:

• BTCPool's golang modules

Architecture (need update)

Install

1. Install Zookeeper, or see INSTALL-ZooKeeper.md

• https://zookeeper.apache.org/

2. Install Kafka, or see INSTALL-Kafka.md

• https://kafka.apache.org/

3. Install BTCPool, see INSTALL-BTCPool.md

Upgrade

Upgrading to BTCPool 2.3.0 requires additional operations on sharelogger, slparser and statshttpd due to incompatible sharelog format changes.

See UPGRADE-BTCPool.md for more information.

Benchmark (outdated)

We have test 100,000 miners online Benchmark. see Benchmark-100000.md

BTCAgent

BTCAgent is a kind of stratum proxy which use customize protocol to communicate with the pool. It's very efficient and designed for huge mining farm.

• AGENT.md
• BTCAgent's HomePage

Testing

You could run simulator to test the system. It will simulate a lots of miners, need to enbale config enable_simulator on your Stratum Server.

License

BTCPool is released under the terms of the MIT license. See LICENSE for more information or see https://opensource.org/licenses/MIT.

Welcome aboard!

BTC.COM Team.

btcpool-abandoned's Issues

nmc btccom-v0.1-beta docker namecoin-cli can't be used for some command

Why use calcWorkerId?

I'm interested in the codes.
I noticed that every worker name has a workerid, which is calced by a hash function.(code comment link to a Birthday_attack wiki page).

Is that just for speed optimization or has any other concerns?

how to insert user in database for testing backend?

add jobId or timestamp into coinbase tx input's script

If merkle root hash is the same between different jobs, could have duplicate share attack in some condition. For bitcoin it's very hard to do the attack but for altcoin it's much easier.

Payout system

Hi! Is there any preferred payout system? PPLNS? PPS? Or another?

linux build - undefined reference to `__gmpn_sub_n'

Hey can you advice on this one, haven't found solution elsewhere

Building CXX object CMakeFiles/unittest.dir/test/gtest/gtest-all.cc.o
[100%] Building CXX object CMakeFiles/unittest.dir/test/TestStratumServer.cc.o
[100%] Built target statshttpd
[100%] Built target nmcauxmaker
[100%] [100%] Built target gbtmaker
Built target simulator
[100%] Built target slparser
[100%] Built target poolwatcher
[100%] Built target sharelogger
..../src/.bitcoin//srcsecp256k1/../libsbitcoin//src//secp256k1bitcoinlibsecp256k1.a//secp256k1(/libsecp256k1_la.-.libssecp256k1.olibs)/:libsecp256k1.a(libsecp256k1_la-secp256k1.o ): In functionIn / secp256k1_num_set_binfunction' :
secp256k1_num_set_bin'/libsecp256k1.awork:/
(/btcpoollibsecp256k1_la/worksrc/-/secp256k1.obtcpool)bitcoin/:src //bitcoinsecp256k1/Insecp256k1//srcsrc //num_gmp_impl.hnum_gmp_impl.hfunction::4949 :: undefinedundefined secp256k1_num_set_binreferencereference 'to:to / __gmpn_set_strwork__gmpn_set_str/'btcpool '. /...//srcsrc/srcbitcoin///secp256k1bitcoinbitcoin//secp256k1.//.libslibs/secp256k1libsecp256k1.a/(/libsecp256k1_lalibsecp256k1.a-srcsecp256k1.o()libsecp256k1_la/-:secp256k1.o num_gmp_impl.hIn) :function: 49In :functionsecp256k1_num_mod_inverse ' : secp256k1_num_mod_inverse/undefinedwork'/: btcpool//worksrcreference//bitcoinbtcpool //secp256k1srcto// srcbitcoin//__gmpn_set_strnum_gmp_impl.h':secp256k1128/:
src.undefined/ num_gmp_impl.h.reference: 128/:to src/undefinedbitcoin__gmpn_gcdext/ secp256k1'/reference. libs. .//src/libsecp256k1.a(libsecp256k1_la-secp256k1.o): In function bitcoin/secp256k1_num_mod_inverse':
/work/btcpool/src/bitcoinsecp256k1/.libsto/ libsecp256k1.a(__gmpn_gcdextlibsecp256k1_la/-'secp256k1.o secp256k1.)/:. /srcsrcIn/ /bitcoinfunctionnum_gmp_impl.h /secp256k1:secp256k1_num_get_bin/'.:128
libs//:libsecp256k1.awork (/undefinedbtcpool /libsecp256k1_lasrc-referencesecp256k1.o/)bitcoin /to:secp256k1 / srcIn /functionnum_gmp_impl.h__gmpn_gcdext: 34':secp256k1_num_get_bin '
:undefined
.reference/.work /to/ btcpoolsrc/src__gmpn_get_str/'/bitcoinbitcoin /./.secp256k1//secp256k1srcsrc///bitcoinnum_gmp_impl.h.:/34secp256k1libs/:. /undefinedlibsecp256k1.alibs(/ libsecp256k1.areference(libsecp256k1_lalibsecp256k1_la- secp256k1.oto- )secp256k1.o: )__gmpn_get_strIn':
.In. /functionsrc/ bitcoin /function__gmpn_subsecp256k1'/:. libs //usrlibsecp256k1.a/(secp256k1_num_get_binlibsecp256k1_lainclude-/'x86_64:secp256k1.o
-)/linux:- workIngnu //functionbtcpoolgmp.h/: 2097:__gmpn_sub 'undefined: srcreference //bitcointo/ usr/secp256k1include__gmpn_sub_n/'/
x86_64src-linux-gnu/num_gmp_impl.h/gmp.h:34:2097: : undefined reference to __gmpn_sub_nundefined reference' to __gmpn_get_strcollect2: error: ld returned 1 exit status
'collect2: error: ld returned 1 exit status

../src/bitcoin/secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): In function __gmpn_sub': /usr/include/x86_64-linux-gnu/gmp.h:2097: undefined reference to __gmpn_sub_n'
collect2: error: ld returned 1 exit status
make[2]: *** [sserver] Error 1make[2]:
*** [blkmaker] Error 1
make[1]: *** [CMakeFiles/sserver.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/blkmaker.dir/all] Error 2
make[2]: *** [jobmaker] Error 1
make[1]: *** [CMakeFiles/jobmaker.dir/all] Error 2
Linking CXX executable unittest
../src/bitcoin/secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): In function secp256k1_num_set_bin': /work/btcpool/src/bitcoin/secp256k1/src/num_gmp_impl.h:49: undefined reference to __gmpn_set_str'
../src/bitcoin/secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): In function secp256k1_num_mod_inverse': /work/btcpool/src/bitcoin/secp256k1/src/num_gmp_impl.h:128: undefined reference to __gmpn_gcdext'
../src/bitcoin/secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): In function secp256k1_num_get_bin': /work/btcpool/src/bitcoin/secp256k1/src/num_gmp_impl.h:34: undefined reference to __gmpn_get_str'
../src/bitcoin/secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): In function __gmpn_sub': /usr/include/x86_64-linux-gnu/gmp.h:2097: undefined reference to __gmpn_sub_n'
collect2: error: ld returned 1 exit status
make[2]: *** [unittest] Error 1
make[1]: *** [CMakeFiles/unittest.dir/all] Error 2
make: *** [all] Error 2

list_id_api lib missing?

project doesn't have api lib for below?

$ curl http://xxx.xxx.xxx/GetUserList?last_id=0

what's the difference between statushttpd and slparser?

as we know, slparser parse the binary file which is logged from share_log, and save the worker info to the local_status_db; statushttpd consume the share_log and get some worker info to the local_db.
why not use the statushttpd to get all the info and save it to one db? what's the advantage of this framework? Thx.

Build fails

Hello.
I made some errors at compile.

Environment:

ubuntu 18.04 bionic
zmq-v4.2.5,
glog-v0.3.4,
librdkafka-v0.9.1,
libevent-2.1.8-stable,
bitcoin-0.16.0

and compile command is:

git clone https://github.com/btccom/btcpool.git
cd /work/btcpool
mkdir build && cd build
cmake -DCHAIN_TYPE=BTC -DCHAIN_SRC_ROOT=/work/bitcoin-0.16.0 -DCMAKE_BUILD_TYPE=Debug ..
make -j $CPUS

It is recorded as follows.

-- GIT commit hash: 735180e

------------------------------- Install & Package -------------------------------
-- Configuring done
-- Generating done
-- Build files have been written to: /work/btcpool/build
Scanning dependencies of target btcpool
[ 1%] Building CXX object CMakeFiles/btcpool.dir/src/BitcoinUtils.cc.o
[ 3%] Building CXX object CMakeFiles/btcpool.dir/src/GwMaker.cc.o
[ 5%] Building CXX object CMakeFiles/btcpool.dir/src/Utils.cc.o
[ 7%] Building CXX object CMakeFiles/btcpool.dir/src/Zookeeper.cc.o
[ 9%] Building CXX object CMakeFiles/btcpool.dir/src/MySQLConnection.cc.o
[ 11%] Building CXX object CMakeFiles/btcpool.dir/src/Statistics.cc.o
[ 13%] Building CXX object CMakeFiles/btcpool.dir/src/BlockMaker.cc.o
[ 15%] Building CXX object CMakeFiles/btcpool.dir/src/Stratum.cc.o
[ 17%] Building CXX object CMakeFiles/btcpool.dir/src/Common.cc.o
[ 19%] Building CXX object CMakeFiles/btcpool.dir/src/Kafka.cc.o
[ 21%] Building CXX object CMakeFiles/btcpool.dir/src/GbtMaker.cc.o
[ 23%] Building CXX object CMakeFiles/btcpool.dir/src/JobMaker.cc.o
[ 27%] Building CXX object CMakeFiles/btcpool.dir/src/StratumClient.cc.o
[ 27%] Building CXX object CMakeFiles/btcpool.dir/src/StratumServer.cc.o
[ 29%] Building CXX object CMakeFiles/btcpool.dir/src/rsk/RskWork.cc.o
[ 31%] Building CXX object CMakeFiles/btcpool.dir/src/Watcher.cc.o
[ 33%] Building CXX object CMakeFiles/btcpool.dir/src/StratumSession.cc.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
make[2]: *** [CMakeFiles/btcpool.dir/src/GbtMaker.cc.o] Error 4
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/btcpool.dir/build.make:134: recipe for target 'CMakeFiles/btcpool.dir/src/GbtMaker.cc.o' failed
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
make[2]: *** [CMakeFiles/btcpool.dir/src/GwMaker.cc.o] Error 4
CMakeFiles/btcpool.dir/build.make:158: recipe for target 'CMakeFiles/btcpool.dir/src/GwMaker.cc.o' failed
make[1]: *** [CMakeFiles/btcpool.dir/all] Error 2
CMakeFiles/Makefile2:252: recipe for target 'CMakeFiles/btcpool.dir/all' failed
make: *** [all] Error 2
Makefile:129: recipe for target 'all' failed
Removing intermediate container 72b7bf05a1db
The command '/bin/sh -c bash ./install_btcpool.sh' returned a non-zero code: 2

The following error occurred when I used ubuntu 16.04.

-- GIT commit hash: 735180e

------------------------------- Install & Package -------------------------------
-- Configuring done
-- Generating done
-- Build files have been written to: /work/btcpool/build
Scanning dependencies of target btcpool
[ 1%] Building CXX object CMakeFiles/btcpool.dir/src/Kafka.cc.o
[ 3%] Building CXX object CMakeFiles/btcpool.dir/src/JobMaker.cc.o
[ 5%] Building CXX object CMakeFiles/btcpool.dir/src/Statistics.cc.o
[ 7%] Building CXX object CMakeFiles/btcpool.dir/src/StratumServer.cc.o
[ 9%] Building CXX object CMakeFiles/btcpool.dir/src/BitcoinUtils.cc.o
[ 11%] Building CXX object CMakeFiles/btcpool.dir/src/MySQLConnection.cc.o
[ 13%] Building CXX object CMakeFiles/btcpool.dir/src/rsk/RskWork.cc.o
[ 15%] Building CXX object CMakeFiles/btcpool.dir/src/Zookeeper.cc.o
[ 17%] Building CXX object CMakeFiles/btcpool.dir/src/GwMaker.cc.o
[ 19%] Building CXX object CMakeFiles/btcpool.dir/src/StratumClient.cc.o
[ 21%] Building CXX object CMakeFiles/btcpool.dir/src/Utils.cc.o
[ 23%] Building CXX object CMakeFiles/btcpool.dir/src/BlockMaker.cc.o
[ 25%] Building CXX object CMakeFiles/btcpool.dir/src/Stratum.cc.o
[ 27%] Building CXX object CMakeFiles/btcpool.dir/src/GbtMaker.cc.o
[ 29%] Building CXX object CMakeFiles/btcpool.dir/src/Common.cc.o
[ 31%] Building CXX object CMakeFiles/btcpool.dir/src/Watcher.cc.o
[ 33%] Building CXX object CMakeFiles/btcpool.dir/src/StratumSession.cc.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
CMakeFiles/btcpool.dir/build.make:446: recipe for target 'CMakeFiles/btcpool.dir/src/rsk/RskWork.cc.o' failed
make[2]: *** [CMakeFiles/btcpool.dir/src/rsk/RskWork.cc.o] Error 4
make[2]: *** Waiting for unfinished jobs....
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
make[2]: *** [CMakeFiles/btcpool.dir/src/StratumServer.cc.o] Error 4
CMakeFiles/btcpool.dir/build.make:134: recipe for target 'CMakeFiles/btcpool.dir/src/StratumServer.cc.o' failed
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
make[2]: *** [CMakeFiles/btcpool.dir/src/Statistics.cc.o] Error 4
CMakeFiles/btcpool.dir/build.make:110: recipe for target 'CMakeFiles/btcpool.dir/src/Statistics.cc.o' failed
CMakeFiles/btcpool.dir/build.make:206: recipe for target 'CMakeFiles/btcpool.dir/src/JobMaker.cc.o' failed
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
make[2]: *** [CMakeFiles/btcpool.dir/src/JobMaker.cc.o] Error 4
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
make[2]: *** [CMakeFiles/btcpool.dir/src/MySQLConnection.cc.o] Error 4
CMakeFiles/btcpool.dir/build.make:158: recipe for target 'CMakeFiles/btcpool.dir/src/MySQLConnection.cc.o' failed
CMakeFiles/Makefile2:326: recipe for target 'CMakeFiles/btcpool.dir/all' failed
make[1]: *** [CMakeFiles/btcpool.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
Removing intermediate container e7642d8ce1d5
The command '/bin/sh -c bash ./install_btcpool.sh' returned a non-zero code: 2

I want to ask what does it depend on?
And the possible reasons for that.

Thanks.

slparser - | [warn] event_add: event has no event_base set.

When I run command

/work/btcpool/build/run_slparser/slparser -c /work/btcpool/build/run_slparser/slparser.cfg -l /work/btcpool/build/run_slparser/log_slparser

following is the error:

[warn] event_add: event has no event_base set.

###################################################################
blkmaker RUNNING pid 16006, uptime 1:04:50
gbtmaker RUNNING pid 16009, uptime 1:04:50
jobmaker RUNNING pid 16002, uptime 1:04:50
kafka RUNNING pid 16004, uptime 1:04:50
php-web RUNNING pid 16001, uptime 1:04:50
poolwatcher RUNNING pid 16005, uptime 1:04:50
sharelogger RUNNING pid 16008, uptime 1:04:50
slparser FATAL Exited too quickly (process log may have details)
sserver RUNNING pid 16010, uptime 1:04:50
statshttpd RUNNING pid 16003, uptime 1:04:50
###################################################################

Some ideas ?

请问博主found_blocks表中is_orphaned是否与出块和确认块有关？

您好，我在代码中没发现有对found_blocks表中is_orphaned字段的改动，请问这个字段是否是对于出块的确认再改动，我们通过什么机制去确认我们出的块被接受确认出块成功了的，谢谢博主。

Empty sharelogger

Hi everyone.
My sharelogger produce empty files. Is it expected or something strange?

/work/btcpool/build$ ls -l ../data/sharelog/
total 0
-rw-r--r-- 1 root root 0 May 28 12:03 sharelog-2018-05-28.bin
-rw-r--r-- 1 root root 0 May 29 09:33 sharelog-2018-05-29.bin

这个版本有问题呀

I0116 19:22:41.017634 4544 StratumServer.cc:159] start job repository consume thread
I0116 19:22:41.025430 4521 StratumServer.cc:446] update users count: 5
I0116 19:22:41.032752 4521 Kafka.cc:421] add brokers: kafka1:9092,kafka2:9093,kafka3:9094
I0116 19:22:41.032845 4521 Kafka.cc:428] create topic handle: ShareLog
I0116 19:22:41.063551 4521 Kafka.cc:421] add brokers: kafka1:9092,kafka2:9093,kafka3:9094
I0116 19:22:41.063633 4521 Kafka.cc:428] create topic handle: SolvedShare
I0116 19:22:41.094233 4521 Kafka.cc:421] add brokers: kafka1:9092,kafka2:9093,kafka3:9094
I0116 19:22:41.094477 4521 Kafka.cc:428] create topic handle: NMCSolvedShare
I0116 19:22:41.124956 4521 Kafka.cc:421] add brokers: kafka1:9092,kafka2:9093,kafka3:9094
I0116 19:22:41.125020 4521 Kafka.cc:428] create topic handle: CommonEvents
I0116 19:25:48.418354 4521 StratumSession.cc:235] client connect, ip: 192.168.1.2
E0116 19:25:48.418558 4521 StratumSession.cc:317] decode line fail, not a json string
I0116 19:25:48.419358 4521 StratumServer.cc:982] got an error on the socket: Connection reset by peer
I0116 19:25:48.419924 4521 StratumSession.cc:235] client connect, ip: 192.168.1.2
W0116 19:25:49.423190 4521 StratumServer.cc:125] getLatestStratumJobEx fail

guiminer无法正常测试，sserver.INFO中不再显示以前的remove过期Job了。

打开supervisorctl进程状态错误FATAL，求问怎么解决

错误如下，不知道是哪里没有配置好

请问这个代码可以直接用于LTC矿池吗？

How to set the value of block_version in jobmaker.cfg?

According to the comment，if block_version is 0, jobmaker will use the the version which returned by bitcoind. So, should I always set this value as 0? In which case should I set block_version as the other value instead of 0?

[Notice] United-Bitcoin Hard Forked at Block Height 506400 without any Sound

The new version of United-Bitcoin (UBTC) was just released yesterday, and the hard fork it contains is activated today.

If you are using BTCPool's source code to dig UBTC and forgot upgrade your United-Bitcoin node, you may want to switch your miners to mine other blockchain as soon as possible to avoid loss of rewards.

Your United-Bitcoin nodes may have synced to a wrong branch/height and need a -reindex to fix, it will take some hours.

For more information, please visit the official github releases of United-Bitcoin:

https://github.com/UnitedBitcoin/UnitedBitcoin/releases

v1.1.0.0
We do a new upgrade at block height 506400 and get some features:

    change block inteval from 10 minutes to 1 minute
    change block reward to 1 (reduce to 80% of former reward)
    coinbase tx need to get 7200 confirmations to get matured

Error block submit in bitcoin testnet:{"result":"inconclusive","error":null,"id":"1"}

Testing in bitcoin testnet and got several:
{"result":"inconclusive","error":null,"id":"1"}

Is it a known issue?

poolwatcher.conf missing from install/supervisord

this file is referenced in INSTALL.md. And there doesn't appear to be any other docs on the contents of this file.

missing libgmp in CMakeLists.txt

1. typo in CMakeLists.txt: THRID_LIBRARIES => THIRD_LIBRARIES

2. I have to add "gmp" to THIRD_LIBRARIRES to build btcpool on ubuntu 16.04.1 LTS. Not sure if it's my fault though.

error message:

/usr/bin/ld: ../src/bitcoin/secp256k1/.libs/libsecp256k1.a(libsecp256k1_la-secp256k1.o): undefined reference to symbol '__gmpn_sub_n'
//usr/lib/x86_64-linux-gnu/libgmp.so.10: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
CMakeFiles/blkmaker.dir/build.make:110: recipe for target 'blkmaker' failed
make[2]: *** [blkmaker] Error 1
CMakeFiles/Makefile2:474: recipe for target 'CMakeFiles/blkmaker.dir/all' failed
make[1]: *** [CMakeFiles/blkmaker.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Kafka queued.max.messages.kbytes configuration

Hello.
I found a problem.
When kafka was initialized, The following configuration is overridden.

https://github.com/btccom/btcpool/blob/4acfdabed0fd3b49b34f92ee709927527e80f8e4/src/Kafka.cc#L91

defaultOptions_["queued.max.messages.kbytes"] = RDKAFKA_QUEUED_MAX_MESSAGES_KBYTES;

And its default value is

https://github.com/btccom/btcpool/blob/4acfdabed0fd3b49b34f92ee709927527e80f8e4/src/Kafka.h#L60

#define RDKAFKA_QUEUED_MAX_MESSAGES_KBYTES   "60000000"

When I use librdkafka latest edition (0.11.4). It's going to make this mistake.

E0707 03:40:49.472179    20 Kafka.cc:138] kafka set conf failure: Configuration property "queued.max.messages.kbytes" value 60000000 is outside allowed range 1..2097151
, key: queued.max.messages.kbytes, val: 60000000

Librdkafka has changed its range of values.

The associated commit is cc43f451b46e41fc3ce00e3b4e5c9bafd11f7b0e and Issues #1304

So should consider changing the configuration to compatibility with the librdkafka.

`sserver` will crash after `mining.authorize` in a 32bit OS

BTCPool has never been run on 32-bit operating systems, so we do not know if it is compatible with 32-bit systems.

But now we have a case. A BTCPool built and run in a 32-bit Ubuntu 16.04 and shows it is not compatible with 32-bit operating systems.

In this case, every thing is OK but sserver. The sserver can start, listen, and receive connections, until a miner to request authorization.

telnet 127.0.0.1 3333
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
{"id": 1, "method": "mining.subscribe", "params": []}
{"id":1,"result":[[["mining.set_difficulty","01000000"],["mining.notify","01000000"]],"01000000",8],"error":null}
{"params": ["wxx.miner1", "password"], "id": 2, "method": "mining.authorize"}
{"id":2,"result":true,"error":null}
{"id":null,"method":"mining.set_difficulty","params":[16384]}
{"id":null,"method":"mining.notify","params":["0","024b2b13987744b4dde4b9bd6152cbf489c4dbe07fa5d2c316da174700000000","01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff1e036c791204174deb592f7970742e636f6d2f","ffffffff02087a7a09000000001976a9147ec01c9c7c66e2901a07a5e025da405dfda34a0c88ac0000000000000000266a24aa21a9edbbac1ce1bbadfed5422536fca8f1e3031d1513417d16ea111d62d0563d79b4b600000000",["7f686aa7c3c774ac1c5cfe7a4c5b8c87a5a9ca537ad907c0ad574438055a72c9","8f07b01db81c78752807480040455913de162389ce18d611edc82a8975ca0822","be632cbeb2f377a0ae39a0dfc3ae2d0b7205f5730a38a13ca2cb98fb247bc4c3","73ea95c65ecec4173fc855934b59d4698630fd0c376dac8b388394b5911fd894","94875313e6f39a2846488c5d64a9c6cd3e7f24f40088903b94d36f5eaf4622c3","331bab3ab914758ff5d7732fe018f76cf45a02b58d093ade925672c48498f6d7","2d059281cdeb5d47e508e5854dc85f77e9897153ef1d890fdb665203a6534597"],"20000000","1973e180","59eb4d17",true]}
// `sserver` segmentation fault
Connection closed by foreign host.

I will not fix this issue because I do not need to run BTCPool on a 32-bit operating system.

BTCPool is only tested in 64-bit operating systems. I recommend that anyone who wants to run BTCPool choose a 64-bit operating system.

BLOCK_REWARD is a const in Stratum.h

// TODO: update when next Halving
#define BLOCK_REWARD 1250000000ll

We may forget to update after next halving.

Really need two databases?

Hello. Now in mysql schema used bpool_local_db and bpool_local_stats_db. Will it be good to combine them into one?

Error when building solution

I've downloaded the master branch and followed the instructions explained in INSTALL-BTCPool.md. Ran cmake for the release build with 4 jobs. When I try to run the make command I'm getting the following errors:

• JobMaker.cc:88:51: error: ‘IsValidDestinationString’ was not declared in this scope
• JobMaker.cc:93:57: error: ‘DecodeDestination’ was not declared in this scope

To avoid those error I just commented the #ifdef CHAIN_TYPE_SBTC in BitcoinUtils.cc and BitcoinUtils.h .

sharelogger append more shares to bin file

Hello. I found a strange things.
If the sharelogger program is stopped and then restart, it will reread all share messages from kafka (from offset 0) and append them to the share bin files.
The question is: these share messages has been written to bin files before. They are supposed to be discarded.
Is that normal?
Or it is just a error caused by kafka setup ?
Thanks.

关于btcpool跨国算力融合共享StratumJob的问题请教

比如有两个矿场分别位于**和加拿大，我在**部署了btcpool，我想把加拿大的算力融合在一起共享kafka中的StratumJob，是在加拿大部署sserver远程连接**的kafka还是将zookeeper+kafka分别部署在**和加拿大进行partitions，sserver只使用特定kafka的partitions？
PS：这套程序的模块化和模块间解耦可以当作教科书。

Why docker used for bitcoind?

Hi, I found docker was used to contain bitcoind progress. I want to know any advantage for that?

Problem compiling with bitcoin0.16.0

I'm trying to compile the pool against bitcoin-0.16.0 and i'm getting the following types of compilation errors.

/src/GbtMaker.cc:179: undefined reference to `base_blob<256u>::ToString[abi:cxx11]() const'
/src/Stratum.cc:245: undefined reference to `base_blob<256u>::GetHex[abi:cxx11]() const'

Some searching has led me down the path of using libdb4.8, but even after installing that i'm still getting the same compilation errors. These methods do exist in the bitcoin source, what am I doing wrong?

sserver: accidental deadlock when release StratumSession from consume thread

This happened to two different servers in two days. The phenomenon is that the TCP connection can be established, but the server will not make any response to your request.

---

It seems like libevent: Deadlock when calling bufferevent_free from an other thread

I'm experiencing a deadlock on libevent-2.0.19 while calling bufferevent_free frome thread A, while thread B is in event_base_dispatch.

And a reply:

Ouch. This is a known bug. This fix is going to be hard. I wrote
about it here:
http://archives.seul.org/libevent/users/Feb-2012/msg00053.html

---

GDB infomations:

(gdb) thread 1
[Switching to thread 1 (Thread 0x7ff2055b3780 (LWP 19479))]
#0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
(gdb) bt
#0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1  0x00007ff2034b8e42 in __GI___pthread_mutex_lock (mutex=0xf73fc0) at ../nptl/pthread_mutex_lock.c:115
#2  0x00007ff2038e6fac in _bufferevent_incref_and_lock (bufev=bufev@entry=0x124a910) at bufferevent.c:582
#3  0x00007ff2038e805a in bufferevent_writecb (fd=44, event=<optimized out>, arg=0x124a910) at bufferevent_sock.c:212
#4  0x00007ff2038dc4c9 in event_process_active_single_queue (activeq=0xe817d0, base=0xd64f20) at event.c:1350
#5  event_process_active (base=<optimized out>) at event.c:1420
#6  event_base_loop (base=0xd64f20, flags=flags@entry=0) at event.c:1621
#7  0x00007ff2038dd5f7 in event_base_dispatch (event_base=<optimized out>) at event.c:1450
#8  0x000000000043919e in Server::run (this=<optimized out>) at /work/btcpool/src/StratumServer.cc:1021
#9  StratumServer::run (this=<optimized out>) at /work/btcpool/src/StratumServer.cc:792
#10 0x0000000000434557 in main (argc=<optimized out>, argv=<optimized out>) at /work/btcpool/src/sserver/StratumServerMain.cc:173

(gdb) thread 7
[Switching to thread 7 (Thread 0x7ff1f96bf700 (LWP 19485))]
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
(gdb) bt
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x00007ff2036ccfa5 in evthread_posix_cond_wait (_cond=0xd5f5c0, _lock=0xd62310, tv=<optimized out>) at evthread_pthread.c:156
#2  0x00007ff2038da58d in event_del_internal (ev=<optimized out>) at event.c:2220
#3  event_del (ev=ev@entry=0x124a9a8) at event.c:2188
#4  0x00007ff2038e85c7 in be_socket_destruct (bufev=0x124a910) at bufferevent_sock.c:592
#5  0x00007ff2038e7472 in _bufferevent_decref_and_unlock (bufev=0x124a910) at bufferevent.c:622
#6  0x00007ff2038e79cb in bufferevent_free (bufev=0x124a910) at bufferevent.c:681
#7  0x0000000000448627 in StratumSession::~StratumSession (this=0x1155640, __in_chrg=<optimized out>) at /work/btcpool/src/StratumSession.cc:261
#8  0x00000000004398b2 in Server::sendMiningNotifyToAll (this=0xd25258, exJobPtr=std::shared_ptr (count 4, weak 0) 0x7ff1e804f240) at /work/btcpool/src/StratumServer.cc:1054
#9  0x0000000000439a89 in JobRepository::sendMiningNotify (this=this@entry=0xd28a40, exJob=std::shared_ptr (count 4, weak 0) 0x7ff1e804f240) at /work/btcpool/src/StratumServer.cc:316
#10 0x000000000043c2bc in JobRepository::consumeStratumJob (this=this@entry=0xd28a40, rkmessage=rkmessage@entry=0x7ff1f0001df0) at /work/btcpool/src/StratumServer.cc:271
#11 0x000000000043c500 in JobRepository::runThreadConsume (this=0xd28a40) at /work/btcpool/src/StratumServer.cc:176
#12 0x00007ff2031e5c80 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#13 0x00007ff2034b66ba in start_thread (arg=0x7ff1f96bf700) at pthread_create.c:333
#14 0x00007ff20294b3dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:74
#15 0x0000000000000000 in ?? ()

It looks like thread 7 want to call bufferevent_free() for a session when thread 1 is writing to it.

I now want to know if the new version of libevent has solved this problem. Otherwise, we need to change the code to prevent this problem.

The libevent I used:

ii  libevent-2.0-5:amd64                   2.0.21-stable-2ubuntu0.16.04.1  amd64        Asynchronous event notification library
ii  libevent-core-2.0-5:amd64              2.0.21-stable-2ubuntu0.16.04.1  amd64        Asynchronous event notification library (core)
ii  libevent-dbg:amd64                     2.0.21-stable-2ubuntu0.16.04.1  amd64        Asynchronous event notification library (debug symbols)
ii  libevent-dev                           2.0.21-stable-2ubuntu0.16.04.1  amd64        Asynchronous event notification library (development files)
ii  libevent-extra-2.0-5:amd64             2.0.21-stable-2ubuntu0.16.04.1  amd64        Asynchronous event notification library (extra)
ii  libevent-openssl-2.0-5:amd64           2.0.21-stable-2ubuntu0.16.04.1  amd64        Asynchronous event notification library (openssl)
ii  libevent-pthreads-2.0-5:amd64          2.0.21-stable-2ubuntu0.16.04.1  amd64        Asynchronous event notification library (pthreads)

Account earn stats by month/year

Can we have a monthly earning API call?
e.g GET /account/earn-stats/?month=January&year=2018

This would be great if implemented.
Thanks

13123

123123123

URL of github link in "build BTCPool that linking to Bitcoin"

"git clone https://github.com/bitcoin/bitcoin.git" seems wrong,
should be this one?
"git clone https://github.com/btccom/btcpool.git"

btcpool包含那些内容？

各位好，btcpool这个开源库包含下图中的那些功能？我阅读了所有的文档，没有发现收益分配规则等相关的内容
https://github.com/btccom/btcpool/raw/master/docs/btcpool.png

[warning, fixed] A poolwatcher bug may cause Bitcoin losing from coinbase TX

A poolwatcher bug was introduced into BTCPool's master branch since commit 00a7b00 and fixed by commit 558a4a6.

The bug is on this line:
https://github.com/btccom/btcpool/blob/00a7b003d21057823b33d29d6ffb085e4bd282f6/src/BitcoinUtils.cc#L42

In order to be compatible with the Amount object of BitcoinCash 0.17, the displacement operation is replaced by division.

However, this operation increased the halving times by one. So the poolwatcher will think that the current block reward is 6.25 BTC instead of the correct value of 12.5 BTC.

Due to the lack of test cases, the bug had not been discovered for a long time. Until yesterday I began to test the new method for calculating mining earnings in deveth branch, and found the bug.

The fix is reverting the code to the previous version and changing the type of variables to compatible with BitcoinCash 0.17.
https://github.com/btccom/btcpool/blob/558a4a679334dc19c4ed975f29559c6c44c88494/src/BitcoinUtils.cc#L47

And a testcase was added for this function:
https://github.com/btccom/btcpool/blob/2cbf1fd2712ba79103150669898002dcb3b76806/test/TestBitcoinUtils.cc#L31

But I mistakenly believe that the bug only affected the poolwatcher's log records, so it was not deployed in time after the problem was fixed. So pool.btc.com mined a block with 6.25 BTC as reward:
https://btc.com/526591

This caused 6.25 BTC to be permanently destroyed from the Bitcoin network.

If you are using the affected poolwatcher version, please update it in time.

【UI】手机端打开首页导航背景缺失

手机端网页打开，首页导航背景条显示缺失

slparser的hashrate_t字段不适用于以太坊

https://github.com/btccom/btcpool/blob/c656fd7a08927abb7300f7af86e458be60929a4e/src/ShareLogParser.inl#L950

该字段采用以下函数进行哈希速率运算，只适用于比特币及pdiff与其相同的DCR，并不适用于以太坊。

// we use G, so never overflow
inline double share2HashrateG(uint64_t share, uint32_t timeDiff) {
  //    G: 1000000000.0
  // 2^32: 4294967296.0
  return share * (4294967296.0 / 1000000000.0 / timeDiff);
}
inline double share2HashrateT(uint64_t share, uint32_t timeDiff) {
  return share2HashrateG(share, timeDiff) / 1000.0;
}
inline double share2HashrateP(uint64_t share, uint32_t timeDiff) {
  return share2HashrateG(share, timeDiff) / 1000000.0;
}

Compile error when using the 'do-everything' bash script

[ 97%] Built target blkmaker
[ 98%] Building CXX object CMakeFiles/unittest.dir/test/gtest/gtest-all.cc.o
In file included from /work/btcpool/test/TestCommon.cc:27:0:
/work/btcpool/test/gtest/gtest.h: In instantiation of ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = long unsigned int; T2 = int]’:
/work/btcpool/test/gtest/gtest.h:19007:23: required from ‘static testing::AssertionResult testing::internal::EqHelper<lhs_is_null_literal>::Compare(const char*, const char*, const T1&, const T2&) [with T1 = long unsigned int; T2 = int; bool lhs_is_null_literal = false]’
/work/btcpool/test/TestCommon.cc:137:3: required from here
/work/btcpool/test/gtest/gtest.h:18971:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if (expected == actual) {
^
In file included from /work/btcpool/test/TestStratumSession.cc:25:0:
/work/btcpool/test/gtest/gtest.h: In instantiation of ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = long int; T2 = long unsigned int]’:
/work/btcpool/test/gtest/gtest.h:19007:23: required from ‘static testing::AssertionResult testing::internal::EqHelper<lhs_is_null_literal>::Compare(const char*, const char*, const T1&, const T2&) [with T1 = long int; T2 = long unsigned int; bool lhs_is_null_literal = false]’
/work/btcpool/test/TestStratumSession.cc:122:3: required from here
/work/btcpool/test/gtest/gtest.h:18971:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if (expected == actual) {
^
/work/btcpool/test/gtest/gtest.h: In instantiation of ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = long unsigned int; T2 = int]’:
/work/btcpool/test/gtest/gtest.h:19007:23: required from ‘static testing::AssertionResult testing::internal::EqHelper<lhs_is_null_literal>::Compare(const char*, const char*, const T1&, const T2&) [with T1 = long unsigned int; T2 = int; bool lhs_is_null_literal = false]’
/work/btcpool/test/TestStratumSession.cc:418:5: required from here
/work/btcpool/test/gtest/gtest.h:18971:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]

min difficulty

Is there a existing way to set a minimum or starting difficulty ?

I am receiveing:
Received mining.set_difficulty... Pool difficulty too low (provided=128, minimum=250000)
I can set it with d=diff in password section but still sometimes drop bellow my min difficulty and so stops mining

Why not send new stratum job when namecoin block height has changed?

I found that when namecoin block height has changed jobmaker doesn't build a new stratum job, and the stratum server will send the new job to the miners only when the hash of bitcoin block has changed.
This will cause that the miners are always calculating the hash of the old namecoin block, and blkmaker will get rpc fail (error code: 500) when calling rpc getauxblock(blockhash, auxpow), because the current height of namecoin has already changed.
I want to know why not send the new job with isClean=true to the miners to calculate with the new namecoin block height?

怎么设置多个gbtMaker

在btcpool架构图里面，是可以布置多个bitcoind节点的，所以我们需要设置数量相同的GBTMaker用来同步。所以，是不是意味着我们需要布置多个btcpool?毕竟一个btcpool服务器里面只有一个gbtMaker?

Proposed efficient getblocktemplate replacement

Bitcoin Unlimited is proposing a replacement for getblocktemplate that sends block candidates in O(log2(blocksize)). It does this by not providing every transaction in the block. Instead it gives the mining pool the block header and the merkle branch to the coinbase transaction (similar to stratum). Two new RPC commands are defined, getminingcandidate and submitminingsolution. To make it extremely easy to integrate into your pool, these RPCs use same json-rpc protocol as getblocktemplate/submitblock, and the same RPC parameters where possible. For example, the header returned from getblocktemplate is formatted very similarly to getblock, and the return (error) values for submitminingsolution are the same as submitblock (plus one additional error).

Please take a look, give feedback, and hopefully you can integrate this RPC into your pool so that large blocks can be communicated efficiently between bitcoind and the pool.

BitcoinUnlimited/BitcoinUnlimited#1139

Documentation:
First section of: https://github.com/gandrewstone/BitcoinUnlimited/blob/minerRpc/doc/miner.md

NiceHash's DaggerHashimoto order always be DEAD when pool's EthereumStratum/1.0.0 protocol enabled

This is a bug report I submitted to NiceHash. I look forward to receiving timely feedback here.

---

Hello, I am a developer of a Ethereum (DaggerHashimoto) mining pool. At present, my pool has encountered compatibility issues with NiceHash. I have done a lot of work, but still can't make the order work. Maybe it's a issue in NiceHash and not in my pool.

The problem is that when the pool enabled the EthereumStratum/1.0.0 protocol, the order status is always DEAD. When the pool disabled the protocol (so that the normal Stratum protocol is used), the order status returns to normal.

As we know, EthereumStratum/1.0.0 is a protocol that proposed by NiceHash to improve the efficiency of mining in Ethereum: https://www.nicehash.com/sw/Ethereum_specification_R1.txt

I think I have implemented the protocol by following the specification completely. And my Nonce size has been set to 2 bytes, and the lowest difficulty has been set to 1. NiceHash's pool validator did not detect any compatibility issues. And the ethminer (https://github.com/ethereum-mining/ethminer/releases) that also supports the EthereumStratum/1.0.0 protocol can successfully mine in my pool.

So, I have no idea. I hope you can investigate this issue with me.

The pool's setting (the main pool server that 100% follows the EthereumStratum/1.0.0 specification):
Pool host: *********
Pool port: 3333
Pool user: stiger.x
Pool pass: x
Algorithm: DaggerHashimoto

The backup pool's setting (there are extra fields in protocol texts):
Pool host: us-eth.ss.btc.com
Pool port: 3333
Pool user: stiger.x
Pool pass: md=4294967296
Algorithm: DaggerHashimoto

None of the above two pool settings can work.

The attachment contains some screenshots:

• order always dead

• always connecting to remote pool

• no compatibility issues with verificator

• mining successfully with ethminer

• protocol text in telnet

SegWit signalling

Hi, im trying to get an understanding of why miners are not signalling segwit yet. Can you share any details as to why btc.com pool is not signalling yet?

Thank you in advance, and my apologies if this is the improper place to ask. Have a nice day.

*** Check failure stack trace: ***

For now while just testing I am running only one zookeper with --replication-factor 1

since launch all instanced died

supervisor> status
blkmaker FATAL Exited too quickly (process log may have details)
gbtmaker FATAL Exited too quickly (process log may have details)
jobmaker FATAL Exited too quickly (process log may have details)
sharelogger RUNNING pid 23082, uptime 0:06:06
slparser FATAL Exited too quickly (process log may have details)
sserver FATAL Exited too quickly (process log may have details)
statshttpd FATAL Exited too quickly (process log may have details)

When I run it manually

---

root@localhost:/work/btcpool/build# /work/btcpool/build/run_sserver/sserver -c /work/btcpool/build/run_sserver/sserver.cfg -l /work/btcpool/build/run_sserver/log_sserver
F1214 22:36:24.403488 23670 Kafka.cc:192] Failed to acquire metadata: Local: Timed out
*** Check failure stack trace: ***
@ 0x7f8e858b8bcd google::LogMessage::Fail()
@ 0x7f8e858bab48 google::LogMessage::SendToLog()
@ 0x7f8e858b8763 google::LogMessage::Flush()
@ 0x7f8e858bb47e google::LogMessageFatal::~LogMessageFatal()
@ 0x4477c5 KafkaConsumer::checkAlive()
@ 0x437ad8 JobRepository::setupThreadConsume()
@ 0x438f4d Server::setup()
@ 0x439916 StratumServer::init()
@ 0x431d40 main
@ 0x7f8e839fdf45 __libc_start_main
@ 0x433aa3 (unknown)
Aborted

Hashrate?

Hi, guys.
How is possible to count hashrate of miners? I see in database only accepted/rejected shares. But as I know also share difficulty needed to count hashrate, insn't it?

Thanks.

too large delay from kafka to receive topic 'StratumJob'

The pools is well working with bitcoin codebase, but I am facing issues to run it for BCH

the following is in my sserver log

E0809 17:42:42.101991 5858 StratumServer.cc:221] too large delay from kafka to receive topic 'StratumJob'

Other components seems to run and report well. Some ideas ?

getLatestStratumJobEx fail

Hello!
First of all thanks for creating such mining pool =)

But during a testing we cannot connect to stratum server

Serverside logs have many of this:

E0524 19:03:46.582401 22602 StratumSession.cc:328] decode line fail, not a json string
I0524 19:03:46.769662 22602 StratumServer.cc:1139] got an error on the socket: Connection reset by peer
I0524 19:03:46.948123 22602 StratumSession.cc:246] client connect, ip: 188.167.167.2
W0524 19:03:47.953395 22602 StratumServer.cc:128] getLatestStratumJobEx fail
I0524 19:03:49.422773 22602 StratumServer.cc:1139] got an error on the socket: Connection reset by peer
I0524 19:06:18.259013 22602 StratumSession.cc:246] client connect, ip: 188.167.167.2
E0524 19:06:18.262748 22602 StratumSession.cc:328] decode line fail, not a json string
I0524 19:06:18.440995 22602 StratumServer.cc:1139] got an error on the socket: Connection reset by peer
I0524 19:06:18.647907 22602 StratumSession.cc:246] client connect, ip: 188.167.167.2
I0524 19:06:20.646862 22602 StratumServer.cc:1136] socket closed
I0524 19:06:34.655304 22602 StratumSession.cc:246] client connect, ip: 188.167.167.2
E0524 19:06:34.659281 22602 StratumSession.cc:328] decode line fail, not a json string
I0524 19:06:34.840478 22602 StratumServer.cc:1139] got an error on the socket: Connection reset by peer
I0524 19:06:35.034485 22602 StratumSession.cc:246] client connect, ip: 188.167.167.2
I0524 19:06:37.049279 22602 StratumServer.cc:1136] socket closed

May be its a known issue and there is an easy answer?

API is not working and/or documentation needs improved

After trying multiple different GET requests the only response I manage to get is this:
{"err_no":20001,"err_msg":"permissions denied"}

This is the path that the documentation provides:
https://${Endpoint}/${Version}/${Path}

For example:
https://us-pool.api.btc.com/v1/realtime/hashrate

Even after passing in the access key and puid I do not get any results. Any suggestions or improvements would be greatly appreciated.

关于子账户实时算力的问题请教

请问用什么方法计算子账户的实时算力、24小时算力和矿池的总算力？谢谢先