Coder Social home page Coder Social logo

omnilaboratory / obd Goto Github PK

View Code? Open in Web Editor NEW
214.0 9.0 21.0 93.54 MB

OmniBOLT daemon, a golang implementation of OmniBOLT spec, the smart assets lightning network.

License: MIT License

Go 99.25% Dockerfile 0.09% Makefile 0.14% Python 0.11% Shell 0.41% Ruby 0.01%
lightning-network blockchain-technology bitcoin usdt omnilayer

obd's Introduction

OmniBOLT Daemon | Smart Asset Lightning Network

OmniBOLT-banner

OBD implements the OmniBOLT specification, and it is an open source, off-chain decentralized platform, build upon BTC/OmniLayer network, implements basic multi hop HTLC payment, multi-currency atomic swap, and more off-chain contracts on the network of smart assets lightning channels.

None Custodial OmniBOLT Daemon

To know how obd works, jump to the OmniBOLT - Architecture.

The latest features and ETA is here: OmniBOLT - Features and Roadmap.

Quick Start on Regtest:

Users can get quickly started with graphic or command line tools:

Polar

The graphic terminal is provided by Polar, and is customized for obd: Polar - GUI Terminal Customized.

Polar helps Lightning Network application developers quickly build networks locally on their computers. Here is a short video demo: https://twitter.com/omni_bolt/status/1549709303921410048?s=20&t=-M9Y4L0Bw_VialiSVPgqmA

Docker

Docker helps people to quickly interact with obd and omnicore via command line tools. OBD uses docker-compose to package obd, omnicored, and btcd together to make deploying these daemons easily. Please check the docker-compose.yml config file for all the configurations under:

https://github.com/omnilaboratory/lnd/tree/obd/docker/obtest

We compiled and deployed images for your testing:

  • obd docker: ccr.ccs.tencentyun.com/omnicore/ob-lnd:0.1.0

Now we can:

Mint tokens

Using omnicore to issue tokens with fixed amount or manageable supply:

For example, use omni_sendissuancefixed to issue 1000000 Quantum Miner tokens on the Bitcoin mainchain:

$ omnicore-cli "omni_sendissuancefixed" \
    "3Ck2kEGLJtZw9ENj2tameMCtS3HB7uRar3" 2 1 0 "Companies" "Bitcoin Mining" \
    "Quantum Miner" "" "" "1000000"

Please visit https://github.com/OmniLayer/omnicore/blob/master/src/omnicore/doc/rpc-api.md to learn how to use omnicore to manage tokens on-chain.

Build obd network, and Lightning pay tokens

test-shell-template.md instructs how to start with command line tool to interact with obd, including:

The testing asset id is --asset_id = 2147483651.

To issue assets on the Bitcoin/Omnilayer mainnet, you should deploy an omnicore full node and execute the cli to mint. For non-developers, we recommend you to visit the official https://www.omniwallet.org/ (https://github.com/OmniLayer/omniwallet) for easier and quicker managing your assets.

Backend and Faucet(Regtest, Testnet)

The omnicore proxy offers the backend public anonymous omni/bitcoin services for obd nodes.
It is specified in the parameter omnicoreproxy.rpchost when an OBD node starts. For example, as in the docker/lnd/start-a.sh, docker/obtest/docker-compose.yml, the regnet.oblnd.top:18332 is where the proxy deployed:

command: >-
      lnd-debug --noseedbackup --trickledelay=5000 --alias=alice
      --externalip=alice --tlsextradomain=alice --tlsextradomain=alice
      --listen=0.0.0.0:9735 --rpclisten=0.0.0.0:10009 --restlisten=0.0.0.0:8080
      --bitcoin.active --bitcoin.regtest --bitcoin.node=omnicoreproxy
      --omnicoreproxy.rpchost=regnet.oblnd.top:18332 
      --omnicoreproxy.zmqpubrawblock=tcp://regnet.oblnd.top:28332
      --omnicoreproxy.zmqpubrawtx=tcp://regnet.oblnd.top:28333

The proxy decouples the lightning node and the full Bitcoin/Omnilayer node, to lower the barriers of OBD deployment, especially for mobile nodes.

The complete white-listed services are: https://github.com/omnilaboratory/omnicore-proxy/blob/master/whitelist_proxy/whitelist_proxy.go.

OBD supports three networks: regtest, testnet and mainnet; Each network supports three backends: bitcoind, omnicore proxy, and neutrino. For a neutrino backend, an omnicore proxy must be specified.

The network is specified by param: --bitcoin.xxxxxx:  

#regtest 
--bitcoin.regtest
#testnet 
--bitcoin.testnet
#mainnet 
--bitcoin.mainnet

The backend is specified by: --bitcoin.node=[bitcoind|omnicoreproxy|neutrino|]:
(Currently, we only support these 3 modes, each has different connection type and certification method.)

  • bitcoind
  --bitcoind.rpchost="$BTC_HOST_ADDRESS_PORT"
  --bitcoind.rpcuser="$RPCUSER"
  --bitcoind.rpcpass="$RPCPASS" 
  --bitcoind.zmqpubrawblock=tcp://"$BTC_HOST_ADDRESS":28332 
  --bitcoind.zmqpubrawtx=tcp://"$BTC_HOST_ADDRESS":28333
  • omnicoreproxy
    --omnicoreproxy.rpchost="$OMNI_HOST_ADDRESS_PORT" 
    --omnicoreproxy.zmqpubrawblock=tcp://"$OMNI_HOST_ADDRESS_PORT":28332 
    --omnicoreproxy.zmqpubrawtx=tcp://"$OMNI_HOST_ADDRESS_PORT":28333
  • neutrino
    --neutrino.connect="$BTC_HOST_ADDRESS"
    --omnicoreproxy.rpchost="$OMNI_HOST_ADDRESS_PORT"
    --neutrino.feeurl=https://nodes.lightning.computer/fees/v1/btc-fee-estimates.json #only mainnet

Depolyed Backends

On each network, not all 3 backends are deployed. We currently maintain the following backends for the community:

  • regtest: ~3 blocks per 2 minutes.
  * omnicoreproxy
    * Asia:  
      `$omni_host_adress_port`=43.138.107.248   
      `$omni: http://43.138.107.248:9090/swaggerTool/?surl=http://43.138.107.248:8090/openapiv2/foo.swagger.json  

  * neutrino
    * Asia:  
      `$omni_host_adress_port`=43.138.107.248   
      `$omni_host_adress_port`=43.138.107.248:18332   
      faucet: http://43.138.107.248:9090/swaggerTool/?surl=http://43.138.107.248:8090/openapiv2/foo.swagger.json    
    * neutrino.db downloading list: https://cache.oblnd.top/neutrino-regtest/  
      When downloading neutrino.db, add date to url to get correct file. For example: https://cache.oblnd.top/neutrino-regtest/neutrino.db?date=2022-12-22 
    * Put the downloaded database file under `${LNDIR}/data/chain/bitcoin/regtest/`  
  • testnet ~1 blocks per 2 to 18 minutes.
  * neutrino
    * Asia:
      `$BTC_HOST_ADDRESS`=192.144.199.67  
      `$OMNI_HOST_ADDRESS_PORT`=192.144.199.67:18332  
      token faucet: http://43.138.107.248:9090/swaggerTool/?surl=http://192.144.199.67:8090/openapiv2/foo.swagger.json  
      token-property id: 2147485160  
      token-owner: mvd6r2KRoaMVr7Y9mDe8pDxe5a5iZLJHN9  
      
    * Other countries and regions:  
      `$BTC_HOST_ADDRESS`=testnet.oblnd.top  
      `$OMNI_HOST_ADDRESS_PORT`=192.144.199.67:18332  
      token faucet: http://43.138.107.248:9090/swaggerTool/?surl=http://192.144.199.67:8090/openapiv2/foo.swagger.json  
      token-property id: 2147485160      
      token-owner: mvd6r2KRoaMVr7Y9mDe8pDxe5a5iZLJHN9  

    * btc-testnet faucet: https://testnet-faucet.com/btc-testnet/  
    * neutrino.db downloading list: https://cache.oblnd.top/neutrino-testnet/    
      When downloading neutrino.db, add date to url to get correct file. For example: https://cache.oblnd.top/neutrino-testnet/neutrino.db?date=2022-12-22
    * We generate database file at 8:00 UTC+8 every day.   
    * Put the downloaded database file under `${LNDIR}/data/chain/bitcoin/testnet/`  
  • mainnet  to be done.

Liquidity nodes

Liquidity nodes offer token/BTC inbound/outbound liquidity to users' nodes, especially for OBWallet mobile nodes.

  • regtest: 0386790984cda19a179486bf45f7a1d7dc58964605b928e3d36cd7806ce3d31cdb@otest:9735
  • testnet: 025767c2a772bb48f04117625c2da759d55d3e287c101602452c5228c975111594@192.144.199.67:9735
  • mainnet: TO BE ADDED

Community

Discord: http://discord.gg/2QYqzSMZuy
Slack: https://join.slack.com/t/omnibolt/shared_invite/zt-ad732myf-1G7lXpHPkFH_yRcilwT4Ig

Video Tutorials

Video tutorials can be found here:

Mainnet warning

OBD is still in an early stage, and sendToMany in omnicore is still under development. We do not recommend running it on mainnet with real money just yet, which may lead to loss of funds unless you want to take a reckless adventure.

Security

If you discover a vulnerability, weakness, or threat that can potentially compromise the security of obd, we ask you to keep it confidential and submit your concern directly to the team.

obd's People

Contributors

blocktink avatar coreyphillips avatar feiben avatar healergyl avatar iamgamelover avatar johng avatar neocarmack avatar obdcoder avatar progressortch avatar wangchun avatar wxf4150 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

obd's Issues

Support neutrino backend for mobile devices.

Mobile devices have limited space and bandwidth, so that syncing full Bitcoin/Omnilayer database is not possible.

obd currently uses omnicore-proxy as a full node backend, but omnicore does not support neutrino mode yet. So that if obd runs on a mobile phone, it will download full blocks data to the device, which is too much.

The solution is: maintain two full nodes as backends, one is netrino bitcoin node, which is used to sync data to mobile obd wallet. And the other is omnicore full node, which is used to manipulate on-chain omni assets.

Tracker-OBD grpc communication module refactored

issues:

  1. service-interface :fixed
    old service-interfce:
// old service/NodeAccountService api
    Login(obdClient *ObdNode, msgData string) (retData interface{}, err error)
	Logout(obdClient *ObdNode) (err error)
	UserLogin(obdClient *ObdNode, msgData string) (retData interface{}, err error)
	UserLogout(obdClient *ObdNode, msgData string) (err error)
	UpdateUserInfo(obdP2pNodeId, obdClientId, userId string) (retData interface{}, err error)
	UpdateUsers(obdClient *ObdNode, msgData string) (err error)
	GetNodeInfoByP2pAddress(context *gin.Context)
	GetUserState(context *gin.Context)
	GetUserP2pNodeId(context *gin.Context)
	GetAllUsers(context *gin.Context)
	GetAllObdNodes(context *gin.Context)

	// service/channel api:
   func (manager *channelManager) updateChannelInfo(obdP2pNodeId string, msgData string) (err error)
   func (manager *channelManager) GetChannelState(context *gin.Context)
   func (manager *channelManager) GetChannels(context *gin.Context)

	// service/htlc api:
	func (manager *htlcManager) getPath(msgData string) (path interface{}, err error)
	func (manager *htlcManager) updateHtlcInfo( msgData string) (err error)
	func (manager *htlcManager) GetHtlcCurrState(context *gin.Context)

The old intrerfaces: have no document; input/output parameters are week types; mixed gin-handler and data-api service

New service definition applies protobuf. Details are in tracker/tkrpc/info-tracker.proto

New interfaces:

service InfoTracker {
rpc HeartBeat (stream UpdateNodeInfoReq) returns (EmptyRes);
//map to old func Logout Login
rpc UpdateNodeInfo (UpdateNodeInfoReq) returns (EmptyRes);
//map to old func userLogout userLogin
rpc UpdateUserInfo (UpdateUserInfoReq) returns (EmptyRes);
//map to old func updateUsers
rpc UpdateUserInfos (UpdateUserInfosReq) returns (EmptyRes);
//map to old func: GetUserState , GetUserP2pNodeId
//Request:SetUserInfoReq{user_id,node_Id}
//old GetUserP2pNodeId use request:SetUserInfoReq{user_id} ;  may not work when user login on multi node
rpc GetUserInfo (UpdateUserInfoReq) returns (UserInfo);
//Map to old function GetAllUsers
rpc GetUserInfos (ListReq) returns (UserInfosRes);
//map old function GetAllObdNodes
rpc GetNodes (ListReq) returns (NodeInfosRes);
// map to old ChannelService.updateChannelInfo

rpc UpdateChannelInfo(ChannelInfo)returns(ChannelInfo);
rpc UpdateChannelInfos(UpdateChannelInfosReq)returns(EmptyRes);
//map to old ChannelService.GetChannelState
rpc GetChannelInfo(SimpleFilter)returns(ChannelInfo);
rpc GetChannels(ListReq)returns (NodeInfosRes);

// map old function updateHtlcInfo
rpc UpdateHtlcInfo(HtlcInfo)returns (HtlcInfo);
//map old function getPath
rpc HtlcGetPath(HtlcGetPathReq)returns (HtlcGetPathRes);
//map old function GetHtlcCurrState
rpc GetHtlcInfo(GetHtlcInfoReq)returns (HtlcInfo);
}

//obd-user-info
message UserInfo {
    int32 id = 1;
    //user_peer_id
    string  user_id = 2;
    //node_peer_id
    string  node_id = 3;
    // 1 online 2 offline
    int32 is_online = 4;
    string access_ip = 5;
    //unix timestamp in seconds ; google.protobuf.Timestamp will more better,but sql database not support protobuf.Timestamp.
    int64  updated_at = 6;
    int64  created_at = 7;
}
...
...

Strong-typed data structure. Added/updated some comments as document in the proto definition file, and the document is displayed synchronously in proto genrated go-files, and will be automatically integrated in swagger document too, or obd js-sdk which invokes the rpc-service.

  1. (fixed) one user connects to multiple obd-nodes ( tracker? ) which call getUserP2pNodeId from remote trackers. This will not work. Fixed: now an obd only connects one tracker when sync/query data.

  2. (fixed) Tracker side updateUsers does not set userinfo.ObdP2pNodeId. userInfo should sync this field.

  3. (fixed) omnibolt-report-final.pdf refer: OBD uses not maintained dependencies ( ile-rotatelogs, asdine/storm, etc.). Fixed: storm is delete, tracker uses gorm database model now.

  4. (fixed) Tracker/dao/pojo.go UserInfo.ObdNodeId fieldname hard to understand.

  5. (fixed) func (manager *htlcManager) getPath(obdClient *ObdNode, msgData string) (path interface{}, err error) at tracker/service/htlc_service.go lines 40.
    the retured paramter path may be a string or a map; When consume returned value, you should use reflect to detect type of the return value at lightclient/connect_tracker.go lines 107, which is a bad desgin

    replyMessage := bean.ReplyMessage{}
    err = json.Unmarshal(message, &replyMessage)
    if err == nil {
    switch replyMessage.Type {
    case enum.MsgType_Tracker_GetHtlcPath_351:
    v := reflect.ValueOf(replyMessage.Result)
    requestMessage := bean.RequestMessage{}
    requestMessage.Type = replyMessage.Type
    requestMessage.Data = ""
    if v.Kind() == reflect.Map {
    dataMap := replyMessage.Result.(map[string]interface{})
    requestMessage.RecipientUserPeerId = dataMap["senderPeerId"].(string)
    requestMessage.Data = dataMap["h"].(string) + "_" + dataMap["path"].(string) + "_" + tool.FloatToString(dataMap["amount"].(float64), 8)
    //requestMessage.Data = dataMap["h"].(string) + "_" + dataMap["path"].(string)
    }
    htlcTrackerDealModule(requestMessage)
    case enum.MsgType_Tracker_Connect_301:

  6. (fixed) userState mananger, userOnlineOfOtherObdMap .fixed
    old code

    func handleUserStateStream(stream network.Stream) {
    rw := bufio.NewReadWriter(bufio.NewReader(stream), bufio.NewWriter(stream))
    str, err := rw.ReadString('~')
    if err != nil {
    return
    }
    if str == "" {
    return
    }
    if str != "" {
    str = strings.TrimSuffix(str, "~")
    log.Println("handleUserStateStream", str)
    params := strings.Split(str, "_")
    if len(params) > 1 {
    if _, ok := userOnlineOfOtherObdMap[params[0]]; ok == true {
    if _, ok = userOnlineOfOtherObdMap[params[0]][params[1]]; ok == true {
    delete(userOnlineOfOtherObdMap[params[0]], params[1])
    } else {
    if userOnlineOfOtherObdMap[params[0]] == nil {
    userOnlineOfOtherObdMap[params[0]] = make(map[string]string)
    }
    userOnlineOfOtherObdMap[params[0]][params[1]] = params[2]
    }
    } else {
    if userOnlineOfOtherObdMap[params[0]] == nil {
    userOnlineOfOtherObdMap[params[0]] = make(map[string]string)
    }
    userOnlineOfOtherObdMap[params[0]][params[1]] = params[2]
    }
    }
    }
    _ = stream.Close()
    }

    it's hard to understand the the meaning of the var-params at line 225 from the context, you must seek the message source to understand the exact meaning of this variable, which is hard to maintain the source code. Concurrent map access is a problom too, which is now discarded.

the new code version :

func (s *ImpInfoServer) UpdateUserInfo(ctx context.Context, req *tkrpc.UpdateUserInfoReq) (*tkrpc.EmptyRes, error) {
	uinfo := new(tkrpc.UserInfo)
	err := Orm.FirstOrCreate(uinfo, tkrpc.UserInfo{UserId: req.UserId, NodeId: req.NodeId}).Assign(tkrpc.UserInfo{IsOnline: req.IsOnline}).Error
	return &tkrpc.EmptyRes{}, err
}
  1. (fixed) Two duplicated function: tracker/service/p2p_service.go sendChannelUnlockInfoToObd SendChannelLockInfoToObd

    func sendChannelLockInfoToObd(channelId, userId, obdP2pNodeId string) bool {
    findID, err := peer.Decode(obdP2pNodeId)
    if err == nil {
    findPeer, err := kademliaDHT.FindPeer(ctx, findID)
    if err == nil {
    stream, err := hostNode.NewStream(ctx, findPeer.ID, bean.ProtocolIdForLockChannel)
    if err == nil {
    rw := bufio.NewReadWriter(bufio.NewReader(stream), bufio.NewWriter(stream))
    request := bean.TrackerLockChannelRequest{UserId: userId, ChannelId: channelId}
    marshal, _ := json.Marshal(request)
    _, _ = rw.WriteString(string(marshal) + "~")
    err = rw.Flush()
    if err == nil {
    str, err := rw.ReadString('~')
    if err != nil {
    return false
    }
    if str == "" {
    return false
    }
    if str != "" {
    str = strings.TrimSuffix(str, "~")
    log.Println("OnSendChannelLockInfoToObd", str)
    if str == "1" {
    return true
    }
    }
    }
    _ = stream.Close()
    }
    }
    }
    return false
    }

    func sendChannelUnlockInfoToObd(channelId, userId, obdP2pNodeId string) bool {
    if len(obdP2pNodeId) == 0 {
    return false
    }
    findID, err := peer.Decode(obdP2pNodeId)
    if err == nil {
    findPeer, err := kademliaDHT.FindPeer(ctx, findID)
    if err == nil {
    stream, err := hostNode.NewStream(ctx, findPeer.ID, bean.ProtocolIdForUnlockChannel)
    if err == nil {
    rw := bufio.NewReadWriter(bufio.NewReader(stream), bufio.NewWriter(stream))
    request := bean.TrackerLockChannelRequest{UserId: userId, ChannelId: channelId}
    marshal, _ := json.Marshal(request)
    _, _ = rw.WriteString(string(marshal) + "~")
    err = rw.Flush()
    if err == nil {
    str, err := rw.ReadString('~')
    if err != nil {
    return false
    }
    if str == "" {
    return false
    }
    if str != "" {
    str = strings.TrimSuffix(str, "~")
    if str == "1" {
    return true
    }
    }
    }
    _ = stream.Close()
    }
    }
    }
    return false
    }

  2. (to to)channelInfo will be updated on both sides of tracker and obd-server; When an obd starts, obd's channels-Info data will cover the data from the tracker, the tracker's update will lost. The sync mechanism between obd and tracker causes data error.

  3. (to to) channelInfo's fields on tracker are different to the obd.
    When obd commits channelInfo to a tracker, now we have the code to convert:

    //infoRequest is the data submit to tracker
	infoRequest.PeerIdA = channelInfo.PeerIdA
	infoRequest.PeerIdB = channelInfo.PeerIdB

	infoRequest.IsAlice = false
	if commitmentTx.Id > 0 {
		if commitmentTx.Owner == channelInfo.PeerIdA {
			infoRequest.IsAlice = true
			infoRequest.AmountA = commitmentTx.AmountToRSMC
			infoRequest.AmountB = commitmentTx.AmountToCounterparty
		} else {
			infoRequest.AmountB = commitmentTx.AmountToRSMC
			infoRequest.AmountA = commitmentTx.AmountToCounterparty
		}
	}

We should modify channelInfo's fields to be the same as the tracker. It will be easy when updates the sync mode between the obd and tracker.

channelInfo.IsAlice should be a function, it's a computed field. For example:

func (c *channelInfo)IsAlice(currentUserPeerID) bool{
	return c.PeerIdA==currentUserPeerID
}
  1. (to to) func (manager *htlcManager) getPath(msgData string) (path interface{}, err error) shoud have a document: the main flow,and how to use the returned value.

    func (manager *htlcManager) getPath(obdClient *ObdNode, msgData string) (path interface{}, err error) {
    manager.mu.Lock()
    defer manager.mu.Unlock()
    log.Println("getPath", msgData)
    if tool.CheckIsString(&msgData) == false {
    return "", errors.New("wrong inputData")
    }
    pathRequest := &bean.HtlcPathRequest{}
    err = json.Unmarshal([]byte(msgData), pathRequest)
    if err != nil {
    return "", err
    }
    if tool.CheckIsString(&pathRequest.RealPayerPeerId) == false {
    return "", errors.New("wrong realPayerPeerId")
    }
    if tool.CheckIsString(&pathRequest.PayeePeerId) == false {
    return "", errors.New("wrong SendeePeerId")
    }
    if pathRequest.Amount < tool.GetOmniDustBtc() {
    return "", errors.New("wrong amount")
    }
    manager.createChannelNetwork(pathRequest.RealPayerPeerId, pathRequest.PayeePeerId, pathRequest.PropertyId, pathRequest.Amount, nil, true)
    resultIndex := manager.getPathIndex()
    retNode := make(map[string]interface{})
    retNode["senderPeerId"] = pathRequest.RealPayerPeerId
    retNode["h"] = pathRequest.H
    retNode["amount"] = pathRequest.Amount
    retNode["path"] = ""
    if resultIndex != -1 {
    splitArr := strings.Split(manager.openList[resultIndex].ChannelIds, ",")
    path := ""
    for i := len(splitArr) - 1; i > -1; i-- {
    path += splitArr[i] + ","
    }
    path = strings.TrimSuffix(path, ",")
    retNode["path"] = path
    }
    log.Println("return path info", retNode)
    return retNode, nil
    }

  2. (fixed) obd/service/user.go line 60 calls noticeTrackerUserLogin().sendMsgToTracker() .

    obd/service/user.go

    Lines 20 to 76 in 5829315

    func (service *UserManager) UserLogin(user *bean.User) error {
    if user == nil {
    return errors.New(enum.Tips_user_nilUser)
    }
    if user.IsAdmin {
    if tool.CheckIsString(&user.Mnemonic) == false || bip39.IsMnemonicValid(user.Mnemonic) == false {
    return errors.New(enum.Tips_common_wrong + "mnemonic")
    }
    changeExtKey, err := HDWalletService.CreateChangeExtKey(user.Mnemonic)
    if err != nil {
    return err
    }
    user.PeerId = tool.GetUserPeerId(user.Mnemonic)
    user.ChangeExtKey = changeExtKey
    }
    userDB, err := dao.DBService.GetUserDB(user.PeerId)
    if err != nil {
    return err
    }
    var node dao.User
    err = userDB.Select(q.Eq("PeerId", user.PeerId)).First(&node)
    if node.Id == 0 {
    node = dao.User{}
    node.PeerId = user.PeerId
    node.P2PLocalPeerId = user.P2PLocalPeerId
    node.P2PLocalAddress = user.P2PLocalAddress
    node.CurrState = bean.UserState_OnLine
    node.CreateAt = time.Now()
    node.LatestLoginTime = node.CreateAt
    node.CurrAddrIndex = 0
    err = userDB.Save(&node)
    } else {
    node.P2PLocalPeerId = user.P2PLocalPeerId
    node.P2PLocalAddress = user.P2PLocalAddress
    node.CurrState = bean.UserState_OnLine
    node.LatestLoginTime = time.Now()
    err = userDB.Update(&node)
    }
    noticeTrackerUserLogin(node)
    if err != nil {
    return err
    }
    loginLog := &dao.UserLoginLog{}
    loginLog.PeerId = user.PeerId
    loginLog.LoginAt = time.Now()
    _ = userDB.Save(loginLog)
    user.State = node.CurrState
    user.CurrAddrIndex = node.CurrAddrIndex
    user.Db = userDB
    return nil
    }

  • issue 1: if some err occurs ,we MUST not invoke it (L60). If it noticeTrackerUserLogin just record a audit-log, we can ignore the err and invoke it.
  • issue 2: obd/service is just a database/data operation module, it MUST not call sendMsgToTracker to send async message; obd/lightclient module is async message center, now 95% async messages are recieved and sent here, all the client-websocket-conn p2p-conn are initialized here; obd/service/htlc_tx_forward.go PayerRequestFindPath invoke "sendMsgToTracker(enum.MsgType_Tracker_GetHtlcPath_351, pathRequest)" have the same probolem.
  • now i have fixed the two issue, and obd/service.sendMsgToTracker have disbled, and below messages submit by grpc-conn now,no need to sync userInfo/channelInfo/htlcInfo to tracker with complex async message Goroutines. the old msgList:
    • MsgType_Tracker_UpdateChannelInfo_350
    • MsgType_Tracker_UserLogin_304
    • MsgType_Tracker_UserLogout_305
    • MsgType_Tracker_UpdateHtlcTxState_352
  1. (to to) may be we should design the obd-tracker info sync-mode and security first. Maybe new mechanism won't need the above work.

The data sync mechanism between obd and tracker need update.

This issue is the bug 12 and 13 in issue 53

  1. (fixed) obd/service/user.go line 60 calls noticeTrackerUserLogin().sendMsgToTracker() .

    obd/service/user.go

    Lines 20 to 76 in 5829315

    func (service *UserManager) UserLogin(user *bean.User) error {
    if user == nil {
    return errors.New(enum.Tips_user_nilUser)
    }
    if user.IsAdmin {
    if tool.CheckIsString(&user.Mnemonic) == false || bip39.IsMnemonicValid(user.Mnemonic) == false {
    return errors.New(enum.Tips_common_wrong + "mnemonic")
    }
    changeExtKey, err := HDWalletService.CreateChangeExtKey(user.Mnemonic)
    if err != nil {
    return err
    }
    user.PeerId = tool.GetUserPeerId(user.Mnemonic)
    user.ChangeExtKey = changeExtKey
    }
    userDB, err := dao.DBService.GetUserDB(user.PeerId)
    if err != nil {
    return err
    }
    var node dao.User
    err = userDB.Select(q.Eq("PeerId", user.PeerId)).First(&node)
    if node.Id == 0 {
    node = dao.User{}
    node.PeerId = user.PeerId
    node.P2PLocalPeerId = user.P2PLocalPeerId
    node.P2PLocalAddress = user.P2PLocalAddress
    node.CurrState = bean.UserState_OnLine
    node.CreateAt = time.Now()
    node.LatestLoginTime = node.CreateAt
    node.CurrAddrIndex = 0
    err = userDB.Save(&node)
    } else {
    node.P2PLocalPeerId = user.P2PLocalPeerId
    node.P2PLocalAddress = user.P2PLocalAddress
    node.CurrState = bean.UserState_OnLine
    node.LatestLoginTime = time.Now()
    err = userDB.Update(&node)
    }
    noticeTrackerUserLogin(node)
    if err != nil {
    return err
    }
    loginLog := &dao.UserLoginLog{}
    loginLog.PeerId = user.PeerId
    loginLog.LoginAt = time.Now()
    _ = userDB.Save(loginLog)
    user.State = node.CurrState
    user.CurrAddrIndex = node.CurrAddrIndex
    user.Db = userDB
    return nil
    }
  • issue 1: if some err occurs ,we MUST not invoke it (L60). If it noticeTrackerUserLogin just record a audit-log, we can ignore the err and invoke it.
  • issue 2: obd/service is just a database/data operation module, it MUST not call sendMsgToTracker to send async message; obd/lightclient module is async message center, now 95% async messages are recieved and sent here, all the client-websocket-conn p2p-conn are initialized here; obd/service/htlc_tx_forward.go PayerRequestFindPath invoke "sendMsgToTracker(enum.MsgType_Tracker_GetHtlcPath_351, pathRequest)" have the same probolem.
  • now i have fixed the two issue, and obd/service.sendMsgToTracker have disbled, and below messages submit by grpc-conn now,no need to sync userInfo/channelInfo/htlcInfo to tracker with complex async message Goroutines. the old msgList:
    • MsgType_Tracker_UpdateChannelInfo_350
    • MsgType_Tracker_UserLogin_304
    • MsgType_Tracker_UserLogout_305
    • MsgType_Tracker_UpdateHtlcTxState_352
  1. (to to) may be we should design the obd-tracker info sync-mode and security first. Maybe new mechanism won't need the above work.

closing channel reports error

OBAndroid calls closeChannel but gets error: rpc error: code = Unknown desc = unsupported script type

close_pending {
...

eea0c6b4525f649f5ffb8020712ea06

Random crashes

2020/07/16 21:57:16 schedule.go:31: timer 10m 2020-07-16 21:57:16.001173196 +0100 BST m=+601.028695263
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8c5592]

goroutine 54 [running]:
github.com/omnilaboratory/obd/service.sendRdTx()
        /home/johng/repos/obd/service/schedule.go:99 +0x152
github.com/omnilaboratory/obd/service.(*scheduleManager).StartSchedule.func1()
        /home/johng/repos/obd/service/schedule.go:32 +0x126
created by github.com/omnilaboratory/obd/service.(*scheduleManager).StartSchedule
        /home/johng/repos/obd/service/schedule.go:21 +0x35

There doesn't seem to be consistent trigger.

Integrate Circuit Breaker

Background

Circuit breaker allows nodes to protect themselves from being flooded with htlcs. With circuitbreaker a maximum to the number of in-flight htlcS can be set on a per-peer basis. Known and trusted peers for example can be assigned a higher maximum, while a new channel from a previously unseen node may be limited to only a few pending htlcs.

Project intro

Circuit Breaker

TO DO

Assess how to make Circuit Breaker support OmniBOLT channel.

channelInfo's fields on tracker are different to the obd.

This issue is the tenth bug in issue 53: #53

  1. channelInfo's fields are different from the obd.
    when obd commits channelInfo to a tracker, now we have the code to convert:
    //infoRequest is the data submit to tracker
	infoRequest.PeerIdA = channelInfo.PeerIdA
	infoRequest.PeerIdB = channelInfo.PeerIdB

	infoRequest.IsAlice = false
	if commitmentTx.Id > 0 {
		if commitmentTx.Owner == channelInfo.PeerIdA {
			infoRequest.IsAlice = true
			infoRequest.AmountA = commitmentTx.AmountToRSMC
			infoRequest.AmountB = commitmentTx.AmountToCounterparty
		} else {
			infoRequest.AmountB = commitmentTx.AmountToRSMC
			infoRequest.AmountA = commitmentTx.AmountToCounterparty
		}
	}

We should modify channelInfo's fields to be the same as the tracker. It will be easy when updates the sync mode between the obd and tracker.

channelInfo.IsAlice should be a function, it's a computed field. For example:

func (c *channelInfo)IsAlice(currentUserPeerID) bool{
	return c.PeerIdA==currentUserPeerID
}

Using TLV to enable lightning network to be aware of OmniBOLT TXs

TLV -- Type Length Value

https://github.com/lightningnetwork/lightning-rfc/blob/master/01-messaging.md#type-length-value-format

A TLV (Type-Length-Value) format is used to allow for the backwards-compatible addition of new fields to existing LN message types.

Motivation

OmniBOLT transaction can be embeded into LN messages, so that current ln network can benifit from the liquidity of Omnilayer assets.

Rationale

  • OmniLayer issues assets on Bitcoin network, it can be used as a settlement system, which has proven mechanism to provent double spent or any other malicious activities onchain.

  • OmniBOLT transactions are constructed on top of raw omni transactions, and can be used as a quick circulation network. It has it own communication spec, but need to be compatible with existing LN network.

  • TLV is a right place to put OmniBOLT tx info in. A lnd node will be omni asset aware, if it work with obd, in which case obd works like a plugin to lnd.

A simple workflow is:

  1. lnd receives a message, which contain a TLV field, then lnd redirect it to its obd plugin.
  2. obd parses the value of TLV field,
  3. if it is an OmniBOLT message, obd will execute it.

[Lightning-dev] CVE-2020-26895: LND Low-S Tx-Relay Standardness

Background

CVE-2020-26895 was fully disclosed on Oct 20, 2020.

LND v0.10.0-beta, released on April 29,2020, fixed this vulnerability.

Description of vulnerability

High S signature causes signature malleability related to ECDSA signature encoding . To fix signature malleability, Low S signature is propsed in BIP-0146, and high s signature are no longer accepted by btccore.

Affected Component

OBD core, client sdk

Platform

All

Proof-of-concept

CVE-2020-26895 fully disclosed this vunerability.

Vulnerability reproduction output

N/A

Fix

To OBD node, If a signature from client passing to ECDSA verification does not pass the Low S value check and is not an empty byte array, the entire script evaluates to false immediately. OBD shall reject this signature.

Reference

BIP-0146
CVE-2020-26895
Low S signature to fix

Malleability Attack

Bitcoin raw transactions are malleable, if without SegWit. That is to say a transaction can be modified, without invalidate it, but without access to the relevant private keys.

BIP-0062

So that OBD HTLC transactions built upon omni raw transactions are maleable, one participant ( Alice ) can broadcast a modified elder commitment transaction with totally different TxID without trigger the following ED, BR, HERD transactions, but pays himself the same money.

His counterparty (Bob) has no knowledge of this actitvity, and even if Bob knows, his punishment BR transaction can never be broadcast because Alice broadcaseted an transaction with totally different TxID.

When Omnicore supports SegWit, obd shall update all the currently being constructed transacitons.

Support regtest

Currently the only supported options are testnet/mainnet. Regtest is very useful for testing locally and should be supported.

One option this can be supported is in the chain configuration for btcd. chaincfg.Params. There is an explicit simnet variable that obd doesn't use

if config.ChainNode_Type == "test" {

static channel backup is insufficient

This is from an auditor:

As to https://omnilaboratory.github.io/obd/#/safety-guidelines
Since you are using LND as base instead of C-Lightning v(^^) users need to ensure that their storage is reliable.
C-Lightning includes a hook that allows storage to be replicated by any mechanism, see >>https://lightning.readthedocs.io/BACKUP.html for the options users have in practice.

To my knowledge, LND has not corresponding hook.
Static Channel Backups may be insufficient; if Omni requires the LND-derived daemon to keep track of additional metadata to >>ensure correctness, then the data in the Static Channel Backups may be insufficient (depends on how you structure that data).
And you may need to modify the SCB mechanism to include the extra data required by Omni.

My suggestion is to urge your users (or at least the ones with significant amounts of stored value), in the safety guidelines, to >>use some kind of RAID system, such as ZFS (mirror or any RAID-Z), mdadm (RAID1 mode), or BTRFS (raid1, raid1c3, or >>raid1c4) and to monitor the storage continuously so that any storage failure is detected.

This is in addition to any SCB they might regularly make (and you need to augment the SCBs with Omni-specific data as well).
See the above-linked BACKUP.html file for C-Lightning for how I suggest you present the safety guidelines.

Compile obd for running on mobile and call by java on android

Currently lack of document on compiling mobile node and steps to invock by java side on android. We don't need docker on mobile devices.

The doc must include:

  1. System requiroment. Is the "root" necessary. Maybe a tested device list.
  2. Compilation enviroment preparation: for example: android sdk, gomobile version, makefile, etc.
  3. Commands to compile: correct output, solutions for possible errors.
  4. Package for java developers.
  5. Step-by-step java example code.

API naming

In SignUp, the mnemonic word must not be used as an user ID, it's dangours. The only way that mnemonic word can be used is to restore an account.

Use another way to login, dont use mnemonic word.

ChannelAcceptor --> AcceptChannel

SendBitcoin -1009 --> FundingBTC -1009

NotifySendingBitcoin -3400 --> BTCFundingCreated -3400

ResponseSendingBitcoin -3500 --> BTCFundingSigned -3500

SendOmniAsset 2001 --> FundingAsset -2001

NotifySendingOmniAsset -34 -- AssetFundingCreated -34

ResponseSendingOmniAsset -35 --> AssetFundingSigned -35

PaymentInChannle -351 --> Commitment Transaction Created -351

ResponsePaymentInChannel -352 --> Revoke and Acknowledge Commitment Transaction -352

HTLCPayment -40 --> Add HTLC -40
HTLC can be added(created) or forward to the next node in routing.
"recipient_peer_id": "<user_id>" what is this peer id? Here use mnemonic word as UserID?

ResponseHTLCPayment -41 --> HTLC Signed -41
who signs this request? Alice's OBD or the reciever Bob?

SendH2Middleman -42 --> Deprecated
ResponseByMiddleman -44 --> Deprecated

SendR -46 --> Forward R -46
ResponseSendR -47 --> Verify R and Create HTLC Tx -47

ResponseCloseHTLC -49 --> CloseHTLC Signed -49

ResponseCloseHtlcChannel --51 --> CloseHtlcChannel Signed --51

Alternative construction of HTLC Execution Delivery Transaction

image

The transaction in RED box is the HTLC execution delivery transaction, which is locked by custom locking script.

To unlock this transaction, Bob shall provide three arguments: the preimage R, Alice2 and Bob2's signature. And in the process of HTLC, this preimage R is used to exchange money(token) in a channel.

But current Omnicore does not support custom redeem script in locking and unlocking a transaction. So can we just transform this non-standard transaction to a standard 3-3 multisig transaction, replacing R by another private key of an address(public key), which is already well supported by Omnicore.

This transformation is a little bit different from the original process in the white paper. There is no potential risk if we use 3-3 multiSig transaction, because essentially the two solutions are the same: given a number, image or private key, there is no polynomial algorithm to find its preimage or public key.

p2p communication data should be strongly typed

The data type of p2p communication is not strongly typed. At present, using one ~ char to split the data stream, using gjson to extract json data is not safe enough; it should be possible to use protobuf to do complete and strongly typed data extraction. The data stream separator should be longer to reduce potential segmentation errors, maybe three '~' char will better.

Naively usingN HTLCs introduces an inadvertent American Call Option

This is some early feedback from a potential auditor for OmniBOLT:

I saw in the https://omnilaboratory.github.io/obd/# that you advertise as feature the ability to swap different token types.

Note that if this is implemented naively using HTLCs, you introduce an inadvertent American Call Option.

See here https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-December/001752.html

There may be mitigations against this, though I have not kept track on the state-of-the-art on this problem; I hope you have?

Looking at https://github.com/omnilaboratory/OmniBOLT-spec/blob/master/OmniBOLT-05-Atomic-Swap-among-Channels.md#hashed-timelock-swap-contract

It seems you do use the naive form, the one that is susceptible to the American Call Option problem I pointed out about.

This is fine for an initial implementation when you have no serious activity yet, but note that you should at least think of how to upgrade to a later mechanism that is safe against the American Call Option problem.

Barrier Escrows (which become feasible once we implement PTLCs on Lightning Network) are a good solution, I believe: https://suredbits.com/payment-points-and-barrier-escrows/

So all you need now is to ensure some kind of upgrade mechanism that lets you switch to using Barrier Escrows later and deprecate the current mechanism.

A flaw in creating commitment transaction if without Sendmany

RSMC-C1a-RD1a

During creating C1a, there are two unbroadcast outputs constructed simutanously: 0.Alice2 & Bob 60, and 1. Bob 40. The output 0 sends 60 USDT to the multisig address Alice2 & Bob, while the output 1 sends 40 USDT to Bob's address.

Currently, omnicore does not support sendmany, so we have to construct two temporary transactions based on which the two outputs are created. Here comes the problem: If Alice broadcasts output 0 only, and just ignore the output 1, then Bob will never get his 40 USDT.

This flaw will not occur if we construct the two outputs by sendmany based on one parent transaction, so Alice is not able to just broadcast one of them.

Add omnicore proxy as obd backend

omnicore proxy offers public anonymous omnicore services to anonymous users. It syncs full Bitcoin/Omnilayer database. OBD starts with a param to specify the backend. Direct connection to a Bitcoin/Omnilayer full node is disabled for security consideration.

[Tracker] Incorrect keys in retrieving Node information from storm db

        info := &dao.ObdNodeInfo{}
	err := db.Select(q.Eq("p2p_address", p2pAddress), q.Eq("is_online", true)).First(info)
	if err != nil {
		context.JSON(http.StatusInternalServerError, gin.H{
			"msg": "error p2pAddress",
		})
		return
	}

The following segment always fails with an unknown field error. Replacing the columns with their actual names fixed e.g P2pAddress fixed the issue. An alternative approach would be updating the schema in storm.

Complete the document of the algorithm for finding payment path

this issue is the 11 bug in issue 53: #53

  1. (to to) func (manager *htlcManager) getPath(msgData string) (path interface{}, err error) shoud have a document: the main flow,and how to use the returned value.

func (manager *htlcManager) getPath(obdClient *ObdNode, msgData string) (path interface{}, err error) {
manager.mu.Lock()
defer manager.mu.Unlock()
log.Println("getPath", msgData)
if tool.CheckIsString(&msgData) == false {
return "", errors.New("wrong inputData")
}
pathRequest := &bean.HtlcPathRequest{}
err = json.Unmarshal([]byte(msgData), pathRequest)
if err != nil {
return "", err
}
if tool.CheckIsString(&pathRequest.RealPayerPeerId) == false {
return "", errors.New("wrong realPayerPeerId")
}
if tool.CheckIsString(&pathRequest.PayeePeerId) == false {
return "", errors.New("wrong SendeePeerId")
}
if pathRequest.Amount < tool.GetOmniDustBtc() {
return "", errors.New("wrong amount")
}
manager.createChannelNetwork(pathRequest.RealPayerPeerId, pathRequest.PayeePeerId, pathRequest.PropertyId, pathRequest.Amount, nil, true)
resultIndex := manager.getPathIndex()
retNode := make(map[string]interface{})
retNode["senderPeerId"] = pathRequest.RealPayerPeerId
retNode["h"] = pathRequest.H
retNode["amount"] = pathRequest.Amount
retNode["path"] = ""
if resultIndex != -1 {
splitArr := strings.Split(manager.openList[resultIndex].ChannelIds, ",")
path := ""
for i := len(splitArr) - 1; i > -1; i-- {
path += splitArr[i] + ","
}
path = strings.TrimSuffix(path, ",")
retNode["path"] = path
}
log.Println("return path info", retNode)
return retNode, nil
}

Data sync mechanism between tracker and obd

This is the bug 9 in issue 53: #53

  1. (to to)channelInfo will be updated on both sides of tracker and obd-server; When an obd starts, obd's channels-Info data will cover the data from the tracker, the tracker's update will lost. The sync mechanism between obd and tracker causes data error.

Run tests on PR

We should setup something like Travis CI to run the tests on PR

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.