alice private key,HD chain issue? about omnibolt-spec

[sidhujag]

Its not a refactor, TLV are an extension for general purpose functionality enhancement. They are already used internally for tx data, but an extended field was added for external data

Any reference link available for this enhancement? except the official RFC:

https://github.com/lightningnetwork/lightning-rfc/blob/master/01-messaging.md#type-length-value-format

lightning/bolts#754 which is merged recently and referenced from lightning/bolts#72

Great to chat with you aswell, I am working in the similar field so that is why I have questions, hopefully with mindshare we can solve these problems and bring this technology to the world in meaningful ways.

[neocarmack]

I guess the context we are talking about is when Alice pay Bob some USDT in the channel, she has to send bob the private key of the 2-2 multisig address <Alice2, Bob>, where gets the 60 USDT in our diagram in the chapter 2, commitment_tx.

We leverage HD to generate public/private key pair for any address created in any process of payment. Alice has a wallet, on her desktop or her phone, where she stores her mnemonic words, which is the root that generats all the derivated key pairs. The mnemonic words are the root of the HD tree, and can not be leak to anyone else. Leak of a single pair of public/private key will not jeopardize other information generated by the same set of mnemonic words.

So as long as the mnemonic words are safe and unknown to others, the tree of public/private key pairs are safe. No one can use a private key to reversely decode the mneminic words.

[sidhujag]

The problem isn’t obtaining the master key or anything like that. It’s related to non hardened keys. If you follow spec everything up to account level is hardened and safe. But not below that.

A parent xpub in an HD chain along with any non-hardened child private key can effectively derive all other keys in child level and under.

“Note however that the following properties does not exist:
Given a parent extended public key (Kpar,cpar) and a child public key (Ki), it is hard to find i.
Given a parent extended public key (Kpar,cpar) and a non-hardened child private key (ki), it is hard to find kpar.”

https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki#security

Read more from trezor wiki as well:

https://wiki.trezor.io/Hardened_and_non-hardened_derivation

Is this a new part of the process? If so why? If it is can you not just create htlc instead using preimage r instead of private key r?

In this strategy any money inside of Alice's account may be taken by Bob.

[neocarmack]

Correct. There is no place for non-hardened keys to be applied, especially the temporary private key <Alice 2>, which is hardened and will be handed over to Bob during the next commitment transaction.

An alternative way to keep safe is to apply a set of complete new mnemonic words to generate the temporary key pairs, and only use for once. But this will bring extra complexity in implementation.

In creating HTLC, the private key we use to replace preimage R, whose hashed value is used to lock a HTLC, is hardened ofcourse, otherwise there is a huge risk of leaking private keys of the account.

[neocarmack]

According to BIP44, which uses hardening/private derivation on most levels:
m / purpose' / coin_type' / account' / change / address_index

but not on the change and address index level.

So a better practice is to apply temporary generated seed for every temporary multi-sig address. They dont live in the same HD keys chain. Then there is no chance to compromise any parent private keys.

[sidhujag]

Correct. There is no place for non-hardened keys to be applied, especially the temporary private key <Alice 2>, which is hardened and will be handed over to Bob during the next commitment transaction.

An alternative way to keep safe is to apply a set of complete new mnemonic words to generate the temporary key pairs, and only use for once. But this will bring extra complexity in implementation.

In creating HTLC, the private key we use to replace preimage R, whose hashed value is used to lock a HTLC, is hardened ofcourse, otherwise there is a huge risk of leaking private keys of the account.

Sure. but why go through the hoops when you can simply just use a secret, why use a private key? If you are using a new HD chain now you have to manage that new HD chain and also ensure its a new chain for every transaction. Why not just use a normal secret? If you are using a new HD chain and there is no connection to the one holding funds then just use a random secret number or alpha numeric. There is no convenience benefits in having a separate hd chain over just what LN Bolt suggests as far as I can see.

[neocarmack]

Becasue this two options offer the same security level, but using a public/private key pair instead of R and Hash(R) makes the source code looks consistent and concise. We may just simply apply elliptic curve to generate the temp pub/priv key pair instead of applying a complex HD chain, when creating the multi-sig address.

And custom redeem script, which uses Hash(R) to lock an omnilayer transaction, currently not officially be supported by omnicore v0.18, although there is a way to contruct such a redeem script.

OmniLayer is in a big update that is going to support SegWit. When the update releases, we may be able to invoke official interface to constuct transaction by custom redeem script. And at that time, there is a concise way to simply use a secret key R.

Both of the two options are same to us.

[sidhujag]

Becasue this two options offer the same security level, but using a public/private key pair instead of R and Hash(R) makes the source code looks consistent and concise. We may just simply apply elliptic curve to generate the temp pub/priv key pair instead of applying a complex HD chain, when creating the multi-sig address.

And custom redeem script, which uses Hash(R) to lock an omnilayer transaction, currently not officially be supported by omnicore v0.18, although there is a way to contruct such a redeem script.

OmniLayer is in a big update that is going to support SegWit. When the update releases, we may be able to invoke official interface to constuct transaction by custom redeem script. And at that time, there is a concise way to simply use a secret key R.

Both of the two options are same to us.

oh ok so you were constrained with not being able to use custom redeem scripts in omnicore (although its based on bitcoin? should be no problem) but ok, now it makes sense.

So once you support segwit you can do custom redeem script and not just a standard 2 of 2 multisig.

[sidhujag]

Could you use SIGHASH_NOINPUT and just exchange signatures of the tx prior to broadcast?

[neocarmack]

Could you use SIGHASH_NOINPUT and just exchange signatures of the tx prior to broadcast?

Not yet. After we figure out the real benefit of SIGHASH_NOINPUT, we shall propose to Omni Layer Enhancement (proposal) for tech enhancement.

Currently we simply exchange signatures between peers when contruct transactions, and it works as of now.

There was a proposal to improve the punishment mechanism, called Eltoo, that requires the support of SIGHASH_NOINPUT. But at this stage, i dont see the benifits of Eltoo. Let me think about it after OmniBOLT testnet is online.

[sidhujag]

SIGHASH_NOINPUT is part of the official LN paper, and is required in order to work around issues where commitment transactions required non-malleable way to build off of a SIGHASH_NOINPUT (inputs are not part of sighash) transactions without affecting the transaction hash, so once a final commitment is created it can be published along with the parent once parent confirms and is buried deep enough.

Without SIGHASH_NOINPUT, it is not possible to generate a spend from a transaction without exchanging signatures. SIGHASH_NOINPUT gets around malleability concerns by permitting
the child to spend without signing the input.

This is the order of operations, note the funding tx is offline until the commitments are done. Is your spec publishing the funding first?

1. Create the parent (Funding Transaction)
2. Create the children (Commitment Transactions and all spends from
   the commitment transactions)
3. Sign the children
4. Exchange the signatures for the children
5. Sign the parent
6. Exchange the signatures for the parent
7. Broadcast the parent on the blockchain

With this procedure, one has to use SIGHASH_NOINPUT and this is the one described by the official LN paper.

[neocarmack]

Your are right. SIGHASH_NOINPUT is part of the whitepaper. But we dont follow all the details of the whitepaper, we just follow its principal. Especially in the context of Omnilayer, the funding process is a little more complex:

Creating an omni transaction requires a little BTC as miner fee, otherwise it will fail even if it is unboardcast. So first of all, when we create a channel, we have to fund the channel a little BTC, say 0.5 usd value of BTC, as the miner fee. This is the cost of building a channel, and this is not revockable.

https://api.omnilab.online/#fundingbtc

After that, we are able to create, say 10000 USDT, funding transaction but not broadcast before commitment tx C1a, revockable delivery tx RD1a, and breach remedy tx BR1a(maybe later) are correctly created in the channel, and signed by both participants. At this time, the channel has some real BTC(0.5 usd) in it, and is waiting for the 10000 USDT funding tx to be broadcasted.

These internal transactions are based on the BTC funding transaction as the parent. So this the reason why we dont use SIGHASH_NOINPUT.

BTW, is SIGHASH_NOINPUT officially enbled by btc-core?

[sidhujag]

Your are right. SIGHASH_NOINPUT is part of the whitepaper. But we dont follow all the details of the whitepaper, we just follow its principal. Especially in the context of Omnilayer, the funding process is a little more complex:

Creating an omni transaction requires a little BTC as miner fee, otherwise it will fail even if it is unboardcast. So first of all, when we create a channel, we have to fund the channel a little BTC, say 0.5 usd value of BTC, as the miner fee. This is the cost of building a channel, and this is not revockable.

https://api.omnilab.online/#fundingbtc

After that, we are able to create, say 10000 USDT, funding transaction but not broadcast before commitment tx C1a, revockable delivery tx RD1a, and breach remedy tx BR1a(maybe later) are correctly created in the channel, and signed by both participants. At this time, the channel has some real BTC(0.5 usd) in it, and is waiting for the 10000 USDT funding tx to be broadcasted.

These internal transactions are based on the BTC funding transaction as the parent. So this the reason why we dont use SIGHASH_NOINPUT.

BTW, is SIGHASH_NOINPUT officially enbled by btc-core?

So because segwit is already enabled, if you use p2wsh you dont need sighash_noinput as segwit txs don't have signature component in the txid. However eltoo which everyone thinks is better because its simpler theoretically required sighash_noinput because it builds on commitment txs on each other requiring the use of explicitly not signing inputs.

If you use segwit or moving to it, you are fine but I don't understand what you said about the btc gas tx being done before the omni asset tx.

Why not use the same transaction to send the btc and the asset (which is in the opreturn output). I don't see why it has to be separate. When you create commitment txs do they have to forward some btc as gas too so it can be broadcasted on chain in the event of dispute or settlement? I guess everything would follow the same protocol but the value in the channel would be the asset, but gas is used and forwarded so transactions can settle, but I don't think it introduces any new problems with attaching an asset output representing the value in the channel except the "gas" btc amount

eltoo is starting to wear on me ( its much simpler to design wallets around ) but SIGHASH_NOINPUT is not yet officially enabled. However you will need to be using segwit to get around having to use it.

[sidhujag]

@neocarmack do you use private keys in the commitment txs because omni is account based, or has omni changed to utxo input/output based now? From what I understand the commitment txs would take the input from a previous commitment and create a new CLTV output, and if the asset model is not utxo based that poses problems with the BOLT spec, because you can wait for timeout and just use another output, or some historical output in the same address for spend-ability of that asset. Does using private key alleviate these concerns somehow?

[neocarmack]

Why not use the same transaction to send the btc and the asset (which is in the opreturn output)....

Omnicore currently can not send both BTC and token in one transaction, although when sending tokens, some BTC "dust" will be sent to the destinine address.

[neocarmack]

do you use private keys in the commitment txs because omni is account based, or has omni changed to utxo input/output based now? From what I understand the commitment txs would take the input from a previous commitment and create a new CLTV output, and if the asset model is not utxo based that poses problems with the BOLT spec, because you can wait for timeout and just use another output, or some historical output in the same address for spend-ability of that asset.

I do not exactly understand your point. We do use private keys and use the counterparty's signature as well, in contructing commitment transacitons.

And in contructing the following transactions, like breach remedy tx, htlc timeout tx, htlc execution delivery tx, etc, all these need private keys/signature in contructing. In our implementation of OBD, the commitment tx itself takes the funding transaction as input, not the previous commitment transaciton. So all the commitment txs are not chained, but the sons of one funding transaction.

[sidhujag]

@neocarmack see this question from rusty russell lightning/bolts#72 (comment)

I think omni is account based and not UTXO correct? So I think the issue would apply unless you use UTXO model.

Also note that the official spec now has support for generic TLV's so you can put your asset data into there and perhaps just follow original spec.

[neocarmack]

Also note that the official spec now has support for generic TLV's so you can put your asset data into there and perhaps just follow original spec.

before we develop this omni-asset-aware version of lighnting network, we consulted the lnd developer team if lnd can support smart asset or have a plan on this. The answer is no, because of the huge cost/effort that will be paid. Refactoring the code base to support smart asset is terrible.

After OmniBOLT network is online, we will use atomic swap to communicat lnd network.

[neocarmack]

what prevents someone from putting an older commitment on the chain? You use account model or utxo model for your asset protocol?

The breach remedy(RD) transaction prevents broadcasting older commitmen transaction.

[neocarmack]

BTW, we know RGB, which still uses liquidity of BTC in lnd network. Omni team had cummunication with RGB team last year. Wish they have good luck.

OmniBOLT testnet will be online soon. At that time, we will connect to LND network. We both has advantages: OmniBOLT has liquidity of assets, including stable coins, while LND has a good liquidity of BTC.

[sidhujag]

what prevents someone from putting an older commitment on the chain? You use account model or utxo model for your asset protocol?

The breach remedy(RD) transaction prevents broadcasting older commitmen transaction.

Ok I am just not sure how that locking to an address would work when the commitment CLTV transactions are clearly locking UTXOs, I just feel there is a security vulnerability there but I am not sure what that is at the moment, Rusty Russel was sure that it would be an issue if assets are not locked to UTXOs, Colu seems to be doing so and there was a comment on there if Omni was changing to utxo and it looks like RGB is also utxo based I think.

[sidhujag]

Also note that the official spec now has support for generic TLV's so you can put your asset data into there and perhaps just follow original spec.

before we develop this omni-asset-aware version of lighnting network, we consulted the lnd developer team if lnd can support smart asset or have a plan on this. The answer is no, because of the huge cost/effort that will be paid. Refactoring the code base to support smart asset is terrible.

After OmniBOLT network is online, we will use atomic swap to communicat lnd network.

Its not a refactor, TLV are an extension for general purpose functionality enhancement. They are already used internally for tx data, but an extended field was added for external data

[neocarmack]

Its not a refactor, TLV are an extension for general purpose functionality enhancement. They are already used internally for tx data, but an extended field was added for external data

Any reference link available for this enhancement? except the official RFC:

https://github.com/lightningnetwork/lightning-rfc/blob/master/01-messaging.md#type-length-value-format

[neocarmack]

Its not a refactor, TLV are an extension for general purpose functionality enhancement. They are already used internally for tx data, but an extended field was added for external data

I'm not sure if TLV field can be used to put colored coin information in. What happens when broadcasting transaction to withdraw coin defined in TLV back to BTC network(address)? Do we really need to re-invent another omnilayer to support it using OP_RETURN again? I dont see necessity of doing so.

Lighning network itself has no temper-resistent mechanism. What if someone modifies the data of TLV, and there is no protective protocol like omnilayer to prevent broadcasting dirty data?

I remember before we decide to develop OmniBOLT, we studied and consulted lnd team, about the custom field for adding omni asset data in. But no positive answer at that time.

I guess i need some time to figure it out. Some of these problems seems unsolvable.

[sidhujag]

It’s just a spec, every node I assume verifies the transaction including opreturn data to color the outputs. What it is about is using the official spec whilst encoding data in tlv so it can be quickly filtered. That is based on my intuition though. Rgb guys seemed pretty adamant that the tlv extension capability was needed so it can widely support multi coin ln networks.. maybe their transaction is not standard, but if every tx is parsed and verified prior to relay to another node you are right you shouldn’t need extra info. But then I’m not sure you even need asset Id field in there either? You can color that locally once you parse txs if you need to filter txs by asset.

[neocarmack]

So if RBG heavily rely on TLV extention, then they shall put more effort on the other end of the whole workflow: the consensus layer of colored coin, where currently solved by omnilayer/btc, or similar protocols. Because on lighnting network itself, no more works can be done.

If one node involved in a hop of transferring money, it shall verify the data, but how? There is no chain, no consensus algorithm, can be introduced into the process. Even watch tower, i dont think it is a solution. The only check point is when broadcasting, let POW to check. So that's where omnilayer waiting for.

Lots of doubtable questions on that technical path...............

Maybe i did not fully understand your point..........But its great to have chat with you.

[sidhujag]

So if RBG heavily rely on TLV extention, then they shall put more effort on the other end of the whole workflow: the consensus layer of colored coin, where currently solved by omnilayer/btc, or similar protocols. Because on lighnting network itself, no more works can be done.

If one node involved in a hop of transferring money, it shall verify the data, but how? There is no chain, no consensus algorithm, can be introduced into the process. Even watch tower, i dont think it is a solution. The only check point is when broadcasting, let POW to check. So that's where omnilayer waiting for.

Lots of doubtable questions on that technical path...............

Maybe i did not fully understand your point..........But its great to have chat with you.

I meant that on each hop the transaction in question can be parsed to get the colouring information regarding the asset transfer and if accepted (if their channel is of the same asset and has liquidity then hop is valid)

Edit: I might retract this, because the wire protocol is different it does not relay the entire transaction but amount, payment hash and hop information. I guess that is why you have asset id as it is passed along on wire and used to detect which other channels are open with that asset as well. Maybe with TLV extension you can sign a payload which will allow one to verify it when recreating the commitment tx locally.

[sidhujag]

@neocarmack See my response on lightning/bolts#72 about using LN with multiple-assets.. it only makes sense for single asset networks.. hopefully you have designed with it in mind (that LN network only work on single assets at a time)

[neocarmack]

Edit: I might retract this, because the wire protocol is different it does not relay the entire transaction but amount, payment hash and hop information. I guess that is why you have asset id as it is passed along on wire and used to detect which other channels are open with that asset as well. Maybe with TLV extension you can sign a payload which will allow one to verify it when recreating the commitment tx locally.

Emmm, we will try TLV extention to embed OmniBOLT tx info into a general lnd message, so that lnd network will have the liquidity of omnilayer assets.

But the top priority on our schedule is release our first omnibolt testnet, on May 30.

[neocarmack]

@neocarmack See my response on lightning/bolts#72 about using LN with multiple-assets.. it only makes sense for single asset networks.. hopefully you have designed with it in mind (that LN network only work on single assets at a time)

It's true, LN channel only work on one asset at a time. We have not finalize the design yet. What we focus is how to communicate with existing lnd network, not just embed omnibolt transaction into lnd messages.

I dont know which one will be our best choice.

[neocarmack]

@neocarmack See my response on lightning/bolts#72 about using LN with multiple-assets.. it only makes sense for single asset networks.. hopefully you have designed with it in mind (that LN network only work on single assets at a time)

Let me study it.

[sidhujag]

@neocarmack See my response on lightningnetwork/lightning-rfc#72 about using LN with multiple-assets.. it only makes sense for single asset networks.. hopefully you have designed with it in mind (that LN network only work on single assets at a time)

It's true, LN channel only work on one asset at a time. We have not finalize the design yet. What we focus is how to communicate with existing lnd network, not just embed omnibolt transaction into lnd messages.

I dont know which one will be our best choice.

You mean some nodes will propagate and maybe even participate not knowing or caring that it’s an omni asset

[neocarmack]

You mean some nodes will propagate and maybe even participate not knowing or caring that it’s an omni asset

Correct. Embedding info into TLV field just like sending message via some instant message systems. LND has no knowledge of any command contained in TLV.

I consulted lnd community, the answer is that we can subscribe invoice to monitor the event of payment ( which is still unclear to me). But this is not what we want.

I put the technology guide in this repo, hope this can help you see the whole picture of OmniBOLT, especially what we are currently trying to communicate with LND:

https://github.com/omnilaboratory/OmniBOLT-spec/tree/master/docs