joinmarket-org / joinmarket Goto Github PK

Right now the taker class requires a wallet class, when it really only needs access to the private keys of its own addresses in order to sign, and a function for finding the address and value of a utxo.
So there needs to be an abstract superclass which just has get_key_from_addr(self, addr). The class can be called AddressToKeyMapping or something.
This will be useful for when eventually making the electrum plugin, taker class will just need private keys that electrum gives it.

Imagine if after every coinjoin, the taker calculated a measure related to how 'good' the coinjoin was. This would take into account the response time, whether the connection dropped halfway through and some other things.

When the taker then went to choose again who to coinjoin with, it would take this measure into account along with the offered price. This measure could be called 'rating' or 'reputation' or a word to that effect. Takers which use the market a lot would build up a detailed ranking of makers.

Makers would now have an incentive to have a low-latency reliable internet. Slower than average responding to requests, or connections cutting out, would reduce the maker's income. It should help solve Mike Hearn's user experience criticism of CoinJoin from his Merge Avoidance blog post. People operating coinjoins from a flaky network should be gradually forced from the market.

Problem: This creates a disincentive for using tor by makers. Solution is that tor yield generators will simply have to offer lower prices.

Also it could be an incentive for yield generators to use Bitcoin Core's json_rpc interface instead of a blockchain explorer like blockr.io, solving Issue #55 Because it's faster to look up txID's from your hard disk or chainstate than to download them from a website.

Also it could act as a barrier to DOS'ing against the taker.

There would need to be some time decay element involved in the measure, to give makers second chances and slowly forgive their past mistakes. Also an increase in reputation by time, to encourage makers which are permanently around.

Reputations would be associated with the IRC or subspace nickname. If a yield-generator ended up with such a bad reputation they could change nicks although they would again start from zero, far behind others who may have built up long reputations.

More graphs would be useful. Perhaps graphs like those on http://bitcoinity.org/markets which show the depth of the orderbook at a given coinjoin amount.

This would need talking to some trader / algo types, and investors and heavy users to see exactly what kind of information they'd find useful

A few enhancements is probably needed to better match incentives and costs.

Right now it costs the same % to coinjoin a small amount and a large amount, you would expect larger amounts would cost more because fewer individuals own larger amounts of bitcoins.
Although individuals with small amounts of bitcoins might think it's less worth it to invest them in this scheme, so it's not obvious to me what will cost more.
Also coinjoining large amounts can be done more cheaply by splitting them up into smaller amounts.

The marginal cost to the yield generator is proportional the amount of UTXOs used, not the coinjoin amount. So we may see an algorithm that takes that into account.

To a more descriptive name. Perhaps get_encryption_box() or get_encryption_object()

Let's not do it right now, every time there's a commit I need to merge it into my msgchan branch.

For people testing a potentially-buggy new software, they must have confidence that they can easily get their money out of any seeds / master private keys.

wallet-tool.py needs a method which prints to terminal all the private keys, which can be easily imported into another wallet and spent.

I just watched some yield generators appear to quit because of a netsplit. Once the netsplit is over they appear to join, but do not announce their orders.
This is potentially a huge problem for a long-running taker bot.

Perhaps it can be solved with a timer that periodically clears the orderbook and requests orders again.

Need to change all stored nicks on IRC when a bot changes nick.

Right now if the nickname is already used, the algorithm is to nick = nick + '_'
This is pretty subpar. One reason is it can uncloak a user if they are relying on NickServ's cloak.

A better way would be to have multiple possible nicknames, the bot tries all of them and gives up if all are taken.

Need to detect tx that would make dust and avoid them, such as posting offers which dont lead to dust.

A field specifying the minimum number of other makers should be added to the order format. It will be used by patientsendpayment type bots. Yield-generator bots would of course set the field to zero.

Needs #13 and a few others done first.

Give users the option to query bitcoind with a local copy of the blockchain, as well as http query blockchain explorers.

Bitcoind needs to be run with -txindex=1 so you can look up an arbitrary txid.

this is the third ECDSA impl with flagrant timing sidechannels i've seen in like 18 hours

belcher: fwiw, I believe that pybitcointools is a completely naieve implementation, written by a single person, which has never been peer review, which is internally undocumented and completely without tests, which was the authors first cryptographic code, by an author who'd previously written a number of completely broken wallet tools (e.g. reading the mouse position three times in a tight loop and adding math.random() to generate a private key)

ok, so it sounds like the project should be moving towards using python-bitcoinlib instead of pybitcointools

yes

The yield generator bot should periodically print out reports, showing the amount of successful coinjoins, average coinjoin amount, amount of fees earned, time running, calculated % yield and so on.

It would be very useful information for investors, they could compare bitcoin with other assets.

It might also be useful to output information about successful coinjoins to a csv file, the investor could open it with a spreadsheet program and spot patterns or something useful.

Other electrum plugins I've seen fit into one python file, this project has about 12 files which seems impractical.

Perhaps the joinmarket application could run as separately and the electrum plugin merely uses some kind of interprocess communication.

DOS's against takers might be solved by issue #57

DOS's against makers needs discussion.

Now the maker never commits to certain UTXOs belonging to the taker until the taker has actually broadcast them. So if two takers simultaneously fill the maker's offer, the maker might well send them both the same UTXO and only one transaction would actually succeed.

As long as the maker has enough bandwidth, I don't think there's any resource that a DOSer could consume.

Income mode is where the yield generator periodically sends all it's profits to an outside address.

With income mode, an investor in bitcoins could give the bot his master public key and receive payouts at a regular interval. These payouts would represent a passive income stream for the investor, analogous to stock dividends, real estate rent payments, bond coupons or cash interest payments.

Needless to say bitcoin as an asset class is rather different from those.

Contrast this with the current mode of operation, I'm calling accumulation mode, where all the profits are held in the same wallet and used to earn even more.

The other day I was trying to do a coinjoin with many parties, the taker sent out so much data that the IRC server killed it due to flood.

To fix there needs to be a thread which waits on a queue of data and sends it down the IRC socket, with the appropriate rate limiting.

Just a reminder, as we discussed there are changes that will give a big ECC speedup.

Longer term look into adding dependency via install script to keep up to date.

The protocol needs a way to announce version numbers, in case we ever update it that the bots can know their incompatibility. Best way is probably to say !ver [version] in IRC as the first command, and the other bots wont respond unless your bot has first said !ver.
!ver 1!orderbook
!ver 1!relorder [data]

Maker's orders should have an option for requiring a certain number of other makers. For instance patientsendpayment.py will announce an order that can only be filled if two other makers are also taking part. Yield-generator.py would of course be happy with any number of makers.

Compressed public keys should be used. Reduces size, lower miner fees required, bitcoind uses them by default. Only issue is if a coinjoin takes place with both compressed and uncompressed pubkeys it is obvious which output belongs to whom, so this is a hard protocol break.

Opened to start a discussion.

Here is a first attempt.

(redacted: moved to https://github.com/chris-belcher/joinmarket/wiki/Joinmarket-Messaging-Protocol)

This will eventually need to be improved with some ideas we've come up with in the past few weeks. This issue is about collecting together the improvements.

The name of open orders should be changed to 'offers'. An offer is what the maker announces, an order is when a taker and maker(s) agree to do a coinjoin. In other words, an offer is an unfilled order. The concepts are different and need different names.
#29 Add minimum maker count

#9 (comment) something for sendmany
#20 utxo auctioning

Setting up bitcoind is annoying and I bet many people wont do it. Using blockr.io is a poor alternative.

A good idea might be to use BitcoinJ's SPV verification to obtain the transaction history and UTXOs without privacy violations.

According to the libnacl documentation:

"Every encryption routine requires a nonce. The nonce is a 24 char string that must never be used twice with the same keypair. If no nonce is passed in then a nonce is generated based on random data. If it is desired to generate a nonce manually this can be done by passing it into the encrypt method."

(from https://libnacl.readthedocs.org/en/latest/topics/public.html ).

It would be good to double check and evaluate whether the nonce defaults are correct and safe, especially to have another pair of eyes look at it.

(libnacl is the Python binding we use to libsodium: http://doc.libsodium.org/index.html)

Electrum does this when listing addresses and its much neater.

tl;dr If you were designing a tumbler using joinmarket primatives that's hard to unmix, how would you do it?

One popular application of this project will probably be completely breaking the link between coins. The advantages over centralized tumblers are cost and no counterparty risk.

Examples of users might be people who simply bought bitcoins with a very privacy-invading method, such as passing AML/KYC from an exchange, and wish to have privacy in all their purchases again. Some bitcoin users also just need it as a simple medium of exchange, buying bitcoins with traceable fiat and immediately spending them on goods and services. Example would be a anonymous buyer of a domain name, VPS hosting, email, VPN provisions. Users also might be those who engage in capital flight or want to store bitcoins without anyone knowing. They are the kind who would use tumbler.py

Another kind of people use bitcoin like a bank account and do most of their daily spending and earning with it. They would best be served by an electrum plugin that coinjoins every transaction they do, instead of tumbler.py

We need a discussion on what exactly a tumbler bot should do. Repeatedly doing coinjoins will be easy to unmix by looking for a similar sized output as in https://www.reddit.com/r/DarkNetMarkets/comments/2rhaqc/deanonimyzing_bitcoinfog_and_other_tumblers/

So clearly the coins being mixed need to be split up into many different sizes. They should reach the user's clean wallet in several different addresses, although the user should take care not to recombine the outputs. Ideally the service provider (exchange, payment processor, marketplace, etc) should offer up two or three bitcoin addresses to a user, who can feed those addresses into the tumbler bot. We might need a drive of awareness-raising to convince admins to provide multiple addresses to deposit.

It's not clear to me if repeated coinjoin sweeps is worth it, since an observer on the blockchain can easily see an amount that had been coinjoin swept several times. It might be better to do a single coinjoin but with many other participants.

Also we should think carefully about the time interval between different coinjoins. If the blockchain observer sees a ton of coinjoins one very soon after another, it will be more likely they belong to the same person. Random time intervals are needed, trouble is the usd/btc volatility means you cant take too long to tumble.

Perhaps we need to think about how to describe configuration to users. Perhaps a size of anonymity set meaning if you set that to 10, there are 10 other 'people' or 'wallets' who might also be confused with you.

I mean in terms of the transaction sizes, output amounts, timings, number of participants and so on.

At some point the authentication of the encryption between peers must be checked. I've noticed some apparently unfinished things.

E.G.
https://github.com/chris-belcher/joinmarket/blob/4b935e2d89a9e8d646836bbb4c15bc86ec7bb74d/taker.py#L38

signing_btc_add and signing_btc_pub don't seem to be used anywhere after that

(low priority but worth doing at some point)

In commit 46c8547

I've recoded the classes to use the MessageChannel base class. All I did was move the IRC specific code into irc.py and replace it with functions and callbacks in the Maker and Taker classes.
It was successful with Taker and OrderbookWatch, but for Maker and yield generator it appears to get the encryption wrong. The taker on the other side fails to decrypt the !ioauth command and crashes. Before that, the !auth command is correctly received by the maker and decrypted.

I have tested the old and new (which uses MessageChannel) takers on the old kind of makers and it works.
I've tried a few things but no longer know what to do to try to debug this.

The project should be open to any suggestions from anyone coding algorithms for takers or makers on the interface.

Right now for takers you extend the Taker class, create a CoinJoinTX object and pretty much everything is done for you.
For makers you extend Maker, override create_my_orders() oid_to_order() on_tx_unconfirmed() on_tx_confirmed() and everything else is done for you.

The Freenode network requires the SASL extension to IRC in order to connect through tor.
https://blog.freenode.net/2010/01/connecting-to-freenode-using-tor-sasl/

As far as I can tell, there is no need for the keyfile. Keys in libsodium do not have to be persistent across different instances, since its all authenticated.

The keyfile clutters up the working directory and requires an extra command line argument in some cases.

This code is needed so that Tor or an ssh tunnel can be used.

The configuration should go in the joinmarket.cfg file.

Should be a fairly easy addition for keen contributers.

Need a kind of program that parses the blockchain, finds likely coinjoin tx, calculates the fee paid and creates charts showing the market price of coinjoins. Along with the volume of orders.

An important indicator at the state of the market.

Just an idea:

Enforce some encapsulation of private data in such a way that it's impossible that later coding errors lead to involuntary leaking of it.

This is important for the incentives / sybil defence.

Creates a rate limit on sybil attackers. They now have to own more bitcoins and tie them up for longer.

Also reduces the chances of a tx being invalidated.

create_combination(li, n) i coded for the sendpayment sweep already exists in python itertools.

Dont always pick the lowest cost order, instead have a probability density function that chooses them randomly. So most of the time you pick the lowest and sometimes you take higher ones this represents your uncertainty in sybil attackers, the cheapest may not always be the best
i.e. randomly chosen makers, weighted by the price they offer

Need ideas for which form this PDF would take. Exponentially decaying probably, but with what decay constant?

For a start, if a taker comes along and exactly fills amount desired by patientsendpayments, they should pay very little.

Will then probably want to incentivize higher amounts so that the patientsendpayments bot has to pay less when it gives up and becomes a taker.

Related to #19
A bot that keeps track of the orderbook and creates graphs. Graphs like total offered bitcoins vs time, number of orders vs time, number of counterparties vs time and so on.

Need a new order type which publishes a UTXO and offers a fee for using it for coinjoins. People who have desirable coins might achieve higher prices. For example freshly mined coins.

Although in the long run with increased fungibility, the price of these UTXOs should approach the price of a normal coinjoin.

to reproduce

run gui-taker.py
local its page in your browser
kick a yield generator
reload the page
note the orders of the kicked bot are still there

patientsendpayment.py posts small cheaper orders, would be good if sendpayments.py took advantage of them by checking if it can send the payment cheaper by filling several small orders.

If joinmarket is to be used by institutions (e.g. an exchange) there will need to be a coinjoin version of sendmany.

I think this can be simply implemented by replacing the fields cj_amount, cj_address in the procotol with lists. Then huge transactions containing multiple coinjoins could be made.

What are the privacy implications? The yield-generators and makers on the other side of the transaction will likely recombine their coinjoin outputs later so it will be clear those are owned by the same people.

When the wallet synchronisation runs at startup, it picks up utxos for the wallet which are in existing unconfirmed transactions, which can of course lead to pushtx() failure.

I saw this happen for 1bb51f2af3e0a05c37151f337c4a58a8294cc7f78af4cdc06d645530cfd0b98b and aaaf6f710e73740f1e413111b338d4b53a8209d8def2010d09c4adda50bc9150.
aaa was still unconfirmed when I started another sendpayment and outpoint 1 from 1bb was recorded as part of the wallet, so the pushtx() for the new tx failed.

It seems that http://tbtc.blockr.io/api/v1/address/unspent/(address) does not show transactions in the list with zero confirmations (I just tried briefly). If it did, I guess it would explain why that utxo showed up in the wallet.

A command line tool which constructs a coinjoin from a raw transaction. Used for spending exotic inputs with coinjoin. For example somebody who wants to spend from a multisignature address.

You have to be quick! Makers are might doublespend their inputs if someone else comes along to take liquidity from them.

Regarding https://www.reddit.com/r/Bitcoin/comments/2yvy6b/a_regulatory_compliance_service_is_sybil/

Obviously coinjoin doesn't directly help against this IP tracking. But this project can still help.

We should introduce a method where the taker (coinjoin-initiator) can send the fully-signed tx hex to one of the makers (market-makers who wait-to-coinjoin) who will then broadcast it. The taker would choose to broadcast the txhex himself, or send it to one randomly chosen maker to broadcast. In this way the coinjoin tx could come from many IPs instead of just the taker's.

Naturally this kind of bad actor could also be a sybil in the joinmarket. But all communicating is encrypted under private messages, so a passive observer learns nothing.
If they become a sybil actively participating in coinjoins they will be limited in ways already discussed: To coinjoin everybody's transactions the sybil would need to own a large amount of bitcoins, so would be harshly limited by cost. Also their coinjoining activity will stop other sybils learning what's happening. Because of the feature in #14 the sybil will not be guaranteed to coinjoin with everyone just because they offer the lowest price.