openbazaar / openbazaar-go

OpenBazaar 2.0 Server Daemon in Go

openbazaar-go

OpenBazaar Server Daemon in Go

This repository contains the OpenBazaar server daemon which handles the heavy lifting for the OpenBazaar desktop application. The server combines several technologies: A modified IPFS node, which itself combines ideas from Git, BitTorrent, and Kademlia. A lightweight wallet for interacting with several cryptocurrency networks. And a JSON API which can be used by a user interface to control the node and browse the network. Find the user interface for the server at github.com/OpenBazaar/openbazaar-desktop.

Table of Contents

• Install
  • Install prebuilt packages
  • Build from Source
• Dependency Management
  • IPFS Dependency
• Updating
• Usage
  • Options
• Documentation
• Contributing
• License

Install

A typical install of OpenBazaar contains a bundle of the server daemon and user interface. If this is what you are looking for, you can find an installer at https://openbazaar.org/download. If you are looking to run the server daemon by itself or to contribute to development, see below for instructions.

Install Pre-built Packages

The easiest way to run the server is to download a pre-built binary. You can find binaries of our latest release for each operating system here.

Build from Source

To build from source you will need to have Go installed and properly configured. Detailed instructions for installing Go and openbazaar-go on each operating system can be found in the docs package.

Run via Docker image

You can also use our Docker Hub image to run openbazaar-go. latest points to the latest release, but you can run any other version by matching the release tag to the Docker image tag.

docker run -it openbazaar/server:latest [OPTIONS] start [start-OPTIONS]

Dependency Management

We use Godeps with vendored third-party packages.

IPFS Dependency

We are using a fork of go-ipfs in the daemon. The primary changes include different protocol strings to segregate the OpenBazaar network from the main IPFS network and an increased TTL on certain types of DHT data. You can find the full diff in the readme of the forked repo. The fork is bundled in the vendor package and will be used automatically when you compile and run the server. Note that you will still see github.com/ipfs/go-ipfs import statements instead of github.com/OpenBazaar/go-ipfs despite the package being a fork. This is done to avoid a major refactor of import statements and make rebasing IPFS much easier.

Updating

You can either pull in remote changes as normal or run go get -u github.com/OpenBazaar/openbazaar-go.

Usage

You can run the server with go run openbazaard.go start. Ensure you are using at least version 1.10 of Golang, otherwise you might get errors while running.

Options

Usage:
  openbazaard [OPTIONS] start [start-OPTIONS]

The start command starts the OpenBazaar-Server

Application Options:
  -v, --version                   Print the version number and exit

Help Options:
  -h, --help                      Show this help message

[start command options]
      -p, --password=             the encryption password if the database is encrypted
      -t, --testnet               use the test network
      -r, --regtest               run in regression test mode
      -l, --loglevel=             set the logging level [debug, info, notice, warning, error, critical] (default: debug)
      -f, --nologfiles            save logs on disk
      -a, --allowip=              only allow API connections from these IPs
      -s, --stun                  use stun on µTP IPv4
      -d, --datadir=              specify the data directory to be used
      -c, --authcookie=           turn on API authentication and use this specific cookie
      -u, --useragent=            add a custom user-agent field
      -v, --verbose               print openbazaar logs to stdout
          --torpassword=          Set the tor control password. This will override the tor password in the config.
          --tor                   Automatically configure the daemon to run as a Tor hidden service and use Tor exclusively. Requires
                                  Tor to be running.
          --dualstack             Automatically configure the daemon to run as a Tor hidden service IN ADDITION to using the clear
                                  internet. Requires Tor to be running. WARNING: this mode is not private
          --disablewallet         disable the wallet functionality of the node
          --disableexchangerates  disable the exchange rate service to prevent api queries
          --storage=              set the outgoing message storage option [self-hosted, dropbox] default=self-hosted
          --forcekeypurge         repair test for issue OpenBazaar/openbazaar-go#1593; use as instructed only

Documentation

Documentation of the OpenBazaar protocol has not been formalized yet. If you would like to help, please reach out on Slack or via a new issue on GitHub.

openbazaar-go exposes an HTTP API which permits high-level interactions on the network and the internal wallet. Find the HTTP API documentation at https://api.docs.openbazaar.org.

Contributing

Contributions are definitely welcome! Please read the Contributing Guidelines before starting.

License

MIT.

openbazaar-go's Issues

Feature request: Build source with go get

It is currently not possible to build openbazaar-go with go get. This hampers development because it makes it difficult to set up CI tools like Travis, Coveralls, and godocs. It also creates a barrier to entry to new developers who would like to contribute

It would be great if a contributor could download and build just by running:

go get -u github.com/OpenBazaar/openbazaar-go/...

What does `pb` stand for?

There's a pb folder, with .pb.go files inside. What does pb stand for?

Consider adding phone to profile pb

See here. There's website, email, but no phone number.

Phone call are still used for retail (and other) businesses.

Listing POST returns nothing on success which is invalid JSON

A successful POST to /ob/listing/ returns nothing. This results in an error on the client because an empty response body is not valid json:

If the response will not return anything useful (like the created listing, or the listing id / hash), the expectation is that it would at a minimum return an empty object (which is valid json).

What is the `repo` directory?

Does repo stand for "repository"? Is it a GitHub repo?

On a PUT on the listing API, if you change the slug, a listing with the old slug still remains.

Steps to reproduce:
1.) Send a POST to create a new listing with the slug 'check-yo-self'.
2.) Send a PUT to update the existing listing and change the slug to 'check-yo-self-no-mo'.

Observed Results:
You will not have two listings separate listings. With the second process, you essentially forked your listing.

Expected Results:
There would only be one listing with a slug of 'check-yo-self-no-mo'.

What is test_compile.sh?

What is test_compile.sh and how do I run it?

This is the output I'm currently getting

Justins-MacBook-Air:openbazaar-go justin$ ./test_compile.sh
?       github.com/OpenBazaar/openbazaar-go [no test files]
?       github.com/OpenBazaar/openbazaar-go [no test files]
?       github.com/OpenBazaar/openbazaar-go [no test files]
?       github.com/OpenBazaar/openbazaar-go [no test files]
?       github.com/OpenBazaar/openbazaar-go [no test files]
?       github.com/OpenBazaar/openbazaar-go [no test files]
?       github.com/OpenBazaar/openbazaar-go [no test files]
?       github.com/OpenBazaar/openbazaar-go [no test files]
?       github.com/OpenBazaar/openbazaar-go [no test files]
?       github.com/OpenBazaar/openbazaar-go [no test files]
?       github.com/OpenBazaar/openbazaar-go [no test files]
?       github.com/OpenBazaar/openbazaar-go [no test files]
cat: *.cover.out: No such file or directory

Suggested changes to the Listing proto

I think there is room for improvement in the message protocol generally. I make suggestions below for the Listing proto. If useful, I will comments on other parts of the message protocol.

[Please see this file for reference.]

1. string condition -> Why is this a free text string? We currently have fixed options NEW, USED_GOOD, USED_EXCELLENT, USED_POOR and REFURBISHED.
2. bool freeShipping -> This is redundant when we have Price domestic and Price international, and doesn't allow for free domestic but paid international.
3. message Price -> This is needlessly complicated. Why is there two cases for bitcoin and fiat? It should simply be currencyCode (for which BTC is an option) and price. The simpler design also allows for other 'currencies' like ether or gold to be added.
4. repeated string moderators -> Why are moderators strings? Shouldn't it be repeated ID moderators?
5. CATEGORY_NOT_SET -> Setting a category should be mandatory, no?
6. repeated CountryCode shippingRegions -> The name CountryCode mismatches with "regions", which should include continents (North America, South America, Europe, Asia, etc.)
7. With the Options message, it seem like Price domestic and Price international are redundant and needlessly restrictive. For example, a vendor can have three shipping options as Options. (For example, "Standard", "First class" and "Prime" shipping, each an option.)
8. I can see people getting creative with Options, so maybe add a string description for each option?
9. Shouldn't Options be renamed Option (singular)?

Break down openbazaard.go into smaller files

In the OB1.0 OpenBazaar-Server repo, the file openbazaard.py was very difficult to work with because it was long and linear (lots of "spaghetti"). openbazaard.go seems to be headed in a similar direction, with close to 500 lines of code.

Does it make sense to break down openbazaard.go into smaller files?

Some APIs are PUTs that should be POSTs.

In HTTP, by convention, POST is used to create, whereas PUT is used to update. There are some APIs which are create actions, yet they use the verb PUT. Can they be updated to use POST?

PUTAvatar
PUTHeader
PUTImage

Proof of follow

It seems that a node can fake its followers given that a "follower" is just a string without a proof-of-follow. Is that correct?

If so, why not add signed proof-of-follow messages?

Does libp2p not include stun?

I can see code for stunning here. My understanding was that the p2p networking was handled by IPFS's libp2p. Is that correct? Does libp2p not handle stun natively?

Issue with OpenSSL while the process installing go-sqlchiper

I have the gcc properly installed. But looks like there seems to be an issue while installing go-sqlchiper. Has anyone tried to install the new release on windows yet. Any help is greatly appreciated. Thanks

go get github.com/OpenBazaar/openbazaar-go

github.com/OpenBazaar/openbazaar-go/vendor/github.com/xeodou/go-sqlcipher

....\workspace\src\github.com\OpenBazaar\openbazaar-go\vendor\github.com\xeodou\go-sqlcipher\sqlite3-binding.c:16824:26: fatal error: openssl/rand.h: No such file or directory
#include <openssl/rand.h>
^
compilation terminated.

There is not well-functioning update Listing API.

Currently, there is only a POST API available to create / update a listing. It is in-adequate in regards to updating a listing for a couple of reasons:

1.) There is no way for the API consumer to express their intention on whether they want to update an existing listing vs. create a new one. For example, if your intention is to create a new listing, and you send a POST with a slug that, unbeknownst to you, is already associated with an existing listing, you will be overwriting the existing listing.

Expected result:
In this case, the expectation would be that the API returns an error like 'Slug already exists'.

2.) If you attempt to change the slug of an existing listing, the old listing still remains and a "copy" of the new one is created with a new slug (and any other fields you updated). The listing is essentially forked.

Expected result:
When you change the slug of an existing listing, the slug is updated "in place" so that only the one listing remains, but with an updated slug.

Failure to fetch public key from the DHT

It appears some attempts at fetching public keys from the DHT are failing and causing IPNS queries where the peer is not online and the key is not cached to fail. Records are being fetched from the DHT correctly so it's a bit strange the keys are not.

IPFS v0.4.3 should be out soon. When it's released I will rebase and see if it's still a problem. If so I'll do some deeper investigating.

ipfs repo needs migration

Just pulled and got this error

Justins-MacBook-Air:openbazaar-go justin$ go run openbazaard.go start
________                      __________
\_____  \ ______   ____   ____\______   \_____  _____________  _____ _______
 /   |   \\____ \_/ __ \ /    \|    |  _/\__  \ \___   /\__  \ \__  \\_  __ \
/    |    \  |_> >  ___/|   |  \    |   \ / __ \_/    /  / __ \_/ __ \|  | \/
\_______  /   __/ \___  >___|  /______  /(____  /_____ \(____  (____  /__|
        \/|__|        \/     \/       \/      \/      \/     \/     \/

OpenBazaar Server v2.0 starting...
19:02:22.300 [Execute] [ERROR] ipfs repo needs migration.
ipfs repo needs migration.
exit status 1

Would it be possible to add documentation about how to do the "ipfs repo migration"?

DRY-up jsonapi.go

The file jsonapi.go has a lot of repeated code. For example, the line

w.Header().Add("Content-Type", "application/json")

appears 33 times in just that file. jsonapi.go needs DRYying up.

go fmt issues with openbazaar.go

There are 3 formatting issues for openbazaar.go. Below is the diff on my local machine.

diff --git a/openbazaard.go b/openbazaard.go
index 7580c81..02144fa 100644
--- a/openbazaard.go
+++ b/openbazaard.go
@@ -3,6 +3,7 @@ package main
 import (
        "errors"
        "fmt"
+       bstk "github.com/OpenBazaar/go-blockstackclient"
        "github.com/OpenBazaar/openbazaar-go/api"
        "github.com/OpenBazaar/openbazaar-go/core"
        "github.com/OpenBazaar/openbazaar-go/ipfs"
@@ -35,7 +36,6 @@ import (
        proto "gx/ipfs/QmZ4Qi3GaRbjcx28Sme5eMH7RQjGkt8wHxt2a65oLaeFEV/gogo-protobuf/proto"
        "gx/ipfs/QmZy2y8t9zQH2a1b8q2ZSLKp17ATuJoCNxxyMFG5qFExpt/go-net/context"
        ipfslogging "gx/ipfs/Qmazh5oNUVsDZTs2g59rq8aYQqwpss8tcUWQzor5sCCEuH/go-log"
-       bstk "github.com/OpenBazaar/go-blockstackclient"
        "os"
        "os/signal"
        "path"
@@ -44,7 +44,6 @@ import (
        "sort"
        "strconv"
        "strings"
-
 )

 var log = logging.MustGetLogger("main")
@@ -352,7 +351,7 @@ func (x *Start) Execute(args []string) error {
                Datastore:      sqliteDB,
                Wallet:         wallet,
                MessageStorage: storage,
-               Resolver: bstk.NewBlockStackClient(resolverUrl),
+               Resolver:       bstk.NewBlockStackClient(resolverUrl),
        }

        var gwErrc <-chan error

Can we agree on sane versioning?

Versioning is important communication for collaborative dev work. Part of what made OpenBazaar-Server hard for others to work on was the versioning was inconsistent and poorly updated.

Ideally, the various version numbers appears in a single place and there are clear guidelines for updating them, e.g. http://semver.org

For example at the moment we have

fmt.Println("OpenBazaar Server v2.0 starting...")

from here and

"version": "0.2.0"

from here and

var ProtocolOpenBazaar protocol.ID = "/openbazaar/app/1.0.0"

from here.

Can we agree on a clear versioning process?

Improving ratings for OB2.0

In OpenBazaar-Server it looks like the transaction address and amount are part of a given rating. Why isn't a hash of the transaction (not the txid as that can mutate) included? Indeed, the pair (address, amount) is not always enough to uniquely identify a Bitcoin transaction.

Secondly, it looks like OpenBazaar-Server does not actually verify the existence of the Bitcoin transaction. I suspect since OB2.0 has an SPV node, such verification will be done for OB2.0. Is that correct?

Finally, it looks like the OpenBazaar-Server rating object doesn't contain a timestamp. Please consider including a timestamp.

License missing in package.json

See here. The MIT license is missing.

Misc contracts.proto comments

1. double price -> Prefer using integers for prices, especially given JavaScript and Go handle floats differently.
2. string slug -> What is a slug? Maybe add a descriptive comment
3. Why do Coupons have a hash?
4. Tax tax -> Should this be repeated Tax tax?
5. string moderator -> Should this be more structured, e.g. ID moderator?
6. Why is the Metadata separated out from the Item? It's confusing for the listing data to be spread out. Can these two be merged?
7. Why does Image have a fileName? I don't see how this is helpful.
8. I suggest adding a description to Image. Different images can show different aspects of a listing, and image descriptions can provide context.
9. string condition -> Having constraints on the condition makes it manageable (e.g. the current the five options New, Good, Excellent, Poor, Refurbished add consistency throughout)
10. ID vendorID -> Can the naming be simplified to ID vendor? Same with ID buyerID.
11. Shouldn't be included in the seeded listing. (referring to inventoryCount) -> Why not make this public?
12. Price pricePerUnit -> Can the naming be simplified to Price price?
13. string estimatedDelivery -> Generally freetext strings produce messy listings that are hard for machines to parse/understand. Should this be a timestamp, i.e. uint64 estimatedDelivery?
14. string chaincode -> What is chaincode? Should it be chainCode?
15. Method method -> This seems redundant given we have string moderator right below. (No moderator chosen means direct payment.)
16. Why do we have bytes listingHash but string hash for Image and Coupon?
17. It's a bit confusing that Listing and Order both have Item, but those are very different.

Is this project testable already?

Stress tests and integration tests

Many people, including myself, felt that OpenBazaar-Server was/is very buggy (the reason I pinpoint OpenBazaar-Server is that the stability bugs would manifest in headless mode also).

I've tried reporting such stability issues (for example here on May 22). That particular report (which IMO should have been top-priority) was never replied to, categorised or prioritised. For similar reports from me and others, the responses were along the lines of "can't reproduce", "not enough information", "help wanted".

IMO, the stability issues of OB1.0 are killing OB1.0, and those can be attributed to two things:

1. (technical) Extreme lack of rigorous testing, in particular stress testing (and integration testing). The superficial/easy-to-fix bugs were fixed very quickly, but little effort was done to find and understand the bugs that are subtle, hard-to-reproduce, take time to trigger, affect only some platforms, etc. Those are found with rigorous stress tests and integration tests.
2. (managerial) Releases were rushed. The non-OB1 community had very little time to review the release candidates (usually just a few days), and there's no formal review process (e.g. like Bitcoin's ACK, NACK, utACK and Concept ACK). Also, the developers should test their code as they write it. Testing is too important to offload to volunteers after the fact.

I hope my rant can help improve development and release processes for OB2.0. I will personally take on some of the testing debt that openbazaar-go has started to accumulate.

The listing API overwrites an existing listing that has the given slug.

Scenario #1

Steps to reproduce:
1.) Send a POST to create a new listing with the slug 'check-yo-self'.
2.) Send another POST to create a new listing with the slug 'check-yo-self'.

Observed Results:
The second post creates a listing that overwrites the one from the first post.

Expected Results:
On the second POST an error is returned saying 'That slug already exists' and the second listing is not saved.

Scenario #2

Steps to reproduce:
1.) Send a POST to create a new listing with the slug 'hello-world'.
2.) Send a POST to create a new listing with the slug 'bye-world'.
3.) Update the 'bye-world' listing by sending a PUT with a slug of 'hello-world' and a currentSlug of 'bye-world'.

Observed Results:
The 'hello-world' listing is overwritten with the updated contents of the 'bye-world' listing.

Expected Results:
If you attempt to update the slug of a listing to a slug that already exists, an error is returned saying 'That slug already exists' and the listing is not saved.

Encrypt outgoing online messages (two-layer encryption)

(This issue was discussed with @cpacia during the OpenBazaar Partners Call on Thursday August 25.)

Duo Search wants to proxy OpenBazaar orders (and other similar p2p messages) from a buyer visiting www.duosear.ch to a vendor's node, without Duo Search knowing the contents of the message (to preserve privacy). At the moment there is a single TLS-like layer of encryption between two nodes for outgoing online messages, as shown in Diagram 1:

Diagram 1
*********

         TLS
 Buyer <-----> Vendor
(node)         (node)

To allow for proxying of p2p messages while preserving privacy, we suggest adding a second layer of AES encryption similar to what already exists for offline messages (see SendOfflineMessage):

ciphertext, cerr := n.EncryptMessage(p, messageBytes)

(As stated in a comment for the function EncryptMessage: encrypt outgoing offline messages with the long lived identity key. We suggest generalising that encryption to online messages.)

The second layer of AES encryption for outgoing online messages allows for the setup as shown in Diagram 2:

Diagram 2
*********

                     AES
      <------------------------------>
           TLS                TLS
  Buyer  <-----> Duo Search <-----> Vendor
(browser)          (node)           (node)

As can be seen in Diagram 2, the AES encryption is end-to-end, overarching the whole path from buyer to vendor, including the two TLS hops. In this setup, the AES encryption (and decryption) is client-side in the buyer's browser using JavaScript, similar to what Blockchain.info does for its wallet. Here the Duo Search node is a dumb proxy of AES encrypted messages.

(Incidentally, having AES for both outgoing online and offline messages makes the code neater and more consistent.)

Comments and questions welcome; cc @hoffmabc @tyler-smith

Bad form database initialisation

In db.go, the initialisation code contains this

    if password != "" {
        sqlStmt = "PRAGMA key = '" + password + "';"
    }

This means that some passwords can mess up the database, in particular passwords with the character ".

"cannot use sqliteDB"

go get is failing for me:

Justins-MacBook-Air:openbazaar-go justin$ go get -u github.com/OpenBazaar/openbazaar-go
# github.com/OpenBazaar/openbazaar-go
./openbazaard.go:393: cannot use sqliteDB (type *db.SQLiteDatastore) as type "github.com/OpenBazaar/openbazaar-go/repo".Datastore in field value:
    *db.SQLiteDatastore does not implement "github.com/OpenBazaar/openbazaar-go/repo".Datastore (missing Purchses method)
./openbazaard.go:416: cannot use sqliteDB (type *db.SQLiteDatastore) as type "github.com/OpenBazaar/openbazaar-go/repo".Datastore in argument to service.SetupOpenBazaarService:
    *db.SQLiteDatastore does not implement "github.com/OpenBazaar/openbazaar-go/repo".Datastore (missing Purchses method)
./openbazaard.go:418: cannot use sqliteDB (type *db.SQLiteDatastore) as type "github.com/OpenBazaar/openbazaar-go/repo".Datastore in argument to "github.com/OpenBazaar/openbazaar-go/net".NewMessageRetriever:
    *db.SQLiteDatastore does not implement "github.com/OpenBazaar/openbazaar-go/repo".Datastore (missing Purchses method)
./openbazaard.go:421: cannot use sqliteDB (type *db.SQLiteDatastore) as type "github.com/OpenBazaar/openbazaar-go/repo".Datastore in argument to "github.com/OpenBazaar/openbazaar-go/net".NewPointerRepublisher:
    *db.SQLiteDatastore does not implement "github.com/OpenBazaar/openbazaar-go/repo".Datastore (missing Purchses method)

The same error is happening with go run openbazaard.go start. Fails on both go 1.6.3 and 1.7.

Profile POST is requiring a PUT even when the fetch 404s

When I fetch a Profile and it 404s, this indicates to the client that the Profile was never created. On a subsequent POST, sometimes the we get the following error:

If we refetch the Profile, we still get a 404:

But, a subsequent POST responds with an error to use PUT:

With this behavior, it's very hard for the client to determine whether they should POST or PUT. Can the discrepancy be fixed where the fetch is 404ing, but the POST is still demanding a PUT?

Alternatively, if it's just easier to eliminate PUTs on the endpoint, that would probably work.

What's the dependency strategy?

There seems to be dependency-related stuff all over the repository:

• The vendor folder
• The Godeps folder which seems to be related to this dependency tool
• The gx folder which seems to be related to this package manager
• The Makefile has six (!) undocumented dependency-related commands (godep, toolkit_upgrade, gx_upgrade, gxgo_upgrade, deps, vendor)

What's the dependency strategy (i.e. can it be documented)? Can the current dependency structure be simplified?

runtime error: slice bounds out of range

Trying to get openbazaar-go running. I get an error with go get.

Justins-MacBook-Air:ob-go justin$ go get -u github.com/OpenBazaar/openbazaar-go
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
panic(0x53b3c0, 0xc820010090)
    /usr/local/go/src/runtime/panic.go:481 +0x3e6
main.vendoredImportPath(0xc821127000, 0xc821151fc1, 0x1e, 0x0, 0x0)
    /usr/local/go/src/cmd/go/pkg.go:466 +0x713
main.loadImport(0xc821151fc1, 0x1e, 0xc8208d3180, 0x7d, 0xc821127000, 0xc8208c3788, 0xc82114f470, 0x1, 0x1, 0x1, ...)
    /usr/local/go/src/cmd/go/pkg.go:336 +0xe52
main.(*Package).load(0xc821127000, 0xc8208c3788, 0xc82104a700, 0x0, 0x0, 0x4)
    /usr/local/go/src/cmd/go/pkg.go:953 +0x4397
main.loadImport(0xc820fe8b11, 0x29, 0xc820ff6190, 0x4c, 0xc820ff3800, 0xc8208c3788, 0xc820fe9770, 0x1, 0x1, 0x1, ...)
    /usr/local/go/src/cmd/go/pkg.go:377 +0x84e
main.(*Package).load(0xc820ff3800, 0xc8208c3788, 0xc820e61180, 0x0, 0x0, 0xc)
    /usr/local/go/src/cmd/go/pkg.go:953 +0x4397
main.loadImport(0x7fff5fbffbc2, 0x23, 0xc820014064, 0x24, 0x0, 0xc8208c3788, 0x0, 0x0, 0x0, 0x0, ...)
    /usr/local/go/src/cmd/go/pkg.go:377 +0x84e
main.loadPackage(0x7fff5fbffbc2, 0x23, 0xc8208c3788, 0x0)
    /usr/local/go/src/cmd/go/pkg.go:1639 +0x13d8
main.packagesAndErrors(0xc820aa7930, 0x1, 0x1, 0x0, 0x0, 0x0)
    /usr/local/go/src/cmd/go/pkg.go:1684 +0x531
main.packagesForBuild(0xc820aa7910, 0x1, 0x1, 0x0, 0x0, 0x0)
    /usr/local/go/src/cmd/go/pkg.go:1700 +0x82
main.runGet(0x885fe0, 0xc820aa7910, 0x1, 0x1)
    /usr/local/go/src/cmd/go/get.go:131 +0x3f2
main.main()
    /usr/local/go/src/cmd/go/main.go:181 +0x783

Irregularly, but steadily getting IPNS pinned errors.

I don't know what the exact steps to reproduce are, because it doesn't always happen. But, it has been happening enough to multiple developers that I think it requires attention. Anyhow, often times, after wiping your data and starting with a fresh server, POSTs on the profile are failing with a 'not pinned' error:

Then, a subsequent fetch returns a 404, whereas a subsequent PUT fails with 'Profile already exists. Use PUT':

Once again, i think the crux of the problem is that the Profile is erroneously 404ing, when the file does indeed exists. It shouldn't 404 in this case, otherwise the client has no reliable way of knowing whether to onboard or not and whether to save a profile via PUT or POST.

What is dropbox?

There's a folder called dropbox (see here).

Is that dropbox.com? Is it for testing?

Error pollutes shell in red

See screenshot. I got an error and the my shell went red.

Inconsistent casing

There's a mix of camel casing and snake casing in the project. For example, see here.

    string textColor              = 14;

    uint32 follower_count         = 15;

Will openbazaar-go also work with libbitcoin?

As an alternative, the Go implementation btcd could be considered.

Add Country Code to ShippingAddress

To determine if a user has an address that matches the countries a listing can ship to, we need to have the country code for each saved address of the user.

Can we add a countryCode field (with a string type) to the ShippingAddress data?

Listing API returns invalid JSON when quoting a field.

The listing POST API is returning the following error:

which is invalid JSON because item needs to be quoted with single quotes or have its double quotes escaped.

Expected Results:
The API returns valid JSON.

Also, I imagine this may be a scenario happening in other APIs. If so, can it also be addressed there please?

Build/documentation issue

I've followed the README.md to the letter, but failed. How can I get openbazaar-go to run?

Justins-MacBook-Air:ob-go justin$ go get github.com/OpenBazaar/openbazaar-go
    # github.com/OpenBazaar/openbazaar-go/vendor/github.com/ipfs/go-ipfs/core/corehttp
src/github.com/OpenBazaar/openbazaar-go/vendor/github.com/ipfs/go-ipfs/core/corehttp/gateway_handler.go:241: invalid operation: ! string
# github.com/OpenBazaar/openbazaar-go/vendor/github.com/xeodou/go-sqlcipher
src/github.com/OpenBazaar/openbazaar-go/vendor/github.com/xeodou/go-sqlcipher/sqlite3-binding.c:16824:10: fatal error: 'openssl/rand.h' file not found

I suggest making README.md slightly more verbose with instructions on installing dependencies. (My understanding was there would be a focus on documentation for OB2.0.)

How should Go Report Card issues be treated?

The Go Report Card for this repo flags 44 issues. How should Go Report Card issues be treated? (E.g. which classes of issues are ignored, which should be addressed.)

A piece of documentation in docs would be great.

ZeroMQ dependency

The ReadMe states

You will need ZeroMQ version 4.0.1 or above installed with libsodium enabled.

I understood from the OpenBazaar 2.0 global hangout that the ZeroMQ dependency was dropped. Is that correct?

If it's not, would it be possible to expand upon the ZeroMQ installation steps?

Testing out openbazaar-go

Ok, I've got openbazaar-go running

Justins-MacBook-Air:openbazaar-go justin$ go run openbazaard.go start
________                      __________
\_____  \ ______   ____   ____\______   \_____  _____________  _____ _______
 /   |   \\____ \_/ __ \ /    \|    |  _/\__  \ \___   /\__  \ \__  \\_  __ \
/    |    \  |_> >  ___/|   |  \    |   \ / __ \_/    /  / __ \_/ __ \|  | \/
\_______  /   __/ \___  >___|  /______  /(____  /_____ \(____  (____  /__|
        \/|__|        \/     \/       \/      \/      \/     \/     \/

OpenBazaar Server v2.0 starting...
generating 4096-bit RSA keypair...done
15:56:02.029 [DoInit] [INFO] initializing openbazaar node at /Users/justin/OpenBazaar2.0
15:56:04.087 [Execute] [INFO] Peer ID:  QmcQSSjQZi2bHGbR3e7xU9UYEjKwAiWoGbzVdbXftnTjsy
15:56:04.087 [printSwarmAddrs] [INFO] Swarm listening on /ip4/127.0.0.1/tcp/4001
15:56:04.088 [printSwarmAddrs] [INFO] Swarm listening on /ip4/127.0.0.1/udp/4001/utp
15:56:04.088 [printSwarmAddrs] [INFO] Swarm listening on /ip4/192.168.0.16/tcp/4001
15:56:04.088 [printSwarmAddrs] [INFO] Swarm listening on /ip4/192.168.0.16/udp/4001/utp
15:56:04.088 [printSwarmAddrs] [INFO] Swarm listening on /ip4/78.221.67.24/tcp/4001
15:56:04.088 [printSwarmAddrs] [INFO] Swarm listening on /ip6/::1/tcp/4001
15:56:04.088 [printSwarmAddrs] [INFO] Swarm listening on /ip6/::1/udp/4001/utp
15:56:04.101 [Execute] [INFO] Starting bitcoin wallet...
15:56:04.101 [serveHTTPGateway] [INFO] Gateway/API server listening on /ip4/127.0.0.1/tcp/8080
15:56:04.131 [SetupOpenBazaarService] [INFO] OpenBazaar service running at /openbazaar/app/1.0.0

The README.md file doesn't have further steps after go run openbazaard.go start, so now what? How can I play around with the server? What can I do?

I notice there's an API server listing on 127.0.0.1:8080 but a naive GET request responds with 404 page not found without further information or documentation.

Tor integration

As of August 26, the openbazaar-go codebase doesn't seem to have code for Tor integration. The intent of this issue is to push forward discussion about Tor integration, as this one of the most requested features for OpenBazaar. I'll start with a couple questions:

1. Is Tor integration planned for the OB2.0 initial release?
2. I tried to find a Tor library in Go but couldn't find anything satisfactory. Which Tor library is planned to be used for the integration?

Opt-in proof of transaction

For the purposes of generating OpenBazaar statistics, I think it would be useful for buyers to be able to submit an opt-in proof of transaction, similar to leaving a rating (with is both a proof of transaction and a rating). This would effectively be an "empty rating".

Panic: interface conversion

I've started from scratch, following instructions here. A few seconds after running I get a panic.

Justins-MacBook-Air:ob-go justin$ go get github.com/OpenBazaar/openbazaar-go
Justins-MacBook-Air:ob-go justin$ cd $GOPATH/src/github.com/OpenBazaar/openbazaar-go
Justins-MacBook-Air:openbazaar-go justin$ go run openbazaard.go start
________                      __________
\_____  \ ______   ____   ____\______   \_____  _____________  _____ _______
 /   |   \\____ \_/ __ \ /    \|    |  _/\__  \ \___   /\__  \ \__  \\_  __ \
/    |    \  |_> >  ___/|   |  \    |   \ / __ \_/    /  / __ \_/ __ \|  | \/
\_______  /   __/ \___  >___|  /______  /(____  /_____ \(____  (____  /__|
        \/|__|        \/     \/       \/      \/      \/     \/     \/

OpenBazaar Server v2.0 starting...
17:36:11.677 [Execute] [INFO] Peer ID:  QmcQSSjQZi2bHGbR3e7xU9UYEjKwAiWoGbzVdbXftnTjsy
17:36:11.681 [printSwarmAddrs] [INFO] Swarm listening on /ip4/127.0.0.1/tcp/4001
17:36:11.681 [printSwarmAddrs] [INFO] Swarm listening on /ip4/127.0.0.1/udp/4001/utp
17:36:11.681 [printSwarmAddrs] [INFO] Swarm listening on /ip4/192.168.0.16/tcp/4001
17:36:11.681 [printSwarmAddrs] [INFO] Swarm listening on /ip4/192.168.0.16/udp/4001/utp
17:36:11.681 [printSwarmAddrs] [INFO] Swarm listening on /ip4/78.221.67.24/tcp/4001
17:36:11.681 [printSwarmAddrs] [INFO] Swarm listening on /ip6/::1/tcp/4001
17:36:11.681 [printSwarmAddrs] [INFO] Swarm listening on /ip6/::1/udp/4001/utp
17:36:11.697 [Execute] [INFO] Starting bitcoin wallet...
17:36:11.698 [fetchCurrentRates] [INFO] Fetching bitcoin exchange rates
17:36:11.698 [serveHTTPGateway] [INFO] Gateway/API server listening on /ip4/127.0.0.1/tcp/8080
panic: interface conversion: interface is nil, not string

goroutine 228 [running]:
panic(0x4ddc8a0, 0xc822485f00)
    /usr/local/go/src/runtime/panic.go:481 +0x3e6
github.com/OpenBazaar/openbazaar-go/repo.GetAPIUsernameAndPw(0xc8224a0ea0, 0x22, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /Users/justin/Desktop/duomoney/ob-go/src/github.com/OpenBazaar/openbazaar-go/repo/config.go:28 +0x25c
github.com/OpenBazaar/openbazaar-go/api.newRestAPIHandler(0xc822181260, 0x0, 0x0, 0x0, 0xc8224a0a80, 0x0, 0x0)
    /Users/justin/Desktop/duomoney/ob-go/src/github.com/OpenBazaar/openbazaar-go/api/restapi.go:51 +0x325
github.com/OpenBazaar/openbazaar-go/api.makeHandler(0xc822181260, 0xc820419a01, 0xc820159ea0, 0x1b, 0x0, 0xc822494700, 0xc8223fef40, 0x0, 0xc822479030, 0x6b4b490, ...)
    /Users/justin/Desktop/duomoney/ob-go/src/github.com/OpenBazaar/openbazaar-go/api/gateway.go:21 +0x111
github.com/OpenBazaar/openbazaar-go/api.Serve(0xc822496900, 0xc822181260, 0x4d62701, 0xc820159ea0, 0x1b, 0x0, 0xc822494700, 0xc8223fef40, 0x0, 0xc822479030, ...)
    /Users/justin/Desktop/duomoney/ob-go/src/github.com/OpenBazaar/openbazaar-go/api/gateway.go:47 +0xed
main.serveHTTPGateway.func1(0xc8224968a0, 0xc822496900, 0xc822181260, 0x680ccf0, 0xc8224917a0, 0xc8224a0a50, 0x5, 0x5)
    /Users/justin/Desktop/duomoney/ob-go/src/github.com/OpenBazaar/openbazaar-go/openbazaard.go:451 +0xb6
created by main.serveHTTPGateway
    /Users/justin/Desktop/duomoney/ob-go/src/github.com/OpenBazaar/openbazaar-go/openbazaard.go:453 +0xa24
exit status 2

Understanding the dependencies

There are dependencies in vendor/github.com/. I'm trying to get my head around those so I did a bit of research, and maybe this is useful for someone else. I've added several questions along the way.

jessevdk/go-flags -> Helper
mitchellh/go-homedir -> Helper
mattn/go-isatty -> Helper
mattn/go-colorable -> Logging
fatih/color -> Logging (do we need two color libraries?)
natefinch/lumberjack -> Logging
op/go-logging    -> Logging (do we need two logging libraries?)
OpenBazaar/spvwallet -> Bitcoin
tyler-smith/go-bip39 -> Bitcoin
btcsuite -> Bitcoin
boltdb/bolt -> DB
xeodou/go-sqlcipher -> DB (why both sql and bolt?)
gorilla/websocket -> Client-server communication
ipfs/go-ipfs -> Core tech
golang/protobuf -> Protocol specification?
dropbox/dropbox-sdk-go-unofficial -> Is OB2.0 using DropBox?
ccding/go-stun -> Isn't this handled by libp2p?

How are tests run?

I have tried running go test in two different directories, both times unsuccessfully.

• From $GOPATH/src/github.com/OpenBazaar/openbazaar-go I get:

Justins-MacBook-Air:openbazaar-go justin$ go test
?       github.com/OpenBazaar/openbazaar-go [no test files]

• From $GOPATH I get:

Justins-MacBook-Air:ob-go justin$ go test
can't load package: package .: no buildable Go source files in /Users/justin/Desktop/duomoney/ob-go

A short testing guide in docs would be helpful.

Order states

On the #openbazaar-2_0 Slack channel, @cpacia asks:

what do you think about these order states in 2.0:

0. Pending (new order, has not been accepted by the vendor)
1. Confirmed (vendor has accepted the order and is awaiting payment)
2. Funded (buyer has funded the order)
3. Fulfilled (vendor has shipped the order/sent the digital good)
4. Complete (buyer has closed the order, released the funds, sent the rating).
6. Disputed (under active dispute)
7. DisputeResolved (the dispute has been closed by the moderator, waiting for one of the parties to accept the payout)
8. DisputeClosed (the payout transaction has been broadcast, the dispute is over). 
9. Refunded (vendor closed the order and refuned the buyer)

My initial thoughts below:

1. This is a critical part of the protocol spec. Can we formalise it (e.g. into an OBIP)?
2. What are the state transitions? We need a state diagram to communicate the full story.
3. There are three parties (buyer, vendor, moderator) involved in an order. Do each of the parties have the exact same set of possible states? Seems unlikely given things are not perfectly symmetric across parties (e.g. see point 4. below).
4. It seems that state synchronisation has not been taken into account. For example, the buyer's order can be in state Pending while for the vendor it is in state Confirmed. The vendor then needs to communicate the confirmation, leading to the two substates ConfirmedNotAcknowledged and ConfirmedAcknowledged for the vendor.
5. What happens when an order is under-funded? Don't we need an UnderFunded state?
6. What happens when an order is over-funded? Don't we need an OverFunded state?
7. I was under the impression that the buyer and vendor can bilaterally decide to use a different moderator. Is that still part of the protocol?
8. I was under the impression that the moderator could split the amount in escrow between buyer and vendor. DisputeResolved mentions "waiting for one of the parties". It reads as if there is waiting involved; what are the wait states (e.g. PayoutWaiting and PayoutAccepted)?
9. Is there any difference between "online" and "offline" orders? For example, in the offline mode, how do the buyer and seller agree on exchange rates?
10. In the offline mode, do buyers have to wait for the seller to come back online before funding the order? That sounds like poor UX.
11. As part of the state transitions, it would be useful to have the automatic timeouts (if any) documented.

Instead of ~/.OpenBazaar2 please use ~/Library/Application Support for OS X

Instead of ~/.OpenBazaar2 please use ~/Library/Application Support as the destination for the user folder in OS X.

Settings api attributes are capitalized.

In order to be consistent with our other APIs (and common API design), can we please not capitalize the attributes on the settings API (and any other API that capitalizes them)?