interledger-deprecated / five-bells-ledger Goto Github PK

View Code? Open in Web Editor NEW

27.0 27.0 21.0 2.38 MB

Open-source reference ledger optimized for use with the Interledger protocol

License: Other

JavaScript 96.69% Shell 0.74% PLpgSQL 2.57%

ledger

five-bells-ledger's People

Stargazers

Watchers

Forkers

matthewphinney mrajvanshy vhpoet alandotcom naoitoi gip austbot nocenter brettmollin mwshortt scratch28 mduo13 destenson dappelt jancarloviray andreasp1994 cross-chain-deprecated mwveliz

five-bells-ledger's Issues

Automatically upload docs

The API documentation npm run apidoc should automatically be uploaded to Github Pages.

Setting LEDGER_ADMIN_PASS should update the admin user's password

If the admin user already exists, LEDGER_ADMIN_PASS should update their password.

Prevent duplicate subscriptions

Multiple subscriptions with the same event, subject and target should be rejected with a 422 error code.

Use binary search tree for TimeQueue

TimeQueue uses a priority queue as the underlying data structure, but exposes includes and remove methods in the API. Queues are not meant to support these methods, and they will run in linear time. We should instead use a self balancing binary search tree, such as a red-black BST (https://github.com/vadimg/js_bintrees), which supports max, min, includes, insert, and delete in log time, and also provides an iterator over the elements.

Switch to PBKDF2 for password hashing

Following the recommendations from OWASP we can use PBKDF2 for password hashing.

This would allow us to remove the dependency on the compiled bcrypt module. We don't need to include another module, since Node ships with PBKDF2 support.

Require auth for GET /accounts/:id

GET /accounts/:id should require auth either from the user (:id) or from an admin. I.e. you can see only your own balance unless you're an admin. Then you can see anyone's balance.

Document LEDGER_ADMIN_(USER|PASS)

These config options should be documented in the README.

LEDGER_ADMIN_USER should default to 'admin'

The idea is that typically you'd only ever set LEDGER_ADMIN_PASS.

Authorized credits

There is a common use case we've talked about which is: As a recipient I want to approve all incoming transfers.

For example:

As a bank I don't want transfers incoming on the ILP level that Ripple Connect doesn't expect.
As an ILP-enabled wallet provider I don't want transfers incoming on the ILP level that don't have appropriate metadata for a higher-level protocol, such as sender information.

Functional specification

To allow for this, we add a new field to credits called authorized. Like the authorized field on debits, it can only be set by the owner of the account that that credit references (or an admin.) If this feature is turned on, a transfer cannot progress to prepared unless all debits and credits are authorized.

The feature is disabled by default (to avoid a breaking change) and can be enabled globally with an environment variable LEDGER_FEATURE_CREDIT_AUTH=true.

Three-layer Architecture

Split code into three layers:

Service layer: HTTP server and HTTP request/response handling
Domain layer: Business logic
Persistence layer: Database access
See: https://en.wikipedia.org/wiki/Multilayered_architecture

Currently there is not a clean segregation of responsibilities; transfers.putResource does all three in a single function. The directory structure can be refactored to:

/src/server - HTTP router, route handlers, HTTP parameter extraction, HTTP response formatting, converting return values and exceptions to HTTP status codes, etc
/src/domain - Business logic
/src/data - Database access

Layer 1 can depend on layer 2, layer 2 can depend on layer 3, but no other dependencies should be allowed.

Not all environment variables are documented

LEDGER_ED25519_SECRET is not in readme

Add escrow account

Currently, when a transfer is prepared, we apply debits to the source account. But we don't apply any corresponding credits right away. In other words, the money "disappears" from the ledger. To be correct from a pure accounting perspective, debits and credits must always be applied together.

In other words, when a transfer is prepared we should apply credits to an "escrow" accounts (and create corresponding Entry rows in the database.) And when that transfer is executed or rejected we should apply debits to the escrow account.

Also in the code, debits and credits should always be applied together, i.e. by the same function.

DB Issue on startup

five-bells-ledger » LEDGER_DB_URI=sqlite://:memory: node app.js
2016-01-13T21:47:34.959Z expiry monitor DEBUG checking for transfers to expire
2016-01-13T21:47:34.976Z db DEBUG Executing (default): PRAGMA busy_timeout = 0;
2016-01-13T21:47:34.981Z db DEBUG Executing (default): PRAGMA journal_mode = WAL;
2016-01-13T21:47:34.982Z db DEBUG Executing (default): PRAGMA synchronous = off;
2016-01-13T21:47:34.989Z db DEBUG Executing (default): SELECT `primary`, `name`, `balance`, `connector`, `password`, `public_key`, `is_admin`, `created_at`, `updated_at` FROM `Accounts` AS `Account` WHERE `Account`.`name` = 'hold' LIMIT 1;
2016-01-13T21:47:34.993Z app CRIT SequelizeDatabaseError: SQLITE_ERROR: no such table: Accounts
    at Query.formatError (/Users/alan/Projects/bundle/node_modules/five-bells-ledger/node_modules/sequelize/lib/dialects/sqlite/query.js:328:14)
    at afterExecute (/Users/alan/Projects/bundle/node_modules/five-bells-ledger/node_modules/sequelize/lib/dialects/sqlite/query.js:100:29)
    at replacement (/Users/alan/Projects/bundle/node_modules/five-bells-ledger/node_modules/sqlite3/lib/trace.js:20:31)
    at Statement.errBack (/Users/alan/Projects/bundle/node_modules/five-bells-ledger/node_modules/sqlite3/lib/sqlite3.js:16:21)
2016-01-13T21:47:35.982Z db DEBUG Executing (default): SELECT `id`, `subscription_id`, `transfer_id`, `retry_count`, `retry_at`, `created_at`, `updated_at` FROM `Notifications` AS `Notification` WHERE (`Notification`.`retry_at` IS NULL OR `Notification`.`retry_at` < '2016-01-13 21:47:35.976 +00:00');

Implement replyToTransfer functionality for better connector error handling

Right now if you send a transfer to a connector and the connector doesn't like it, you won't hear anything back. We should implement the replyToTransfer feature (if this is going to be supported by the ledger as opposed to implemented purely on the plugin level) to improve this type of error handling.

Add pretend_state to `/transfers/:id/state` for preimage algorithms

Notification IDs are leaking information

After #80, notification IDs will be public. The notification ID is currently a serial integer. This means it's leaking information about the number of notifications that are happening.

We should change that to some kind of unique ID. Could be a UUID or a shorter time-based ID like Snowflake.

Implement signature verification and condition verification in PUT /transfers/:id

Signature verification:
https://github.com/interledger/five-bells-ledger/blob/e750a7c7339c9a3f679a5479236ff0e5ca4f05e7/src/controllers/transfers.js#L372

Condition verification:
https://github.com/interledger/five-bells-ledger/blob/e750a7c7339c9a3f679a5479236ff0e5ca4f05e7/src/controllers/transfers.js#L399

Notifications should have their own ID

Current:

{
  "id": "http://eur-ledger.example/subscriptions/52a42d6f-8d9c-4c05-b31c-cccc8bbdb30d",
  "event": "transfer.update",
  "resource": {
    // ...
  }
}

Proposed:

{
  "id": "http://eur-ledger.example/subscriptions/52a42d6f-8d9c-4c05-b31c-cccc8bbdb30d/notifications/140",
  "subscription": "http://eur-ledger.example/subscriptions/52a42d6f-8d9c-4c05-b31c-cccc8bbdb30d",
  "event": "transfer.update",
  "resource": {
    // ...
  }
}

Edit: Changed notification URI to start with subscription URI.

Dead code due to condition being always false

The !subscriptions expression at https://github.com/interledger/five-bells-ledger/blob/master/src/lib/notificationWorker.js#L89 always evaluate to false since subscriptions is an array in this context. Easy to fix but not sure of the side-effects.

    let subscriptions = yield transaction.from('subscriptions')
      .whereIn('subject', affectedAccountUris)
      .whereIn('event', ['transfer.update', 'transfer.*', '*'])
      .select().then()
    if (!subscriptions) {
      return
    }

Add ability for admin to disable accounts

Since account deletion would rewrite history, the alternative is to disable accounts. Disabled accounts are not able to send or receive transfers. They do not count as valid users who can authenticate. Without authentication they are unable to see their own balance, or change any properties of their account. Disabled accounts should not be included in the planned /connectors endpoint.

Existing transfers involving disabled accounts in their debits or credits may not be prepared. Transfers that are already prepared should complete normally, even when they involve disabled accounts.

Only admins can disable or re-enable accounts.

Generate Swagger docs from api-doc

It would be useful to make swagger docs for our API available so that it's easy to generate client code for the ledger

API Docs - Put account - error if no body sent

Received error on put account when no body was sent, as specified in API docs. Added body {"name":"brett"} and account was created.

branch = master
API docs: https://interledger.org/five-bells-ledger/apidoc/#api-Account-PutAccount

Error
Request:
curl -X PUT -H "Authorization: Basic YWRtaW46dGVzdGluZw==" -H "Content-Type: application/json" -d '' "http://localhost:3000/accounts/brett"

Response:
{
"id": "InvalidBodyError",
"message": "Body did not match schema Account",
"validationErrors": [
{
"message": "Missing required property: name",
"params": {
"key": "name"
},
"code": 302,
"dataPath": "",
"schemaPath": "/required/0",
"subErrors": null,
"stack": "Error\n at new ValidationError (/Users/brett/five-bells-ledger/node_modules/tv4/tv4.js:1461:12)\n at ValidatorContext.createError (/Users/brett/five-bells-ledger/node_modules/tv4/tv4.js:359:14)\n at ValidatorContext.validateObjectRequiredProperties (/Users/brett/five-bells-ledger/node_modules/tv4/tv4.js:993:22)\n at ValidatorContext.validateObject (/Users/brett/five-bells-ledger/node_modules/tv4/tv4.js:960:11)\n at ValidatorContext.validateAll (/Users/brett/five-bells-ledger/node_modules/tv4/tv4.js:603:11)\n at Object.api.validateMultiple (/Users/brett/five-bells-ledger/node_modules/tv4/tv4.js:1597:12)\n at /Users/brett/five-bells-ledger/node_modules/five-bells-shared/lib/validator.js:81:31\n at Object.setAccount (/Users/brett/five-bells-ledger/src/models/accounts.js:66:55)\n at next (native)\n at onFulfilled (/Users/brett/five-bells-ledger/node_modules/co/index.js:65:19)\n at /Users/brett/five-bells-ledger/node_modules/co/index.js:54:5\n at Object.co (/Users/brett/five-bells-ledger/node_modules/co/index.js:50:10)\n at Object.toPromise (/Users/brett/five-bells-ledger/node_modules/co/index.js:118:63)\n at next (/Users/brett/five-bells-ledger/node_modules/co/index.js:99:29)\n at onFulfilled (/Users/brett/five-bells-ledger/node_modules/co/index.js:69:7)\n at /Users/brett/five-bells-ledger/node_modules/co/index.js:54:5\n at Object.co (/Users/brett/five-bells-ledger/node_modules/co/index.js:50:10)\n at Object.toPromise (/Users/brett/five-bells-ledger/node_modules/co/index.js:118:63)\n at next (/Users/brett/five-bells-ledger/node_modules/co/index.js:99:29)\n at onFulfilled (/Users/brett/five-bells-ledger/node_modules/co/index.js:69:7)\n at process._tickCallback (internal/process/next_tick.js:103:7)"
}
]
}

Worked

Request:
curl -X PUT -H "Authorization: Basic YWRtaW46dGVzdGluZw==" -H "Content-Type: application/json" -d '{"name":"brett"}' "http://localhost:3000/accounts/brett"

Response
{
"name": "brett",
"id": "http://brett-lm:3000/accounts/brett",
"balance": "NaN",
"minimum_allowed_balance": "0"
}

Handle multiple transfers submitted at the same time

See interledger-deprecated/five-bells-shared#64

Ability to request back notifications

When sending out notifications, the ledger should also allow GET requests back at an URL of the form GET /subscriptions/:id/notifications/:id. This provides another way for a client to check if a notification is authentic, other than HTTP-Signature (#81) which may be difficult to verify in some languages where no preexisting implementation is available.

Notifications are already persisted in the database. For now we can leave that as is: The rule could be, a client who is verifying a notification must request it before they respond to the notification POST request. That way, the database row will still exist. In the future we may want to persist notifications longer and clean them up after some delay.

Request:

GET /subscriptions/52a42d6f-8d9c-4c05-b31c-cccc8bbdb30d/notifications/140

Response:

{
  "id": "http://eur-ledger.example/subscriptions/52a42d6f-8d9c-4c05-b31c-cccc8bbdb30d/notifications/140",
  "subscription": "http://eur-ledger.example/subscriptions/52a42d6f-8d9c-4c05-b31c-cccc8bbdb30d",
  "event": "transfer.update",
  "resource": {
    // ...
  }
}

Interledger setup

Hi,

I would like to try Interledger with my custom ledgers. Could you, please, clarify how to start?
I wanna run something like red.ilpdemo.org + wallet in private net.

In my opinion, I should use ilp-client to make a payments (and receive payments) from client side. Also, I need a private Interleger with five-bells-connector and five-bells-ledger to authorize clients and transfer assets. But what is ilp-core, ilp-plugin-bells, five-bells-sender and other. I little bit confused how to use it all together, and what code I need to implement to work with Interledger.

Thanks advance.

EntryGroups key is too long

Error when running the migrations on MySQL:

SQL query:

ALTER TABLE `Entries` ADD INDEX `idx_entry_groups` (`account`, `entry_group`)

MySQL said:

#1071 - Specified key was too long; max key length is 767 bytes

The issue is that account is a 1024-byte varchar field. We should have a smaller integer ID as the primary key for accounts which the database can use. That will avoid this error and be much more space-efficient.

Can't subscribe to /accounts/*/transfers

Schema validation fails here on a *.

Enable ledger to be looked up from account identifier

Right now when doing pathfinding we need to pass source_account, source_ledger and source_username, even though source_account combines the two other pieces of information. We're supposed to treat URIs as opaque so we can't assume that we can parse the username or ledger from the source_account.

My proposal would be to allow anyone (without authentication) to GET the account URI and return an object that contains the ledger and account.

One thing that would enable would be that your "account" that you share with people could actually be a service not tied to your ledger that would return your up to date payment info. If the standard behavior was for people to GET that endpoint first, you'd be able to update what that points to whenever you want to switch where you hold your money. This would be very useful for cases when you want to embed payment information in some data that can't be easily updated when you move accounts.

The first potential problem with this is that that endpoint would become a target for hackers because if they could change where that points to they could redirect where your money goes.

The second potential problem is that you might not want anyone to be able to find out if an account exists. That could be addressed by adding functionality to the ledgers to enable account holders to create aliases for their account that they could give to specific parties.

Ledger notifications should contain the resulting balance

When the ledger sends out notifications about transfers, it would be useful to include the old and new balances, similar to "metadata" in rippled.

Let's use this issue to discuss what the JSON format should look like.

Implement auth for subscriptions

We need to implement a permissions scheme so that only users who own an account can subscribe to events for that account. For the code, see here.

@justmoon Should admin users be allowed to subscribe to events on any account?

Transfer to self

User asd is doing a transfer of 1000 to himself. Screenshot is the before/after database.

Logs

2015-12-08T18:54:44.293Z koa INFO   <-- PUT /transfers/96d7b518-9571-473c-8c99-b35ab9c2c588
2015-12-08T18:54:44.316Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'asd' LIMIT 1;
2015-12-08T18:54:44.321Z transfers DEBUG putting transfer ID 96d7b518-9571-473c-8c99-b35ab9c2c588
2015-12-08T18:54:44.321Z transfers DEBUG asd -> asd : 1000
2015-12-08T18:54:44.322Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): START TRANSACTION;
2015-12-08T18:54:44.322Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ;
2015-12-08T18:54:44.323Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): SET autocommit = 1;
2015-12-08T18:54:44.324Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): SELECT "id", "ledger", "debits", "credits", "part_of_payment", "state", "execution_condition", "execution_condition_fulfillment", "cancellation_condition", "cancellation_condition_fulfillment", "expires_at", "proposed_at", "pre_prepared_at", "prepared_at", "pre_executed_at", "executed_at", "rejected_at", "created_at", "updated_at" FROM "Transfers" AS "Transfer" WHERE "Transfer"."id" = '96d7b518-9571-473c-8c99-b35ab9c2c588';
2015-12-08T18:54:44.326Z updateState DEBUG updating transfer state from undefined to proposed
2015-12-08T18:54:44.341Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'asd' LIMIT 1;
2015-12-08T18:54:44.346Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'asd' LIMIT 1;
2015-12-08T18:54:44.348Z updateState DEBUG updating transfer state from proposed to pre_prepared
2015-12-08T18:54:44.353Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'hold' LIMIT 1;
2015-12-08T18:54:44.359Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): INSERT INTO "EntryGroups" ("updated_at","created_at") VALUES ('2015-12-08 18:54:44.358 +00:00','2015-12-08 18:54:44.358 +00:00') RETURNING *;
2015-12-08T18:54:44.361Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'asd' LIMIT 1;
2015-12-08T18:54:44.362Z account balances DEBUG sender asd balance: 3001 -> 2001
2015-12-08T18:54:44.364Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): INSERT INTO "Entries" ("entry_group","transfer_id","account","balance","updated_at","created_at") VALUES (95,'96d7b518-9571-473c-8c99-b35ab9c2c588',2,2001,'2015-12-08 18:54:44.363 +00:00','2015-12-08 18:54:44.363 +00:00') RETURNING *;
2015-12-08T18:54:44.368Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): UPDATE "Accounts" SET "balance"=2001,"updated_at"='2015-12-08 18:54:44.367 +00:00' WHERE "primary" = 2
2015-12-08T18:54:44.371Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): INSERT INTO "Entries" ("entry_group","transfer_id","account","balance","updated_at","created_at") VALUES (95,'96d7b518-9571-473c-8c99-b35ab9c2c588',22,-1850,'2015-12-08 18:54:44.370 +00:00','2015-12-08 18:54:44.370 +00:00') RETURNING *;
2015-12-08T18:54:44.374Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): UPDATE "Accounts" SET "balance"=-1850,"updated_at"='2015-12-08 18:54:44.373 +00:00' WHERE "primary" = 22
2015-12-08T18:54:44.375Z updateState DEBUG updating transfer state from pre_prepared to prepared
2015-12-08T18:54:44.375Z updateState DEBUG updating transfer state from prepared to pre_executed
2015-12-08T18:54:44.376Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'hold' LIMIT 1;
2015-12-08T18:54:44.378Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): INSERT INTO "EntryGroups" ("updated_at","created_at") VALUES ('2015-12-08 18:54:44.377 +00:00','2015-12-08 18:54:44.377 +00:00') RETURNING *;
2015-12-08T18:54:44.380Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'asd' LIMIT 1;
2015-12-08T18:54:44.382Z account balances DEBUG recipient asd balance: 3001 -> 4001
2015-12-08T18:54:44.386Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): INSERT INTO "Entries" ("entry_group","transfer_id","account","balance","updated_at","created_at") VALUES (96,'96d7b518-9571-473c-8c99-b35ab9c2c588',2,4001,'2015-12-08 18:54:44.383 +00:00','2015-12-08 18:54:44.383 +00:00') RETURNING *;
2015-12-08T18:54:44.395Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): UPDATE "Accounts" SET "balance"=4001,"updated_at"='2015-12-08 18:54:44.391 +00:00' WHERE "primary" = 2
2015-12-08T18:54:44.398Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): INSERT INTO "Entries" ("entry_group","transfer_id","account","balance","updated_at","created_at") VALUES (96,'96d7b518-9571-473c-8c99-b35ab9c2c588',22,-3850,'2015-12-08 18:54:44.397 +00:00','2015-12-08 18:54:44.397 +00:00') RETURNING *;
2015-12-08T18:54:44.405Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): UPDATE "Accounts" SET "balance"=-3850,"updated_at"='2015-12-08 18:54:44.403 +00:00' WHERE "primary" = 22
2015-12-08T18:54:44.406Z updateState DEBUG updating transfer state from pre_executed to executed
2015-12-08T18:54:44.406Z expiry monitor DEBUG unwatch transfer: 96d7b518-9571-473c-8c99-b35ab9c2c588
2015-12-08T18:54:44.415Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): INSERT INTO "Transfers" ("id","ledger","debits","credits","state","expires_at","proposed_at","pre_prepared_at","prepared_at","pre_executed_at","executed_at","updated_at","created_at") VALUES ('96d7b518-9571-473c-8c99-b35ab9c2c588','http://vahe-2.local:3000','[{"account":"asd","amount":"1000","authorized":true}]','[{"account":"asd","amount":"1000"}]','executed','2016-06-16 00:00:01.000 +00:00','2015-12-08 18:54:44.326 +00:00','2015-12-08 18:54:44.349 +00:00','2015-12-08 18:54:44.375 +00:00','2015-12-08 18:54:44.375 +00:00','2015-12-08 18:54:44.406 +00:00','2015-12-08 18:54:44.409 +00:00','2015-12-08 18:54:44.409 +00:00') RETURNING *;
2015-12-08T18:54:44.419Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): SELECT "id", "owner", "event", "subject", "target", "created_at", "updated_at" FROM "Subscriptions" AS "Subscription" WHERE (("Subscription"."event" = 'transfer.update' OR "Subscription"."event" = 'transfer.*' OR "Subscription"."event" = '*') AND "Subscription"."subject" IN ('http://vahe-2.local:3000/accounts/asd', 'http://vahe-2.local:3000/accounts/asd', '*'));
2015-12-08T18:54:44.422Z db DEBUG Executing (1c0a232a-b2b3-4342-bdbe-f2a380a173d9): COMMIT;
2015-12-08T18:54:44.425Z transfers DEBUG changes written to database
2015-12-08T18:54:44.425Z notificationWorker DEBUG scheduling notifications
2015-12-08T18:54:44.426Z koa INFO   --> PUT /transfers/96d7b518-9571-473c-8c99-b35ab9c2c588 201 133ms -
2015-12-08T18:54:44.429Z db DEBUG Executing (default): SELECT "id", "subscription_id", "transfer_id", "retry_count", "retry_at", "created_at", "updated_at" FROM "Notifications" AS "Notification" WHERE ("Notification"."retry_at" IS NULL OR "Notification"."retry_at" < '2015-12-08 18:54:44.427 +00:00');

Test endpoint responses against JSON schemas in unit tests

In each REST-level unit test, test the response against the schemas in: https://github.com/interledger/five-bells-shared/tree/master/schemas

Connectors and transaction states

Hello,

I read more detailed information about connectors and I have couple of questions.

Who are manage connectors?
I mean, is it predefined connectors by Interledger administrator? Or every clients who are implemented Ledger plugin could be a connector? Who is determine relationships between connectors?
How other connectors will know about me?
E.g someone provided me Interledger network and I would like to use it to receive the assets. I have to have an account. Where is my account stored? Does any connectors have my account details?
Cryptography escrow.
Is it means that every transaction are stored on each connectors's ledger? What information are stored into the connector's ledger? Could I use the connector or Interledger to get transfer status if my client was stopped unexpectedly?

Thanks advance

Dedicated APIs for getting ledger's public key + hash to be used in hash pre-image scheme

Right now the /transfers/:id/state endpoint is used for this, which is pretty hacky. A dedicated endpoint would be preferable.

This is also used to get the hash that will be used in the hash pre-image signature scheme. That should also be separated into a different API. Theoretically that is provided by the recipient. Right now the ledger is acting as the recipient's agent. However, even if it continues to do so that behavior should be made more modular and separated out.

Automatic account creation should be moved into a class

This code:

https://github.com/interledger/five-bells-ledger/blob/47717ff9d2292cc7b03fe9de5d5d43f31c943463/app.js#L101-L121

Should be moved to a class.

Prevent transfer from and to the same account

App should be a class

This code:

https://github.com/interledger/five-bells-ledger/blob/47717ff9d2292cc7b03fe9de5d5d43f31c943463/app.js#L30-L128

Should live in a class called App, so that I can easily include the ledger as a module and still start it.

For an example, see the notary:
https://github.com/interledger/five-bells-notary/blob/db831cfabf49ef82947846ba87de019ff48c5c87/app.js
https://github.com/interledger/five-bells-notary/blob/ec60a2cff187f3bcb906fa3fd52826d65557108c/src/lib/app.js

Websocket Endpoint always returns 301

I'm trying to connect to the RED ledger via websocket, and it doesn't seem to be working using the command found in the docs.

I'm using:

wscat --auth dfuelling:not-my-real-password -c ws://red.ilpdemo.org/ledger/accounts/dfuelling/transfers

... and it always returns "Error: unexpected server response (301)"

Is this the right endpoint?

Support reliable notifications

We are about to remove reliable notification support via REST hooks. Do we need a new facility (e.g. using MQTT/mosca) to replace it?

Ability to turn on/off auth schemes

It would be good to turn off HTTP Basic auth entirely via the configuration.

LEDGER_AUTH_BASIC_ENABLED=0 - Turn off Basic auth
LEDGER_AUTH_HTTP_SIGNATURE_ENABLED=0 - Turn off http-signature based auth

Signed, timestamped receipt of fulfillment

When the ledger receives a fulfillment, the entity submitting it will want to know whether the event they were trying to trigger happened (e.g. the connector in Universal mode wants to know the source transfers were executed). There are two ways of achieving this:

the ledger could return in response all of the resources modified / events triggered by this fulfillment that the submitter is authorized to see
the ledger could return a signed, timestamped receipt of the fulfillment

@justmoon and I discussed this and agreed that the second option is the better, more general solution to the problem. If the submitter of a condition fulfillment gets a signed, timestamped receipt they can check that that the timestamp is before the expiry of whatever they wanted to trigger and be sure that the receiving system should trigger that event.

This means that the ledger must ensure that if it says that a fulfillment was received before the expiry time of a transfer that it fulfills, that transfer will be executed (even if the ledger only gets to processing the transfer / marking it as executed after the expiry time).

In the case of the connector in Universal mode submitting the fulfillments to the source transfers, the basic behavior is that the connector will check for the HTTP 200 status code and timestamp on the fulfillment it gets back from the ledger to make sure it is before the transfer's expires_at time. This gives the connector proof that the transfer will be executed.

If the connector wants an explicit acknowledgement that the specific transfer was executed, they can request the transfer resource from the ledger. To improve the performance of this, the ledger can use HTTP/2 Server Push to push the transfer resource to the connector after the transfer is processed and (potentially) before the connector requests it.

Notifications should be signed with HTTP-Signature

All outgoing notifications from the ledger should be signed with HTTP-Signature. This allows the recipient to easily verify their authenticity.

Delete (or deactivate?) transferExpiryMonitor

If one of the ledgers goes offline right before receiving the notary's message to execute and stays offline until after the expiration time, the sender or connector will lose funds because one escrow will go through and the other will be cancelled. In atomic mode, we can never let transfers expire in the ledger; only the notary can process expirations.

If we go with deleting, we can delete transferExpiryMonitor.js, timerWorker.js, and timeQueue.js; and updateState can be merged into controllers/transfers.js. Also, accountBalances.js then only depends on controllers/transfers.js, so it can be moved into a subdirectory of controllers to follow the weak layering principle. At that point we can move notificationWorker.js to services to eliminate the "lib" directory.

SQLite transaction compatibility code should be moved to five-bells-shared/lib/db.js

This code:

https://github.com/interledger/five-bells-ledger/blob/47717ff9d2292cc7b03fe9de5d5d43f31c943463/app.js#L85-L99

Should be moved to five-bells-shared/lib/db.js into a static generator method called init.

Hold account balance bug

This is the "Accounts" state before and after asd initiates a transfer of 1 to Bob. All cool, balances update correctly, except for the hold account which keeps going down.

Logs

2015-12-08T18:06:40.201Z koa INFO   <-- PUT /transfers/1272a287-1dc2-4e70-8d96-c97a3a115dfe
2015-12-08T18:06:40.206Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'asd' LIMIT 1;
2015-12-08T18:06:40.213Z transfers DEBUG putting transfer ID 1272a287-1dc2-4e70-8d96-c97a3a115dfe
2015-12-08T18:06:40.214Z transfers DEBUG asd -> bob : 1
2015-12-08T18:06:40.214Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): START TRANSACTION;
2015-12-08T18:06:40.215Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ;
2015-12-08T18:06:40.216Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): SET autocommit = 1;
2015-12-08T18:06:40.218Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): SELECT "id", "ledger", "debits", "credits", "part_of_payment", "state", "execution_condition", "execution_condition_fulfillment", "cancellation_condition", "cancellation_condition_fulfillment", "expires_at", "proposed_at", "pre_prepared_at", "prepared_at", "pre_executed_at", "executed_at", "rejected_at", "created_at", "updated_at" FROM "Transfers" AS "Transfer" WHERE "Transfer"."id" = '1272a287-1dc2-4e70-8d96-c97a3a115dfe';
2015-12-08T18:06:40.219Z updateState DEBUG updating transfer state from undefined to proposed
2015-12-08T18:06:40.221Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'asd' LIMIT 1;
2015-12-08T18:06:40.224Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'bob' LIMIT 1;
2015-12-08T18:06:40.226Z updateState DEBUG updating transfer state from proposed to pre_prepared
2015-12-08T18:06:40.228Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'hold' LIMIT 1;
2015-12-08T18:06:40.232Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): INSERT INTO "EntryGroups" ("updated_at","created_at") VALUES ('2015-12-08 18:06:40.231 +00:00','2015-12-08 18:06:40.231 +00:00') RETURNING *;
2015-12-08T18:06:40.234Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'asd' LIMIT 1;
2015-12-08T18:06:40.235Z account balances DEBUG sender asd balance: 532 -> 531
2015-12-08T18:06:40.237Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): INSERT INTO "Entries" ("entry_group","transfer_id","account","balance","updated_at","created_at") VALUES (78,'1272a287-1dc2-4e70-8d96-c97a3a115dfe',2,531,'2015-12-08 18:06:40.236 +00:00','2015-12-08 18:06:40.236 +00:00') RETURNING *;
2015-12-08T18:06:40.242Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): UPDATE "Accounts" SET "balance"=531,"updated_at"='2015-12-08 18:06:40.241 +00:00' WHERE "primary" = 2
2015-12-08T18:06:40.245Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): INSERT INTO "Entries" ("entry_group","transfer_id","account","balance","updated_at","created_at") VALUES (78,'1272a287-1dc2-4e70-8d96-c97a3a115dfe',22,-344,'2015-12-08 18:06:40.244 +00:00','2015-12-08 18:06:40.244 +00:00') RETURNING *;
2015-12-08T18:06:40.249Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): UPDATE "Accounts" SET "balance"=-344,"updated_at"='2015-12-08 18:06:40.248 +00:00' WHERE "primary" = 22
2015-12-08T18:06:40.250Z updateState DEBUG updating transfer state from pre_prepared to prepared
2015-12-08T18:06:40.250Z updateState DEBUG updating transfer state from prepared to pre_executed
2015-12-08T18:06:40.251Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'hold' LIMIT 1;
2015-12-08T18:06:40.253Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): INSERT INTO "EntryGroups" ("updated_at","created_at") VALUES ('2015-12-08 18:06:40.252 +00:00','2015-12-08 18:06:40.252 +00:00') RETURNING *;
2015-12-08T18:06:40.255Z db DEBUG Executing (default): SELECT "primary", "name", "balance", "identity", "password", "public_key", "is_admin", "created_at", "updated_at" FROM "Accounts" AS "Account" WHERE "Account"."name" = 'bob' LIMIT 1;
2015-12-08T18:06:40.257Z account balances DEBUG recipient bob balance: 1468 -> 1469
2015-12-08T18:06:40.259Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): INSERT INTO "Entries" ("entry_group","transfer_id","account","balance","updated_at","created_at") VALUES (79,'1272a287-1dc2-4e70-8d96-c97a3a115dfe',3,1469,'2015-12-08 18:06:40.258 +00:00','2015-12-08 18:06:40.258 +00:00') RETURNING *;
2015-12-08T18:06:40.268Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): UPDATE "Accounts" SET "balance"=1469,"updated_at"='2015-12-08 18:06:40.266 +00:00' WHERE "primary" = 3
2015-12-08T18:06:40.272Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): INSERT INTO "Entries" ("entry_group","transfer_id","account","balance","updated_at","created_at") VALUES (79,'1272a287-1dc2-4e70-8d96-c97a3a115dfe',22,-346,'2015-12-08 18:06:40.270 +00:00','2015-12-08 18:06:40.270 +00:00') RETURNING *;
2015-12-08T18:06:40.276Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): UPDATE "Accounts" SET "balance"=-346,"updated_at"='2015-12-08 18:06:40.275 +00:00' WHERE "primary" = 22
2015-12-08T18:06:40.277Z updateState DEBUG updating transfer state from pre_executed to executed
2015-12-08T18:06:40.277Z expiry monitor DEBUG unwatch transfer: 1272a287-1dc2-4e70-8d96-c97a3a115dfe
2015-12-08T18:06:40.282Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): INSERT INTO "Transfers" ("id","ledger","debits","credits","state","expires_at","proposed_at","pre_prepared_at","prepared_at","pre_executed_at","executed_at","updated_at","created_at") VALUES ('1272a287-1dc2-4e70-8d96-c97a3a115dfe','http://vahe-2.local:3000','[{"account":"asd","amount":"1","authorized":true}]','[{"account":"bob","amount":"1"}]','executed','2016-06-16 00:00:01.000 +00:00','2015-12-08 18:06:40.219 +00:00','2015-12-08 18:06:40.227 +00:00','2015-12-08 18:06:40.250 +00:00','2015-12-08 18:06:40.250 +00:00','2015-12-08 18:06:40.277 +00:00','2015-12-08 18:06:40.278 +00:00','2015-12-08 18:06:40.278 +00:00') RETURNING *;
2015-12-08T18:06:40.286Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): SELECT "id", "owner", "event", "subject", "target", "created_at", "updated_at" FROM "Subscriptions" AS "Subscription" WHERE (("Subscription"."event" = 'transfer.update' OR "Subscription"."event" = 'transfer.*' OR "Subscription"."event" = '*') AND "Subscription"."subject" IN ('http://vahe-2.local:3000/accounts/asd', 'http://vahe-2.local:3000/accounts/bob', '*'));
2015-12-08T18:06:40.287Z db DEBUG Executing (11d1aca3-6d78-4e24-9f07-3104a498bd24): COMMIT;
2015-12-08T18:06:40.288Z transfers DEBUG changes written to database
2015-12-08T18:06:40.289Z notificationWorker DEBUG scheduling notifications
2015-12-08T18:06:40.290Z koa INFO   --> PUT /transfers/1272a287-1dc2-4e70-8d96-c97a3a115dfe 201 89ms -
2015-12-08T18:06:40.292Z db DEBUG Executing (default): SELECT "id", "subscription_id", "transfer_id", "retry_count", "retry_at", "created_at", "updated_at" FROM "Notifications" AS "Notification" WHERE ("Notification"."retry_at" IS NULL OR "Notification"."retry_at" < '2015-12-08 18:06:40.290 +00:00');

LEDGER_DB_SYNC issue

Restarting the ledger that has a LEDGER_DB_SYNC environment variable throws an exception

CREATE INDEX "L_XPK_ACCOUNTS" ON "L_ACCOUNTS"
  ("ACCOUNT_ID" ASC) - relation "L_XPK_ACCOUNTS" already exists

P. S. LEDGER_DB_SYNC is not documented

CircleCI tests should test against MySQL, Postgres and SQLite

We keep running into errors that only appear with certain databases. Let's improve the CI tests to run the whole suite on these three types of database:

MySQL
Postgres
SQLite

interledger-deprecated / five-bells-ledger Goto Github PK

five-bells-ledger's People

Stargazers

Watchers

Forkers

five-bells-ledger's Issues

Functional specification

Recommend Projects

Recommend Topics

Recommend Org