golemfactory / golem-client

Client for Brass Golem Network protocol.

golem-client

Client for Brass Golem Network.

Subprojects

• actix-wamp (api docs) - Asynchronous client library for WAMP.

• golem-rpc-api (api docs) - Typesafe binding for Brass Golem RPC services.

• golem-rpc-api-macro - Procedural macros for binding Brass Golem RPC endpoints.

• golemcli - command line interface for Brass Golem (re)implemented in Rust.

GolemCLI

Install or Upgrade

On Unix:

curl -sSf https://golemfactory.github.io/golem-client/install/golemcli-update.sh | bash

Compilation prerequisites (Windows)

This project builds under Windows (validated under VC toolchain), after following prerequisites are installed:

• rustup upgrade - get latest stable Rust toolset
• Install Perl (eg. https://www.activestate.com/products/activeperl/downloads/)
• Install OpenSSL (https://slproweb.com/products/Win32OpenSSL.html)

golem-client's Issues

change subcommand name: res -> cache

res command looks like an unnecessary command in the main command list. It can be also confused with setting amount of resources for the provider.
Better description of res - Manage resource storage

mainnet/testnet info

In regular golemcli you need to type golemcli debug rpc golem.mainnet in order to know whether golem node is connected to mainnet or testnet.

1. There should be more intuitive way to check whether golem node is connected to mainnet or testnet.
2. I think it would be nice to have such info also in welcome message.

golemcli subtask restart not working

Separate `exit` interactive shell from main commands (new line)

golemcli status

Implement provider info and print
provider_state: {'status': 'Idle'}

TODO: 360 view about golem node.

Running status

• Process is running/stoped/stoping (on shutdown?)
• Components is Starting/Running/Failed
• mainet / testnet
• golem version
• node name
• disk usage

Network status

• is connected
• ports status
• number of nodes in the network

Account status

• GNT / ETH
• 0x1308f7345c455ED528bC80C37C7EC175Abe502B4

Providers status

• is task working ?

• processing status

• pending payments

• Subtasks accepted (in session): 0

• Subtasks rejected (in session): 0

• Subtasks failed (in session): 0

• Subtasks computed (in session): 0

• Subtasks in network: 0

Requestor status

• is any task active?
• tasks progress

CLI concent section

Commands:

• switch
• terms

CLI tasks section

Commands:

• abort Abort a task
• create Create a task from file. Note: no client-side validation is performed yet. This will change in the future
• delete Delete a task
• dump Dump an existing task
• purge Deletes all tasks
• restart Restart a task
• restart_subtasks Restart given subtasks from a task
• show Show task details
• stats Show statistics for tasks
• subtasks Show sub-tasks
• template Dump a task template
• unsupport Show statistics for unsupported tasks

Other tasks:

• initial integration tests

Show dates for incomes

As an end user i want to know when a payment was received so i can get a better picture of when the task was payed.

From public chat we where investigating some "missing" payments, now we used the debug rpc call to get the dates, would be nice to have golemcli incomes ... also print the end ( and start? ) dates of the payments

missing subtasks_count in tasks template

golem-rpc-api: make data_dir optional

connect_to_app currently requires the data dir path to be provided to it. It can be made optional and have the default use the same logic as in golemcli (CliArgs#get_data_dir).

This would allow for removing duplicate code and simplify the API's usage from consuming apps.

CLI incomes section

• add --full flag, and present public key
• in not full mode, present ETH address

original sample json (not correct, bc of nil)

[
  [
    {
      "subtask": "1e12a7e4-50a3-11e9-9521-1bac4bb5328e",
      "payer": "1bac4bb5328ec9fc70b336e7c720dbb0c9cd23b8782c8bfc66f2a946ab036b8b6b71775cb04ad08d606f652ba476d9692c77cbd135f534d68f54c12e3edc039a",
      "value": "33333333333333334",
      "status": "awaiting",
      "transaction": nil,
      "created": 1553699819.286437,
      "modified": 1557242132.961853
    },
...
    {
      "subtask": "b5bf888a-4fba-11e9-a993-1bac4bb5328e",
      "payer": "1bac4bb5328ec9fc70b336e7c720dbb0c9cd23b8782c8bfc66f2a946ab036b8b6b71775cb04ad08d606f652ba476d9692c77cbd135f534d68f54c12e3edc039a",
      "value": "10",
      "status": "confirmed",
      "transaction": "8062f4dccade73a728589f9e0db65792a6551b65d30f9fea1968a13d578052ba",
      "created": 1553599935.011557,
      "modified": 1557242132.963763
    },

rendered as

payer               status    value
------------------  --------  ---------------
1bac4bb5..3edc039a  awaiting  0.03333333 GNT
...
1bac4bb5..3edc039a  confirmed  0.00000000 GNT
1bac4bb5..3edc039a  confirmed  0.01877905 GNT

Can we publish new releases of golem-rpc-api and golem-rpc-macro?

The latest versions of the above crates are 0.2.0 while in crates.io only 0.1.0 are available. Could anyone with owner privs for those crates publish the latest crates to crates.io? This is very desirable since 0.2.0 migrated the API to futures 0.3.

cc @tworec @prekucki @etam

CLI account withdraw - should require confirmation

upgrade for #2
resolves golemfactory/golem#3984

run benchmark from cli is not working

./golemcli.py envs recount BLENDER
RPC: call error: <twisted.python.failure.Failure autobahn.wamp.exception.ApplicationError: ApplicationError(error=<wamp.error.runtime_error>, args=['expected str, bytes or os.PathLike object, not NoneType'], kwargs={}, enc_algo=None)>
[10:01:12.117877] ERROR: Exception: ApplicationError(error=<wamp.error.runtime_error>, args=['expected str, bytes or os.PathLike object, not NoneType'], kwargs={}, enc_algo=None)

New password handling

Test 1

Type weak password

$ ./golemcli.py account unlock
No account found, generate one by setting a password
This command will time out in 30 seconds.
Password:
Password is not strong enough. Please use capitals, numbers and special characters.

TODO:

use https://github.com/shssoichiro/zxcvbn-rs min password length is 5 and min password score is 2

account_name = getpass.getuser() or ''
            result = zxcvbn.zxcvbn(pswd, user_inputs=['Golem', account_name])
            # print(result['score'])
            if result['score'] < MIN_SCORE:
                return "Password is not strong enough. " \
                    "Please use capitals, numbers and special characters."

Test 2

Catch password confirmation.

No account found, generate one by setting a password
This command will time out in 30 seconds.
Password:
Confirm password:
Password and confirmation do not match.

Test 3

Password catch timeout

$ ./golemcli.py account unlock
No account found, generate one by setting a password
This command will time out in 30 seconds.
Password:[11:36:15.569613] ERROR: Command timed out

add pub/sub support for actix-wamp

golemcli rpc on locked account

• env.hw.caps
• env.hw.preset
• env.hw.presets
• env.hw.virtualization
• golem.mainnet
• golem.password.key_exists
• golem.password.set
• golem.password.unlocked
• golem.terms
• golem.terms.accept
• golem.terms.show
• golem.version

Detect mainnet or testnet

Using HTTPS cert check if given connection is for mainnet or testnet.

Original Issue

When running golemapp --mainnet the command returned by the terminal is Waiting for a password to unlock the account. Run golemcli account unlock and enter your password. Which is untrue for the mainnet version, as it should state Waiting for a password to unlock the account Run golemcli --mainnet account unlock and enter your password. The current command that is shown by CLI is valid only for the testnet version

• Either change the info as above
• Or detect for a user which version he is running and don’t require --mainnet flag for any commands in the command line interface after previously entering testnet or mainnet

Incomes and Payments filtering

Incomes and Payments should have additional arguments printed out at first, as it is not obvious that user can filter them with additional commands

Incomes

• All
• Awaiting
• Confirmed
• Total (or just print sum under each above)

Payments

As above

golemcli Concent

concent on
concent off
concent status
concent terms (show|accept|status)

concent deposit payments - lists payments
concent deposit - show balance

incomes: unknown variant arbitraged_by_concent

golem version: 0.22.1
cli version: 0.3.0

listing incomes fails with:

unknown variant `arbitraged_by_concent`, expected one of `awaiting`, `sent`, `confirmed`, `overdue`
err: ProcessingError { context: "", cause: Error("unknown variant `arbitraged_by_concent`, expected one of `awaiting`, `sent`, `confirmed`, `overdue`", line: 0, column: 0) }

CLI deposit_payments section

Tasks:

• generate table
• tests

golemcli network blacklist

Implement option to unblock node, as currently there is only option to block someone. Unblocking must be done manually by user in his local files

# - shows blacklist
golemcli network blacklist 
# added removes from blacklist
golemcli network blacklist { --add <node_id> | --del <node_id> }

Align golemcli with feedback from UI/UX team

We need to plan our work according to

• golemfactory/golem#3015 command feedback improvements based on @jamuszyn's report
• golemfactory/golem#3984 account withdraw confirmation
• and last but most important and much broader @jamuszyn's feedback (protected doc) >>

REST Spec (align swagger with latest docs)

use users directory for bash-completion

See "Where should I install my own local completions?" at https://github.com/scop/bash-completion#faq

If bash-completion is installed (this needs to be checked what's the default for supported distributions), then the installer script should put completion in users directory, not global.

add "are you sure" in golemcli acl setup

Add rpc calls to `apps.*`

For the task-api i have added new rpc commands to golem core in https://github.com/golemfactory/golem/pull/5103

Please add these calls to golemcli, i think the mapping can be 1-1:

• golemcli apps list => apps.list
• etc..

Account shutdown should be moved to the main command just above help

golemcli subtasks moved to tasks

subtasks from main command list are duplicated from tasks command line. And it seems that they should be removed, as they lack all the commands that are needed (available in tasks commands)

TODO: consider tasks subtasks vs tasks subtask for single suntasks details

CLI: tasks subtasks: remove ETA

related to https://github.com/golemfactory/golem/issues/2562

actix-wamp caller role impl

Spec: https://wamp-proto.org/_static/gen/wamp_latest.html
Scope: caller role, with wamp-cra from the advanced profile
Supported messages:

• Session handling: HELLO, WELCOME, ABORT, GOODBYE
• Authentication: CHALLENGE, AUTHENTICATE, ERROR
• RPC calls: CALL, RESULT, ERROR

Features:

• basic support
• error handling
• ssl certificate fingerprint detection

CLI network section

Commands:

• block - Block provider
• connect - Connect to a node
• dht - Show known nodes
• show - Show connected nodes
• status - Show client status

Other tasks:

• tests

CLI account section

• basic version
• tests

Parse Error Responses

ERROR [golem_rpc_api::rpc] on comp.task.create unable to parse: [Array([Null, Object({"error_details": Object({"missing_funds": Array([Object({"available": Number(0), "currency": String("GNT"), "required": Number(1000000000000000000)}), Object({"available": Number(0), "currency": String("ETH"), "required": Number(652000000000000)})])}), "error_msg": String("Not enough funds available.\nRequired GNT: 1.000000, available:
0.000000\nRequired ETH: 0.000652, available: 0.000000\n"), "error_type": String("NotEnoughFunds")})])]: invalid type: map, expected a string
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ProcessingError { context: "", cause: Error("invalid type: map, expected a string", line: 0, column: 0) }', src/libcore/result.rs:997:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

golemcli presets support

Command line

Showing the current resource configuration.

$ golemcli res show
---
cpu_cores: 2
disk: 306M
memory: 1.2G

Updateing resource configuration.

$ golemcli res update --cores 3 --mem 2g
---
cpu_cores: 2
disk: 306M
memory: 2G

golemcli concent deposit - error on null deposit.

$ golemcli concent deposit 
ERROR [golem_rpc_api::rpc] on pay.deposit_balance unable to parse: [Null]: invalid type: null, expected struct DepositBalance
ERROR [golemcli] : invalid type: null, expected struct DepositBalance

DEBUG [actix_wamp::connection] send message [48, 6080252602300624, {}, "pay.deposit_balance"]
TRACE [actix_wamp::connection] got message =[50, 6080252602300624, {}, [nil]]

WASM task JSON template has incorrect structure

Template JSON:

{
  "type": "WASM",
  "compute_on": "cpu",
  "name": "simple wasm",
  "timeout": "0:10:00",
  "subtask_timeout": "0:06:40",
  "bid": 0.1,
  "resources": [],
  "concent_enabled": false,
  "subtasks_count": 1,
  "options": {
    "js_name": "",
    "wasm_name": "",
    "input_dir": "",
    "output_dir": "",
    "subtasks": [
      {
        "name": "",
        "exec_args": [],
        "output_file_paths": []
      }
    ]
  }
}

How it should like (ignore values):

{
    "type": "wasm",
    "name": "flite",
    "bid": 1,
    "subtask_timeout": "00:10:00",
    "timeout": "00:10:00",
    "options": {
        "js_name": "flite.js",
        "wasm_name": "flite.wasm",
        "input_dir": "<abs-path-to-wasm-store>/flite/in",
        "output_dir": "<abs-path-to-wasm-store>/flite/out",
        "subtasks": {
            "subtask1": {
                "exec_args": [
                    "intro.txt",
                    "intro.wav"
                ],
                "output_file_paths": [
                    "intro.wav"
                ]
            }
        }
    }
}

Under subtasks key there should be yet another object (not an array).

enable password unlock and terms accept before/instead of actual command

current CLI returns

[17:23:53.237455] ERROR: Exception: get_connected_peers not mapped

in such a case. Allow user to input password or accept terms immediately

Choose Use Case (Application) for API demo

• Try to do Demand/offer spec for example Application current standars.

CLI -y to enable scripting usage

-y option means automatic answering yes for all interactive questions

CLI envs section

Commands:

• disable Disable environment
• enable Enable environment
• perf_mult Gets accepted performance multiplier
• perf_mult_set Sets accepted performance multiplier
• recount Recount performance for an environment
• show Show environments

golemcli command output without 'completed in x s.'

implement requirements from golemfactory/golem#3015
output does not need to contain execution time.

reference branch at main golem repo: https://github.com/golemfactory/golem/tree/feature-improve-cli

CLI debug section

Commands:

• exposed_procedures
• rpc

Other tasks:

• tests

Group settings set/show

Group settings set into
It seems that num_cores max_memory_size and max_resource_size are not working as provider resources settings.
There should be added an option to change custom_preset values - it is currently unavailable
General
Provider
Requestor