golemfactory / ya-client

Specification for REST API in yagna

Yagna public API

Public Yagna REST APIs client binding with Data Model and specifications in OpenAPI format.

REST API documentation is available here.

GSB API

GSB API documentation page.

Browse Rest API here:

https://golemfactory.github.io/ya-client

ya-client's Issues

ya-client REST api with crypto

Progress Reporting

• extend ExeScriptCommandResult with filed for progress reporting for deploy & transfer.
• extend deploy & transfer command with progress option.

Market unsubscribe result type

Both unsubscribe's should result with () and now results with String.
Comply with yaml which specifies204 code

define maximum timeout for REST API endpoints

ya-client/specs/common.yaml

Line 14 in f7e2c69

minimum: 0.0

arose when reviewing
https://github.com/golemfactory/yagna/pull/808/files#diff-ff490246b403bbf6555dcfcf6cb38d31758d138d6fadbe6cd529de60d4374a99R361

Base URLs are truncated to the last '/'

Base URLs for yagna APIs specified in the command line or via environment vars are truncated to the last '/', which may lead to confusing agent behaviour.

For example, with the environment variable YAGNA_MARKET_URL set to http://127.0.0.1:5001/market-api/v1" the agent will use just http://127.0.0.1:5001/market-api/`, without the trailing v1 as the base URL which will lead to errors.

This happens because of the Url::join method from the url crate treats v1 above as referring to a file:

use url::{Url, ParseError};

fn test() -> Result<(),ParseError> {

    let base_url_str = "http://0.0.0.0:6000/market-api/v1";
    {
        let base_url = Url::parse(base_url_str)?;
        let url = base_url.join("operation")?;
        assert_eq!(url.as_str(), "http://0.0.0.0:6000/market-api/operation");
    }
    {
        let base_url = Url::parse(&format!("{}/", base_url_str))?;
        let url = base_url.join("operation")?;
        assert_eq!(url.as_str(), "http://0.0.0.0:6000/market-api/v1/operation");
    }
    Ok(())
}

I think all base URLs that are not terminated with "/" should have the '/' character appended.

Add `410` error code to DELETE allocation docs

Why:

• We forgot about this when making changes to allocation behavior

What:

• Update documentation for DELETE /allocation/{allocation_id} endpoint in payment api

Support non-default configuration for Payment API

Currently only default configuration can be used because ProviderApiConfig and RequestorApiConfig have private fields and no constructor. Preferred manner for handling this is:

1. Implement a config builder class so that only non-default config values need to be specified.
2. Implement a function which reads config values from environment variables.

readOnly properties in OpenAPI spec are not supported by the openapi-generator

Notify SDK or address the issue

Define BB10 Event API specification

Refactor of Event flow from all Functional APIs into a dedicated Event API.

Event API shall follow the specs as defined here.

Request body with message should be required in terminate agreement

API: Market
Endpoint: POST /agreements/{agreementId}/terminate (on both provider and requestor APIs)

For the above endpoint, only agreementId parameter is marked as required in the API spec. Request body is marked as optional (and is generated as such). However, using this endpoint I found out that the request body is required and must include a message field.

Otherwise, the server returns a 400 with either Json deserialize error: EOF while parsing a value at line 1 column 0 or Json deserialize error: missing field message at line 1 column 2, depending on whether we pass an empty request body or not.

spec does not specify eventType ...

... while its well known enum. Instead string is used.

Required attributes, defaults, typing

I don't really know ya-client so I'm not sure how to state the issue correctly and what really needs fixing here.

Nevertheless, let's take a look for example at ya_market.models.proposal.Proposal and the proposal_id attribute (other Proposal attributes and other classes have similar "problems"):

class Proposal(object):
    openapi_types = { 
        'proposal_id': 'str',
    }   
        
    attribute_map = {
        'proposal_id': 'proposalId',
    }       
        
    def __init__(self, properties=None, constraints=None, proposal_id=None, issuer_id=None, state=None, timestamp=None, prev_proposal_id=None, local_vars_configuration=None):  # noqa: E501   
        self._proposal_id = None
        self.proposal_id = proposal_id
        
   @property
   def proposal_id(self):
       return self._proposal_id

   @proposal_id.setter
   def proposal_id(self, proposal_id):
       if self.local_vars_configuration.client_side_validation and proposal_id is None:  # noqa: E501
           raise ValueError("Invalid value for `proposal_id`, must not be `None`")  # noqa: E501

       self._proposal_id = proposal_id

All of this together adds up to "If we have self.local_vars_configuration.client_side_validation then proposal_id is required and must not be None".

So the main question is - what is this self.local_vars_configuration.client_side_validation & do we really need an option to initialize a Proposal without proposal_id? Because there is a cost behind such construction: static type checkers are not clever enough to understand this. So e.g. writing:

class OfferProposal(object):
    def __init__(self, subscription: "Subscription", proposal: models.ProposalEvent):
        self._proposal: models.ProposalEvent = proposal

    @property
    def id(self) -> str:
        return self._proposal.proposal.proposal_id

makes mypy sad:

Incompatible return value type (got "Optional[str]", expected "str")

--> it would be nice to somehow improve the typing here.

[sgx] ya-client check atestation

in SecureActivityRequestorApi::from_response

ya-client rest api for SGX

Collect events - maxEvents spec allows 0

It doesn't make sense to query 0 events, but I implemented it like this since spec requires ;)
Needs changes in spec

getExecBatchResults `timeout` parameter`

@mfranciszkiewicz you are removing timeout in this PR, but the comment is unchanged and still requires to specify timeout

Originally posted by @tworec in #68 (comment)

Fix ya-client CI

Spinned out of golemfactory/yagna#2294, not doing the whole task as it wasn't planned and we don't need that yet (yagna has a different workaround)

missing fields in event related endpoint spec

https://github.com/golemfactory/ya-client/blob/master/specs/payment-api.yaml#L1412-L1417 Is missing invoiceId
https://github.com/golemfactory/ya-client/blob/master/specs/payment-api.yaml#L1493-L1498 Is missing debitNoteId

Once fixed make sure to fix :

https://github.com/golemfactory/golem-js/blob/master/src/payment/payments.ts#L80-L82
https://github.com/golemfactory/golem-js/blob/master/src/payment/payments.ts#L129-L131

Findings from using swagger-gen

While working on the requestor agent for the python integration tests i found some improvements to the swagger files:

• date formats are not provided consistently
  • Demand.properties['golem.srv.comp.expiration'] = int, posix time in ms ( i think )
  • AgreementProposal.valid_to = string, in python str(datetime.now) requires an extra Z at the end
• app key security is not provided consistently ( fixed by #13 )
• StatePair is not implemented in activity-api.yml, but available in model/... https://github.com/golemfactory/ya-client/blob/master/model/src/activity/activity_state.rs#L54

Collect events timeout optional value unspecified

Collect events can have optional timeout. It is unspecified what happens if timeout is not set.
I would assume it means 0 timeout, and call should return immediately.
Probably we should mention this in spec.

Check specs/common.yaml file

Verify proposed changes for Event API

• check feasibility and consistency of proposed specifications

relates to #53

url_format! causes compiler warning since 1.64.0

Warning: named_arguments_used_positionally
Triggered by: url_format!("{}", arg);
Why: Macro expansion changes all arguments into named ones, and Rust warns about positional uses of named macros since 1.64.0.

replace `chrono::Date` with `chrono::DateTime` in `ya-client`

chrono deprecated Date in 0.4.23, but changing it will affect ya-client public API.

Payment API (Unified): Propagate Accounts listing endpoints to Unified API branch

As in title

Move swagger python client generation to ya-client

For the yagna-integration level 1 project i have generated the swagger models and client side library.

It seems they belong in the ya-client repository, next to the rust models.

• Move the model files, client api and readme files about generation to ya-client
• Add auto generation / validation ( generate with [email protected] ) and apply 3 git patches in a github job

Specification review changes to list Agreements

Apply changes requested by @stranger80 in PR:
#124

Agreement Events serves to serve as audit log for Agreement

We agreed that agreement events serves a purpose of audit log for Agreement.
It is to be decided how to present this log/history for particular Agreement. Possibilities that we see:

• support filtering collectAgreementEvents by agreementId
• extend getAgreement endpoint to contain related events or at least termination reason.
• create new endpoint for getting agreement history

Originally posted by @tworec in #61 (comment)

Add topic golem to the repo

Let's strive to always populate the topic golem here on Github to show how much of an ecosystem we have.

https://github.com/topics/golem

Please support it by adding the topic golem to this repo.

ActivityState.state spec doesn't match implementation

In spec it's described as a string with value from an enum.
In implementation it's pub struct StatePair(pub State, pub Option<State>);