ncatstranslator / relay Goto Github PK

• Motivation: UX/UI studies show that if an end user needs to wait longer than some short amount of time (say, 0.5 seconds), then they will get frustrated with the system unless they are aware of what is happening in the background while they wait for their results.

• ReasonerStdAPI already has a log feature that ARAs utilize to show their reasoning steps.

• If ARA's can share with the ARS this log and/or partial results in ReasonerStdAPI (instead of waiting until the very end after all reasoning has taken place), then the eventual ARS GUI can take advantage of this to show the end user "what's happening in the background" as they wait for results.

• As an additional bonus, the ARS having access to this information as it comes in will demonstrate to an end user how integrated the system as a whole has become.

Note: this feature request is motivated by a discussion with @edeutsch, @MarkDWilliams and Christine Colvis at the April 2020 Kickoff Relay.

https://github.com/NCATSTranslator/ReasonerAPI/releases/tag/v1.0.0-beta

Should be in place by 1/15 like the ARA and KP teams

webyrd/mediKanren#46

Change model to add new field to turn them on/off

biothings/biothings_explorer#164

http://smart-api.info/registry?q=230691056df158545fd38bb73379d9c3

RTXteam/RTX#1136

Noel I think you developed the status page, so assigning to you for now

Noel Southall (Link Broker) 10:58 AM
Wanted to follow-up about that ARS status page error for ARAX --- currently, I was using a very old snapshot from SmartAPI, so we updated it.
However, we list the endpoint for the ARAX agent in ARS as: https://arax.ncats.io/devED/api/arax/v1.0/query?bypass_cache=false
and that SmartAPI entry that you pointed me to lists:
https://arax.ncats.io/api/arax/v1.0
So, that's another problem with the current mapping. Should we further update the ARS registration for the ARAX agent to this SmartAPI url?

Eric Deutsch (Expander Agent) 10:59 AM
yes please!
If we update info in our SmartAPI registration, should we also let you know?
Or is the new information pulled automatically?

Noel Southall (Link Broker) 11:02 AM
Well, we hadn't completely settled on where the best metadata is in the SmartAPI entry .... so I was linking it on URL, but that's clearly not a good idea. Can I update the agent name to map to the SmartAPI resource name too - then I might be able to catch updates to a URL?

RTXteam/RTX#1127

@patrickkwang mentioned having testable queries for the KPs that are hooked up to Strider, and that he could commit those and some simple python code for testing to the Architecture repo . Once that is done, I will also add the Fix It queries, the ARA status queries, and the user engagement queries

for existing and new registrations

In:
https://github.com/NCATSTranslator/Relay/wiki/Use-Cases

several of the JSON snippets use curly quotes, which is not valid JSON and makes it difficult to copy-paste and use/test.

Can all the examples be made valid JSON?

From the Last Message link in the https://ars.transltr.io/ars/app/status page:

"logs": [ "No connection adapters were found for 'hhttps://translator.broadinstitute.org/genetics_provider/trapi/v1.0/query'", "Tr_ars.url: hhttps://translator.broadinstitute.org/genetics_provider/trapi/v1.0/query" ] }, "url": "hhttps://translator.broadinstitute.org/genetics_provider/trapi/v1.0/query", "ref": "b589286f-5d74-412c-b506-a68d1fed324b" } }

hhttps://translator.broadinstitute.org/genetics_provider/trapi/v1.0/query
should be
https://translator.broadinstitute.org/genetics_provider/trapi/v1.0/query

The presentation of the Use Cases is currently

1. variable in format
2. lacks English descriptions

I recommend that
A. A template for "best practice" of submitting a Use Case is created
B. More English descriptive text is added. It is hard to reverse engineer on the fly by eyeballing a use case what the intended question is, in some cases.

For example, the template might be

TITLE: Compound-Disease-Disease (no specific names of entities should be in the title)
SPECIFIC EXAMPLE IN THIS USE CASE: rheumatoid arthritis - lung cancer - compound
English Description: This is a query to find a compound that is known to treat one disease that also treats a related disease.
Key identifier mappings to help interpret this query and that can be changed to customize the query (optional): DOID:7148 = rheumatoid arthritis
Query: (do not use curly quotes!)
{ "query_message": { "query_graph": { "nodes": [ { "node_id": "n0", "curie": "", "type": "biolink:ChemicalSubstance" }, { "node_id": "n1", "curie": "DOID:7148", "type": "biolink:Disease" }, { "node_id": "n2", "curie": "", "type": "biolink:Disease" }, { "node_id": "n3", "curie": "", "type": "biolink:Gene" } ], "edges": [ { "edge_id": "e0", "source_id": "n0", "target_id": "n1" }, { "edge_id": "e1", "source_id": "n1", "target_id": "n2" }, { "edge_id": "e2", "source_id": "n2", "target_id": "n3" } ] }, "query_options": { "psev-context": "DOID:7148", "query_kps": "true" } } }

Note that the above is my attempt to understand one of the current use cases. I trust that my best interpretation shows that I did not interpret it correctly. :)

ranking-agent/strider#22

Minor suggestion... Currently API calls (e.g., https://ars.transltr.io/ars/api/messages/fc7414d4-1f64-48f6-a4be-0775a6486227) use Content-Type: text/html in the response header. So that my browser plugin identifies it correctly as json for pretty formatting, would be nice if that were switched to Content-type: application/json.

(A bit confused because it looks like it should already be set up that way in https://github.com/NCATSTranslator/Relay/blob/master/tr_sys/tr_ars/api.py#L78 ?)

RTXteam/RTX#1135

Despite TRAPI 1.0 SMART API registration for MolePro and Genetics KP(@marcdubybroad), ARS still calls Reasoner API 0.9.2; please use the following endpoints:

https://translator.broadinstitute.org/molepro/trapi/v1.0/
https://translator.broadinstitute.org/genetics_provider/trapi/v1.0/

Agents will not be individually dockerized but included in the main ARS

for existing and new registrations

Would you add CORS support in the ARS?

Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Methods: GET, POST, OPTIONS

Review list for tools that may be ready for ARS Registration (SmartAPI and TRAPI): https://docs.google.com/spreadsheets/d/1n9X8EnSCYxgnJHs4zuNqpwwuJG4oSnbqTmvBPn7_G9M/edit?usp=sharing

• Not TRAPI Compliant
• Pending (which indicates we need to register)
• Date (which indicates completed)

ARS allows an enum of message statuses. This should be aligned with the recent decisions on Reasoner API response statuses discussed at:
https://docs.google.com/document/d/12GRjcAqXQfp557kAcVEm7V0mE3hKGxAY5B6ZPavp6tQ/edit
which is a follow-up to:
NCATSTranslator/ReasonerAPI#147

High level thoughts:

• live instance to allow people to browse queries and answers
• includes all queries that have ever been submitted and their answer graphs
• static answer graphs - includes version of the answer graph

This topic was brought up at the architecture meeting on July 16th prompted by David (ARAx) presentation where he referred to user intentions. We propose we discuss this interface to ask user preferences. One example is to ask a user about preference towards innovative but not necessarily scientifically validated results as those to receive highest ranks or towards well-validated results. This interface relates to the iterative process between the ARS and the ARAs as referred to in the FOA.

ARAGORN's use case query functions when sent directly to their service but times out at the ARS level.

RTXteam/RTX#1167

currently Travis tests do not cover other agents.

Include:

1. Meat Grinder Session with Tyler b.
2. Round 1 User Engagement with AK

Discussed storing both human readable and as well as adding to Travis

v1.0 API is the following
https://smart-api.info/ui/99592dd5fc3a69d3ed27692f3f278e47

Thanks,
Marc Duby

https://drive.google.com/drive/folders/1xelbvQsn6Oph51iTyLkIhr_W6e2IEjAa?usp=sharing

Use Case: Can we repurpose a drug for any indication?

Submitting this test query
https://slack-files.com/TSCGQ3XGB-F01ARUUBH8S-b59bab246b

Can take some time to execute. It appears that the connection times out before a response is received.
One example response can be seen in this message thread:
https://ars.transltr.io/ars/api/messages/0925ff3c-7cb2-4b12-abd4-e4f22c10e3d2

Update status to unknown or error if connection is lost.

Currently, ARS service agent/actors POST queries to resources synchronously and holds that connection open as long as is needed for the service action to be performed. The preferred behavior would be to provide a callback for services to POST responses and updates directly to the ARS, with the agent/actor service just replying with an initial status of 'in progress' or something.

As new versions of the Translator API are released, we will need to know which versions to POST and what type of response to expect back. This most likely requires and update to the Agent/Actor model - to allow specific properties, like API version.

POSTing to, e.g.

curl -d @tr_sys/tr_ars/actor_update.json http://localhost:8000/ars/api/actors > response2.htm

will only create new actors instead of updating them. You can use the Django admin interface in the interim, but delete/update for actors/agents should be fixed.

One of the test queries we currently use to probe service status is a one-hop query, and this is currently returning a non-status 200 response for improve agent. We have yet to fully converge on a standard for error responses from Reasoner API services, but the current convention is to return a status 200 response if the message was received and is TRAPI compliant. See, for example, NCATSTranslator/ReasonerAPI#140

There are various scenarios where the ARS could malfunction (Celery down, etc) that need to give a message on the Status Page (https://ars.transltr.io/ars/app/status). Currently it looks like the actors/agents aren't working - which isn't a true reflection of the status

alpha status page for ARS https://ars.transltr.io/ars/app/status shows actors with responses but without SmartAPI registration as yellow --- this is not a current requirement for an ARA.

While SmartAPI is the current, preferred resource discovery convention in Translator, we should be more thoughtful in separating behavior/status on the ARS status page in particular.

https://docs.google.com/document/d/1zU0ljQ_gNDziZVLXYpVCKJ4HT1nu5lywyVN4MlMLfU8/edit?usp=sharing

per team meeting on 1/13 should be deprecated by 1/21