speciesfilegroup / taxonworks_api

Documentation for TaxonWorks APIs

taxonworks_api

Documentation for the TaxonWorks workbench API, served at https://api.taxonworks.org. Application code is here.

Building

Documentation uses RAML, it is built with gulp:

Clone this repository, then:

npm install

npx gulp apidoc

Documentation is in /docs. Open /docs/index.html to view locally, or see running an api console.

Deploying

Merge development into main. Github actions do the rest.

Changelog

See CHANGELOG.md. Note this changelog documents changes in the documentation, not the underlying code.

Contributing

See CONTRIBUTING.md.

License

https://opensource.org/licenses/NCSA.

Code of conduct

Please see CODE_OF_CONDUCT.md.

taxonworks_api's Issues

configure CI generation

Most of the workflow seems to be doable via this pattern

• setup our TW Bot to provide the personal access token

• figure out how to encrypt and supply that token to Travis

• setup hook to Travis that is fired for only a specific type/class of commits (see infinite loop) (i.e. only when a .raml file changes should we trigger Travis)

• setup a local build script to generate the HTML from RAML (#10)

• integrate that build script into .travis.yml

• ensure that when Travis commits /doc back to this repo

• ... profit

Review and add or revise endpoints documentation.

Correlate new API routes and recent functional changes with API documentation.

Add OpenAPI and Insomnia config?

I used oasraml-cli to convert the TaxonWorks api.raml to OpenAPI 3.0, which can be imported into the Insomnia client. This might be useful to provide to users, because it makes exploring the API easier. I've attached these files here.

Under manage environments, the following variables need to be configured within the Insomnia project environment:

{
  "user_token": "",
  "project_token": "",
  "project_id": "",
  "base_path": "/api/v1",
  "host": "sandpaper.taxonworks.org",
  "scheme": "https",
  "taxon_id": ""
}

consider implications of JSON-LD

Should we return this format, etc.

https://json-ld.org/

As a User I want the Content endpoints documented

As a curator I want instructions as to how open a API endpoint for my project(s)

Include documentation near the top of api.taxonworks.org

choose a markup for the API

Very likely RAML https://raml.org/.

As a user I want to access identifiers tied to community data with my `token` alone

Presently you must still also use a project_token. Might be tricky to tease this out.

See also related ideas of sharing People and Sources without a token, or having a service to signup to token use only, but not login us.

define a basic response pattern

• differentiate b/w HTTP errors and application errors
• differentiate b/w data elements and response metadata
• differentiate b/w RESTful and non-restful endpoints (yes there will be both)
• determine if any elements belong in header vs. body

As a would-be user I want to sign up for a token, but not otherwise use the UI/interfaces

As a user, one or more simple examples of the taxon_names endpoint would be nice

Rename/add `keyword_ids[]=` to `keyword_id_and[]=` and `keyword_id_or[]=` throughout

/collecting_events add missing attributes from `api_params` (e.g. wkt)

Double check that params are all included in the documentation.

RAML pattern examples

A list of RAML documented endpoints for reference:

• https://github.com/raml-apis

gulp-raml2html works only with RAML 0.8

gulp-raml2html hasn't been updated in a while, and it doesn't support RAML 1.0. I looked at this (from the raml2html module page):
https://gist.github.com/iki/784ddd5ab33c1e1b726b

However, I couldn't get this to work either - both with 0.8 and 1.0. The files weren't getting parsed correctly, and the output HTML page just had the default theme with API Documentation as the header.

/collection_objects - describe that all collecting_event attributes can also be referenced

Don't repeat attributes, just reference the corresponding /collecting_events documentation.

point api.taxonworks.org to this resource

As a user, reading the documentation, I can not copy/paste the URLS

it's hard to copy/paste the examples and there's double quotes around the url and as well as double quotes within for the strings, so that doesn't work AFAICT

determine if there is a lossless route from RAML to Swagger

@devarsh1997 can you research and answer the question, "If I write my API docs in RAML, then can I convert them losslessly to swagger"?

As a user I want more informative error messages in some cases

e.g., a request to an endpoint that doesn't exist https://sfg.taxonworks.org/api/v1/foobar returns 404 correctly but the returned payload is html - for the API ideally that would be the same content type as successful requests

Clarify max/min filter per values [was: Taxon missing when pulling all data from /taxon_names]

With the following query
.../taxon_names?token=foo&project_token=bar&per=15000
I can roughly get all the data from /taxon_names, since 15000 is larger than the total number of elements.
However, the taxon named Itequahy is returned when we filter the query by name but is NOT returned in the 9MB result of the query above.

As a DevOp I need to document `authors` param for taxon name filter

Identical to OTUs in function.

Auto-generate HTML docs from RAML

• setup a structure for housing the input RAML files (check out how others are setting this up)
• setup a structure for housing the build scripts (/build ?)
• build the conversion script (node/webpack? grunt/gulp? - maybe help from José)
• ensure the build script has proper success/failure responses that can be interpretted and used by the travis environment
• Demonstrate local build server (analagous to jekyll server) that listens to raml file changes and does local builds

As a developer I need to have API token generation documented

/collecting_events - document collecting event attributes

All collecting event attributes can be queried against (with exact or wildcarded options). Add the list of attributes to the documenation. There are enumerated in Queries::CollectingEvent::Filter::ATTRIBUTES.

As a user I want to see method not allowed responses where applicable

Method not allowed http code could be used for this example of calling POST on a route that presumably only allows GET: curl -v -X POST https://sfg.taxonworks.org/api/v1/user_authenticated

Some gulp tasks error

OS X latest. Started with npm install. If we don't need these tasks let's just destroy them.

• gulp apijson fails
• gulp apiyaml fails

Preceeding both with npx also fails.

/people `&include_roles` will be changed to `extend[]=roles`

Update when these closed:
SpeciesFileGroup/taxonworks#2532
SpeciesFileGroup/taxonworks#2528

Minor issues when getting taxon_names

This has been a really smooth API to work with.

Two very minor issues:

1. When I query for species I get subspecies too. This may be a feature but it bit me when looking through the records.
2. There are duplicate names in the DB. For example: "Eusynthemis guttata" is both under IDs 715483 and 709907.

https://sfg.taxonworks.org/api/v1/taxon_names?project_token=iu4ty6tOdWg_-cceoQjhyQ&taxon_name_id=707405&descendants=true&nomenclature_group=Species&per=10000

Alphabetize params list

For each endpoint let's go ahead and set the standard to be an alphabetized list of params.

As a user I want to authenticate with my token in a header

@LocoDelAssembly Maybe we have this already, I have to look again. If we have it's just a matter of updating the doc.

consider auto-generating components of the base API file

We have demonstrated generating routes based on model properties, it's worked very nicely for our annotations layer. We could follow this pattern, perhaps in a Rake task, to generate the components of the API that are included in the description.

Example route generating code. This loops all data models, checking to see whether they inherit from an annotating class, and adding RESTFUL getters if so. The code could be modified to produce type lists in RAML, for example.

  # Generate shallow routes for annotations based on model properties, like
  # otu_citations GET    /otus/:otu_id/citations(.:format)    citations#index
  ApplicationEnumeration.data_models.each do |m|
    Shared::IsData::Annotation::ANNOTATION_TYPES.each do |t|
      if m.send("has_#{t.to_s}?")
        n = m.model_name
        match "/#{n.route_key}/:#{n.param_key}_id/#{t}", to: "#{t}#index", as: "#{n.singular}_#{t}", via: :get, constraints: {format: :json}, defaults: {format: :json}
      end
    end 
end

Update documentation to includes

Search for extend_response_with and update endpoint to include valid extend[], embed[] values.

As a user I want documentation for `/collection_objects/123/dwc`

Example

Work up CONTRIBUTING.md

• It should describe the format of PR
• It should describe the pattern for accepting the PR

etc.

Expose remaining nomenclature based endpoints

The index endpoint for the following should suffice:

• TaxonNameRelationship
• TaxonNameClassification
• Source
• Person
• Citation

consider serializing to json-api

• https://blog.codeship.com/building-a-json-api-with-rails-5/

As a developer I want advice on what branch to commit to

We anticipate a lot more work on this repo shortly. Do we start commiting to development, then release to master?

Ping @LocoDelAssembly

Boolean vs. String params

Using type: Boolean forces the enum to true or false (true booleans). Technically our back end handles strings, converting them to true or false there. We also handle Capitalized and upcase values as well.

Should we just declare our booleans to be string, so we can provide examples as enums?

consider the implications of using Swagger code generation for native wrappers

https://github.com/swagger-api/swagger-codegen

If this code is solid then we get "free" native language wrappers on our API if we document it using swagger. This is consideration for what we use.

Note that RAML to Swagger (and other converters) appear to exist- https://github.com/LucyBot-Inc/api-spec-converter.

consider implications of GraphQL

Should we aim towards a graphQL based interface?

http://graphql.org/