zooniverse / panoptes-javascript-client

A Javascript client for accessing the Panoptes API

Panoptes Client

A Javascript client for accessing the Panoptes API.

Installation

You can install the client from NPM:

npm install panoptes-client

and use it with:

ES5

apiClient = require('panoptes-client/lib/api-client');
auth = require('panoptes-client/lib/auth');
oauth = require('panoptes-client/lib/oauth');
talkClient = require('panoptes-client/lib/talk-client');

ES6

import apiClient from 'panoptes-client/lib/api-client'
import auth from 'panoptes-client/lib/auth'
import oauth from 'panoptes-client/lib/oauth'
import talkClient from 'panoptes-client/lib/talk-client'

Documentation

The documentation for the library is available at https://zooniverse.github.io/panoptes-javascript-client/. If there's anything missing, submit a PR!

Resource access

The Panoptes API is built on the very generically named JSON API Spec. This client leans heavily on this library to make it easy to access different resources that the API offers.

Conventions

This project adheres to Semantic Versioning, and follows the changelog format set out at Keep a CHANGELOG.

Running the tests

Tests (via tap) exist for the auth module, and can be run with npm run test.

Publishing

1. Create a new branch for the new version.
2. Add the new version to the changelog with summary of changes.
3. Update package.json and package-lock.json with the new version number (manually or with npm --no-git-tag-version version [major|minor|patch]).
4. Commit and push the changes to GitHub. Create a pull request, have it reviewed, and merge it.
5. Using GitHub's web interface, create a new tag and release with the new version number and summary of changes.
6. Pull the new version of the package locally, confirm it reflects the new version.
7. Run tests with npm test, confirm that they pass.
8. Run npm publish --dry-run to check that the package is ready to publish, confirm no errors or issues.
9. Run npm publish to publish the new version to npm.

License

Copyright 2015 Zooniverse

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

panoptes-javascript-client's Issues

`auth.signIn()` runs forever in Node

On a successful login, the process doesn't exit (I'm guessing there's a promise somewhere that never resolves). An unsuccessful one quits as normal.

`_handleNewBearerToken()` florps on `.slice()`

Weird Glitch Detected?

I ran across this weird glitch while working on some ASM updates. Essentially, after returning to an idle webpage after 2 hours, the Panoptes Client goes, "Hmm, time to renew the bearer token" then suddenly goes "BLARP I'm dead"

While I fully expected the token to die after a lengthy timeout (this is why I went on a 2 hour tea-drinking binge, after all - for science! ) I didn't quite expect to see how it died: the code crashes at oauth._handleNewBearerToken()'s console.log() line, which attempts to .slice() an undefined

Debug Details

Setup: OSX+Chrome, panoptes-client 2.9.2

Steps taken:

1. At 5pm I am logged in to Anti-Slavery Manuscripts, development, on localhost:3000. Computer is put to sleep. (ASM window is not closed)
2. Two hours are spent attempting to create the perfect blend of chamomile and chrysanthemum.
3. At 7pm computer is reactivated. ASM window is viewed again.
4. I notice the following new logs in the console, timestamped at 7pm:

- Deleting bearer token
- Found existing Panoptes session
- Panoptes session has expired
- TypeError: Cannot read property 'slice' of undefined
    at Model._handleNewBearerToken (oauth.js?13a0:260)
    at Model.eval (oauth.js?13a0:276)
    at <anonymous>

Expected: After 2 hours of non-activity, login token (user session) should expire - any attempt to conduct user-only Panoptes requests should fail with a 401 or similar.
Actual: Exactly as expected, but with weird bonus console errors.

The ASM page when accessed after 7pm. Note the console messages.

The ".slice() of undefined" error traced to the oath.handleNewBearerToken() code

Analysis:

• The error seems to happen at the line: console.log('Got new bearer token', tokenDetails.access_token.slice(-6));
• Presumably, oauth._handleNewBearerToken() received the tokenDetails object, but for some reason it was missing the .access_token property. (I was unable to trace what the value of tokenDetails was, unfortunately.)
• This is worrying since the console.log() is the first line in the _handleNewBearerToken() function, meaning everything else in the method would have died, and the method returned nothing.
• That said, I'm pretty sure the bigger concern is that the tokenDetails in the method didn't conform to expected data structures. And I'm not sure what's up with that.
• Also, I'm not sure if this is an issue that's already been solved by PR #82 (which is the basis for panoptes-client 2.9.3), what with the this._deleteBearerToken() no longer triggering before return this._handleNewBearerToken(tokenDetails) in oauth._refreshBearerToken() 🤷‍♂️
• Either way, the code may need to be made robust enough not to crash at the first console.log() statement.

Status

Unknown priority? Help, I have no idea if this is a minor issue, an edge case, or something that needs to be taken care of.

This may be tangentially related to #81, as any attempt to create a better token-refresh system may need to be aware of possible crashing in _handleNewBearerToken's unexpected tokenDetails.

"Got new bearer token" no longer logs distinct tokens

panoptes-javascript-client/lib/oauth.js

Line 232 in e7115da

console.log('Got new bearer token', tokenDetails.access_token.slice(-6));

The start of all bearer tokens is going to be the same for all. Better slice from the end.

String Interpolation Issues

The client has an issue accepting the following string interpolation:
projectPreferences.update({preferences.tutorials_completed_at.${this.props.tutorial.id}: now});

This came up in a ZRC PR.

Unnecessary save to localstorage

https://github.com/zooniverse/panoptes-javascript-client/blob/master/lib/oauth.js#L61

...this should be ls.set, but it works anyway. Which means that we probably don't need this line.

User sessions are cached in memory forever

auth.checkCurrent() returns a cached _currentUserPromise here, if it's already been set on page load.

panoptes-javascript-client/lib/auth.js

Lines 169 to 186 in bbda9c2

	checkCurrent: function() {
	if (!this._currentUserPromise) {
	console.log('Checking current user');
	this.update({
	_currentUserPromise: this._getBearerToken()
	.then(function() {
	return this._getSession();
	}.bind(this))
	.catch(function() {
	// Nobody's signed in. This isn't an error.
	console.info('No current user');
	return null;
	}),
	});
	}
	return this._currentUserPromise;
	},

The problem with this is that the stored user session can't be reset by logging out in another tab. This code will never check the Panoptes API for a stale user session, as long as this._currentUserPromise is defined.

To test this out, run auth.checkCurrent() in two tabs, re-running it on visibility change, then log out in one of the tabs. The other tab will still return a Panoptes user from auth.checkCurrent(), even though its session cookie is no longer valid.

auth.checkBearerToken() returns undefined after refreshing a token

auth.checkBearerToken() returns undefined after refreshing a token, but it should return the refreshed token. The refreshed token is correctly set in the client, so subsequent calls do work.

// check an expired token. This should set token to the refreshed token but actually returns undefined.
let token = await auth.checkBearerToken()
// token is now refreshed. This sets it correctly.
token = await auth.checkBearerToken()

The underlying cause is that _refreshBearerToken returns undefined. It should return a Promise that resolves to the refreshed token.

panoptes-javascript-client/lib/auth.js

Lines 64 to 91 in 9438015

	_refreshBearerToken: function() {
	if (this._tokenRefreshPromise === null) {
	console.log('Refreshing expired bearer token');
	var url = config.host + '/oauth/token';
	var data = {
	grant_type: 'refresh_token',
	refresh_token: this._refreshToken,
	client_id: config.clientAppID,
	};
	this._tokenRefreshPromise = makeHTTPRequest('POST', url, data, config.jsonHeaders)
	.then(function(request) {
	var token = this._handleNewBearerToken(request);
	console.info('Refreshed bearer token', token.slice(-6));
	}.bind(this))
	.catch(function(request) {
	console.error('Failed to refresh bearer token');
	apiClient.handleError(request);
	})
	.then(function() {
	this._tokenRefreshPromise = null;
	}.bind(this));
	}
	return this._tokenRefreshPromise;
	},

Trailing slashes in OAuth redirect URIs should be stripped

...as it causes issues in Safari - see https://github.com/zooniverse/ancient-lives-frontend/issues/102

Calling oauth.signOut when there is no user signed in should not produce an error.

This is interfering with my ability to use Mocha to test code that uses or talks to oauth, because keeping the same oauth instance around results in a lot of useless calls to signOut.

Expired tokens used in talk client

I finally tracked down zooniverse/Panoptes-Front-End#1052. It turns out that the talk client is using an expired token.

Just from the bug reports, it appears that the api client has a valid token while the talk client is using an expired token.

Considering talkClient.headers = apiClient.headers;, I'm not sure how this happens. Possibly related to #15? Any idea @brian-c?

refreshBearerToken interval never called

Tokens consistently expire in long-running scripts.

Cookie not passed back through iFrame depending on settings

On projects using the oauth module on Safari, if Privacy > Cookie and website data is set to Allow from current website only -- or Always block -- the cookie returned from the sign in process is blocked, causing login to fail.

Next step is to check out this behaviour on other browsers, but I'm guessing any restrictive cookie sessions will produce the same result.

Potential fix could be falling back to a full-page redirect for now...

Thanks to @astopy for finding this one.

suppress log messages

It'd be nice to have an option we could set to keep console.log from being spammed.

Keeping config in one place

The OAuth module keeps its own app ID on the class, and there's another one stored in the _config property on the client. Should everything use the same config object instead?

auth.signOut() doesn't send Accept headers

auth.signOut() should send Accept: application/json but that header isn't being added to the request.

First-party auth doesn't return login errors

Login errors aren't returned as rejected promises, which leads to them being handled as resolved promises with undefined arguments. See zooniverse/Panoptes-Front-End#6236

Compare the error handler for auth.signIn here:

panoptes-javascript-client/lib/auth.js

Lines 227 to 230 in 49a56f4

	.catch(function(request) {
	console.error('Failed to sign in');
	apiClient.handleError(request);
	});

with the error handler for auth.register here:

panoptes-javascript-client/lib/auth.js

Lines 151 to 154 in 49a56f4

	.catch(function(request) {
	console.error('Failed to register');
	return apiClient.handleError(request);
	});

Apps needing updated PJC

List that have been deployed to production:

• Panoptes-Front-End zooniverse/Panoptes-Front-End#5561
• front-end-monorepo zooniverse/front-end-monorepo#1323
• $ ASM zooniverse/anti-slavery-manuscripts#301
• $ Scribes zooniverse/scribes-of-the-cairo-geniza#181
• Zoonvierse Classrooms zooniverse/classroom#175
• Pandora zooniverse/pandora#149
• NfN field book zooniverse/notes-from-nature-field-book#92
• wildcam gorongosa zooniverse/wildcam-gorongosa#255 (Note app was developed against production so there is no staging WG or app setup in doorkeeper)
• $ PFE lab (for organizations) zooniverse/pfe-lab#196
• Transcription viewer (https://shaunanoordin.github.io/zooniverse-react-transcriber/app/)
• Annotate zooniverse/AnnoTate#236
• WG lab (https://lab.wildcamgorongosa.org/) zooniverse/wildcam-gorongosa-education#330

$ = Uses ZRC components (will have to be updated once ZRC has been updated with new PJC)

Other that have been updated:

• Text Editor zooniverse/alice#59
• Zooniverse React Components zooniverse/Zooniverse-React-Components#69
• Vote (https://vote.zooniverse.org) does anyone use this? Nope!

Add OAuth flow to auth

Would be super cool to have the Mammoth Bats-style OAuth login available as a method in the auth module

Add sugar client tests

https://github.com/zooniverse/Sugar-Client/tree/master/test

Choose a style standard

@aweiksnar suggested https://github.com/feross/standard

Do we want to keep using ES6, or revert to ES5?

Client crashes if response.body is null

panoptes-javascript-client/lib/api-client.js

Line 25 in 0a40d49

} else if (Array.isArray(response.body.errors)) {

If the response body is null, the client crashes here with unable to read property errors of null.

Client doesn't pass back HTTP response status codes with errors

This piece of code triggers a 422: unprocessable entity from Panoptes by posting a malformed classification.

classification.metadata = {}
classification.save().catch( function(e){ console.log(e); } );

e comes back as Error: {"metadata"=>"did not contain a required property of 'workflow_version'"} so it's impossible to tell this failed with a 422. Consequently, it's not possible to write client code that decides on an error handling strategy based on the HTTP response eg. queue failed classifications in case of a 500 or 503, but not in case of a 404 or 422.

Remove `test-mac` once testling gets a fix?

I wasn't sure why there were separate test and test-mac commands, but I've just found tape-testing/testling#108. We should probably remove the command and add PhantomJS as a devDependency once it's fixed.

Specify a node-version in the package.json

So we're all on the same one

3rd party cookies disabled can't renew a token - auth code overhaul required.

Browsers with third party cookies disabled won't be able to renew a token via the iframe after #75.

After #75 is deployed, these users will get a token from hash fragment of the page redirection of the implicit oauth flow and then it'll be set for re-use via local storage. The implementation in #75 will use a timeout function to clear the expiring tokens and attempt to get a new one via the iframe.
This will fail and those users may be left with an inconsistent UI state depending on how the calling code detects the underlying logged-in state change, most likely signalling logged in status but appearing to the API as unauthenticated.

The above will be poor UX on our behalf and will cause issues with tracking seen subjects and recents in the API among other things. Also noting that these users will be logged in to talk as it is on the zooniverse.org domain. The users can make notes on the subjects they classified but probably won't have that subject tracked in their seen subjects and may be served them again (this riles users no end and wastes effort).

This library needs an overhaul in how it manages and re-uses tokens and this code can be shared between the supported authentication flows we currently use:

1. Auth.js uses the password credentials flow and gets a refresh token
2. Oauth.js uses the implicit flow and does not get a refresh token

We should split out the token retrieval to different strategies patterns and consolidate the token storage and management code where we can. The different token flows above will require different token renewal strategies as well:

1. flows with a refresh token can get a new access token directly
2. flows without a refresh token will have to re-authenticate or rely on an existing session to gain a new token.

The updated code will also provide management hooks that the including app can use to configure and customize the authentication flows as well. This will allow events like failing to refresh a token to bubble up to the calling app to better manage the experience of our users on sites that use implicit flows.

Load Promise, XMLHttpRequest polyfills if needed

Originally via #8.

Currently, if either the Promise or XMLHttpRequest polyfills are missing, the client will silently exit. It should include both of these as dependencies and conditionally load them if needed.

Auth module API isn't documented

Docs on the gh-pages branch document the APIs for APIClient and OAuth but Auth is missing.

Suggestion: Document how to choose staging or production

I heard that it's the NODE_ENV variable and confirmed that in the code (

panoptes-javascript-client/lib/config.js

Line 56 in 0713172

'try setting NODE_ENV to "staging" instead of "'+envFromShell+'".');

) but it doesn't seem to be mentioned in https://zooniverse.github.io/panoptes-javascript-client/

Might PR this myself later but just noting it for now.

Add method to set Panoptes environment

Mobile apps don't have access to NODE_ENV, so we can't set a target environment. We should create a method like config.setEnvironment('FOOBAR') so it can be set from within a project

Add tests for OAuth module

Will need some kind of in-browser test to allow for iFrame usage etc

`Staging` undefined

I was getting this when trying to access https://preview.zooniverse.org/wge/

Unfortunately I can reproduce that anymore and instead I get this now

It looks like something is not right with the use of iframe in the AuthModule. WGE is using this branch of the client zooniverse/panoptes-javascript-client#d97cbc1.

@rogerhutchings thoughts?

Rails 6 Upgrade: Updating password from PFE mentions returns error that user is not signed in

Errors when updating password from Profile Settings Page from Rails 6 on Staging. See image below.

When changing password via Settings, error Error: You need to sign in or sign up before continuing. comes from the PUT request

Mentioned in PR: zooniverse/panoptes#4114

lib/auth.js still uses some calls to apiClient.put and apiClient.post, which weren’t converted over to makeCredentialHTTPRequest in #121. Converting them will probably close this.

API client errors with 'data is null' for some Talk searches

panoptes-javascript-client/lib/json-api-client/index.js

Lines 94 to 97 in 8cd40d7

	for (j = 0, len1 = ref2.length; j < len1; j++) {
	resourceData = ref2[j];
	this.type(typeName).create(resourceData, headers, response.meta);
	}

Some Talk API searches return an array of comments that include null entries. Those crash here, when resourceData is null.

Example URL:
https://www.zooniverse.org/talk/search?page=3&query=depression

Oauth sign-in fails in Chrome when third party cookies blocked

Linked to #30

The oauth flow works and the access_token is returned in the URL fragment however that redirect response tries to set a cookie as well. I assume this cookie set failure causes the code / a promise to fail even though we have all the information we need to authenticate the user.
Steps to recreate:

1. Activate block third party cookies in chrome
2. try to login to shakespearesworld.org and check the response from panoptes signin oauth flow.

Happy to help debug this as well

Add addLink and removeLink to ResourcePromise

Might help with zooniverse/panoptes#4104 and zooniverse/Panoptes-Front-End#6400.

Some resources (i.e. subject sets, tutorials) aren't unlinking from workflows as expected.
Here's the PFE code to unlink a subject set from a workflow - https://github.com/zooniverse/Panoptes-Front-End/blob/master/app/pages/lab/workflow-components/subject-set-linker.jsx#L50-L75 - note the use addLink and removeLink in the updateLink function.

Using npm version with node is brokened

/Users/roger/Projects/zooniverse/pjctest/node_modules/panoptes-client/lib/utils.js:7
if (!exists(window) && !exists(window.console)) {
            ^
ReferenceError: window is not defined

This has all been commented out on GitHub (https://github.com/zooniverse/panoptes-javascript-client/blob/master/lib/utils.js), but we should probably release a new version

Do we want to keep using JSON-API-Client as a dependency, or do something different?

OAuth.init() no longer checks for an existing Panoptes session

Prior to 2.9.0, OAuth.init() checked for an existing Panoptes session and used that if you were already logged in. With 2.9, you must always sign in with each new instance of the client, even if you already have a valid session.

oAuth Sign In possibly broken on front ends with Webpack 5

Functionality Issue

If a Custom Front End (e.g. Classrooms) upgrades from Webpack 4 to Webpack 5, there's a chance that Sign In via oAuth is broken.

Symptoms:

• A user might click the Sign In link, sign in on the oAuth page, but when they're redirected back to the front end, they're still not signed in.
  • Sanity check: prior to Webpack 5, sign ins worked fine, and the oAuth application is set up correctly.
• when the CFE loading, it no longer checks zooniverse.org/api/me to find out who's signed in.

This can be a result of Webpack 5 removing default polyfills for the Node.js url library, which causes a hidden chain of failures when signing in.

Probable triggers:

• Front end project upgraded from Webpack 4 to Webpack 5, AND...
• ...the Webpack config contains resolve.fallback = { url: false }, which is usually added to solve build issues on webpack-dev-server 4 onwards.

Dev Notes

Full failure chain:

1. Webpack 5 removes fallbacks for Node.js's url module
2. panoptes-client uses json-api-client uses normalizeurl 0.1.3
3. normalizeurl 0.1.3 expects the Node.js url module. If that doesn't exist, it falls back to its own parsing code.
4. normalizeurl's fallback parsing code is limited and throws a 'Cannot normalize absolute and protocol-relative urls passed as a string without the node.js url module' error whenever you try to get zooniverse.org/api/me
5. panoptes-client hides that error, so it just looks like you can't login for some mysterious reason

normalizeurl 1.0.0 fixes the fallback parsing code that was borked in 0.1.3.

Status

Solution in progress. Solution cannot be applied at either PJC or json-api-client, AFAICT.

• Workaround (if you need things fixed right now): 1. add resolve.fallback = { url: require.resolve("url") } and 2. add url to package.json's dependencies
• Long term (by end of week): I'm in the process of upgrading panoptes-client and json-api-client, so a few version bumps should fix the problem for all projects.

Solution must be performed at the front end project which uses Webpack 5

1. add resolve.fallback = { url: require.resolve("url") } to the Webpack config.
2. add url to package.json's dependencies

Figure out why this isn't working in Node

I just tried accessing a resource in a subject upload script I'm writing in Node, and the client silently exits. I don't suppose you've tried anything like this @chrissnyder / @brian-c ?

Tests are broken

I'm not sure how it happened, but the test index.js file is looking for a module that doesn't exist, so the tests cannot run. Quickest thing to do is to take out that linked line, but I'm not sure how that happened to begin with.

Client creates a new token for each page load/refresh

I'm not sure what's going on, but if you look at the Network requests you'll see a POST /oauth/token every time, which then generates a new token on the Panoptes side. Shouldn't the client, after login, stick the access token, refresh token and expiry time somewhere in a cookie/localstorage and keep using that?

Calling save() on fetched incomplete classification results in GET, not PUT

api.type('classifications/incomplete')
  .get()
  .then(classifications => {
    classifications[0].save(); // GET https://panoptes-staging.zooniverse.org/api/classifications/77449
  });

Re-saving complete classifications results in a PUT as expected (though obviously the API rejects the save), as does resaving other types, e.g. subjects.

I'm experiencing this on OW but I'll put together a test case to rule out project-specific weirdness.

Clean up NPM

We've got a bunch of different libs on NPM for Panoptes: https://www.npmjs.com/search?q=panoptes

2 hour OAuth token refresh timer is probably unreliable

panoptes-javascript-client/lib/oauth.js

Lines 245 to 247 in 965580b

	var refresh = this._refreshBearerToken.bind(this);
	var timeToRefresh = (tokenDetails.expires_in * 1000) - TOKEN_EXPIRATION_ALLOWANCE;
	this._bearerRefreshTimeout = setTimeout(refresh, timeToRefresh);

The OAuth module sets a 2 hour javascript timer when it receives a new token, basically to remind itself to refresh the token when that timer expires. @shaunanoordin raised a good point about this in conversation on Slack. Suppose the timer starts, runs for an hour or so, then my computer goes to sleep. Is timer state preserved when the computer wakes again? If the computer wakes after the timer has expired, will the refresh function still run?

A better solution would be to either hand off responsibility for checking the token to the client app, as per #80, or use a much shorter timer eg. check token validity every few minutes or so, renewing the timer (if the token is still valid) until the token expires or the browser tab closes.

Add support for authorization code flow in auth module

zooniverse/planetary-response-network uses the authorization code flow instead of implicit because our app is a server app. Currently we're doing the code/access token gaining manually (well, via PassportJS). We then stuff the access token into the apiClient in the same way as the auth module does (apiClient.headers.Authorization = 'Bearer ACCESS_TOKEN_HERE'). This is OK, but it means we have to handle the refresh token stuff ourselves, instead of getting it for free with the auth module, which we would if we were using implicit flow.

Is there an appetite for a PR adding support for code flow? I know most of the Panoptes apps are front end so implicit works fine for them.

Ability to copy a Federated Projects 'template project' workflows via the client

This is part of the Notes from Nature Year 2 effort, see zooniverse/Panoptes-Front-End#2968

This particular request is so that BioSpex, the content management system about half of Notes from Nature researchers use, will be used as a portal through which to create subject sets, pick a template workflow, etc. in order to add a new project to Notes from Nature federated project. See 'Plans to Support Biospex' at https://docs.google.com/document/d/1_9yhO_i7wQf_eph70H5pucD9EeUHXtBCSnI5WbDKnYU/edit