g3i / lxhive

A lightweight Experience API Learning Record Store (LRS)

• Current release: 0.10.1
• Supports xAPI spec <= 1.0.3

Important note: lxHive changes ownership. Our organisation name changes from Brightcookie to G3 International. Please update your remotes in your fork.

# given your linked remote's name is "upstream"
git remote set-url upstream <TODO>

Important note: Current version (0.10.0) is not compatible with older lxHive versions. It is not possible to upgrade a legacy lxHive instance. This incompatibility is due to :

• different server requirements
• a changed database model
• behavioural changes (permissions)

See the changelog for more details. At the moment there are no plans to release an open source database migration script. Please get in touch with us should you need to migrate data.

1. Introduction

lxHive is a fast and lightweight open source xAPI conformant Learning Record Store (LRS). lxHive logs and returns activity statements as defined in the Experience API specification (formerly TinCan API) currently at xAPI Version 1.0.3.

The Experience API (also referred to as 'xAPI') is a learning software specification that allows online learning content and systems to interact allowing recording and tracking of all types of learning experiences. It is designed to replace the legacy SCORM Standard and is steered by the US Dept. of Defense ADL (Advanced Distributed Learning). It allows for the efficient aggregation and analysis of learning data as well as allowing learning designers a flexible and intelligent way to design better learning experiences. The Experience API is able to accept learning experiences from any device and/or medium (mobile, tablet, desktop), both in an offline as well as online mode.

The results of learning experiences are stored in a Learning Record Store (LRS). The LRS is defined as part of the Experience API Specification and controls at its core the following functions:

1. Authentication of authorised users
2. Validation of compliance to the xAPI Standard
3. The storage of learning related data
4. Retrieval of learning related data

The application uses MongoDB and PHP and should be easy to install on any web server. It supports Basic Authentication, OAuth 2.0 (Authorization Code Grant) and supports pluggable file storage mechanisms.

2. License

• GNU GPL v3

3. Document storage endpoints

endpoint	xAPI version	PUT	POST	GET	DELETE	Notes	Links
/about	1.0.2	-	-	x	-	(JSON) info about LRS	xAPI, section 7.7
/statements	1.0.2	x	x	x	-	(JSON) create, retrieve xAPI statements	xAPI, section 7.2
/activities	1.0.2	-	-	x	-	(JSON) retrieve s single activity	xAPI, section 7.5
/activities/state	1.0.2	x	x	x	x	(JSON) CRUD - state(s) of an activity	xAPI, section 7.4
/activities/profile	1.0.2	x	x	x	x	(JSON) CRUD - profile(s) of an activity	xAPI, section 7.5
/agents	1.0.2	-	-	x	-	(JSON) retrieve a single agent	xAPI, section 7.6
/agents/profile	1.0.2	x	x	x	x	(JSON) CRUD - profile(s) of an actor	xAPI, section 7.6

• see our wiki for a complete list of lxHive endpoints

4. Installation

Requirements

• PHP >= 5.5.9, with MongoDB extension installed
• (optional) PHPUnit to run tests.
• .htaccess enabled (or similar HTTP rewrite function)
• Composer installed
• Mongo DB installed (requires version >= 3.0)
• OpenSSL

Notes:

• Make sure you have set the date.timezone setting in your php.ini
• lxHive >= 0.10.0 supports PHP 7.x
• since lxHive 0.10.0 we switched the PHP Mongo driver from mongo (deprecated) to mongodb

Setup

• Note: Check out our Wiki for a more comprehensive [step-by-step guide].(https://github.com/g3i/lxHive/wiki/Step-by-step:-Install-lxHive-and-setup-authentication-for-your-app)

1. Application install and set-up

1. Install dependencies via composer install --no-dev -o.
2. Point your server's DocumentRoot directive to the public folder
3. Set up your database & client account:

# Browse to application root
$ cd /<path_to_application_root>
# View available commands
$ ./X
# Run the setup
$ ./X setup
# Create a new user
$ ./X user:create

Notes:

• As an administrator you should first create a new user and authentication token for yourself and assign the super role to it.

2. Create autentication records for your app

./X auth:basic:create     # Create a new basic auth token
./X oauth:client:create   # Or create a new OAuth client (human login)

3. Set-up file Storage and extended config

1. Optionally: Further customise your configuration in src/xAPI/Config/Config.yml

   • TODO: explain options

2. Set up you local file system:

   • Create your storage directory as defined in Config.yml > filesystem.local.
   • Create the required files and log sub directories.
   • Assign the appropiate r\w permissions for your system.

Default file storage structure:

[lxHivE]
    |_ [storage]
    |   |_ [files]
    |   |_ [logs]
    |
    ...

3. Development

Documentation

• Contributing guidelines
• lxHive Wiki
• Compile code documentation: run sh generate-docs.sh from project root (file must be executable)

Unit testing

• Instructions

Benchmarking

• Instructions

4. Contributors

The G3 International team

• Jakob Murko - systems architect, lead developer
• Leo Gaggl - creator, mentor, conformance
• Joerg Boeselt - lead developer, project and community manager, tests, conformance
• Matthew Smith - alpha prototype

lxhive's Issues

change support to xApi 1.0.2

There are many changes to 1.0.3 which is still a working draft. We initially thought we can keep up with the changes but that proved not to be the case.

The current version of the specification is 1.0.2. 1.0.3 is currently being developed by the specification working group.

current Config:

xAPI.latest_version: 1.0.3
xApi.supported_versions: [1.0.3]

Task: Change default support to 1.0.2 in Config.yaml and docs and brand websites

@sraka1 I don't expect anything breaking with these changes? Users can still keep their 1.0.3 configs.

Enable custom oAuth Themes

• expose optional template replacement folder path(s) to config.yaml
• provide example template folder.
• document required vars
• onRequest: check if custom tpl exists and include this instead of the lxHive template

With this the owner has full freedom to customize markup and content of the template and can also replace/extends css and js. Maybe we can even omit the config and do the check by default (for a defined user theme directory)

See also discussion in #8

Return own agent profile data on tempAccessToken/oAuth response

provide user agent profile id (better, the person object) as specified in the specs so it can be loaded straight away on application init

move twig .Cache folders into storage folders

This error stops oauth logins. when you vhost from user accounts (centos)

{"error_message":"Unable to create the cache directory (\/www\/lxHive\/src\/xAPI\/View\/V10\/OAuth\/.Cache\/5\/7)."}

We need to move the twig cache folders out into public/storage.

context.registration MUST be an uuid

We are currently accepting strings as well.

node test -g 'Context: Registration'

PHP 7 compatibility

Hi,

Are there any plans to support PHP 7? You would need to switch to the new mongodb driver in order to do so, as the old legacy driver is not supported in PHP7:

http://stackoverflow.com/questions/34569112/using-legacy-mongodb-driver-with-php7

Get activity state API issue

Get activity state API is getting "Access-Control-Allow-Origin" error issue

POST https://api.experience.at/activities/state?method=GET
quiz.html:1 XMLHttpRequest cannot load https://api.experience.at/activities/state?method=GET. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://mygpcommunity.com.au' is therefore not allowed access. The response had HTTP status code 500.

./X auth:basic:delete fails to remove token from collection

basicTokens

./X auth:basic:list 
[
    {
        "_id": {
            "$id": "5919145ccaa9a74a108b4567"
        },
        "userId": {
            "$id": "5919135bcaa9a7de0e8b4567"
        },
        "key": "<removed>",
        "secret": "<removed>",
        "expiresAt": null,
        "createdAt": {
            "sec": 1494815836,
            "usec": 0
        },
        "name": "client1",
        "description": "",
        "scopeIds": [
            {
                "$id": "59190908caa9a7450d8b4570"
            }
        ]
    },
    {
        "_id": {
            "$id": "591a465bcaa9a784678b4567"
        },
        "userId": {
            "$id": "591a459fcaa9a738678b4567"
        },
        "key": "<removed>",
        "secret": "<removed>",
        "expiresAt": null,
        "createdAt": {
            "sec": 1494894171,
            "usec": 0
        },
        "name": "testing",
        "description": "testing",
        "scopeIds": [
            {
                "$id": "59190908caa9a7450d8b4570"
            }
        ]
    }
]

trying to delete clientt1 token

./X auth:basic:delete
Please enter the the client ID of the token you wish to delete: 5919135bcaa9a7de0e8b4567
Are you sure (y/n): y
Supertoken successfully deleted!

but it is still there

./X auth:basic:list 
[
    {
        "_id": {
            "$id": "5919145ccaa9a74a108b4567"
        },
        "userId": {
            "$id": "5919135bcaa9a7de0e8b4567"
        },
        "key": "<removed>",
        "secret": "<removed>",
        "expiresAt": null,
        "createdAt": {
            "sec": 1494815836,
            "usec": 0
        },
        "name": "client1",
        "description": "",
        "scopeIds": [
            {
                "$id": "59190908caa9a7450d8b4570"
            }
        ]
    },
    {
        "_id": {
            "$id": "591a465bcaa9a784678b4567"
        },
        "userId": {
            "$id": "591a459fcaa9a738678b4567"
        },
        "key": "<removed>",
        "secret": "<removed>",
        "expiresAt": null,
        "createdAt": {
            "sec": 1494894171,
            "usec": 0
        },
        "name": "testing",
        "description": "testing",
        "scopeIds": [
            {
                "$id": "59190908caa9a7450d8b4570"
            }
        ]
    }
]

console command $ ./X user:create throw errors.

After enter the email address and password, error raised as below:

PHP Warning:  Invalid argument supplied for foreach() in /home/vagrant/Code/lxHive/src/xAPI/Console/UserCreateCommand.php on line 58
PHP Stack trace:
PHP   1. {main}() /home/vagrant/Code/lxHive/X:0
PHP   2. Symfony\Component\Console\Application->run() /home/vagrant/Code/lxHive/X:50
PHP   3. Symfony\Component\Console\Application->doRun() /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Application.php:126
PHP   4. Symfony\Component\Console\Application->doRunCommand() /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Application.php:195
PHP   5. Symfony\Component\Console\Command\Command->run() /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Application.php:874
PHP   6. API\Console\UserCreateCommand->execute() /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257

Warning: Invalid argument supplied for foreach() in /home/vagrant/Code/lxHive/src/xAPI/Console/UserCreateCommand.php on line 58

API\Service\User -- line 166

public function fetchAvailablePermissions()
{
   $collection  = $this->getDocumentManager()->getCollection('authScopes');
   $cursor      = $collection->find();

    $this->cursor;

    return $this;
 }

change to

public function fetchAvailablePermissions()
{
   $collection  = $this->getDocumentManager()->getCollection('authScopes');
   $cursor      = $collection->find();

    $this->cursor = $cursor;

    return $this;
 }

Which xAPI version does lxHive support

It is not clear to me which spec lxHive supports?
In the README.md are mentioned 2 different specs.

Is it 1.0.2 or 1.0.3?

500 status response for GET /statements with format: canonical

GET /statements

Here the params

{
            agent:{
                account: {
                    "homePage": "<url>",
                    "name": <name>"
                }
            },
            verb: 'http://adlnet.gov/expapi/verbs/approved',
            format: "canonical"
        }

Query (decoded)

• http://t/statements?agent={"account":{"homePage":"","name":""}}&verb=http://adlnet.gov/expapi/verbs/approved&format=canonical

returns status 500 {"error_message":"Document has no method \"renderCanonical\""}

see also: https://github.com/Brightcookie/lxHive/search?utf8=%E2%9C%93&q=canonical
This seems to be missing indeed.

./X terminal: graceful handling when user (email) not found during auth:create

This applies for all authentication create commands in ./X when entering a non-registered user email

currently

$ ./X auth:basic:create
Please enter a name: Not existing user
Please enter a description: Trying to authenticate a non-existing user
Please enter the expiration timestamp for the token (blank == indefinite): 
Please enter enter the e-mail of the associated user: [email protected]
# WILL THROW NOTICE
# SHOULD SKIP AND OFFER RETRY
PHP Notice:  Undefined index: [email protected] in /var/www/lxHive/src/xAPI/Console/BasicTokenCreateCommand.php on line 95
PHP Stack trace:
PHP   1. {main}() /var/www/lxHive/X:0
PHP   2. Symfony\Component\Console\Application->run() /var/www/lxHive/X:90
PHP   3. Symfony\Component\Console\Application->doRun() /var/www/lxHive/vendor/symfony/console/Application.php:126
PHP   4. Symfony\Component\Console\Application->doRunCommand() /var/www/lxHive/vendor/symfony/console/Application.php:195
PHP   5. Symfony\Component\Console\Command\Command->run() /var/www/lxHive/vendor/symfony/console/Application.php:878
PHP   6. API\Console\BasicTokenCreateCommand->execute() /var/www/lxHive/vendor/symfony/console/Command/Command.php:259
Please select which scopes you would like to enable (defaults to super). Separate multiple values with commas (without spaces). If you select super, all other permissions are also inherited: 
  [0] super
  [1] statements/write
  [2] statements/read
  [3] statements/read/mine
  [4] attachments
  [5] state
  [6] profile
  [7] define
  [8] all/read
  [9] all
 > 9 #NOTE THAT I CAN STILL ENTER VALUES
PHP Catchable fatal error:  Argument 4 passed to API\Service\Auth\Basic::addToken() must be an instance of API\Document\User, null given, called in /var/www/lxHive/src/xAPI/Console/BasicTokenCreateCommand.php on line 133 and defined in /var/www/lxHive/src/xAPI/Service/Auth/Basic.php on line 58
PHP Stack trace:
PHP   1. {main}() /var/www/lxHive/X:0
PHP   2. Symfony\Component\Console\Application->run() /var/www/lxHive/X:90
PHP   3. Symfony\Component\Console\Application->doRun() /var/www/lxHive/vendor/symfony/console/Application.php:126
PHP   4. Symfony\Component\Console\Application->doRunCommand() /var/www/lxHive/vendor/symfony/console/Application.php:195
PHP   5. Symfony\Component\Console\Command\Command->run() /var/www/lxHive/vendor/symfony/console/Application.php:878
PHP   6. API\Console\BasicTokenCreateCommand->execute() /var/www/lxHive/vendor/symfony/console/Command/Command.php:259
PHP   7. API\Service\Auth\Basic->addToken() /var/www/lxHive/src/xAPI/Console/BasicTokenCreateCommand.php:133
$ #EXITED

Correct some "bad request" responses

1. xAPI request without Authorization header: 400 Bad Request
   Currently 401 - however, this is a bad request as a required header is missing
2. xAPI request with un-supported X-Experience-API-Version (e.g 99.99.9): 400 Bad request, X-Experience-API-Version is not supported
   Currently 404, the suggested response is stolen from https://lrs.adlnet.gov/xAPI/
3. xAPI request with clear (un-encoded) Authorization credentials (e.g. Authorization:Basic non-extising:user): 400 or 401 (?)
   Currently 500
   This is an edge-case. ADL returns a 500 as well. Both systems seem to stumble over the decoding algorithm. However, I think we should be more specific as this is a common error. I suggest rejecting it with a 400, as this is technically a bad request.

Added Jasmine tests ('Bad requests' section).

temporary access token workflow

As discussed, the temporary token workflow is currently broken.
https://github.com/Brightcookie/lxHive/wiki/Issuing-temporary-access-tokens

Integrate setup of auth scopes in ./Xsetup:db

Two issues:

1. this is frequently forgotten by admins
2. oauth:client:create and auth:basic:create allow creation of clients without checking if scopes exist

/* INVALID token, created before ./X setup:oauth  */
{
    "_id" : ObjectId("57abb1953fab4bd2428b4567"),
    "key" : "TMTLA5AsOjeWJptoHU7CvyTjvYIygypn6lNXS48R",
    "secret" : "t46dUpSBancxetwGAAFKQojmUBQs3X3qZYuBM9NY"
}

/* VALID */
{
    "_id" : ObjectId("5886fc52a097fa1bfb03cdf2"),
    "userId" : ObjectId("57abb14b3fab4ba8428b4567"),
    "key" : "l1xYmcZLQALwDd7EUwV7hjiqOhsDna9rBcPWRisR",
    "secret" : "rhI3O7vYNNacDfBYbxucyDej9NVsfcwEbZpoK0D3",
    "expiresAt" : null,
    "createdAt" : ISODate("2017-01-24T07:03:46.737Z"),
    "name" : "test",
    "description" : "tes",
    "scopeIds" : [ 
        ObjectId("58859423a097fa1dd72a3504")
    ]
}

I am aware that this is largely addressed already in the development of the new release, however since progress lags behind I will solve this for the current

POST /statements with attachment binaries - catch errors

Invalid /statements multipart requests currently cause 500 errors when parsing the binary parts. See example below.

I think we need to implement a parser Validation layer where we catch the most common cases and respond with an xAPI compliant Bad request error instead.

Example response with 500

{"error_message":"file_put_contents(\/www\/lxHive\/storage\/files\/): failed to open stream: Is a directory"
}
//reason: sha2 hash not found??
//500 errors and revealing internals in the message is not great.

global:

• encapsulationBoundary string misses -- prefix
• encapsulationBoundaries are not closed
• trailing -- closure is missing

parts:

• if no data body string between encapsulationBoundarties (e.g. only headers)
• if double CRLF is missing between headers and body string
• if X-Experience-API-Hash header is missing or empty
• if Content-Type header is missing or empty

context.contextActivities - always store array (even if legacy submitted )

specs: https://github.com/adlnet/xAPI-Spec/blob/1.0.2/xAPI.md#4162-contextactivities-property

a) It is allowed to post any of the types as a single activity instead an array:

Single Activity Objects are allowed as values so that 0.95 Statements will be compatible with 1.0.0.

b) but is is required always to return arrays of activities

The LRS MUST return every value in the contextActivities Object as an array, even if it arrived as a single Activity Object.

We do a) but store them as singles and fail on b) as we return the singles as we had stored them.

test: node test -g 'Context: contextActivities'

Implement 'canonical' statement format

The 'canonical' format has been marked as not implemented.

The format requires the usage of appropriate languages from the language maps, see Language Filtering Requirements for Canonical Format Statements.

Currently not a priority feature.

oAuth 2: add password credential grant method

http://tools.ietf.org/html/rfc6749#section-4.3

With this method it would be possible to authenticate via client (form, middleware) without the redirection step to the Authorisation server.

• I suggest that should be restricted to scopes below super as the security barriers are considerably lower

./X auth:basic:create issue

token is created always for first user in users collection, regardless which email selected

include creating filestore in installation process

@sraka1 I wonder if we can include the creation of the file storage folders (including perms) during the installation or setup process. This seems to be a frequent stumbling block.

As a paradigm I suggest the filestore should be always required, regardless if the user switches the config to aws or another adaptor. (writing cache files, like lock files, logs)

1. option: composer script

    "scripts": {
        "post-package-install": [
            "php -r \"mkdir('./storage/files/',0770, true);\""
        ]
    }
   // 0770 recommended by https://www.drupal.org/node/244924

2. option during ./X setup command via League\Flysystem adapter

$config = Yaml::parse(<template-config.yml>);
$fs  = API\Util\Filesystem::generateAdapter($config['filesystem']); //creates folder(s)

Option 1 is IMO the most elegant. I tested both on apache/linux however I am not too sure if it works well with other env. Thoughts?

Errors in the Wiki

Under xApi -> xAPI introduction, it says:

sponsored by the Office of the United Stated Under Secretary of Defense for Personnel and Readiness.

Shouldn't it read

United States

?

Further down, under 'Using the configuration files', 2 methods of overwriting YAML configuration files are mentiones, which are actually only one method 'Config.production.yml':

The defaults are in Config.yml, however you can override them in Config.production.yml or Config.production.yml files.

What would the second one be?

Under 'Development guidelines', it reads:

[...] and fix any bugs you might.

I assume 'you might find' is, what you mean, right? :-)

Extended GET endpoints (PLUS)

For reporting purposes we need to add some custom API for fetching statements and documents

Examples (this is a quick shot):

GET /plus/agents
//  all agents within within the scope of my APIkey

• activities search:

GET /plus/acitivties?where={"key":"value"} 
// a mongo query,  within the scope of my APIkey

• statements search

GET /plus/statements?mlimit=5&order=ASC 
// mlimit == Mongo limit (not xAPI limit), returns first 5 smt, plus pagination

• statements search

/plus/statements?key=result&where={"key":"value"} // a mongo query
/plus/statements?key=authority&where={"key":"value"} // a mongo query
/plus/statements?key=context&where={"key":"value"} // a mongo query

These queries should be available only for certain scopes - maybe we create a dedicated scope?

Learninglocker has a similar approach:

• http://docs.learninglocker.net/statements_api/
• http://docs.learninglocker.net/analytics_api/

Asynchronous queueing

Some tasks like the one described here could easily be pushed back to a queue for later processing, improving the API performance.

1. Implement a message based queue - see 1, 2, 3, 4, 5, etc.
2. Potential background tasks should be identified and workers written for them

oAuth2 accessTokenPost Service response - include client name and description

For apps to display. These are human data anyway.

Error message "The helper "question" is not defined." for old symfony/console

Problem:
"symfony/console": "2.3.*" causes ./X setup:db on a fresh install to display error message:

"The helper "question" is not defined."

Solution/Fix:
"symfony/console": "2.6.*"

./X terminal - Multi-selects: add hint for multi-select

We should add a hint for multi-select options to the terminal output. See example for permissions below. Currently it is not stated that you can select multiple values as comma-separated input.

$ ./X user:create
Please enter an e-mail: [email protected]
Please enter a password: ********
Please select which permissions you would like to enable (defaults to super). If you select super, all other permissions are also inherited: 
  [0] super
  [1] statements/read
  [2] statements/read/mine
  [3] attachments
  [4] state
  [5] profile
  [6] define
  [7] all/read
  [8] all
 > 1,3,4
User successfully created!
Info:{
// user object
}

How about:

Please select which permissions you would like to enable (defaults to super). Separate multiple values with commas. If you select super, all other permissions are also inherited

Implement 'ids' statement format

The 'ids' format is currently marked as not implemented.

Desribed in spec as:

If 'ids', only include minimum information necessary in Agent, Activity, Verb and Group Objects to identify them. For anonymous groups this means including the minimum information needed to identify each member.

Logger enhancements

1. In the light of #18, wouldn't make it more sense to use syslog for production mode and leave Monolog for devel? (performance, stability)
2. Maybe we should replace Monolog::StreamHandler with Monolog::RotatingFileHandler and expose $maxFiles to the yml. So old log files > $maxFiles are getting cleared.

./X auth:basic:create exits ungracefully if user doesn't exist

In order to create t token on needs first a valid user

correct workflow

./X user:create
./X auth:basic:create #use user.email to match

However frequently admins forget the first step or the email is typed wrong. In this case the script exists with an PHP Exception.

./X auth:basic:create
Please enter a name: asthon
Please enter a description: ashton developer
Please enter the expiration timestamp for the token (blank == indefinite): 
Please enter enter the e-mail of the associated user: [email protected]
# Undefined index here!!
PHP Notice:  Undefined index: [email protected] in /home/experience/www/lxHive/src/xAPI/Console/BasicTokenCreateCommand.php on line 95
Please select which scopes you would like to enable (defaults to super). Separate multiple values with commas (without spaces). If you select super, all other permissions are also inherited: 
  [0] super
  [1] statements/write
  [2] statements/read
  [3] statements/read/mine
  [4] attachments
  [5] state
  [6] profile
  [7] define
  [8] all/read
  [9] all
 > 0
#Ungraceful exit here!!
PHP Catchable fatal error:  Argument 4 passed to API\Service\Auth\Basic::addToken() must be an instance of API\Document\User, null given, called in /home/experience/www/lxHive/src/xAPI/Console/BasicTokenCreateCommand.php on line 133 and defined in /home/experience/www/lxHive/src/xAPI/Service/Auth/Basic.php on line 58
[joerg@adlx08 sandbox-lxhive]$ ./X

TODO

• exit with error message (user [email protected] not found!)
• document required workflow more prominently

Documents API: Concurrency validation is not backwards compatible (xAPI Version header, config.yml)

Submitted requests who are requiring eTags in xAPI 1.03. should not require this:

• when sent as former xAPI versions.
• the LRS is configured for support lower than 1.0.3
• when sent in legacy (IE) mode (POST ?method=)

Currently any affected request are validated as 1.0.3, regardless of X-Experience-API-Version header or supported version settings in the Config.yml

Endpoints:

/agents
/activities/state
/activities

Config.yml, set to 1.0.1 but still having issue:

name: myGPcommunity
mode: development
database:
    host_uri: 'mongodb://127.0.0.1'
    db_name: lxHive
xAPI:
    latest_version: 1.0.1
    statement_get_limit: 100
    default_statement_get_format: exact
    supported_versions: [1.0.1,1.0.0]
    supported_auth_scopes: [{ name: super, description: 'Administrate the LRS' }, { name: statements/read, description: 'Read statements' }, { name: statements/read/mine, description: 'Read own statements' }, { name: attachments, description: 'Include attachments' }, { name: state, description: 'Access the State API' }, { name: profile, description: 'Access the Profile API' }, { name: define, description: '(Re)define new Activities' }, { name: all/read, description: 'Full read-only access' }, { name: all, description: 'Full access' }]
    oauth: { reauthorize_time: 0, reauthenticate_time: 2592000, branding: { enabled: false, logo_path: null, header: null, footer: null, css_path: null } }
filesystem:
    exposed_url: /attachments
    in_use: local
    local: { root_dir: '%app.root%/storage/files' }

Also: we need to honour the legacy request mode in the same way:

i.e POST https://api.experience.at/activities/state?method=GET (returns a 500 with irrelevant CORS error message)

ADL example of CORS PUT request: https://github.com/adlnet/xAPI-Spec/blob/1.0.3/xAPI.md#appendix-g-cross-domain-request-example

Since applications like storyline send in legacy mode and don't supply a version param we have in this case to revert to the minimal supported version as defined in config.yml

@kienv @MarcusFabriczy can give you more details

/statements X-Experience-API-Consistent-Through header

We are missing this response header:

• https://github.com/adlnet/xAPI-Spec/blob/master/xAPI.md#723-getstatements

The LRS MUST include the header "X-Experience-API-Consistent-Through", in ISO 8601 combined date and time format, on all responses to Statements requests, with a value of the timestamp for which all Statements that have or will have a "stored" property before that time are known with reasonable certainty to be available for retrieval. This time SHOULD take into account any temporary condition, such as excessive load, which might cause a delay in Statements becoming available for retrieval.

Installation on wamp

i had configured and done the installation as it is. but when i try to run the localhost in port 82 using apache http server i receive a blank page with no error sign.

followed these steps:

Install dependencies via composer install.
Point your server's DocumentRoot directive to the public folder (httpd.conf) apache http server

Sets up my database & client account:

Browse to application root

$ cd /<path_to_application_root>

View available commands

$ ./X

Set up database

$ ./X setup:db

Set up OAuth scopes

$ ./X setup:oauth

Create a new user

$ ./X user:create

Activity Profile API - returned objectType is always "Activity"

I think the activity objectTypesare returned incorrectly. They always appear as objectType "Activity", even if they are of one of the other allowed Activity types

Example:

Statement: ee438260-59d9-489f-ad8a-59cedcf0f21c
POSTS an object

{
    id: "e79cbe5e-be22-4ef9-b16d-0bd64f6a6fb8",
    objectType: "StatementRef"
}

GET http://dev.experience.at/activities?activityId=e79cbe5e-be22-4ef9-b16d-0bd64f6a6fb8
however returns

{"id":"e79cbe5e-be22-4ef9-b16d-0bd64f6a6fb8","objectType":"Activity"}

The xAPI states that an Object can be of these types:

• Activity,
• Agent, Group,
• SubStatement
• StatementRef

My understanding is that ONLY Activites are to be stored in the activity profile api.

There is however a booby-trap. The objectType param is optional for type Activity, so it can be omitted on POST but needs to be identified as Acitivity. The other types listed require the objectType param according to the specs.

So the LRS needs IMO first to validate for an Activity this way:

• true, if Object has objectTypeparam && objectType .value === 'Activity'
• true, if objectType param is NOT set, but Object has a param id with a valid IRI

[Notes]

• issue migrated from https://bitbucket.org/brightcookie/closed-xat-beta/issues/73
• xAPI section: https://github.com/adlnet/xAPI-Spec/blob/1.0.3/xAPI.md#414-object
• quick tests: http://thedesignspace.net/misc/xapi/#top

improve uuid validation

According to RFC4122 these are valid representations of the same uuid:

e05aa883-acaf-40ad-bf54-02c8ce485fb0
E05AA883-ACAF-40AD-BF54-02C8CE485FB0
{e05aa883-acaf-40ad-bf54-02c8ce485fb0}
{E05AA883-ACAF-40AD-BF54-02C8CE485FB0}

• legacy cases: https://docs.python.org/2/library/uuid.html

urn:uuid:e05aa883-acaf-40ad-bf54-02c8ce485fb0
urn:uuid:E05AA883-ACAF-40AD-BF54-02C8CE485FB0

We are currently considering only 1) as valid. (Json schema)
While it is easy to correct the regex we need to make sure that all 4 are treated as 1 statement for all read/writes

I think it doesn't violate statement immutability when we accept all, but lowercasing on on smt storage. On all GET (incl extensions) we should lowercase and strip the request id as well.

affected

• statement.id
• statement.object(StatementRef).id
• statement.context.registration

Cannot execute setup:db on a clean installation

I've done a clean installation of lxHive on a Debian machine. All the required tools and configurations seems to be ok, but when I try to setup the DB I get the following error:

zebyte@lamp www/lxhive.com$ ./X -vvv setup:db
Welcome to the setup of lxHive!


  [InvalidArgumentException]
  The helper "question" is not defined.


Exception trace:
 () at /var/www/lxhive.com/vendor/symfony/console/Symfony/Component/Console/Helper/HelperSet.php:79
 Symfony\Component\Console\Helper\HelperSet->get() at /var/www/lxhive.com/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:549
 Symfony\Component\Console\Command\Command->getHelper() at /var/www/lxhive.com/src/xAPI/Console/SetupDbCommand.php:50
 API\Console\SetupDbCommand->execute() at /var/www/lxhive.com/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:246
 Symfony\Component\Console\Command\Command->run() at /var/www/lxhive.com/vendor/symfony/console/Symfony/Component/Console/Application.php:856
 Symfony\Component\Console\Application->doRunCommand() at /var/www/lxhive.com/vendor/symfony/console/Symfony/Component/Console/Application.php:185
 Symfony\Component\Console\Application->doRun() at /var/www/lxhive.com/vendor/symfony/console/Symfony/Component/Console/Application.php:115
 Symfony\Component\Console\Application->run() at /var/www/lxhive.com/X:90

setup:db

Do you have any idea of what could had gone wrong?
Thank you,

José Carlos Medeiros

POST/PUT statement - tolerate "stored" property

We are currently rejecting stored timestamps when writing statements. However 1.0.2 and 1.0.3 have not this requirement. They both state only that an LRS MUST SET the stored property.

• https://github.com/adlnet/xAPI-Spec/blob/1.0.2/xAPI.md#418-stored
• https://github.com/adlnet/xAPI-Spec/blob/master/xAPI-Data.md#stored

While 1.0.2 doesn't consider the case explicitly, 1.0.3 elaborates this case.

The "stored" property MUST be set by the LRS; An LRS SHOULD validate and then MUST overwrite any value currently in the "stored" property of a Statement it receives.

The stored property is also exempt from the statement immutability in both versions. So it is valid to overwrite them on receiving. There is also no such negative test case in both ADL test suites.

To me this makes sense, especially when LRS communicate with each other. There is no need to strip this property.

./X setup:oauth - duplication of scopes when called multiple times

./X setup:oauth
Setting up default OAuth scopes...
OAuth scopes configured!
./X setup:oauth
Setting up default OAuth scopes...
OAuth scopes configured!

after running the command twice I have the scope records duplicated

/* 1 */
{
    "_id" : ObjectId("58bf65bf3fab4b8a378b4567"),
    "name" : "super",
    "description" : "Administrate the LRS"
}

/* 2 */
{
    "_id" : ObjectId("58bf65c03fab4b8a378b4568"),
    "name" : "statements/write",
    "description" : "Write statements"
}

/* 3 */
{
    "_id" : ObjectId("58bf65c03fab4b8a378b4569"),
    "name" : "statements/read",
    "description" : "Read statements"
}

/* 4 */
{
    "_id" : ObjectId("58bf65c03fab4b8a378b456a"),
    "name" : "statements/read/mine",
    "description" : "Read own statements"
}

/* 5 */
{
    "_id" : ObjectId("58bf65c03fab4b8a378b456b"),
    "name" : "attachments",
    "description" : "Include attachments"
}

/* 6 */
{
    "_id" : ObjectId("58bf65c03fab4b8a378b456c"),
    "name" : "state",
    "description" : "Access the State API"
}

/* 7 */
{
    "_id" : ObjectId("58bf65c03fab4b8a378b456d"),
    "name" : "profile",
    "description" : "Access the Profile API"
}

/* 8 */
{
    "_id" : ObjectId("58bf65c03fab4b8a378b456e"),
    "name" : "define",
    "description" : "(Re)define new Activities"
}

/* 9 */
{
    "_id" : ObjectId("58bf65c03fab4b8a378b456f"),
    "name" : "all/read",
    "description" : "Full read-only access"
}

/* 10 */
{
    "_id" : ObjectId("58bf65c03fab4b8a378b4570"),
    "name" : "all",
    "description" : "Full access"
}

/* 11 */
{
    "_id" : ObjectId("58bf65c23fab4b8d378b4567"),
    "name" : "super",
    "description" : "Administrate the LRS"
}

/* 12 */
{
    "_id" : ObjectId("58bf65c23fab4b8d378b4568"),
    "name" : "statements/write",
    "description" : "Write statements"
}

/* 13 */
{
    "_id" : ObjectId("58bf65c23fab4b8d378b4569"),
    "name" : "statements/read",
    "description" : "Read statements"
}

/* 14 */
{
    "_id" : ObjectId("58bf65c23fab4b8d378b456a"),
    "name" : "statements/read/mine",
    "description" : "Read own statements"
}

/* 15 */
{
    "_id" : ObjectId("58bf65c23fab4b8d378b456b"),
    "name" : "attachments",
    "description" : "Include attachments"
}

/* 16 */
{
    "_id" : ObjectId("58bf65c23fab4b8d378b456c"),
    "name" : "state",
    "description" : "Access the State API"
}

/* 17 */
{
    "_id" : ObjectId("58bf65c23fab4b8d378b456d"),
    "name" : "profile",
    "description" : "Access the Profile API"
}

/* 18 */
{
    "_id" : ObjectId("58bf65c23fab4b8d378b456e"),
    "name" : "define",
    "description" : "(Re)define new Activities"
}

/* 19 */
{
    "_id" : ObjectId("58bf65c23fab4b8d378b456f"),
    "name" : "all/read",
    "description" : "Full read-only access"
}

/* 20 */
{
    "_id" : ObjectId("58bf65c23fab4b8d378b4570"),
    "name" : "all",
    "description" : "Full access"
}

after running ./X auth:basic:create, it shows PHP Notice and only lists numbers without textual information

PHP Notice:

PHP Notice:  Array to string conversion in /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Helper/QuestionHelper.php on line 121
PHP Stack trace:
PHP   1. {main}() /home/vagrant/Code/lxHive/X:0
PHP   2. Symfony\Component\Console\Application->run() /home/vagrant/Code/lxHive/X:50
PHP   3. Symfony\Component\Console\Application->doRun() /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Application.php:126
PHP   4. Symfony\Component\Console\Application->doRunCommand() /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Application.php:195
PHP   5. Symfony\Component\Console\Command\Command->run() /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Application.php:874
PHP   6. API\Console\BasicTokenCreateCommand->execute() /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
PHP   7. Symfony\Component\Console\Helper\QuestionHelper->ask() /home/vagrant/Code/lxHive/src/xAPI/Console/BasicTokenCreateCommand.php:78
PHP   8. Symfony\Component\Console\Helper\QuestionHelper->validateAttempts() /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Helper/QuestionHelper.php:58
PHP   9. Symfony\Component\Console\Helper\QuestionHelper->Symfony\Component\Console\Helper\{closure}() /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Helper/QuestionHelper.php:369
PHP  10. Symfony\Component\Console\Helper\QuestionHelper->doAsk() /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Helper/QuestionHelper.php:55
PHP  11. sprintf() /home/vagrant/Code/lxHive/vendor/symfony/console/Symfony/Component/Console/Helper/QuestionHelper.php:121

Please select which scopes you would like to enable (defaults to all available).
  [0] Array
  [1] Array
  [2] Array
  [3] Array
  [4] Array
  [5] Array
  [6] Array
  [7] Array
  [8] Array

because this is multiple choice and the way to list the choice items is to fetch the supported_auth_scopes in config file.

but the QuestionHelper.php -- line 119

foreach ($question->getChoices() as $key => $value) {
                $messages[] = sprintf("  [<info>%-${width}s</info>] %s", $key, $value);
}

so we need to change the config file and find the other way to show the multiple choice items.

Frontend: Exception

Please delete this.

Review authorization scope

as per https://github.com/adlnet/xAPI-Spec/blob/1.0.0/xAPI.md#details-25.

Include StatementRefs in StatementResults

Needs to pass Jasmine tests

https://github.com/adlnet/xAPI-Spec/blob/master/xAPI.md#filter-conditions-for-statementrefs

CORS: expose custom headers to XMLHttpRequest (ajax clients)

Custom response headers are not available within the XMLHttpRequest object (ajax) and thus cannot be evaluated.

Example:

The LRS MUST include the header "X-Experience-API-Consistent-Through"

This header is not currently not available to the (ajax) client via xhr.getAllResponseHeaders()

Research...

CORS allows only exposure a few response headers to be exposed by default

Cache-Control
Content-Language
Content-Type
Expires
Last-Modified
Pragma

the server needs to specifically expose any other header :

• https://www.w3.org/TR/cors/#access-control-expose-headers-response-header
• https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS#Access-Control-Allow-Headers

Monolog file handler: split production and development logs

Currently both modes logging into the same file.

oAuth login error: Object of class MongoDate could not be converted to int

This occurs on a fresh install of lxHive, after creating first oAuth client

POST /oauth/authorize?client_id=vDtf04DQwEeupW59Fb6OOutSQj0GiMGXY4hAfYT7&response_type=code&scope=statements%2Fread&redirect_uri=http%3A%2F%2Flxdash.bo%2Fauth%2FoAuth.php

returns

{"error_message":"Object of class MongoDate could not be converted to int"}

access.log

"POST /oauth/authorize?client_id=vDtf04DQwEeupW59Fb6OOutSQj0GiMGXY4hAfYT7&response_type=code&scope=statements%2Fread&redirect_uri=http%3A%2F%2Flxdash.bo%2Fauth%2FoAuth.php HTTP/1.1" 500 724 "http://lxhive.bo/oauth/authorize?client_id=vDtf04DQwEeupW59Fb6OOutSQj0GiMGXY4hAfYT7&response_type=code&scope=statements%2Fread&redirect_uri=http%3A%2F%2Flxdash.bo%2Fauth%2FoAuth.php" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0"

Client console snapshot:

./X oauth:client:create
 // blah..
OAuth client successfully created!
Info:
{
    "clientId": "vDtf04DQwEeupW59Fb6OOutSQj0GiMGXY4hAfYT7",
    "secret": "qqFeuDlVyI3s2GsUVldYnvjRag36NeeUBtZigSkY",
    "description": "lxDash Admin",
    "name": "lxDash",
    "redirectUri": "http:\/\/lxdash.bo\/auth\/oAuth.php",
    "_id": {
        "$id": "55caa4683fab4b8a208b4567"
    }
}

oAuth login pure.css cdn broken

pure css cdn doesn't not support https (Error code: ssl_error_bad_cert_domain), so the css wont load

• https://yui.yahooapis.com/pure/0.6.0/grids-responsive-min.css

We need either to fetch it from a different cdn or include it locally.

Options:

• replace link with one of the other supported cdn's: http://purecss.io/customize/
• download rollup and include it locally: http://yui.yahooapis.com/pure/0.6.0/pure-min.css
• download/bower release package and include pure.min.css locally

This issue actually relates to a discussion we had about allowing LRS owners to plug-in their own theme/branding of OAuth, so maybe we tackle issue and enhancement in 0.9.1.

• expose a number of twig vars to config.yaml
• expose logo image destination to config.yaml
• provide a custom.css overwrite path and custom.js path

X-Experience-API-Version header missing (MS VB.Net)

I took the code out of my VR environment and put it on to a ASP web page....
and I get the same result

<%
Dim xmlhttp
on error resume next

Set xmlhttp = Server.CreateObject("MSXML2.ServerXMLHTTP")
xmlhttp.setTimeouts 30,500,1000,1000
xmlhttp.Open "post", "http://360evolve.saas.lxhive.com/statement", false
xmlhttp.setRequestHeader "Accept", "/"
xmlhttp.setRequestHeader "Authorization", "Basic p9cTBJSnsq1meX4lLT5ltnchZ5MT2mR0HnY2Y3Iv:NUmljosTsnNcXMeWmTroJbceax9lw48eUNV61eFn"
xmlhttp.setRequestHeader "X-Experience-API-Version", "1.0.1"
xmlhttp.setRequestHeader "Content-Type", "application/json"

if Err.Number Then response.write Err.Description &"..." : Err.clear

xmlhttp.Send
If Err.Number Then
response.write("Something went wrong at the xAPI End")
Err.Clear
Else
response.write(xmlhttp.ResponseText)
End If

On Error Goto 0
set xmlhttp = nothing

%>

the error I keep getting is {"error_message":"X-Experience-API-Version header missing."}

From other forums I think your server is not recognizing the 1.0.1 as the About i responding with a null !

as the about show...null

Issues accessing /public

Hi,

I have gotten a fresh install going and I don't seem to be able to access /public . I've gone back over the installation steps and I followed them perfectly. I've also had a look through the Apache and PHP logs and then are no errors logged.

Any ideas of what may have gone wrong?

Thanks,
Zac

statement.object.definition.choices creates 400 error

I think this might be an issue with the JSON schema.

consider this statement (build from xAPI spec examples):

 {
    "actor":{
        "objectType": "Agent",
        "name":"Project Tin Can API",
        "mbox":"mailto:[email protected]"
    },
    "verb":{
        "id":"http://example.com/xapi/verbs#sent-a-statement",
        "display":{
            "en-US":"sent"
        }
    },
    "object":{
        "id":"http://example.com/xapi/activity/pingpong",
        "definition": {
            "description": {
                "en-US": "Order players by their pong ladder position:"
            },
            "type": "http://adlnet.gov/expapi/activities/cmi.interaction",
            "interactionType": "sequencing",
            "correctResponsesPattern": [
                "tim[,]mike[,]ells[,]ben"
            ],
            "choices": [
                {
                    "id": "tim",
                    "description": {
                        "en-US": "Tim"
                    }
                },
                {
                    "id": "ben", "description": {
                        "en-US": "Ben"
                    }
                },
                {
                    "id": "ells",
                    "description": {
                        "en-US": "Ells"
                    }
                },
                {
                    "id": "mike",
                    "description": {
                        "en-US": "Mike"
                    }
                }
            ]
        }
    }
};

• Statement passes with param object.definition.choices when POSTing to https://lrs.adlnet.gov/xAPI
• Statement fails with param object.definition.choices when POSTing to lxHive
• Statement fails with empty array object.definition.choices when POSTing to lxHive
• Statement passes without param object.definition.choices when POSTing to lxHive

the rather cryptic error:

[2015-09-08 04:58:25] SlimMonoLogger.ERROR: exception 'API\Validator\Exception' with message 'Statements do not validate. Violations: [] object value found, but an array is required [object] the property actor is required [object] the property verb is required [object] the property object is required [object.id] does not match the regex pattern [0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{8} [object] The property definition is not defined and the definition does not allow additional properties [object] failed to match exactly one schema [object] the property objectType is required [object.id] string value found, but a null is required [object.object] matched a schema which it should not [object.definition.choices] array value found, but an object is required [object] the property openid is required [object] the property mbox is required [object] the property mbox_sha1sum is required [object] the property account is required [object] failed to match all schemas [object] The property id is not defined and the definition does not allow additional properties [object] the property member is required [] failed to match exactly one schema ' in /www/lxHive/src/xAPI/Validator/V10/Statement.php:46 Stack trace: #0 /www/lxHive/src/xAPI/Validator/V10/Statement.php(75): API\Validator\V10\Statement->throwErrors('Statements do n...', Array) #1 /www/lxHive/src/xAPI/Resource/V10/Statements.php(122): API\Validator\V10\Statement->validatePostRequest(Object(Slim\Http\Request)) #2 /www/lxHive/public/index.php(217): API\Resource\V10\Statements->post() #3 [internal function]: {closure}('statements') #4 /www/lxHive/vendor/slim/slim/Slim/Route.php(468): call_user_func_array(Object(Closure), Array) #5 /www/lxHive/vendor/slim/slim/Slim/Slim.php(1357): Slim\Route->dispatch() #6 /www/lxHive/vendor/slim/slim/Slim/Middleware/Flash.php(85): Slim\Slim->call() #7 /www/lxHive/vendor/slim/slim/Slim/Middleware/MethodOverride.php(92): Slim\Middleware\Flash->call() #8 /www/lxHive/vendor/slim/slim/Slim/Middleware/PrettyExceptions.php(67): Slim\Middleware\MethodOverride->call() #9 /www/lxHive/vendor/slim/slim/Slim/Slim.php(1302): Slim\Middleware\PrettyExceptions->call() #10 /www/lxHive/public/index.php(251): Slim\Slim->run() #11 {main} [] []

support for media stream context with request_fulluri http

TinCanPHP uses fopen for remote connection.

Opposite to our competitors we are returning always 404 for valid requests. this is due to a path issue, mainly caused by slim routing.

TCPHP sets the context option request_fulluri = 1, which cannot be overwritten

https://github.com/RusticiSoftware/TinCanPHP/blob/master/src/RemoteLRS.php#L89

This changes the $_SERVER['REQUEST_URI']

• request_fulluri = 0 creates $_SERVER['REQUEST_URI'] = '/statements'
• request_fulluri = 1 creates $_SERVER['REQUEST_URI'] = 'http://myslrs.com/statements'

Slim2 doesn't consider this

$requestUri = $_SERVER['REQUEST_URI']; // <-- "/foo/bar?test=abc" or "/foo/index.php/bar?test=abc"

https://github.com/slimphp/Slim/blob/2.x/Slim/Environment.php#L131, comments are original

$requestUri becomes $env['PATH_INFO'], which eventually ** get's a trailing slash prefixed**

$env['PATH_INFO'] = '/' . ltrim($env['PATH_INFO'], '/'); // <-- Ensure leading slash

https://github.com/slimphp/Slim/blob/2.x/Slim/Environment.php#L148, comments are original

so we end up with a route like this, which creates a 404:

// mind the first /!
/http://myslrs.com/statements

Bang!

As far as I can see this is not fixable with Slim2, except we are going to do some url rewriting on htaccess or apache, which leads to all kind of troubles with subdomains/subfolders etc...
We need to check if slim3 deals with this case. I 've seen they have added stream environment class but i think this is rather for consuming