personium / personium-engine

Additional module for personium-core to enable server-side JavaScript execution.

Personium Engine

Additional module for personium-core to enable server-side JavaScript execution.

Documentation

http://personium.io/docs/

Requirements

The component personium-engine works behind of personium-core component. So, you have to launch personium-core and need backends, then you have to configure personium-core to use personium-engine you launched.

Please refer to personium/ansible to prepare them manually.

Launch

There are two options to launch personium-engine. DO NOT make the engine endpoints public. personium-engine is designed to run behind the personium-core.

Using war file

You can use maven to build personium-engine.war file to run on a servlet container.

git clone https://github.com/personium/personium-engine
cd personium-engine
mvn package

After compiling, you can get war file on target folder. ( target/personium-engine.war )

Using docker image

You can build docker container image including personium-engine based on Tomcat image with below command.

docker build . -t personium-engine

After building, you can launch personium-engine in docker container.

docker run -d -p 8080:8080 personium-engine

You can mount volume to use specified configuration.

docker run -d -p 8080:8080 -v /path/to/config.properties:/personium/personium-engine/conf/personium-unit-config.properties personium-engine

Testing

1. Place personium-ex-mailsender jar file and Ext_MailSender.properties file in /personium/personium-engine/extensions. Content of Ext_MailSender.properties is below.

   io.personium.engine.extension.MailSender.smtp.host=localhost
   io.personium.engine.extension.MailSender.smtp.port=1025

2. Launch required software with docker-compose up

3. Launch personium-core on Tomcat9 locally.
   For example, with below comand. ( Set $TOMCAT_DIR environment value to path of directory which Tomcat9 installed, and place personium-core.war on $TOMCAT_DIR/webapps/personium-core.war. )

   JAVA_OPTS="-Djava.util.logging.config.file=$TOMCAT_DIR/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dcatalina.base=$TOMCAT_DIR -Dcatalina.home=$TOMCAT_DIR -Djava.io.tmpdir=$TOMCAT_DIR/temp -Dio.personium.configurationFile=$PWD/src/test/resources/personium-core-unit-config.properties" catalina.sh run
   

4. Execute tests with maven.

   mvn -f pom.xml test
   

License

Personium is licensed under the Apache License, Version 2.0. See LICENSE.txt

personium-engine's Issues

Maintenance of checkstyle and PMD.

Upgrade Rhino version so that ES6 functions can be used

Currently, Luxon does not work since it uses several ES6 functions.

Remove unnecessary properties.

Delete unnecessary items from personium-unit-config-default.properties.

# elasticsearch configurations
io.personium.core.es.hosts=localhost:9300
io.personium.core.es.cluster.name=clusterpersonium
io.personium.core.es.unitPrefix=u0

_p.localbox support

Make boxlocal file operations and OData operations simple.

Add the followings to personium-dao.js.

_p.localbox =  function() {
    try {
        return _p.as('client').cell().box();
    } catch (e) {
        return null;
    };
};

Sample in engine script:

_p.localbox().col('dir1')

ACE list does not return proper role name when "href" is not found.

Relevant Issue: personium/personium-client-java#24

Add new endpoint for executing system scripts.

Enhancement as related to personium/personium-core#89.
Commonly usable scripts is placed in personium-engine module, enable to execute the script without register.

Enables setting of timeout time of engine script.

The engine script timeout time is now fixed at 50 seconds.
I want to be able to set in the Unit property according to the environment and processing content.

An error occurs when reading the TranscellToken created by engine script with core.

This is because the version of lib-common is inconsistent between core and engine.

Schema is not set in Token created by serviceSubjcet.

Cache compiled script objects

Caching compiled script objects will improve the overall performance of engine.

Wrong class name

Latest implementation requires to start with "Packages". Japanese explanation here.

Similar issues

To be fixed (Packages.io.personium.xxx) this time.

1. 'var acl = new io.personium.client.Acl();'
2. 'var ace = new io.personium.client.Ace();'

Others

To be fixed by @SawamiWataru (Packages.io.personium.engine.PersoniumEngineException)

1. 'new io.personium.engine.wrapper._pInputStream(request['input'].stream());'
2. 'new io.personium.engine.wrapper._pJSONObject();'
3. 'new io.personium.engine.adapter._pRequestBodyStream(request['input'].stream());'
4. 'if (e.message !== "io.personium.engine._pEngineException: 404 Not Found")'

Corresponds to authorization code grant flow.

Relevant Issue: personium/personium-core#213

Rename the package name to io.personium.engine.*

Rename the package name to io.personium.engine.*

Box

There is no way to get the box name and schema URL in engine script. The following is the suggested modification.

Wrong BOX URL error.

Relevant Issue :personium/app-personium-calendar#141, personium/personium-client-java#39

Add logging of unknown exception's stacktrace

personium-engine/src/main/java/io/personium/engine/rs/AbstractService.java

Line 334 in facb484

log.warn(" unknown Exception(" + e.getMessage() + ")");

WebDAV file encryption

Same Issue：personium/personium-core#27

getPath function returns Cell URL without ending slash

The following return box URL without an ending slash, different specification from API.
_p.localbox().getPath()

Update Rhino version to 1.7.7.

Engine returns timeout immediately

The code below is executed correctly.

function(request) {
  var Context = org.mozilla.javascript.Context,
  currentContext = Context.getCurrentContext,
  rhinoVersion = currentContext.getImplementationVersion;

  return personium.createResponse(200, rhinoVersion );
};

var _ = require("underscore")._;
var personium = require('personium').personium;
var console = require('console').console;

And the engine returns Timeout , but it doesn't take as long as a few seconds.

Is this correct ?

Loaded method name is invalid in `personium-lib.js `.

The bug between 2 source codes below.

personium-lib.js

PersoniumEngineDao.java

getFileList(), getColList() does not work.

Error message
to Java class "java.lang.Object" is prohibited. (/opt/apache-tomcat-8.0.44/webapps/personium-engine/WEB-INF/classes/js-lib/personium-dao.js#467)

Engine Timeout

After parsing the CSV file, the operations of adding 10000 entries to DB has triggered the Script Timeout Error .

If we cannot set Timeout value dynamically, then maybe we have to find a way to execute time consuming script offline. (AJAX like function in the Engine)

Can not access to other cell by _p.as({cellUrl:mycell}).cell('othercell').

The cause is personium/personium-client-java#8.

Add Cell export import function.

"Require" function may not work from v1.4.x

DaoException '401,Unauthorized' is thrown at _p.as().cell().getToken().

The conditions are as follows.

• _p.as('serviceSubject').cell().getToken()
• _p.as('client').cell().getToken()
• _p.as({accessToken:xxxx}).cell().getToken()

Introduce "_p.asEngine()" method to return an accessor with preset "__engine" accoount and role

Overview

Introduce a new accessor creation method "asEngine()" in the engine library and make
the following code return an accessor with pseudo role "../__/__engine" and pseudo account "__engine"

  _p.asEngine().cell().

Token spec

Engine should then return an access token with the following spec:

Role	Subject
box/_engine	{CellURL}#__engine
../__/_engine	{CellURL}#__engine

With these modifications, box install alone can enable engine to access any box-local resource by distributing ACL to grant "__engine" to access that resource.

Related Core modification

personium/personium-core#431

considerations.

• Is it necessary to accept pseudo-account "__engine" as a "serviceSubject" attribute in the ESC (Engine Service Collection) setting?
• Do we need another pseudo-role, pseudo-account "__rule" for event-rule-triggered execution of engine script?

Upgrade personium-lib-common to 1.5.x

This is necessary to use with core 1.7.18+

Engine Service Library can not register ACL, which is caused by incorrect object names.

There are incorrect object names in personium-dao.js source.

https://github.com/personium/personium-engine/blob/master/src/main/resources/js-lib/personium-dao.js#L287

io._p.* should be io.personium.*

Unify the linefeed code of file to LF.

If data of OData is acquired using filter, correct data may not be acquired

Example:

odata.entitySet("test").query().filter("endDate ge datetimeoffset'2019-04-02T15:00:00.000Z').run()

Even if the above date is changed to the day before or after the target data, the data that can be acquired does not change.

HttpClient close leak.

Related issue:personium/personium-client-java#49

Some REST APIs are not supported in personium-dao.js.

• Message Control
• Service Execution

Add some headers on request to Personium API.

This issue is for personium/personium-core#123.

Uint8Array is not defined error

When I try to use UintXXArray, such as Uint8Array, engine script throw below exception.

Server Error : ReferenceError: "Uint8Array" is not defined

Rhino (current version) seems to support UintXXArray in this page.

https://mozilla.github.io/rhino/compat/engines.html

Remove PMD warnings and Checkstyle errors

Update httpclient lib version.

• Delete unused libraries
• Review library version

Can not access to other cell using serviceSubject.

_p.as('serviceSubject').cell('othercell') does not create Transcell token.

Update Rhino version 1.7.11

Upgrade personium-lib-common to 1.5.2

Update pom.xml.

personium/personium-lib-common#26

Correct to close properly.

301 Redirect is needed for OAuth 2.0 support

In order to support OAuth 2.0, we need to redirect from App Cell to Google Authorization Server.
Currently 302 will not work properly. Must use 301 to move to the server completely.

Tests do not work properly.

Engine Tests are too complicated to run since Engine is supposed to be called from core and again call core .. We probably need to organize and refactor the tests.

Delete unused functions.

SystemScript and so on.

CSV parser support

It would be nice to use a library like this to support parsing and exporting of CSV file within the Engine Script.

Principal cannot be cast to Role

Executing '_p.AclManager.prototype.get' produces the following error.

Server Error : _p.PersoniumException: io.personium.client.Principal$1 cannot be cast to io.personium.client.Role (/opt/apache-tomcat-8.0.44/webapps/personium-engine/WEB-INF/classes/js-lib/personium-dao.js#347)

The root cause is that Principal Class.

Organize PersoniumUnitConfig

Support per-Cell FQDN.

Relevant Issue:personium/personium-core#223