52north / javaps

Next generation standardized web-based geo-processing

javaPS

Build Status

• Develop:

Description

Next generation standardized web-based geo-processing.

Interoperable processing framework for web applications/distributed workflow systems

javaPS enables the deployment of geo-processes on the web in a standardized way. One of its main goals ist the provision of an interoperable processing framework for web applications or distributed workflow systems. It features a pluggable architecture for processes and data encodings.

javaPS implements the OGC WPS specification, version 2.0.

Features:

• General Features
  • Full java-based Open Source implementation.
  • Pluggable framework for algorithms and XML data handling and processing frameworks
  • Built upon the arctic-sea project
  • Streaming de-/encoders for inputs and outputs

Key Technologies:

• Java
• Spring
• Arctic Sea
• OpenAPI

Benefits

• Abstraction of (existing) processing tools
• Support for the majority of standardized geo-data formats
• Lightweight API, following the latest state of the draft OGC API Processes standard

Quick Start

Use git to clone the javaPS repository:

git clone https://github.com/52North/javaPS.git

Then just run mvn clean install on the repositories root directory.

User guide/tutorial

Check out the JavaPS Documentation.

Demo

Try out the latest javaPS alpha-release on on our Geoprocessing Demo Server.

License

This project is licensed under the Apache Software License, version 2.0.

Changelog

Fixed issues ---

• Bugfix, add allowedValues element
• REST Serialize UOM
• Fix missing .jsp file

Dependency updates ---

• Bump junit from 4.12 to 4.13.1
• Bump version.jackson from 2.10.3 to 2.12.1
• Bump guava from 28.2-jre to 29.0-jre
• Update parent and artic-sea versions
• Bump commons-io from 2.6 to 2.7

Other ---

• Bump year

References

• WPS for Tsunami Simulation - WPS service deployed in the RIESGOS project.
• OGC Testbed 15
• OGC Routing Pilot
• WaCoDiS

Contact

• Christian Autermann (c.autermann (at) 52north.org)
• Benjamin Pross (b.pross (at) 52north.org)

Support

You can get support in the community mailing list and forums: https://52north.org/discuss/#mailing-lists

Contribute

Are you are interesting in contributing to javaPS and you want to pull your changes to the 52°North repository to make it available to all? In that case we need your official permission and for this purpose we have a so called contributors license agreement (CLA) in place. With this agreement you grant us the rights to use and publish your code under an open source license. A link to the contributors license agreement and further explanations are available here: https://52north.org/software/licensing/guidelines/

Credits

• Christian Autermann, @autermann
• Benjamin Pross, @bpross-52n

Contributing Organizations

Funding projects

• GLUES
• TaMIS
• OGC Testbeds
• RIESGOS

javaps's Issues

Develop HTML renderings for REST endpoints

Details to be defined (e.g. using Spring templates, ...)

How to publish & call a command line utility as WPS using JavaPS?

I have several algorithms that run as commandline utilities. I have to publish them as WPS processes. Some of these processes need geotiff files as input.

I hava tried to use the GenericFileDataWithGTBinding and GenericFileDataWithGT to get the geotiff file at the server side

 private GenericFileDataWithGT tiff;

 @ComplexInput(identifier = "tiff", minOccurs = 1, maxOccurs = 1, binding = GenericFileDataWithGTBinding.class)
 public void setTiff(GenericFileDataWithGT tiff) {
     this.tiff = tiff;
 }
 @Execute
 public void run() {
    String path = tiff.getBaseFile(true).getAbsolutePath();
    log.debug(tiff.getMimeType());
    log.debug(path);
   // return a GridCoverage2D object, error: the file is invalid. 
    this.result = GeoTiffUtils.read(path); 
 }

I hava tested the following codes using wps-client-lib, but the file received at the server side is invalid, it is larger than the original file and cannot read by gdal. Besides, the received file is end with ..tmp, not .tmp or .tif

ExecuteRequestBuilder builder = new ExecuteRequestBuilder(describeProcessDocument);
File file = new File("J:/demos/zts.tif");
//FileInputStream is = new FileInputStream(file);
byte[] bytes = FileUtils.readFileToByteArray(file);
String s = Base64.encodeBytes(bytes);
//also tested using FileInputStream directly, got the same error
builder.addComplexData("dem", s, null, "base64", "image/tiff");

I'm not sure it's the client side error or server side error. The exception at the server side shown that it was caused by the invalid file.

Thanks very much!

---

I have tested the similar code (use GenericFileDataWithGTBinding and GenericFileDataWithGT to get the geotiff) using https://github.com/52North/WPS version 3.6.3, and I got the valid file!

JavaPS eating Errors

After deploying JavaPS to Tomcat I noticed that I am missing a lot of stracktrace errors. Specifically when I have custom algorithm and it throws an error, this error is never printed in the javaps.log, catalina.out or any other log file that I can find. To work around this issue I am wrapping my execute methods in try catches so that I am able to catch almost all errors.

catch (Exception e) {
            StringWriter sw = new StringWriter();
            PrintWriter pw = new PrintWriter(sw);
            e.printStackTrace(pw);
            log.error(sw.toString());
}

This code allows me to print errors to the javaps.log file. However it would be much nicer if the stacktrace/errors were not eaten at the JavaPS level. Is there a way the codebase can be improved to not eat algorithm errors?

Maven install failure on version 1.3.1

Current packages specified in POM are Arctic Sea v7.2.0, however the package resolver cannot find up-to-date versions for some packages, resulting in build failure

related stack trace:

[INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary for 52°North javaPS 1.4.0-SNAPSHOT: [INFO] [INFO] 52°North javaPS .................................... SUCCESS [ 3.986 s] [INFO] 52°North javaPS Processing API Implementation ...... FAILURE [ 14.714 s] [INFO] 52°North Iceland Streaming Interfaces .............. SKIPPED [INFO] 52°North javaPS Service Implementation ............. SKIPPED [INFO] 52°North javaPS KVP Binding ........................ SKIPPED [INFO] 52°North javaPS POX Binding ........................ SKIPPED [INFO] 52°North javaPS Webapp ............................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 19.036 s [INFO] Finished at: 2019-10-09T14:36:37-04:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project engine: Could not resolve dependencies for project org.n52.wps:engine:jar:1.4.0-SNAPSHOT: The following artifacts could not be resolved: org.n52.arctic-sea:shetland:jar :7.2.0-SNAPSHOT, org.n52.arctic-sea:iceland:jar:7.2.0-SNAPSHOT: Could not find artifact org.n52.arctic-sea:shetland:jar:7.2.0-SNAPSHOT -> [Help 1]

May have to do with 7.2.0 versions not being in sonatype directories. For now, our development will use a JavaPS version prior to 1.3.1

[WPS4R] Read scripts from symbolic links

I created this issue at the (orphaned?) WPS repository:

52North/WPS#270

To be honest, I am not 100% sure, if it is better described here ... however, this repository hosts the "new generation WPS", so it my get more attention here 😄

Integration of the REST/JSON binding into the main code base

Preparation

• analyse javaps-rest repository (https://github.com/autermann/javaps-rest)
• check if upstream of javaps-rest (https://github.com/bpross-52n/javaps-rest) diverged from autermann --> GitHub compare / Pull Request (autermann/javaps-rest@master...bpross-52n:master)
• merge changes if required

Integration

• create new module under javaPS: rest (subdirectory + module in pom.xml, with the contents of the javaps-rest module
• adjust webapp/pom.xml to include the new module as a dependency
• if required (~= javaps does not start ;-) ) --> re-integrate spring beans (javaps-rest/src/main/resources/components) into webapp/src/main/resources/components

Environment

• execute the work in a dedicated feature branch feature/rest-module

Testing

• execute in root folder: mvn jetty:run
• evaluate if http://localhost:8080/javaps/rest (or http://localhost:8080/rest) provides the JSON view of the resources (similar to e.g. https://testbed.dev.52north.org/eopad/rest/ )

Implement authorization

Certain endpoints/operations need a security layer, e.g. getting the list of jobs, executing processes and deploying new processes.

Engine.run() should throw OutputEncodingException

It is possible that a OutputEncodingException occurs in the Engine.run() method. This is not handled and this leads to follow-up exceptions.

Integrate transactional REST features into main code base

Details to be defined.

Error when requesting raw data asynchronously

There is no streamwriter that supports the class GetResultResponse and MediaType /. StreamingRawWriter only supports the class ExecuteResponse.

Concurrency issue with LocalAlgorithmRepository

In the LocalAlgorithmRepository, algorithms are instantiated only once at the initialization of the repo. The same instance is used by all executions of the algorithm. This leads to race conditions.

Proposed solution: Each execution needs its own algorithm instance.