convert- Example Application Package
This repo provides a simple example Application Package in accordance with the OGC Best Practice for Application Packages.
The convert application package comprises these parts:
- Application Implementation
Abashscript (convert.sh) that implements the application functionality - Application Container Image
A docker container image that packages the convert script - Application CWL
A CWL file that describes the application so that it can be deployed and executed in an OGC API Processes service
The script is a wrapper around the convert utility from ImageMagick. The script implements a resize function that takes an image as input, and outputs a resized version of the image.
convert.sh takes the following command-line arguments...
Usage: convert.sh <function>
where,
<function> The operation to perform - currently only `resize` is supported
Usage: convert.sh resize [--url <url> | --stac <stac-catalog>] <size>
where,
<url> The url to the input image - http/https or file-system path
<stac-catalog> Directory containing a STAC catalog - root file `catalog.json`.
The first asset in the first stac item is used as the input.
<size> The amount to resize = e.g. `50%`
`--url` and `--stac` are mutually exclusive - exactly one must be supplied.
The 'current working directory' of the application execution is the directory into which all outputs must be placed by the application. This location is established by the CWL runner, and is consistent with the approach defined by the OGC Best Practice for Application Packages.
The convert.sh script names the resized output file by reusing the name of the input file with the postfix -resize.
The outputs must be presented as a STAC Catalog. Thus, the application must accompany the output image with STAC Catalog and STAC Item files that reference the image asset.
The resize --url operation is performed by directly invoking the convert utility with the supplied url and size. For example...
convert https://eoepca.org/media_portal/images/logo6_med.original.png 50%
In this case the argument supplied as --stac <stac-catalog> is an input directory within which it is expected to find a static STAC catalogue. The file <stac-catalog>/catalog.json is expected to exist, from where the input asset can be discovered. For simplicity, the first asset in the first STAC item is taken.
Once the input image has been identified then the convert utility is invoked similarly as for the --url case.
For example...
convert <stac-catalog>/eoepca-logo.png 50%
The application script convert.sh is packaged into a container image, along with its dependencies.
Dockerfile...
FROM ubuntu
RUN apt-get update && apt-get -y install curl imagemagick file jq
WORKDIR /app
COPY ./convert.sh /app
ENV PATH="/app:${PATH}"
In accordance with the OGC Best Practice for Application Packages, the application is described by a CWL file.
The CWL application description comprises two parts:
CommandLineTool
Describes the tool to be executed -convert.shis this caseWorkflow
Provides the entry-point to the application package, in accordance with the OGC Best Practice for Application Packages
Two variants of the Application Package are provided to reflect use of either --url or --stac - see files convert-url-app.cwl and convert-stac-app.cwl.
It would be possible to express these exclusive inputs in a single application package (ref. records and type unions) - but for simplicity separate application packages are used.
Describes the convert.sh script in terms of its inputs as command-line arguments, and outputs.
Example for --url...
- class: CommandLineTool
id: convert
baseCommand: convert.sh
inputs:
fn:
type: string
inputBinding:
position: 1
url:
type: string
inputBinding:
position: 2
prefix: --url
size:
type: string
inputBinding:
position: 3
outputs:
results:
type: Directory
outputBinding:
glob: .
requirements:
DockerRequirement:
dockerPull: eoepca/convert:latest
The baseCommand is required to identify the 'executable' to invoke - assumed to be on the PATH within the container environment.
The inputs reflect the stated command-line options for <function>, <url> and <size> - all of type string. The <url> is specified to require the --url prefix.
The outputs are described as being all files that are presented in the output directory.
The application container image for execution is identified as eoepca/convert:latest.
The CommandLineTool for --stac varies only by replacing the url input with the stac input, which is of type Directory, and with the command-line prefix --stac...
stac:
type: Directory
inputBinding:
position: 2
prefix: --stac
Provides the entry-point to the application package, and includes the step to invoke convert.sh via the defined CommandLineTool.
Example for --url...
- class: Workflow
doc: Convert URL
id: convert-url
label: convert url app
inputs:
fn:
type: string
url:
type: string
size:
type: string
outputs:
- id: stac
type: Directory
outputSource:
- convert/results
steps:
convert:
run: "#convert"
in:
fn: fn
url: url
size: size
out:
- results
The workflow matches the inputs of the convert.sh script, which are passed through to the CommandLineTool.
The workflow has a single step, which is the invocation of convert - the outputs of which are mapped as a the outputs of the workflow.
As before, the Workflow for --stac varies only by replacing the url input with the stac input, which is of type Directory...
stac:
type: Directory
Signficantly, this input is of type Directory, which triggers the ADES to perform a stage-in of the inputs from the identified source.
During stage-in the ADES interprets the source, retrieves the identified assets and presents them as a STAC catalog in an input directory for the executing application.
The deafult ADES deployment uses the stars Spatio Temporal Asset Router Services tool to perform the stage-in. stars understands a variety of sources, such as atom feed from an OpenSearch URL, which it is able to use as a source of assets, from where they are retrieved, enumerated and described as a local static STAC calaog for application input.
In the simplest case, the source can be provided as an existing STAC catalog, or even a single STAC item.
For testing and experimentation, the application package can be executed locally using cwltool.
cwltool --outdir out convert-url-app.cwl#convert \
--fn resize \
--url "https://eoepca.org/media_portal/images/logo6_med.original.png" \
--size "50%"
The script run-locally-url.sh is provided as a helper to perform the above CWL workflow execution.
For convenience, the script emdeds a python virtual environent that includes the cwltool.
cwltool --outdir out convert-stac-app.cwl#convert \
--fn resize \
--stac ./stac \
--size "50%"
NOTE
Since the application package is executed outside the context of the ADES stage-in functionality, the--stacmust be provided as 'pre-staged-in' STAC catalog directory.
For this purpose, a simple STAC catalogue has been created in thestac/directory of this repository.
The script run-locally-stac.sh is provided as a helper to perform the above CWL workflow execution.
For convenience, the script emdeds a python virtual environent that includes the cwltool.
The application package can be deployed and executed on an ADES instance.
See also the Deployment Guide Quick Example for more context.
curl --request POST \
--url https://ades-open.<domain>/<user>/wps3/processes \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--data '{"executionUnit": {"href": "https://raw.githubusercontent.com/EOEPCA/convert/main/convert-url-app.cwl","type": "application/cwl"}}'
curl --request POST \
--url https://ades-open.<domain>/<user>/wps3/processes/convert-url-0_1_1/execution \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--header 'prefer: respond-async' \
--data '{"inputs": {"fn": "resize","url": "https://eoepca.org/media_portal/images/logo6_med.original.png","size": "50%"},"response":"raw"}'
curl --request POST \
--url https://ades-open.<domain>/<user>/wps3/processes \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--data '{"executionUnit": {"href": "https://raw.githubusercontent.com/EOEPCA/convert/main/convert-stac-app.cwl","type": "application/cwl"}}'
Demonstration of ADES invocation with inputs as STAC catalog and alternatively as STAC item - noting that, after ADES stage-in, the application will always receive a STAC catalog as input.
Input as STAC catalog...
curl --request POST \
--url https://ades-open.<domain>/<user>/wps3/processes/convert-stac-0_1_1/execution \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--header 'prefer: respond-async' \
--data '{"inputs": {"fn": "resize","stac": "https://raw.githubusercontent.com/EOEPCA/convert/main/stac/catalog.json","size": "50%"},"response":"raw"}'
NOTE input STAC Catalog
catalog.json
Input as STAC item...
curl --request POST \
--url https://ades-open.<domain>/<user>/wps3/processes/convert-stac-0_1_1/execution \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--header 'prefer: respond-async' \
--data '{"inputs": {"fn": "resize","stac": "https://raw.githubusercontent.com/EOEPCA/convert/main/stac/eoepca-logo.json","size": "50%"},"response":"raw"}'
NOTE input STAC Item
eoepca-logo.json
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent