indeedeng / iwf

iWF is an API orchestration platform offering an orchestration coding framework and service for building resilient, fault-tolerant, scalable long-running processes

iWF project - main & server repo

iWF will make you a 10x developer!

iWF is an API orchestration platform for building resilient, fault-tolerant, scalable long-running applications. It offers an orchestration coding framework with abstractions for durable timers, async/background execution with backoff retry, KV storage, RPC, and message queues. You will build long-running reliable processes faster than ever.

What is iWF

Use case study/examples

• SAGA pattern
  • Java samples, Golang samples, Python samples
• User sign-up/registry in Python/Java
• Abstracted microservice orchestration in Java/Golang
• Employer & JobSeeker engagement in Java or Golang
• Subscription Workflow in Java or Golang

Basic concepts

• Basic concepts overview
• WorkflowState
• RPC
• Persistence

Advanced concepts

• WorkflowOptions
• WorkflowStateOptions
• Persistence Caching

How to use

As a coding framework, iWF provides three SDKs to use with:

• iWF Java SDK and samples
• iWF Golang SDK and samples
• iWF Python SDK and samples

The iWF SDKs required to run with the server:

Using all-in-one docker image

Run the docker command to start the container for:

• IWF service: http://localhost:8801/
• Temporal WebUI: http://localhost:8233/
• Temporal service: localhost:7233

docker run -p 8801:8801 -p 7233:7233 -p 8233:8233 -e AUTO_FIX_WORKER_URL=host.docker.internal --add-host host.docker.internal:host-gateway -it iworkflowio/iwf-server-lite:latest

To update the server version, use docker pull iworkflowio/iwf-server-lite:latest to get the latest image. Or change latest to specify a version tag.

Using docker image & docker-compose

This is the simpler(preferred) option to run the server locally for development.

Checkout this repo, go to the docker-compose folder and run it:

cd docker-compose && docker-compose up

This by default will run Temporal server with it, again:

• IWF service: http://localhost:8801/
• Temporal WebUI: http://localhost:8233/
• Temporal service: localhost:7233

To update the server version, use docker pull iworkflowio/iwf-server:latest to get the latest image. Or update the docker-compose file to specify a version tag.

How to build & run locally

• Run make bins to build the binary iwf-server
• Make sure you have registered the system search attributes required by iWF server:
  • Keyword: IwfWorkflowType
  • Int: IwfGlobalWorkflowVersion
  • Keyword: IwfExecutingStateIds
  • See Contribution for more detailed commands.
  • For Cadence without advancedVisibility enabled, set disableSystemSearchAttributes to true
• Then run ./iwf-server start to run the service . This defaults to serve workflows APIs with Temporal interpreter implementation. It requires to have local Temporal setup. See Run with local Temporal.
• Alternatively, run ./iwf-server --config config/development_cadence.yaml start to run with local Cadence. See below instructions for setting up local Cadence.

Support

Join our Slack channel!

You can also post in our Discussion, or raise an issue.

Contributing

Check out our CONTRIBUTING page.

Posts & Articles & Reference

• Why I created Indeed Workflow Engine
• Durable Timers in Indeed Workflow Engine
• Cadence community spotlights
  • #1
  • #2
  • #3
• iWF is an abstracted Temporal framework. Same for Cadence.
• How ContinueAsNew is built in iWF

iwf's Issues

Add built in search attributes: iwfExecutingState to help find which states is being executed

This will help removing deprecating code

Support array search attribute like array of keyword

Support multiple completion , force completion and graceful completion

add backend config for both cadence and temporal

Consider store query attributes into workflow output to avoid query closed workflows

Return search attributes in an API

maybe workflow/get API? as it can be retried from describe workflow.

Change state local to workflow local attributes

This will be more generic and more useful

allow query attribute to be private

a private query attribute cannot be queried, we may consider renaming query attribute to something else.

SharedAttribute

Or just Attribute

getWithLongWait allows specify maximum wait time

This will be more friendly to use than cadence temporal Java sdk to wait for results

Test search attribute with temporal docker ,

change README to use docker only because Temporalite doesn’t support search attributes

Add operation guide/doc

Allow next state to specify state execution id and allow merging into next state by execution id

Add config to disable search attributes

Support customize StartToClose timeout for StateStart & StartDecide API in WorkflowStateOption

It's hardcoded to 10s today.

Remove unuseful reset types

Support pagination for search API

Version check on server api and pass interpreter version to context

Support reset by state id

This will be more friendly to use than cadence temporal reset

Build a webUI for IWF server

What we can do more than Cadence Temporal web ui:
Two pages as mvp:
Page one: search for workflow , and click on to the workflow page

page two is workflow page :
1: show histories by workflow state execution
2: show internal info of commands and pending data
3. Allow reset by state and state execution id

remove the hack in docker-compose

the hack is from https://github.com/indeedeng/iwf/pull/72/files

Refactor temporal into another pkg so that it can support cadence with same workflow

support Cadence/Temporal activity options into StateOptions (e.g. backoff retry)

Boolean config to disable using search attributes

documentation about contributions

Add link to design doc

start start/decide activity should include error details from worker response

{
  "message": "500 Internal Server Error",
  "source": "GoSDK",
  "stackTrace": "",
  "cause": null,
  "applicationFailureInfo": {
    "type": "GenericOpenAPIError",
    "nonRetryable": false,
    "details": null
  }
}

This won't be useful enough to debug

Change cancel api to close api to support terminate and fail

More search attribute types

improve integ test assertion to reset the counting data on startup, and return the data in completion

WaitForMore allow reset current result status

Future, maybe V2: use matching in server to invoke start/decide APIs instead of calling user code by rest APIs

1. In memory matching only. Activity task will wait for match and return , only wait for 1-2 sec to reserve the activity timeout for APIs
2. Test only
3. Disabled by default
4. IwfWorkflowType as queue name

Rename decider trigger type to decide api trigger type

Support signalWithStart

Replace worker domain with docker.host.internal for local when running in docker compose

New command type for inter-state communications

Idea 1

• any query / search attributes has changed to a certain value. Query attributes are not readable by server so we may only support watching whether or not a query attribute has changed
• easy to implement but not friendly to use

Idea 2

• workflow defines an internal data channel. Any state can listen to it, or wait for results
• hard to design/implement but easy to understand/use)

fail the decide activity when workflow returns needs more result but there is no more

Implement more config template for docker image

the template is commented out today: https://github.com/indeedeng/iwf/blob/main/config/config_template.yaml

See example of how to use it https://github.com/cadence-oss/cadence-notification/blob/master/Dockerfile#L26

It's missing Cadence and other setup, also need to use templating command to converting the value from template into config files

allow specify timeout for all server api

Support long running task like Cadence Temporal long running activity

long running activities are super powerful in Cadence Temporal as it allow using local machine memory and disk for executing.

However, This will not be considered into V2 of IWF. Because today’s pushing to execute task’s won’t work for long running task. So this requires to implement the polling mechanisms to execute tasks.

Workaround 1 is to use workflow instead but which means you may have to upload the large amount of states into other storage like s3

workaround 2 is combining IWF and Cadence Temporal. Use IWF to start a Cadence Temporal workflow, which just executes a long running activity.

Potentially IWF v1 can just schedule a Cadence Temporal activity and let user to execute the activity by using Cadence SDKs. But this will leak the implementation details of Cadence Temporal to IWF users so it’s considered as a “Hack” and will not be a good idea.

support more workflowStartOptions (e.g. IdReusePolicy)

Api to return workflow result : completed state id/execution id and output

Document conceptual data models for this framework

Reset API for Cadence may not work for gRPC

need to test and verify

Provide batch reset system workflow

Change search attribute model to use strongely typed model

Allow empty next state as dead end decision

Activity should validate command id

Or generate default command id on server side

Improve and fix api error handling

It should return 400 as defined in swagger

Cache data objects if the query handler is not cached

Need to evict cache when history event id changed

Add recordEvent to avoid abusing state local attributes

Sdk: rename state local attributes to state local.
method : set/get attributes
RecordEvent

Api to describe workflow status

Running, completed, failed, timeout , canceled