camunda-community-hub / zeebe-http-worker Goto Github PK

We should allow the retry behavior to be dependent on the status code, see e.g. https://softwareengineering.stackexchange.com/questions/341518/rest-how-to-determine-transient-exceptions

I don't think the worker should make a final decision, but it should also be configurable (e.g. 503 should be retried, but 500 not).

We are trying to use Zeebe 1.1.0 with pyzeebe 3.0.0rc3,

and we want to add signal handler on SIGTERM that calls worker.stop().
But we don't know how to correctly call "worker.stop()" that is "async def" from the signal handler.

Is it possible add simple example that uses "worker.stop()" to documentation for "async" pyzeebe 3.0.0, please?

Hi, I can't see in docker hub the latest versions deployed:
https://hub.docker.com/r/camunda/zeebe-http-worker/tags

Only the SNAPSHOT image was updated.

Would it be possible to have all the other versions deployed individually?

Running the docker image using docker-compose file fails. docker run not reading the environment variables specified in the docker-compose file.
`version: "2"

services:
zeebe:
container_name: zeebe_worker_lock_order
image: camunda/zeebe-http-worker
environment:
- ZEEBE_CLIENT_WORKER_DEFAULTNAME='Lock Order'
- ZEEBE_CLIENT_WORKER_DEFAULTTYPE='LockOrder'
- ZEEBE_CLIENT_BROKER_CONTACTPOINT='127.0.0.1:26500'
Failing with below error
zeebe_worker_lock_order | 2020-01-23 15:17:24.031 WARN 1 --- [ main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Failed to start bean 'zeebeClientLifecycle'; nested exception is java.lang.NullPointerException: target
zeebe_worker_lock_order | 2020-01-23 15:17:24.033 INFO 1 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'applicationTaskExecutor'
zeebe_worker_lock_order | 2020-01-23 15:17:24.049 WARN 1 --- [ main] o.s.b.f.support.DisposableBeanAdapter : Destroy method 'close' on bean with name 'zeebeClientLifecycle' threw an exception: java.lang.NullPointerException
zeebe_worker_lock_order | 2020-01-23 15:17:24.066 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
zeebe_worker_lock_order | 2020-01-23 15:17:24.138 INFO 1 --- [ main] ConditionEvaluationReportLoggingListener :
zeebe_worker_lock_order |
zeebe_worker_lock_order | Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
zeebe_worker_lock_order | 2020-01-23 15:17:24.153 ERROR 1 --- [ main] o.s.boot.SpringApplication : Application run failed
zeebe_worker_lock_order |
zeebe_worker_lock_order | org.springframework.context.ApplicationContextException: Failed to start bean 'zeebeClientLifecycle'; nested exception is java.lang.NullPointerException: target
zeebe_worker_lock_order | at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:185) ~[spring-context-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
zeebe_worker_lock_order | at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53) ~[spring-context-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
zeebe_worker_lock_order | at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360) ~[spring-context-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
zeebe_worker_lock_order | at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158) ~[spring-context-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
zeebe_worker_lock_order | at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122) ~[spring-context-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
zeebe_worker_lock_order | at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:894) ~[spring-context-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
zeebe_worker_lock_order | at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.finishRefresh(ServletWebServerApplicationContext.java:162) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
zeebe_worker_lock_order | at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:553) ~[spring-context-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
zeebe_worker_lock_order | at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE] zeebe_worker_lock_order | at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
zeebe_worker_lock_order | at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
zeebe_worker_lock_order | at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
zeebe_worker_lock_order | at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
zeebe_worker_lock_order | at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) ~[spring-boot-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
zeebe_worker_lock_order | at io.zeebe.http.ZeebeHttpWorkerApplication.main(ZeebeHttpWorkerApplication.java:39) ~[classes!/:0.21.0-alpha4]
zeebe_worker_lock_order | at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
zeebe_worker_lock_order | at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:na]
zeebe_worker_lock_order | at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na]
zeebe_worker_lock_order | at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na]
zeebe_worker_lock_order | at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) ~[app.jar:0.21.0-alpha4]
zeebe_worker_lock_order | at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) ~[app.jar:0.21.0-alpha4]
zeebe_worker_lock_order | at org.springframework.boot.loader.Launcher.launch(Launcher.java:51) ~[app.jar:0.21.0-alpha4]
zeebe_worker_lock_order | at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52) ~[app.jar:0.21.0-alpha4]
zeebe_worker_lock_order | Caused by: java.lang.NullPointerException: target
zeebe_worker_lock_order | at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:787) ~[guava-20.0.jar!/:na]
zeebe_worker_lock_order | at io.grpc.internal.AbstractManagedChannelImplBuilder.(AbstractManagedChannelImplBuilder.java:180) ~[grpc-core-1.24.0.jar!/:1.24.0]
zeebe_worker_lock_order | at io.grpc.netty.NettyChannelBuilder.(NettyChannelBuilder.java:128) ~[grpc-netty-1.24.0.jar!/:1.24.0]
zeebe_worker_lock_order | at io.grpc.netty.NettyChannelBuilder.(NettyChannelBuilder.java:123) ~[grpc-netty-1.24.0.jar!/:1.24.0]
zeebe_worker_lock_order | at io.grpc.netty.NettyChannelBuilder.forAddress(NettyChannelBuilder.java:109) ~[grpc-netty-1.24.0.jar!/:1.24.0]
zeebe_worker_lock_order | at io.zeebe.client.impl.ZeebeClientImpl.buildChannel(ZeebeClientImpl.java:119) ~[zeebe-client-java-0.21.1.jar!/:0.21.1]
zeebe_worker_lock_order | at io.zeebe.client.impl.ZeebeClientImpl.(ZeebeClientImpl.java:76) ~[zeebe-client-java-0.21.1.jar!/:0.21.1]
zeebe_worker_lock_order | at io.zeebe.client.impl.ZeebeClientBuilderImpl.build(ZeebeClientBuilderImpl.java:221) ~[zeebe-client-java-0.21.1.jar!/:0.21.1]
zeebe_worker_lock_order | at io.zeebe.spring.client.config.ZeebeClientSpringConfiguration.lambda$zeebeClientObjectFactory$0(ZeebeClientSpringConfiguration.java:39) ~[spring-zeebe-0.7.1.jar!/:0.7.1]
zeebe_worker_lock_order | at io.zeebe.spring.util.ZeebeAutoStartUpLifecycle.start(ZeebeAutoStartUpLifecycle.java:52) ~[spring-zeebe-util-0.7.1.jar!/:0.7.1]
zeebe_worker_lock_order | at io.zeebe.spring.client.ZeebeClientLifecycle.start(ZeebeClientLifecycle.java:46) ~[spring-zeebe-0.7.1.jar!/:0.7.1]
zeebe_worker_lock_order | at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182) ~[spring-context-5.2.1.RELEASE.jar!/:5.2.1.RELEASE]
zeebe_worker_lock_order | ... 22 common frames omitted
zeebe_worker_lock_order |
Gracefully stopping... (press Ctrl+C again to force)
Stopping zeebe_worker_lock_order ... done

`

Hello! We got a problem with russian characters in responce body.
We get:
{
"source": "StrawberryShake.Core",
"message": "продукт не найден: 5098c195-f96a-4a3d-b5d7-68d62e0a05a3"
}
But in zeebe operate (after zeebe-http-worker):
{
"source": "StrawberryShake.Core",
"message":"\u043F\u0440\u043E\u0434\u0443\u043A\u0442 \u043D\u0435 \u043D\u0430\u0439\u0434\u0435\u043D: 5098c195-f96a-4a3d-b5d7-68d62e0a05a3"
}
In elastic (kibana):
\"message\":\"\\u043F\\u0440\\u043E\\u0434\\u0443\\u043A\\u0442 \\u043D\\u0435 \\u043D\\u0430\\u0439\\u0434\\u0435\\u043D: 5098c195-f96a-4a3d-b5d7-68d62e0a05a3\"

We use zeebe-http-worker in k8s, i try to set LC_ALL: en_US.UTF-8 / c.UTF-8 / ru_RU.UTF-8, but it doesn't works.

Can we try something else?

Simple Docker file, comparable to the current Cloud Worker: https://github.com/camunda-cloud/zeebe-http-worker/blob/master/Dockerfile

In order to run http worker in Camunda Cloud SaaS we need a feature to blacklist certain IPs / domains from being accessed.

Otherwise an attacker could use the running http worker to access/modify information within our cluster. Same will hold true for self-managed deployments.

This may become obsolete for Camunda Cloud SaaS if the network topology changes and each resource only sees the other resources it is supposed to see.

Hello,
I am using the http worker to communicate with my Rest Microservices. One of my services accepts a POST request with a JSON body. I tried to set the body cariables in the headers using the zeebe modeler but it does not work. In the modeler there is no option to set Variables, the only options are Variable Mappings and Headers. How can I set the body of my request using the modeler ?
Thank you

Hello! We had to make a few changes to our community's Maven deployment Action. Specifically, we updated the secret names and deployment credentials.

Please update to the latest release, v. 1.2.1, to make sure your Action works. Thank you!

Introduction of Mustache changed the expression format - we still need to support the existing one

The current HTTP worker requires probably too much resources, especially compared to the "legacy" NodeJs one from the cloud.

Checked out the most recent changes and attempted to run the worker without setting any of the new environmental variables such as ENV_VARS_URL (e.g. http://someUrl/config, default: null)
ENV_VARS_RELOAD_RATE (defaullt 15000), etc.

When worker is invoked, the following exception is raised:

java.lang.RuntimeException: Could not load variables from 'null': URI with undefined scheme
	at io.zeebe.http.EnvironmentVariableProvider.getVariables(EnvironmentVariableProvider.java:91) ~[classes/:na]
	at io.zeebe.http.HttpJobHandler.handle(HttpJobHandler.java:57) ~[classes/:na]
	at io.zeebe.http.ZeebeHttpWorkerApplication.handleFooJob(ZeebeHttpWorkerApplication.java:62) ~[classes/:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
	at io.zeebe.spring.client.bean.MethodInfo.invoke(MethodInfo.java:31) ~[spring-zeebe-0.7.1.jar:0.7.1]
	at io.zeebe.spring.client.config.processor.ZeebeWorkerPostProcessor.lambda$null$1(ZeebeWorkerPostProcessor.java:49) ~[spring-zeebe-0.7.1.jar:0.7.1]
	at io.zeebe.client.impl.worker.JobRunnableFactory.executeJob(JobRunnableFactory.java:44) ~[zeebe-client-java-0.21.1.jar:0.21.1]
	at io.zeebe.client.impl.worker.JobRunnableFactory.lambda$create$0(JobRunnableFactory.java:39) ~[zeebe-client-java-0.21.1.jar:0.21.1]
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
	at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) ~[na:na]
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) ~[na:na]
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
	at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: java.lang.IllegalArgumentException: URI with undefined scheme
	at java.net.http/jdk.internal.net.http.common.Utils.newIAE(Utils.java:244) ~[java.net.http:na]
	at java.net.http/jdk.internal.net.http.HttpRequestBuilderImpl.checkURI(HttpRequestBuilderImpl.java:79) ~[java.net.http:na]
	at java.net.http/jdk.internal.net.http.HttpRequestBuilderImpl.uri(HttpRequestBuilderImpl.java:71) ~[java.net.http:na]
	at java.net.http/jdk.internal.net.http.HttpRequestBuilderImpl.uri(HttpRequestBuilderImpl.java:43) ~[java.net.http:na]
	at io.zeebe.http.EnvironmentVariableProvider.refreshM2mToken(EnvironmentVariableProvider.java:132) ~[classes/:na]
	at io.zeebe.http.EnvironmentVariableProvider.createHttpRequest(EnvironmentVariableProvider.java:102) ~[classes/:na]
	at io.zeebe.http.EnvironmentVariableProvider.getVariables(EnvironmentVariableProvider.java:65) ~[classes/:na]

I am also able to duplicate the problem by commenting out these lines in the unit test:

@SpringBootTest(properties = {
    "ENV_VARS_URL=http://localhost:8089/config", "ENV_VARS_RELOAD_RATE=0",    
    "ENV_VARS_M2M_BASE_URL:http://localhost:8089/token", "ENV_VARS_M2M_CLIENT_ID:someClientId", "ENV_VARS_M2M_CLIENT_SECRET:someSecret", "ENV_VARS_M2M_AUDIENCE:someAudience"})

If these are optional configuration parameters, the worker should not fail.

The purpose of these new environment variables is also not quite clear.

Issue: Jobs are executed if worker boots up and jobs are already waiting. As soon as the worker is started and new instances have been started some jobs are executed but at some point further jobs are not activated and instances are pending.
Environment: Cloud and locally
Steps to reproduce locally:

• Start zeebe and operate locally, the easiest way is to use the docker-compose project: https://github.com/zeebe-io/zeebe-docker-compose. Change the operate port because the http worker uses 8080 as well.

cd operate
docker-compose up

• Start worker: get latest jar from https://github.com/zeebe-io/zeebe-http-worker/releases, reproduced with version Zeebe HTTP Worker 0.21.0-alpha4

java -jar zeebe-http-worker.jar

• Deploy a workflow including a service task with type http and start some instances until jobs are not executed anymore. I used https://github.com/camunda-cloud/camunda-cloud-examples for this - in the example the service task calls the github api and updates the labels

  • For the first time this works like a charme

• Wait two minutes

• Start a new instance again: job is not executed

Version v1.2.1 doesn't respond to /actuator/health: connection refused.
This is because pom.xml doesn't contain dependency on spring-boot-starter-web package.

Configuration keys from variable mappings clobber those from custom headers when merged to form the body of a POST request. For example:

Custom Header:
body: {event_type: "message", client_payload: {message_name: "THROW_YO"}}

Input Variable mapping:
test: body.client_payload.test

Expected Output:

{
    body: {
		event_type: "message", 
		client_payload: {
			message_name: "THROW_YO",
			test: ${value of variables.test}
		}
	}
}

Actual output:

{
	body: {
		client_payload: {
			test:"value"
	}
}

I expect a JSON merge. It's probably non-trivial to write in Java, but if you create a ScriptEngine you can have it convert the Maps to JSON and do an Object.assign.

"JavaScript: JSON done right"

If the HTTP Worker gets a 503, it doesn't raise an incident or return an error. The process just halts.

It looks like there are effectively two ways to retrieve the configuration for a job via the ConfigurationMaps class: the get (or getString) or getConfig methods.

However, the custom headers, job variables, and environment variables are added to the config map in a different order than they are referenced by the get method.

For example, if a job has a custom header "foo" with value "foo" and also a variable "foo" with value "bar", a call to get or getString will return "foo" while using getConfig and then retrieving "foo" from the config will result in a value of "bar".

Here is the code in question.

What is the intended order of preference for these variables?

The endpoint returns an JSON Array with following format:

[{ key: 'testKey0', value: 'testValue0' }, { key: 'testKey1', value: 'testValue1' }]

Example from legacy code: https://github.com/camunda-cloud/zeebe-http-worker/compare/cep004_auth_for_http-worker

Three environment variables that will be set:

• AUTH0_BASE_URL <- der endpunkt von dem man die jwt tokens beziehen kann
• AUTH0_CLIENT_ID <- die client Id fuer den spezifischen worker
• AUTH0_CLIENT_SECRET <- das dazugehoerige secret
• AUTH0_AUDIENCE < (e.g. variableEndpointAudience = process.env.ENV_VARS_URL.replace('https://', '').split(':')[0]; //variables.cloud.dev.ultrawombat.com )

Needs to go into https://github.com/zeebe-io/zeebe-http-worker/blob/master/src/main/java/io/zeebe/http/EnvironmentVariableProvider.java

The worker currently doesn't accept receiving xml, it fails to convert it to Json, to solve this issue, I develop a code which makes SOAP request available, transforming it into Json in the worker and returning it to Zeebe. I will put it as a pull request.

If the body is not configured the default should be that all workflow variables are added to the HTTP REQUEST (like the legacy cloud worker: https://github.com/camunda-cloud/zeebe-http-worker/blob/master/src/worker/job.handler.ts#L105)

Currently the type for which the worker is registered is not configurable and set to http. In some use cases it would be useful to set a custom type.

The worker name is already configurable via the environment variable ZEEBE_WORKER_NAME. The same approach would fit to set the worker type with a new env var ZEEBE_WORKER_TYPE.

Hi everyone!

Zeebe and zeebe-http-worker both are very interesting ambitious solutions 👍

I took one of our services which has a healthcheck API point /api/healthcheck which returns the empty body to test zeebe-http-worker. And I had next error

java.lang.RuntimeException: Failed to deserialize response body from JSON: OK
    at io.zeebe.http.HttpJobHandler.bodyToObject(HttpJobHandler.java:189) ~[classes!/:0.22.0]
    at java.base/java.util.Optional.map(Optional.java:258) ~[na:na]
    at io.zeebe.http.HttpJobHandler.processResponse(HttpJobHandler.java:179) ~[classes!/:0.22.0]
    at io.zeebe.http.HttpJobHandler.handle(HttpJobHandler.java:77) ~[classes!/:0.22.0]
    at io.zeebe.http.ZeebeHttpWorkerApplication.handleFooJob(ZeebeHttpWorkerApplication.java:53) ~[classes!/:0.22.0]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:564) ~[na:na]
    at io.zeebe.spring.client.bean.MethodInfo.invoke(MethodInfo.java:31) ~[spring-zeebe-0.22.0.jar!/:0.22.0]
    at io.zeebe.spring.client.config.processor.ZeebeWorkerPostProcessor.lambda$null$1(ZeebeWorkerPostProcessor.java:49) ~[spring-zeebe-0.22.0.jar!/:0.22.0]
    at io.zeebe.client.impl.worker.JobRunnableFactory.executeJob(JobRunnableFactory.java:44) ~[zeebe-client-java-0.22.0.jar!/:0.22.0]
    at io.zeebe.client.impl.worker.JobRunnableFactory.lambda$create$0(JobRunnableFactory.java:39) ~[zeebe-client-java-0.22.0.jar!/:0.22.0]
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) ~[na:na]
    at java.base/java.lang.Thread.run(Thread.java:832) ~[na:na]

Is it a bug or by design?

Thank you.

depends on https://github.com/camunda/team-connectors/issues/86

Some future features will be much easier to implement when using Spring Boot, e.g. readyness probes or configuration of the whole Zeebe client (using https://github.com/zeebe-io/spring-zeebe).

Running the java application locally works But running the docker image fails with:
08:17:25.624 [main] WARN org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springframework.context.ApplicationContextException: Unable to start ServletWebServerApplicationContext due to missing ServletWebServerFactory bean. 08:17:25.627 [main] WARN org.springframework.beans.factory.support.DisposableBeanAdapter - Destroy method 'close' on bean with name 'zeebeClientLifecycle' threw an exception java.lang.NullPointerException: null at io.zeebe.spring.util.ZeebeAutoStartUpLifecycle.stop(ZeebeAutoStartUpLifecycle.java:62) at io.zeebe.spring.util.ZeebeAutoStartUpLifecycle.stop(ZeebeAutoStartUpLifecycle.java:71) at io.zeebe.spring.client.ZeebeClientLifecycle.close(ZeebeClientLifecycle.java:61) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.springframework.beans.factory.support.DisposableBeanAdapter.invokeCustomDestroyMethod(DisposableBeanAdapter.java:339) at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:273) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:571) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:543) at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:1072) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:504) at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:1065) at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1060) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:563) at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) at io.zeebe.http.ZeebeHttpWorkerApplication.main(ZeebeHttpWorkerApplication.java:35) 08:17:25.632 [main] ERROR org.springframework.boot.SpringApplication - Application run failed org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springframework.context.ApplicationContextException: Unable to start ServletWebServerApplicationContext due to missing ServletWebServerFactory bean. at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:156) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:544) at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) at io.zeebe.http.ZeebeHttpWorkerApplication.main(ZeebeHttpWorkerApplication.java:35) Caused by: org.springframework.context.ApplicationContextException: Unable to start ServletWebServerApplicationContext due to missing ServletWebServerFactory bean. at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.getWebServerFactory(ServletWebServerApplicationContext.java:203) at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.createWebServer(ServletWebServerApplicationContext.java:179) at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:153) ... 8 common frames omitted

In some cases the HTTP Worker needs to able to pass on information about the current job, at least the job id (which is required to correlate later on).
This should be supported via expression language.

Maybe it is already possible, then just double check and document it :-)

At the moment a header of URL doesn’t work on a task, and a header of url does.

Is there any good reason to make them case-sensitive?

It’s just friction from a useability perspective. The user’s intent is unambiguous, regardless of the casing of the header, so the worker should ignore case by calling toLowerCase() on header keys before parsing them.

Hi! When calling the grpc-method "/gateway_protocol.Gateway/ActivateJobs" to the zeebe gateway, we need to add the type of task in the headers, for example: "job-type: http".
Does the current implementation allow this without fixing the java code and compiling?

• make Docker image smaller
• Final name for image
• Confirm Repo privacy/license/publishing to Maven (of the worker)

Like https://github.com/camunda-cloud/zeebe-http-worker/blob/master/src/worker/worker.ts#L25

we are trying to get the zeebe-http-worker into camunda-cloud, these are the env-vars we are setting for the container:

**    Environment:
      DEBUG:                             express:*
      ZEEBE_CLIENT_BROKER_CONTACTPOINT:  zeebe-service
      CLUSTER_NAME:                      XXX
      ENV_VARS_URL:                      https://variables.XXX/private/variables/XXX
      ENV_VARS_M2M_BASE_URL:             https://login.cloud.XXX/oauth/token
      ENV_VARS_M2M_CLIENT_ID:            XXX
      ENV_VARS_M2M_CLIENT_SECRET:        XXX
      ENV_VARS_M2M_AUDIENCE:             variables.XXX
**

10:57:25.975 [main] WARN org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'zeebeHttpWorkerApplication': Unsatisfied dependency expressed through field 'jobHandler'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'httpJobHandler': Unsatisfied dependency expressed through field 'environmentVariableProvider'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'environmentVariableProvider': Unsatisfied dependency expressed through field 'config'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'zeebeHttpWorkerConfig': Unsatisfied dependency expressed through field 'environmentVariablesReloadIntervalMs'; nested exception is org.springframework.beans.TypeMismatchException: Failed to convert value of type 'java.lang.String' to required type 'long'; nested exception is java.lang.NumberFormatException: For input string: "${ENV_VARS_RELOAD_RATE:15000}"
10:57:26.074 [main] WARN org.springframework.beans.factory.support.DisposableBeanAdapter - Destroy method 'close' on bean with name 'zeebeClientLifecycle' threw an exception
java.lang.NullPointerException: null
	at io.zeebe.spring.util.ZeebeAutoStartUpLifecycle.stop(ZeebeAutoStartUpLifecycle.java:62)
	at io.zeebe.spring.util.ZeebeAutoStartUpLifecycle.stop(ZeebeAutoStartUpLifecycle.java:71)
	at io.zeebe.spring.client.ZeebeClientLifecycle.close(ZeebeClientLifecycle.java:61)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.springframework.beans.factory.support.DisposableBeanAdapter.invokeCustomDestroyMethod(DisposableBeanAdapter.java:339)
	at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:273)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:571)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:543)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:1072)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:504)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:1065)
	at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1060)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:563)
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
	at io.zeebe.http.ZeebeHttpWorkerApplication.main(ZeebeHttpWorkerApplication.java:39)
10:57:26.371 [main] ERROR org.springframework.boot.SpringApplication - Application run failed

we aren't setting ENV_VARS_RELOAD_RATE intentionally and were expecting the default would kick in ;-)

Originally came up to Allow HTTP 202 for HTTP as a service task could be to invoke an HTTP endpoint:

If that HTTP endpoint returns a 202 it means, that it will do the processing asynchronously. In this case the following behavior of the HTTP Worker would be beneficial:

• Lock the task for a configured timeout (e.g. 10 minutes), so that the HTTP request is not done again
• Do NOT complete the current job, so that the workflow keeps waiting in the service task
• Some external component needs to call back Zeebe with the job id in order to complete it, so that the workflow moves on
• If this does not happen within the timeout, the HTTP request is retried

This might happen around https://github.com/camunda-cloud/zeebe-http-worker/blob/master/src/worker/worker.ts#L136

A discussion is if we need to have this behavior configurable. Or in other words: Do people want the workflow to continue in case of HTTP 202 (I could imagine that this is the expectation for most people).

Together with #3 this improves the support for orchestration of functions. So for example in AWS it could work like this:

As an alternative you could go for an explicit modeling of the asynchronous indirection:

The downside is that the workflow model gets pretty bloated. While it is technically correct and you could argue that it is good that you can see it is asynchronous under the hood, I am pretty sure a lot of people will freak out about it, especially compared to AWS Step Functions where it is always one box.

Also another challenge with that alternative is the correlation (how to route the message to the right waiting workflow instance). In the 202 scenario described above this is solved by the "jobId" from Zeebe. In the async modeling scenario somebody has to create a unique UUID to be used for correlation. This is currently not out-of-the-box with the HTTP cloud worker.

Instead run an own local Mock Http Server

I try example:

https://pyzeebe.readthedocs.io/en/pre-release-3.0.0/worker_quickstart.html#create-and-start-a-worker

It raises exception:

Traceback (most recent call last):
  File "./pyzeebe-example.py", line 6, in <module>
    worker = ZeebeWorker()
TypeError: __init__() missing 1 required positional argument: 'grpc_channel'

In the earlier version of the cloud worker it was possible to set additonal http headers via the Zeebe Task Headers, e.g.

<bpmn:serviceTask id="ServiceTask_BookHotel" name="Book hotel">
      <bpmn:extensionElements>
        <zeebe:taskDefinition type="CAMUNDA-HTTP" retries="" />
        <zeebe:taskHeaders>
          <zeebe:header key="x-api-key" value="${aws-api-key}" />
          <zeebe:header key="url" value="${aws-lambda-base-url}/hotel/book" />
          <zeebe:header key="method" value="put" />
        </zeebe:taskHeaders>
      </bpmn:extensionElements>
 </bpmn:serviceTask>

This is not possible anymore with the current worker, but it makes a ton of sense to have it. It still need to be decided how exactly this is configured. I would probably simply propose a simple naming convention of the key, e.g. 'header-x-api-key', but haven't gave it another thought.

<bpmn:serviceTask id="ServiceTask_BookHotel" name="Book hotel">
      <bpmn:extensionElements>
        <zeebe:taskDefinition type="CAMUNDA-HTTP" retries="" />
        <zeebe:taskHeaders>
          <zeebe:header key="header-x-api-key" value="${aws-api-key}" />
          <zeebe:header key="url" value="${aws-lambda-base-url}/hotel/book" />
          <zeebe:header key="method" value="put" />
        </zeebe:taskHeaders>
      </bpmn:extensionElements>
 </bpmn:serviceTask>

Should work with an empty map instead.

java.lang.RuntimeException: Could not load variables from 'https://variables.cloud.dev.ultrawombat.com/private/variables/0a34365c-73a5-45cc-9063-a16b192c5fd4': No content to map due to end-of-input at [Source: (String)""; line: 1, column: 0] at io.zeebe.http.EnvironmentVariableProvider.getVariables(EnvironmentVariableProvider.java:91) at io.zeebe.http.HttpJobHandler.handle(HttpJobHandler.java:65) at io.zeebe.http.ZeebeHttpWorkerApplication.handleFooJob(ZeebeHttpWorkerApplication.java:53) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at io.zeebe.spring.client.bean.MethodInfo.invoke(MethodInfo.java:31) at io.zeebe.spring.client.config.processor.ZeebeWorkerPostProcessor.lambda$null$1(ZeebeWorkerPostProcessor.java:49) at io.zeebe.client.impl.worker.JobRunnableFactory.executeJob(JobRunnableFactory.java:44) at io.zeebe.client.impl.worker.JobRunnableFactory.lambda$create$0(JobRunnableFactory.java:39) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input at [Source: (String)""; line: 1, column: 0] at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4344) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4189) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3205) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3188) at io.zeebe.http.EnvironmentVariableProvider.getVariables(EnvironmentVariableProvider.java:83) ... 16 more

Double check with existing Cloud HTTP Worker docs (https://github.com/camunda-cloud/zeebe-http-worker) if this docs need to get updated

Help me with integrating this as repo as dependency and get http zeebe worker registered...

I am able to include as dependency with below..

io.zeebe
zeebe-http-worker
1.2.0

But its not registering as Zeebe worker and still unable to invoke worker... adding worker properties in application.yaml does not help either.. Kindly provide info on this ..

Thanks in advance..

Or at least link to https://docs.camunda.io/docs/next/components/connectors/out-of-the-box-connectors/rest/

In the previous release of this task, the following variables were returned:

• statusCode
• body

Without defining any output mapping, it was possible to reference these variables in downstream paths of a workflow. For example, I used the statusCode in one branch of the exclusive gateway as follows:

statusCode == 200 && someOtherProperty != null

This worked.

With the current release, statusCode is no longer available in scope of the parent process. Meaning this variable can no longer be used in downstream steps or conditions unless you define an output mapping on the http task as follows:

Output Mapping:
source: statusCode
target: statusCode

According to the literature, if output mappings are not defined, variables returned from a task or process will be merged into the parent variable scope.

Please refer to the BPMN model and rename from .txt to .bpmn.

Associated code to run the workflow is as follows:

DeploymentEvent deployment =
				client.newDeployCommand().addResourceFromClasspath("simple-http-process.txt").send().join()

        	OffsetDateTime now = OffsetDateTime.now();

		final Map<String, Object> post = new HashMap<>();
		post.put("userid", "1");
		post.put("title", now.toString() +  ": it's all about clean code");
		post.put("body",  now.toString() + " : read some good books on clean code");

		// FOR REST, you specify payload as parameter like this.  In other words there must be context variable called "body"
		final Map<String, Object> payload = new HashMap<>();
		payload.put("postToCreate", post);

		WorkflowInstanceEvent wfInstance =	client
						.newCreateInstanceCommand()
						.bpmnProcessId("simple-http-process")
						.latestVersion()
						.variables(payload)
						.send()
						.join();

simple-http-process.txt

To connect to a zeebe gateway inside the same namespace (Cloud Zeebe cluster) the env var ZEEBE_GATEWAY_ADDRESS can be used.

Currently this worker uses Moustache for templating. As part of the Connectors topic this approach was questioned. Overall sentiment seemed to be in favor of having templating done with FEEL expressions and using the engine to do it.

At the same time, if the worker is used in a connector and corresponding element template, then this setup allows less flexibility to define in and output mappings. As far as I understand, the element template will add explicit input mappings for URL, method, etc. And because there is an explicit mapping, it doesn't simply copy all variables. So in essence the worker is called with only URL, method, etc. So there is nothing to run the template engine against

In order to be able to move this worker in the Camunda Cloud as basis for the HTTP worker we need to

• read additional variables as JSON
• via an URL configured as environment variable 'ENV_VARS_URL'

See https://github.com/camunda-cloud/zeebe-http-worker/blob/master/src/worker/variable.handler.ts#L6

Probe should be available under
/healthz

It would be very useful to have a metrics HTTP endpoint in the http worker so Prometheus can scrape monitoring related information.

Prometheus offers a java client library for it: https://github.com/prometheus/client_java