siddhi-io / siddhi-io-http Goto Github PK
View Code? Open in Web Editor NEWExtension that allows you to receive and publish events via HTTP and https transports
Home Page: https://siddhi-io.github.io/siddhi-io-http/
License: Apache License 2.0
Extension that allows you to receive and publish events via HTTP and https transports
Home Page: https://siddhi-io.github.io/siddhi-io-http/
License: Apache License 2.0
Description:
Let's assume a scenario where we have siddhi app with HTTP source and we try to deploy another siddhi app with same HTTP source. In that case, it will remove the HTTP source from the first app also.
Suggested Labels:
@bug
Steps to reproduce:
Description:
HTTP sink GET request fails with HTTP 400 because we are not sending in Host header. Need to follow up on this and investigate why we are not sending Host header and what to be expected as per spec
Affected Product Version:
SP 4.x.x
Steps to reproduce:
Can use below Siddhi App to reproduce
@App:name("SiddhiApp")
@App:description("Description of the plan")
-- Please refer to https://docs.wso2.com/display/SP400/Quick+Start+Guide on getting started with SP editor.
@sink(
type='http-request', sink.id='id1', publisher.url='http://api.worldbank.org/v2/countries/all/indicators/SP.POP.TOTL',
method='GET', headers = "'Host:api.worldbank.org'",
@map(type='json'))
define stream FooStream (payloadBody String);
@source(type='http-response', sink.id='id1', http.status.code='200',@map(type='text' ))
@sink(type = 'log', prefix = 'LOGGER')
define stream responseStream2xx(body string);
@source(type='http-response', sink.id='id1', http.status.code='400', @map(type='text'))
@sink(type = 'log', prefix = 'LOGGER')
define stream responseStream4xx(body string);
Description:
When performing a perf test using http request-response flow, following LEAK error message is getting printed.
[2019-04-19 15:46:12,400] ERROR {io.netty.util.ResourceLeakDetector} - LEAK: ByteBuf.release() was not called before it's garbage-collected. Enable advanced leak reporting to find out where the leak occurred. To enable adv
anced leak reporting, specify the JVM option '-Dio.netty.leakDetection.level=advanced' or call ResourceLeakDetector.setLevel() See http://netty.io/wiki/reference-counted-objects.html for more information.
After setting the leak detection level to "ADVANCED", it provides following logs.
[2019-05-31 06:37:24,564] ERROR {io.netty.util.ResourceLeakDetector} - LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records:
#1:
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:273)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:141)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#2:
io.netty.buffer.AdvancedLeakAwareByteBuf.getByte(AdvancedLeakAwareByteBuf.java:154)
io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:360)
io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:202)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:141)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#3:
io.netty.buffer.AdvancedLeakAwareByteBuf.forEachByte(AdvancedLeakAwareByteBuf.java:670)
io.netty.handler.codec.http.HttpObjectDecoder$HeaderParser.parse(HttpObjectDecoder.java:803)
io.netty.handler.codec.http.HttpObjectDecoder.readHeaders(HttpObjectDecoder.java:603)
io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:227)
io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:202)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:141)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#4:
io.netty.buffer.AdvancedLeakAwareByteBuf.forEachByte(AdvancedLeakAwareByteBuf.java:670)
io.netty.handler.codec.http.HttpObjectDecoder$HeaderParser.parse(HttpObjectDecoder.java:803)
io.netty.handler.codec.http.HttpObjectDecoder$LineParser.parse(HttpObjectDecoder.java:852)
io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:208)
io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:202)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:141)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#5:
io.netty.buffer.AdvancedLeakAwareByteBuf.getUnsignedByte(AdvancedLeakAwareByteBuf.java:160)
io.netty.handler.codec.http.HttpObjectDecoder.skipControlCharacters(HttpObjectDecoder.java:568)
io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:202)
io.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:202)
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:141)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#6:
Hint: 'codec' will handle the message from this point.
io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:116)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:141)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
#7:
Hint: 'DefaultChannelPipeline$HeadContext#0' will handle the message from this point.
io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:116)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:141)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
Created at:
io.netty.buffer.AdvancedLeakAwareByteBuf.writeBytes(AdvancedLeakAwareByteBuf.java:634)
io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:345)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:126)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.lang.Thread.run(Thread.java:748)
: 6 leak records were discarded because they were duplicates
Description:
When calling endpoints directly from a web application; first an HTTP OPTIONS request is sent to figure permission to access selected resources from the server. Currently in the server, if any request other than an HTTP POST request is received the request is dropped with the status code 400[1].
And the following error gets logged
[2018-08-20 09:50:38,052] ERROR {org.wso2.transport.http.netty.listener.SourceHandler} - Error while notifying listeners org.wso2.extension.siddhi.io.http.source.exception.HttpSourceAdaptorRuntimeException: Request type is not a type of POST
at org.wso2.extension.siddhi.io.http.source.HTTPConnectorListener.onMessage(HTTPConnectorListener.java:75)
at org.wso2.transport.http.netty.contractimpl.HttpWsServerConnectorFuture.notifyHttpListener(HttpWsServerConnectorFuture.java:77)
at org.wso2.transport.http.netty.listener.SourceHandler.notifyRequestListener(SourceHandler.java:200)
at org.wso2.transport.http.netty.listener.SourceHandler.channelRead(SourceHandler.java:119)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at org.wso2.transport.http.netty.listener.WebSocketServerHandshakeHandler.channelRead(WebSocketServerHandshakeHandler.java:100)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at org.wso2.transport.http.netty.listener.UriAndHeaderLengthValidator.channelRead(UriAndHeaderLengthValidator.java:64)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
at java.lang.Thread.run(Thread.java:748)
Suggested Labels:
Suggested Assignees:
Affected Product Version:
OS, DB, other environment details and versions:
Steps to reproduce:
Related Issues:
Description:
Incorrect test case naming format which cause to skip test at building.
Suggested Labels:
Bug
Suggested Assignees:
@Yasara123
Description:
In the HTTP sink, the refresh token is not considered even though it is specified.
Affected Product Version:
SI 1.1.0
@source(type = 'http', receiver.url = "http://0.0.0.0:8006/ctab",@map(type = 'json'))
define stream CtabUrl (url string);
@source(type='http-request', source.id='StreamSource', receiver.url='http://0.0.0.0:8007/url', connection.timeout='150000', @Map(type='json', @attributes(messageId='trp:messageId')))
define stream UrlStream (messageId string);
Affected Product Version:
Ubuntu
Description:
I noticed that it is written in Siddhi-IO-HTTP introduction
... This extension only works inside the WSO2 Data Analytic Server and cannot be run with standalone siddhi.
However, in the "How To Use" section below. It suggest the method of including it as a java library. I tried to include it in my siddhi project and it reported "NoSuchMethodError" in receiving http message.
(Detailed message is posted in Issues of siddhi library))
Does that mean if I just want to use siddhi as library then I cannot use this extension?
Is there any way I can do to establish http stream listener by using siddhi library?
Thank you in advance!
Suggested Labels:
Suggested Assignees:
Affected Product Version:
Siddhi - 4.1.17
HTTP Extention - 1.0.18
OS, DB, other environment details and versions:
IntelliJ IDEA 2017.3.5 (Community Edition)
Build #IC-173.4674.33, built on March 6, 2018
JRE: 1.8.0_152-release-1024-b15 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Windows 10 10.0
Steps to reproduce:
Related Issues:
Description:
In the existing implementation [1], if the exception throws while initializing the component, its bubble up to OSGi framework, but the problem is carbon logging uses pax logging as logging framework and it's not supported for Eclipse Equinox logging framework. Hence this OSGi bundle exception not captured by the logging framework.
A fix would be: wso2/carbon-secvault#55
Suggested Labels:
v1.x.x
Suggested Assignees:
Affected Product Version:
OS, DB, other environment details and versions:
Steps to reproduce:
Related Issues:
N/A
Description:
Hostname verification should be enabled by default. But, as per the current implementation, there isn't any defined property in the transport level that would trigger this behavior.
Suggested Labels:
wso2sp
Affected Product Version:
wso2sp-4.3.0
Description:
For the following Siddhi app deployment in K8s, I got the following error and the Siddhi runner did not come to the healthy state due to this error.
@App:name("0.InputRoute-passthrough")
@source(type='http', receiver.url='http://0.0.0.0:9090/route', @map(type='json'))
define stream passthroughInputRouteStream(routeNo string);
@sink(type='nats',cluster.id='siddhi-stan',destination = '0.InputRoute_InputRouteStream', bootstrap.servers='nats://siddhi-nats:4222',@map(type='text'))
define stream InputRouteStream(routeNo string);
from passthroughInputRouteStream select * insert into InputRouteStream;
[2020-01-18 02:50:47,196] ERROR {org.wso2.transport.http.netty.contractimpl.listener.states.ReceivingHeaders} - Error while notifying listeners io.siddhi.extension.io.http.source.exception.HttpSourceAdaptorRuntimeException: Request type is not a type of POST
at io.siddhi.extension.io.http.source.HTTPConnectorListener.onMessage(HTTPConnectorListener.java:76)
at org.wso2.transport.http.netty.contractimpl.HttpWsServerConnectorFuture.notifyHttpListener(HttpWsServerConnectorFuture.java:72)
at org.wso2.transport.http.netty.contractimpl.listener.states.ReceivingHeaders.notifyRequestListener(ReceivingHeaders.java:98)
at org.wso2.transport.http.netty.contractimpl.listener.states.ReceivingHeaders.readInboundRequestHeaders(ReceivingHeaders.java:77)
at org.wso2.transport.http.netty.contractimpl.listener.states.ListenerReqRespStateManager.readInboundRequestHeaders(ListenerReqRespStateManager.java:38)
at org.wso2.transport.http.netty.contractimpl.listener.SourceHandler.channelRead(SourceHandler.java:130)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at org.wso2.transport.http.netty.contractimpl.listener.WebSocketServerHandshakeHandler.channelRead(WebSocketServerHandshakeHandler.java:135)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at org.wso2.transport.http.netty.contractimpl.listener.UriAndHeaderLengthValidator.channelRead(UriAndHeaderLengthValidator.java:64)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:141)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Unknown Source)
Affected Product Version:
Siddhi runner 5.1.2
Description:
When the sink receives a 301 response for a request, it does not redirect to the URL it receives.
Affected Product Version:
SP 4.0.0, 4.1.0, 4.2.0, 4.3.0
Description:
If the port we have used to make the HTTP(S) connection is already being used by a different program in the server, an error must be thrown. However, no such error is displayed on the console as of now. The only observation is, the connection messages such as 'Source Listener has created for url http://localhost:5005/inputStream' would not be displayed when the Siddhi application containing the relevant HTTP source configuration is run.
Steps to reproduce:
Consider the following HTTP source configuration (Here, 8005 is assumed to be a port already being used by a different program in the system). Add this configuration to a siddhi app, save it and run on the editor environment (or as worker) in WSO2 Stream Processor. You would notice that no connection messages are displayed on the console. However, no errors are displayed either. When attempting to publish data only, the user would know that the HTTP connection is not established.
@source(type = 'http', receiver.url='http://localhost:8005/inputStream', basic.auth.enabled='false',
@Map(type='text'))
define stream inputStream(name string, age int, country string);
This issue must be resolved by logging proper error messages, when the port used in HTTP source configuration is already used by a different program.
Description:
When making a GET request for a resource, the whole URL sent as the resource that needs to be accessed. Therefore it fails with a 404.
The following is a correct GET request to the same resource.
Suggested Labels:
bug
Suggested Assignees:
@minudika
Affected Product Version:
WSO2SP-4.0.0, WSO2SP-4.1.0
Description:
The query params placed in publisher.url
are not been sent to the server.
Here is the code snippet
@App:name("Http_trial")
@App:description("Description of the plan")
define stream SweetProductionStream (projectCode string, totalBid int );
@sink(type="http",
method="POST",
headers="'Content-Type:application/json','Accept:application/json','Host:192.168.43.79'",
follow.redirect='true',
publisher.url = "http://192.168.43.79:8080/path?abc=xyz",
@map(type='json', validate.json='true', @payload("""{"properties":[{"name":"amount","value":"100"}]}""")))
define stream LowProductionAlertStream (projectCode string, totalBid int);
@info(name = 'query1')
from SweetProductionStream
select *
insert into LowProductionAlertStream;
Packet captured using Wireshark
The request uri seen in the packet details only contains /path
and is missing ?abc=xyz
Is there any other way to specify query params ? Or am I missing something here ?
Affected Product Version:
v1.0.33
OS, DB, other environment details and versions:
Windows 10
and Ubuntu 16.04
Steps to reproduce:
Use WSO2 Stream Processor Studio to create the file.
Create a http sink having query params in publisher.url
Description:
When type='keyvalue', the ‘application/x-www-form-urlencoded’ request header cannot be automatically added
Suggested Labels:
Suggested Assignees:
Affected Product Version: all
OS, DB, other environment details and versions:
Steps to reproduce:
'io.siddhi.extension.io.http.sink.HttpSink#generateCarbonMessage' function, 'if (contentType.contains(mapType)) {
httpHeaders.set(HttpConstants.HTTP_CONTENT_TYPE, contentType);
}' logic vulnerability.
Related Issues:
Description:
Some Oauth providers require the token request body to include the client credentials in the request body as seen in here, currently our implementation only support username and password and the grant_type in the message body.
Description:
After posting to SharePoint Online, I received the following error messages - and can't post to SharePoint Online for a little while after that.
INFO {org.wso2.transport.http.netty.contractimpl.sender.TargetHandler} - Exception occurred in TargetHandler : Connection reset by peer
INFO {org.wso2.transport.http.netty.contractimpl.sender.TargetHandler} - Exception occurred in TargetHandler : Connection reset by peer
INFO {org.wso2.transport.http.netty.contractimpl.sender.TargetHandler} - Exception occurred in TargetHandler : Connection reset by peer
Suggested Labels:
Suggested Assignees:
Affected Product Version:
1.1.0
OS, DB, other environment details and versions:
Steps to reproduce:
Related Issues:
Description:
Siddhi HTTP Sink documentation not clear about SSL parameters to bypass SSL certification errors (i.e: expired or self signed).
Suggested Labels:
Type/Docs, Type/Question
Affected Product Version:
WSO2 Stream Processor 4.3.0
OS, DB, other environment details and versions:
Red Hat Enterprise Linux Server release 7.5 (Maipo)
Steps to reproduce:
Tested using this sink configuration:
@sink(type='http',
publisher.url='https://xxx.xxx.xxx.xxx:5050/v1/executions',
headers="'content-type:application/json','Accept-Encoding:gzip, deflate','Connection:keep-alive','X-Auth-Token: be991689c1394444b4966ed400a7eb80'",
method='POST',
@map(type='json', @payload(""" {"action":"{{action}}","parameters":{"url":"{{managed_object}}"} } """)))
In this case the HTTPS endpoint has a self-signed certificate and the http sink produces this exception:
[2018-11-16 10:54:29,871] WARN {io.netty.channel.DefaultChannelPipeline} - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception. io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1478)
at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:813)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:281)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1215)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1127)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1162)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
... 16 more
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1514)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:966)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:963)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1416)
at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1364)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1272)
... 20 more
Caused by: java.security.cert.CertificateException: No subject alternative names present
at sun.security.util.HostnameChecker.matchIP(HostnameChecker.java:145)
at sun.security.util.HostnameChecker.match(HostnameChecker.java:94)
at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:455)
at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:436)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:252)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1501)
Related Issues:
Description:
Since only HTTP status code 200 is considered as success in current http sink implementation, all other 2XX based codes are considered as errors and retry happens.
Suggested Labels:
bug
Suggested Assignees:
Affected Product Version:
siddhi-io-http
2.2.0
OS, DB, other environment details and versions:
macOS
Steps to reproduce:
Call a backend API that returns a 2XX based status code other than 200.
Related Issues:
N/A
Description:
When using OAuth Token EP, scopes cannot be specified. This needs to be modified.
Affected Product Version:
siddhi-io-http 2.2.0
Description:
When a big XML response is received by @source(type='http-response'...), the response is clipped at certain length, and then the clipped response is tried to be parsed (which fails)
Suggested Labels:
bug
Suggested Assignees:
@minudika
Affected Product Version:
Steps to reproduce:
--Sink stream - get http events
@sink(type = 'http-request', publisher.url = 'http://localhost:3000/lastevent',
sink.id = 'request-sink',
@map(type = 'json'))
define stream postHttpRequestStream(payloadBody String); -- payloadBody is currently not used
--Source stream - incoming http events
@source(type = 'http-response', sink.id = 'request-sink', http.status.code = '200',
@map(type = 'xml',
@attributes(eventName = 'reply/equipmentReplies/movementEvents/sightingEventCodeText')))
define stream httpResponseStream2xx(eventName string);
The response from above URL - localhost:3000/lastevent, in postman is:
[2018-10-18_15-51-26_165] INFO {org.wso2.extension.siddhi.map.xml.sourcemapper.XmlSourceMapper} - Error parsing incoming XML event : trainIIAlphaCode>ICHR</trainIIAlphaCode><sightingEventCode>4050</sightingEventCode><sightingEventCodeText>JUNCTION RECEIVED</sightingEventCodeText><toMarkId>BNSF</toMarkId><trainInd>UP</trainInd></movementEvents><tripPlan><tripPlanId/></tripPlan><waybill><commodity/><destination><city/><splc/><state/></destination><grossWeight>0</grossWeight><netWeight>0</netWeight><tareWeight>0</tareWeight><weightCode>8</weightCode></waybill><requestType>LAST_EVENT</requestType></equipmentReplies></reply></ns2:TracingPlatformRequest>. Reason: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,1] Message: Content is not allowed in prolog.. Hence dropping message chunk (Encoded)
Additional information:
If I keep resizing the XML output from server, by removing XML elements, I reach a point where the error is no more.
I have validated my XML via multiple XML validators online. This is the raw version of the XML output:
<ns2:TracingPlatformRequest xmlns:ns2="http://someUrl/someEndPoint"><TracingPlatformRequestHeader>TracingPlatformRequestHeaderVal</TracingPlatformRequestHeader><reply><equipmentReplies><groupSequenceNumber>0001</groupSequenceNumber><equipment><equipmentId><equipmentInitial>BNSF</equipmentInitial><equipmentNumber>0000000100</equipmentNumber></equipmentId></equipment><umlerDetails><owner>BNSF</owner></umlerDetails><movementEvents><eta><etaDate>Thu Oct 18 2018</etaDate><ediEventCode/><eventCode/><location><splc/></location><reportingScac/></eta><fromMarkId>UP</fromMarkId><loadEmptyStatus>E</loadEmptyStatus><location><splc>536870000</splc></location><reportingRailroadSCAC>BNSF</reportingRailroadSCAC><sightingDate>Thu Oct 18 2018</sightingDate><sightingClmCode>R</sightingClmCode><trainIIAlphaCode>ICHR</trainIIAlphaCode><sightingEventCode>4050</sightingEventCode><sightingEventCodeText>JUNCTION RECEIVED</sightingEventCodeText><toMarkId>BNSF</toMarkId><trainInd>UP</trainInd></movementEvents><tripPlan><tripPlanId/></tripPlan><waybill><commodity/><destination><city/><splc/><state/></destination><grossWeight>0</grossWeight><netWeight>0</netWeight><tareWeight>0</tareWeight><weightCode>8</weightCode></waybill><requestType>LAST_EVENT</requestType></equipmentReplies></reply></ns2:TracingPlatformRequest>
Thanks,
Ravi Pandey
Description:
siddhi sql like this:
@app:name("fileHandleTest")
@app:description('test')
@app:statistics(reporter = 'jmx')
define stream testStream(triggered_time long);
@sink(type='http', publisher.url='http://192.168.1.23:7031/alertCenter/merge/mergeData', method='GET',@Map(type='json'))
define stream OutputStream (triggered_time long);
@info(name = "http_data_deal_with")
from testStream#log("---------dd---------------",true)
select *
insert into OutputStream;
Suggested Labels:
Suggested Assignees:
Affected Product Version:
siddhi-core : <= 4.2.17
siddhi-io-http: <=1.0.42
OS, DB, other environment details and versions:
CentOS Linux release 7.5.1804 (Core)
Steps to reproduce:
step1: start siddhi of this siddhi-sql appliaction,
execute cmd:
lsof -p 7921 | wc -l
321
step2: then stop siddhi;
lsof -p 7921 | wc -l
351
step3: start siddhi of this siddhi-sql appliaction,
execute cmd:
lsof -p 7921 | wc -l
381
step4: then stop siddhi;
execute cmd:
lsof -p 7921 | wc -l
381
lsof -p
the result:
java 7921 root 322u a_inode 0,10 0 6091 [eventpoll]
java 7921 root 323r FIFO 0,9 0t0 325541144 pipe
java 7921 root 324w FIFO 0,9 0t0 325541144 pipe
java 7921 root 325u a_inode 0,10 0 6091 [eventpoll]
java 7921 root 326r FIFO 0,9 0t0 325541145 pipe
java 7921 root 327w FIFO 0,9 0t0 325541145 pipe
java 7921 root 328u a_inode 0,10 0 6091 [eventpoll]
java 7921 root 329r FIFO 0,9 0t0 325541146 pipe
java 7921 root 330w FIFO 0,9 0t0 325541146 pipe
java 7921 root 331u a_inode 0,10 0 6091 [eventpoll]
java 7921 root 332r FIFO 0,9 0t0 325541147 pipe
java 7921 root 333w FIFO 0,9 0t0 325541147 pipe
java 7921 root 334u a_inode 0,10 0 6091 [eventpoll]
java 7921 root 335r FIFO 0,9 0t0 325541148 pipe
java 7921 root 336w FIFO 0,9 0t0 325541148 pipe
java 7921 root 337u a_inode 0,10 0 6091 [eventpoll]
java 7921 root 338r FIFO 0,9 0t0 325541149 pipe
java 7921 root 339w FIFO 0,9 0t0 325541149 pipe
java 7921 root 340u a_inode 0,10 0 6091 [eventpoll]
java 7921 root 341r FIFO 0,9 0t0 325541150 pipe
java 7921 root 342w FIFO 0,9 0t0 325541150 pipe
java 7921 root 343u a_inode 0,10 0 6091 [eventpoll]
Description:
$subject
Stacktrace
[2020-01-09 13:26:16,561] ERROR {io.siddhi.core.stream.input.source.Source} - Error on 'JsonInputMapper'. Error while connecting at Source 'http' at 'PersonalDetailsDefaultMapping'. java.lang.NullPointerException
at io.siddhi.extension.io.http.source.HttpConnectorRegistry.createHttpServerConnector(HttpConnectorRegistry.java:218)
at io.siddhi.extension.io.http.source.HttpSource.connect(HttpSource.java:418)
at io.siddhi.core.stream.input.source.Source.connectWithRetry(Source.java:160)
at io.siddhi.core.SiddhiAppRuntimeImpl.startSources(SiddhiAppRuntimeImpl.java:502)
at io.siddhi.core.SiddhiAppRuntimeImpl.start(SiddhiAppRuntimeImpl.java:427)
at io.siddhi.distribution.editor.core.internal.DebugRuntime.start(DebugRuntime.java:93)
at io.siddhi.distribution.editor.core.internal.DebugProcessorService.start(DebugProcessorService.java:42)
at io.siddhi.distribution.editor.core.internal.EditorMicroservice.start(EditorMicroservice.java:761)
at io.siddhi.distribution.editor.core.internal.EditorMicroservice.startWithVariables(EditorMicroservice.java:781)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.wso2.msf4j.internal.router.HttpMethodInfo.invokeResource(HttpMethodInfo.java:187)
at org.wso2.msf4j.internal.router.HttpMethodInfo.invoke(HttpMethodInfo.java:143)
at org.wso2.msf4j.internal.MSF4JHttpConnectorListener.dispatchMethod(MSF4JHttpConnectorListener.java:218)
at org.wso2.msf4j.internal.MSF4JHttpConnectorListener.lambda$onMessage$58(MSF4JHttpConnectorListener.java:129)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Siddhi App:
/**
{
event : {
"name" : "siddhi.user",
"age" : "25"
"address" : "20, Palm Groove, Colombo, Sri Lanka",
"contact": "+9xxxxxxx1,+9xxxxxxx2"
}
}
**/
@Source(type = 'http', receiver.url = 'http://localhost:8006/PersonalDetails',
@map(type = 'json'))
define stream PersonalDetailsDefaultMapping (name string, age int, address string, contact string);
/**
{
"name" : "siddhi.user",
"age" : "25",
"address" : {
"number": "20",
"lane": "Palm Groove",
"city": "Colombo"
"country": "Sri Lanka",
},
"contact": ["+9xxxxxxx1, +9xxxxxxx2"]
}
**/
@Source(type = 'http', receiver.url = 'http://localhost:8006/PersonalDetailsCustom',
@map(type = 'json',
@attributes(name = 'name', age ='age', addressCountry = 'address.country', contact = 'contact[0]' ) ))
@sink(type='log')
define stream PersonalDetailsCustomMapping (name string, age int, addressCountry string, contact string);
/**
{
person : [
{
"name" : "siddhi.user",
"age" : "25",
"address" : {
"number": "20",
"lane": "Palm Groove",
"city": "Colombo"
"country": "Sri Lanka",
},
"contact": ["+9xxxxxxx1, +9xxxxxxx2"]
},
{
"name" : "siddhi.user",
"age" : "25",
"address" : {
"number": "20",
"lane": "Palm Groove",
"city": "Colombo"
"country": "Sri Lanka",
},
"contact": ["+9xxxxxxx1, +9xxxxxxx2"]
},
]
}
**/
@Source(type = 'http', receiver.url = 'http://localhost:8006/PersonalDetailsMultiple',
@map(type = 'json', enclosing.element = 'person',
@attributes(name = 'name', age ='age', addressCountry = 'address.country', contact = 'contact' ) ))
@sink(type='log')
define stream PersonalDetailsEnclosingElement (name string, age int, addressCountry string, contact string);
Affected Product Version:
Latest
OS, DB, other environment details and versions:
Siddhi tooling daily build 8/1/2020
It is workable without shutting down EventLoop Groups from ServerConnectorController as current implementation. But in that scenario both MSF4j and Siddhi uses same ServerConnectorController and it gives following issues when considering the use case for Siddhi.
This issue is also reference to
wso2/carbon-transports#340
And following mail thread
[Dev][Architecture] Issue in accessing same boss and worker group in HTTP carbon transport from MSF4J and Siddhi together
Description:
Users should be able to configure connection pools in siddhi http sink in order to tune up performance.
When using HTTP call and call response scenario we allow 2 sources with http.status.code = 200 and [2|4]00 to register in Siddhi app. However, when the response arrives, according to our current logic in findAndGetResponseSource
method we randomly pick a source from the map matching the regex and send a response. Please find a sample app to simulate this below.
define stream FooStream (message String, headers String);
@sink(type='http-request',publisher.url='http://localhost:8005/abc',
method='POST',
headers='{{headers}}',sink.id='source-1',
@map(type='json', @payload('{{message}}')))
Define stream BarStream (message String, headers String);
@source(type='http-response', sink.id='source-1', http.status.code='[2|4]00',
@map(type='json',@attributes(name='name', id='id')))
define stream responseStream1(name String, id int);
@source(type='http-response', sink.id='source-1', http.status.code='200',
@map(type='json',@attributes(name='name', id='id')))
define stream responseStream2(name String, id int);
We need to have a robust mechanism to handle such responses which need to be sent to multiple sources
Description:
I am trying to connect to the github api through a siddhi app using the bellow code,
@app:name("RequestResponseSample")
@app:description('This sample siddhi app demonstrates how to use transport properties along with request sink and response source')
define stream regStream (id String);
@sink(type='http-request', publisher.url='http://api.github.com/repos/wso2/product-sp/releases',
method='POST',
chunk.disabled='true',
sink.id='reg-company',
@Map(type='json', @payload("""{"id":"{{id}}"}""")))
define stream requestStream (id string);
-- source to receive successful responses (which have 2xx status codes)
@source(type='http-response' , sink.id='reg-company', http.status.code='200',
@Map(type='json', @attributes(tag_name='$.tag_name')))
define stream responseStream(tag_name string);
-- source to receive the responses with 4xx status codes
@source(type='http-response' , sink.id='reg-company', http.status.code='403' ,
@Map(type='text', regex.A='((.|\n)*)', @attributes(message='A[1]')))
define stream responseStream4xx(message string);
@sink(type='log')
define stream logStream(tag_name string);
from regStream
select id
insert into requestStream;
from responseStream
select *
insert into logStream;
But I am not getting a response. It displays an error message saying that the channel is inactive
Bellow is the response I received when I requested through "Postman"
Description:
While invoking an endpoint with Token endpoint specified in the HTTP sink, the following exception occurs.
ERROR {io.siddhi.core.stream.StreamJunction} - Error in 'SiddhiApp' after consuming events from Stream 'FooStream', null. Hence, dropping event 'Event{timestamp=1598441762269, data=[***], isExpired=false}' java.lang.NullPointerException
at io.siddhi.extension.io.http.sink.updatetoken.HttpRequest.sendPostRequest(HttpRequest.java:127)
at io.siddhi.extension.io.http.sink.updatetoken.HttpsClient.getPasswordGrantAccessToken(HttpsClient.java:95)
at io.siddhi.extension.io.http.sink.HttpSink.getAccessToken(HttpSink.java:817)
at io.siddhi.extension.io.http.sink.HttpSink.setAccessToken(HttpSink.java:844)
at io.siddhi.extension.io.http.sink.HttpSink.sendOauthRequest(HttpSink.java:668)
at io.siddhi.extension.io.http.sink.HttpSink.publish(HttpSink.java:607)
at io.siddhi.core.stream.output.sink.Sink.publish(Sink.java:182)
at io.siddhi.extension.map.json.sinkmapper.JsonSinkMapper.mapAndSend(JsonSinkMapper.java:209)
at io.siddhi.core.stream.output.sink.SinkMapper.mapAndSend(SinkMapper.java:180)
at io.siddhi.core.stream.output.sink.SinkCallback.receive(SinkCallback.java:55)
at io.siddhi.core.stream.StreamJunction.sendEvent(StreamJunction.java:199)
at io.siddhi.core.stream.StreamJunction$Publisher.send(StreamJunction.java:474)
at io.siddhi.core.stream.input.InputDistributor.send(InputDistributor.java:34)
at io.siddhi.core.stream.input.InputEntryValve.send(InputEntryValve.java:45)
at io.siddhi.core.stream.input.InputHandler.send(InputHandler.java:78)
at org.wso2.carbon.siddhi.editor.core.internal.DebuggerEventStreamService.pushEvent(DebuggerEventStreamService.java:70)
at org.wso2.carbon.event.simulator.core.internal.generator.SingleEventGenerator.sendEvent(SingleEventGenerator.java:85)
at org.wso2.carbon.event.simulator.core.impl.SingleApiServiceImpl.runSingleSimulation(SingleApiServiceImpl.java:17)
at org.wso2.carbon.event.simulator.core.api.SingleApi.runSingleSimulation(SingleApi.java:63)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.wso2.msf4j.internal.router.HttpMethodInfo.invokeResource(HttpMethodInfo.java:187)
at org.wso2.msf4j.internal.router.HttpMethodInfo.invoke(HttpMethodInfo.java:143)
at org.wso2.msf4j.internal.MSF4JHttpConnectorListener.dispatchMethod(MSF4JHttpConnectorListener.java:218)
at org.wso2.msf4j.internal.MSF4JHttpConnectorListener.lambda$onMessage$0(MSF4JHttpConnectorListener.java:129)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Affected Product Version:
Steps to reproduce:
The sample Siddhi file is as follows.
@App:name("SiddhiApp")
@sink(type='http',
publisher.url='http://jsonplaceholder.typicode.com/posts',
method="POST",
headers="'Content-Type: application/json'",
oauth.username="***",
oauth.password="***",
consumer.key="***",
consumer.secret="***",
token.url='https://localhost:9443/oauth2/token',
ssl.verification.disabled="true",
@map(
type='json'
))
define stream FooStream (id int);
define stream BarStream(id int);
from BarStream
select *
insert into FooStream;
Description:
In a sink of type 'http-request', if the backend specified in the 'publisher.url' parameter is not reachable, upon simulating an event, the following error is being printed even when a source of 'http-response' is specified to handle responses with 500 status code (using a regex).
ERROR {org.wso2.extension.siddhi.io.http.source.HttpResponseMessageListener} - No source of type 'http-response' for status code '500' has been defined. Hence dropping the response message.
For the same scenario, from SI 1.0.1 onwards, the correct error is being printed. Following is a sample.
ERROR {io.siddhi.core.stream.output.sink.Sink} - Error on 'test_http_status_code'. Dropping event at Sink 'http-request' at 'CreditCardStream' as its still trying to reconnect!, events dropped '{CardNumber=1111}'
Affected Product Version:
SP 4.3.0, SP 4.4.0, SI 1.0.0
Steps to reproduce:
Description:
When invoking an API via http-request
sink, Host
HTTP header is not getting set in the HTTP request. This causes 400 Bad Request
error from the server.
This needs to be fixed to get the hostname from the URL and set into the header.
Affected Product Version:
1.0.39
Description:
This issue is created to keep track of the task: Testcases for OAuth cases.
Related Issues:
#100
Suggested Assignees:
https://github.com/thushaaanthan
Description:
It seems, there are a lot places that we could improve the HTTPSink implementation. It seems like some code segments are bit smelly. Need to check this out ASAP.
Description:
http-request and http-response are used for both incoming and outgoing calls. Hence this causes confusion among users. This need to be renamed to differentiate the call type.
Description:
The Basic Authentication mechanism is not working as expected, and this behaviour is observed in SI-4.X
versions when tested locally.
For example, when we use the http-call
sink method with Basic Auth enabled as shown below, the Base64 encoded username and password is getting malformed.
@sink(type='http-call', sink.id='D1', publisher.url='http://localhost:8281/foobar/1.0', method='POST', basic.auth.username='admin', basic.auth.password='admin',
headers='Content-Type: application/json', ...,
When we checked the wirelogs from the backend service, we observed that the Authorization header value being sent to the backend was incorrect,
"Authorization: Basic UnpooledHeapByteBuf(ridx: 0, widx: 11, cap: 11/11)[\r][\n]"
Seems like there is an issue with the Base64 encoding of the Authorization header at [1]. The encoded header is in bytes and it needs to be converted to string through byte methods which causes the incorrect value to get assigned as the Authorization header. Since the encoded username and password value are invalid, the http-call
sink method fails.
[1]
http-call
sink method with Basic Auth.Affected Product Version:
Description:
When using a request sink and responses source, if the response is a long one, sometimes only a part of the response is captured by the responses source.
Description:
The payload send to get a token here looks like following in current implementation &username=abc&password=123&grant_type=password When this is send to /token end point it will throw an exception since the payload cannot be parced due to the first charactor being '&'" -m "Fixed by changing the payload generation
Suggested Labels:
bug
Suggested Assignees:
Affected Product Version:
siddhi-io-http
2.2.0
OS, DB, other environment details and versions:
macOS
Steps to reproduce:
Call an endpoint protected with WSO2 API manager via http sink. Provide username, password, client_id, client_secreat only and let the access token be generated by http sink. This fails
Related Issues:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.