Comments (8)
If I remember correctly, @mre had the same issue and also had a fix.
I'll check with him.
from gollum.
You have to adjust your grok pattern.
See
gollum/format/groktojson_test.go
Line 27 in 3d7deb2
from gollum.
That (?m) is coming from the go regex engine.
See https://github.com/google/re2/wiki/Syntax
from gollum.
Tested various options. The expression could not be matched.
from gollum.
@abelokonev can you provide a sample from the java stack trace logs you want to process? Would make debugging and writing a unit test a little easier. π You could obfuscate the function names if you like.
from gollum.
For an example:
12.76.2058 23:05:55.365 [qtp1174829684-20166] ERROR com.dze.coreapi.controller.CoreExceptionHandler - INTERNAL ERROR org.springframework.web.bind.ServletRequestBindingException: Missing request header 'Profile' for method parameter of type UserProfile
at org.springframework.web.method.annotation.RequestHeaderMethodArgumentResolver.handleMissingValue(RequestHeaderMethodArgumentResolver.java:82) ~[spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.method.annotation.AbstractNamedValueMethodArgumentResolver.resolveArgument(AbstractNamedValueMethodArgumentResolver.java:96) ~[spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.method.support.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:99) ~[spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.method.support.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:161) ~[spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:128) ~[spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) ~[spring-webmvc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:832) ~[spring-webmvc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:743) ~[spring-webmvc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:961) [spring-webmvc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:895) [spring-webmvc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:967) [spring-webmvc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframetork.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:858) [spring-webmvc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) [javax.servlet-api-3.1.0.jar:3.1.0]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:843) [spring-webmvc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [javax.servlet-api-3.1.0.jar:3.1.0]
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:224) [websocket-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:261) [spring-boot-actuator-1.3.4.RELEASE.jar:1.3.4.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:201) [javamelody-core-1.60.0.jar:1.60.0]
at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:178) [javamelody-core-1.60.0.jar:1.60.0]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:115) [spring-boot-actuator-1.3.4.RELEASE.jar:1.3.4.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:87) [spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) [spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121) [spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:103) [spring-boot-actuator-1.3.4.RELEASE.jar:1.3.4.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) [jetty-security-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) [jetty-servlet-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.Server.handle(Server.java:499) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) [jetty-server-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) [jetty-io-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) [jetty-util-9.2.16.v20160414.jar:9.2.16.v20160414]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) [jetty-util-9.2.16.v20160414.jar:9.2.16.v20160414]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
from gollum.
Sorry for the late reply - vacation.
I think I found your problem.
"FileIn":
Type: "consumer.File"
This is a classic problem when reading from files.
Gollum operates on messages. When reading from a stream that is not already message based you need to define when a message actually ends. In case of a file the standard is newline.
So your multiline Grok is not working because you actually read single lines.
With files this is a quite common problem as most of the times you don't have a sufficient "end of message" separator. Because of this we normally don't use file scraping for multiline output but syslog, sockets, or similar consumers.
So I would recommend to either insert a very distinctive "end of message" separator to your output or to use another communication method.
from gollum.
Thanks for the consultation.
from gollum.
Related Issues (20)
- Support new field types for config struct tags
- Test updated AWS Firehose and Kinesis producer HOT 3
- Update producer.Spooling to use components.BatchedWriterAssembly
- Update producer.StatsdMetrics to use core.BatchedProducer
- Update producer.socket to use components.BatchedWriterAssembly
- Race condition in core.PluginRegistry
- Gollum with Kafka and Fallback Producer crashes at startup if Kafka Broker is unreachable HOT 1
- 0.5.0 ERROR Failed to read config error=yaml: unmarshal errors: HOT 7
- Coloured output can't be disabled with the -lc flag HOT 1
- Feature request: Metrics in Prometheus format HOT 1
- format.MetadataCopy: unexpected change in metadata if msg.data.payload get modified HOT 4
- Missing Root CA Certificates in Docker image breaks AwsS3 Producer HOT 4
- Feature request: Allow overriding Syslog RFC protocols HOT 3
- is documentation outdated? HOT 4
- Syslogd consumer should support setting file permissions
- Windows build contains a Linux ELF executable HOT 2
- Compatibility with future Go versions HOT 3
- Mention the gollum wiki project to avoid confusion HOT 1
- `github.com/CrowdStrike/go-metrics-prometheus` seems gone HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gollum.