lestrrat-go / apache-logformat Goto Github PK
View Code? Open in Web Editor NEWPort of Perl5's Apache::LogFormat::Compiler to golang
License: MIT License
apache-logformat's People
Contributors
Stargazers
Watchers
Forkers
toqoz rschlaikjer ikeikeikeike miyagawa lylephone mpsrig skaji charlysan etsangsplk casualjimapache-logformat's Issues
Panic
Hello! I'm back :)
I've been using this library a really long time and I've never had a problem. Lately I've been adding a GUI to my app where it used to just be simple API calls. I got this panic on day 2 of development. I'm not quite sure what to make of it, I'm not sure how prolific it is, and I don't know if I can reproduce it, but I wanted to share it with you in case you had some ideas. It seems to generate within this library, but I'm willing tp be wrong on that.
Not that I expect you to dig in my code, but just in case you have questions, this is the commit where the panic occurred. Notifiarr/notifiarr@c3ba9c5
I'll definitely let you know if I see it happening again. Thanks!
[ERROR] 2022/01/25 19:08:50 http: panic serving [::1]:54074: bytes.Buffer.WriteTo: invalid Write count
goroutine 3488 [running]:
net/http.(*conn).serve.func1()
/usr/local/Cellar/go/1.17.3/libexec/src/net/http/server.go:1801 +0xb9
panic({0x48aa6c0, 0x4ac89a0})
/usr/local/Cellar/go/1.17.3/libexec/src/runtime/panic.go:1047 +0x266
bytes.(*Buffer).WriteTo(0xc000ca2450, {0x4ade7e0, 0xc000262120})
/usr/local/Cellar/go/1.17.3/libexec/src/bytes/buffer.go:241 +0x106
github.com/lestrrat-go/apache-logformat.(*ApacheLog).WriteLog(0xc000011020, {0x4ade7e0, 0xc000262120}, {0x4afcef0, 0xc000c2a060})
/Users/david/go/pkg/mod/github.com/lestrrat-go/[email protected]/logformat.go:40 +0x13c
github.com/lestrrat-go/apache-logformat.(*ApacheLog).Wrap.func1.1()
/Users/david/go/pkg/mod/github.com/lestrrat-go/[email protected]/logformat.go:58 +0x65
github.com/lestrrat-go/apache-logformat.(*ApacheLog).Wrap.func1({0x4aedbb8, 0xc00061e9a0}, 0xc000154160)
/Users/david/go/pkg/mod/github.com/lestrrat-go/[email protected]/logformat.go:66 +0x1e2
net/http.HandlerFunc.ServeHTTP(0x4925e40, {0x4aedbb8, 0xc00061e9a0}, 0xe)
/usr/local/Cellar/go/1.17.3/libexec/src/net/http/server.go:2046 +0x2f
github.com/Notifiarr/notifiarr/pkg/client.(*Client).stripSecrets.func1({0x4aedbb8, 0xc00061e9a0}, 0xc000207100)
/Users/david/go/src/github.com/Notifiarr/notifiarr/pkg/client/handlers.go:137 +0x15d
net/http.HandlerFunc.ServeHTTP(0x0, {0x4aedbb8, 0xc00061e9a0}, 0x406332e)
/usr/local/Cellar/go/1.17.3/libexec/src/net/http/server.go:2046 +0x2f
net/http.serverHandler.ServeHTTP({0xc000ca37d0}, {0x4aedbb8, 0xc00061e9a0}, 0xc000207100)
/usr/local/Cellar/go/1.17.3/libexec/src/net/http/server.go:2878 +0x43b
net/http.(*conn).serve(0xc0009ba500, {0x4af22d0, 0xc0009c0180})
/usr/local/Cellar/go/1.17.3/libexec/src/net/http/server.go:1929 +0xb08
created by net/http.(*Server).Serve
Custom fields
Is there a way to easily add custom fields to the log string. I would like to add the name of the service, so that I could easily filter between different services from kibana. E.g.
127.0.0.1 - - [28/Oct/2016:18:35:05 -0400] "POST /graphql HTTP/1.1" 200 153340 "http://localhost:3006/playground" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36" "my_app"
That would be parsed by a regex like this:
(?<remote>[^ ]*) (?<host>[^ ]*) (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)" "(?<app_name>[^\"]*)")?
Example: here
Thanks!
Include strftime library in this project
Having to install BOTH git and hg is really annoying.
Expose internal methods, or Redact sensitive data from logs
Any chance you can move logctx and httputil out of the internal/ folder and into a pkg/ folder? This will allow me to write my own Wrap() procedure.
The above is a possible solution to my problem. To avoid the XY Problem, this is my actual issue:
I want to hide sensitive data from logs. Some requests have keys and token in the parameters. I want to redact those before logging them to a file. I feel like this should be very easy if I build the Wrap() method into my own app.
Any ideas or thoughts are appreciated! Thanks!
Broken releases, unable to update module in projects.
The v2 releases do not conform to the go.mod standard. You must release them with a /v2 signature. Here's some light reading to get you acquainted: https://blog.golang.org/v2-go-modules
$ go get -u github.com/lestrrat-go/[email protected]
go get github.com/lestrrat-go/[email protected]: github.com/lestrrat-go/[email protected]: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v2
Love your module, but I want to use %{ms}T and I cannot get v2.0.5. :(
EDIT: For anyone else trying to use the latest code, this worked:
$ go get -u github.com/lestrrat-go/apache-logformat@master
%q should be an empty string if it's empty, not '-'
%q The query string (prepended with a ? if a query string exists, otherwise an empty string).
%{usec}t is handled as elapsed
It seems you get T and t mixed up in the block handler.
%t is request time, and takes usec, msec etc. to get microsecs from the Epoch.
%T is time taken for a request (elapsed) and takes us, ms to get them in micro/millisec.
you're handling %{usec}t as elapsed, which seems wrong to me - it should be displaying microseconds from the epoch.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.