Comments (5)
buexplain
commented on June 13, 2024
1
是客户端主动关闭的连接。
这个问题不是必现的，我尝试了写一个demo去复现它，结果不成功；我只能在我那个项目里面复现它。
Linux下协程数量稳定，没这个问题。
from nbio.
lesismal
commented on June 13, 2024
1
我也写demo windows也尝试了下，遇到过一次一个连接Read阻塞，但是过了一会又返回err了，不确定是不是刚好赶巧了在多轮主动断开、重连的临界时间导致的，修改了一轮重连时间间隔长点之后避免这种临届时间后就没再遇到。
即使是Linux，也可能遇到client设备掉电，比如强行关机，或者手机进电梯地铁、切换信号塔之类的导致原来的连接断开但是没有收到TCP FIN、所以也会一直阻塞可能导致泄露。这些场景下，都是需要应用层通过ReadDeadline或者其他心跳检测的方式来实现server主动断开、这样才能确保不泄露。有了应用层保障，也就不怕这种偶发的问题。
而且从日志上看确实是runtime阻塞了，所以nbio本身暂时不修改这块代码了，后面如果能有办法稳定复现咱再来定位分析搞一波。
from nbio.
lesismal
commented on June 13, 2024
感谢关注和反馈！
已知的这些数组、map的读取是并发安全的，如果为了避免race warnign而加锁会影响性能。
之前有些commit尝试使用//go:norace消除race warnign，但实框架实现中涉及很多这种并发race、要全面消除race warnign会做太多不必要的修改、所以又去掉了。
具体的告警OP可以看下实际代码或实际运行中是否会造成数据错乱或者其他bug，如果确实有问题会进行修复，否则请忽略race warning。
以前也有一些相关issue，有兴趣也可以issue里搜下race相关。
from nbio.
buexplain
commented on June 13, 2024
如果只是竞争告警，倒是无所谓。我又观察了一波，发现疑似协程泄漏了。情况如下：
我服务端启动后，记录下协程堆栈信息，然后并发的发起websocket连接到服务端，当连接达到500多个的时候，客户端并发的关闭连接，客户端进程关闭后，服务端的协程堆栈信息比服务端刚启动的时候多了一条协程。
我感觉它(goroutine 6173 [IO wait])是泄漏了，还请大佬检查一下。
服务端启动后，没有任何连接进来时的协程堆栈
goroutine 59 [running]:
runtime/pprof.writeGoroutineStacks({0xd7a240, 0xc0001b7500})
D:/Go/src/runtime/pprof/pprof.go:703 +0x85
runtime/pprof.writeGoroutine({0xd7a240, 0xc0001b7500}, 0x2)
D:/Go/src/runtime/pprof/pprof.go:692 +0x4b
runtime/pprof.(*Profile).WriteTo(0x10e0f60, {0xd7a240, 0xc0001b7500}, 0xc?)
D:/Go/src/runtime/pprof/pprof.go:329 +0x1bb
net/http/pprof.handler.ServeHTTP({0xc00001c911, 0x9}, {0xd7d660, 0xc0001b7500}, 0x0?)
D:/Go/src/net/http/pprof/pprof.go:259 +0x57f
net/http/pprof.Index({0xd7d660?, 0xc0001b7500}, 0xc000182b00)
D:/Go/src/net/http/pprof/pprof.go:376 +0x12c
net/http.HandlerFunc.ServeHTTP(0xcbbfb8, {0xd7d660, 0xc0001b7500}, 0x0?)
D:/Go/src/net/http/server.go:2122 +0x4e
net/http.(*ServeMux).ServeHTTP(0xc00001c91b?, {0xd7d660, 0xc0001b7500}, 0xc000182b00)
D:/Go/src/net/http/server.go:2500 +0xc6
net/http.serverHandler.ServeHTTP({0xc0000314a0?}, {0xd7d660, 0xc0001b7500}, 0xc000182b00)
D:/Go/src/net/http/server.go:2936 +0x683
net/http.(*conn).serve(0xc0001501b0, {0xd7db38, 0xc001ad8150})
D:/Go/src/net/http/server.go:1995 +0xbd5
created by net/http.(*Server).Serve
D:/Go/src/net/http/server.go:3089 +0x818
goroutine 1 [chan receive]:
main.main()
F:/netsvr/cmd/netsvr.go:38 +0x93
goroutine 50 [chan receive]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a000)
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91
goroutine 7 [chan receive]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a008)
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91
goroutine 35 [chan receive]:
github.com/rcrowley/go-metrics.(*meterArbiter).tick(0x10ed6a0)
F:/netsvr/vendor/github.com/rcrowley/go-metrics/meter.go:239 +0x5c
created by github.com/rcrowley/go-metrics.NewMeter
F:/netsvr/vendor/github.com/rcrowley/go-metrics/meter.go:46 +0x171
goroutine 36 [chan receive]:
netsvr/internal/metrics.init.0.func2()
F:/netsvr/internal/metrics/metrics.go:83 +0x10f
created by netsvr/internal/metrics.init.0
F:/netsvr/internal/metrics/metrics.go:78 +0x726
goroutine 37 [syscall]:
os/signal.signal_recv()
D:/Go/src/runtime/sigqueue.go:152 +0x2f
os/signal.loop()
D:/Go/src/os/signal/signal_unix.go:23 +0x25
created by os/signal.Notify.func1.1
D:/Go/src/os/signal/signal.go:151 +0x51
goroutine 38 [chan receive]:
netsvr/pkg/quit.init.1.func1()
F:/netsvr/pkg/quit/quit.go:62 +0x45
created by netsvr/pkg/quit.init.1
F:/netsvr/pkg/quit/quit.go:61 +0x118
goroutine 39 [select]:
github.com/antlabs/timer.(*timeWheel).Run(0xc0000c3300)
F:/netsvr/vendor/github.com/antlabs/timer/time_wheel.go:293 +0x185
created by netsvr/internal/timer.init.0
F:/netsvr/internal/timer/timer.go:30 +0x10a
goroutine 40 [chan receive]:
netsvr/internal/timer.init.0.func1()
F:/netsvr/internal/timer/timer.go:36 +0x59
created by netsvr/internal/timer.init.0
F:/netsvr/internal/timer/timer.go:32 +0x116
goroutine 41 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0e18, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc0001fe438, 0x10ddb80?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc0001fe298, 0xc0001f9960)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).acceptOne(0xc0001fe280, 0x204, {0xc000a202d0, 0x2, 0x2}, 0xc0001fe298)
D:/Go/src/internal/poll/fd_windows.go:936 +0xda
internal/poll.(*FD).Accept(0xc0001fe280, 0xc0001f9bc8)
D:/Go/src/internal/poll/fd_windows.go:970 +0x308
net.(*netFD).accept(0xc0001fe280)
D:/Go/src/net/fd_windows.go:139 +0x75
net.(*TCPListener).accept(0xc000198078)
D:/Go/src/net/tcpsock_posix.go:148 +0x45
net.(*TCPListener).Accept(0xc000198078)
D:/Go/src/net/tcpsock.go:297 +0x68
net/http.(*Server).Serve(0xc0001b80f0, {0xd7d480, 0xc000198078})
D:/Go/src/net/http/server.go:3059 +0x5a7
net/http.(*Server).ListenAndServe(0xc0001b80f0)
D:/Go/src/net/http/server.go:2988 +0xc5
net/http.ListenAndServe(...)
D:/Go/src/net/http/server.go:3242
main.pprof.func1()
F:/netsvr/cmd/netsvr.go:67 +0x20c
created by main.pprof
F:/netsvr/cmd/netsvr.go:62 +0x2a
goroutine 42 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0f08, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c1b8, 0xc0000679d8?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c018, 0xc000067a60)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).acceptOne(0xc000a1c000, 0x1b0, {0xc000a20000, 0x2, 0x2}, 0xc000a1c018)
D:/Go/src/internal/poll/fd_windows.go:936 +0xda
internal/poll.(*FD).Accept(0xc000a1c000, 0xc000067cc8)
D:/Go/src/internal/poll/fd_windows.go:970 +0x308
net.(*netFD).accept(0xc000a1c000)
D:/Go/src/net/fd_windows.go:139 +0x75
net.(*TCPListener).accept(0xc000a0a030)
D:/Go/src/net/tcpsock_posix.go:148 +0x45
net.(*TCPListener).Accept(0xc000a0a030)
D:/Go/src/net/tcpsock.go:297 +0x68
netsvr/internal/worker.(*Server).Start(0xc000a16010)
F:/netsvr/internal/worker/tcp.go:43 +0xa6
netsvr/internal/worker.Start()
F:/netsvr/internal/worker/tcp.go:125 +0x13d
created by main.main
F:/netsvr/cmd/netsvr.go:35 +0x68
goroutine 20 [chan receive]:
gopkg.in/natefinch/lumberjack%2ev2.(*Logger).millRun(0xc00005e240)
F:/netsvr/vendor/gopkg.in/natefinch/lumberjack.v2/lumberjack.go:379 +0x66
created by gopkg.in/natefinch/lumberjack%2ev2.(*Logger).mill.func1
F:/netsvr/vendor/gopkg.in/natefinch/lumberjack.v2/lumberjack.go:390 +0xef
goroutine 51 [chan receive]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a018)
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91
goroutine 52 [chan receive]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a030)
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91
goroutine 53 [chan receive]:
github.com/lesismal/nbio.(*poller).start(0xc000a06300)
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2
goroutine 54 [chan receive]:
github.com/lesismal/nbio.(*poller).start(0xc000a06360)
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2
goroutine 55 [chan receive]:
github.com/lesismal/nbio.(*poller).start(0xc000a063c0)
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2
goroutine 56 [chan receive]:
github.com/lesismal/nbio.(*poller).start(0xc000a06420)
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2
goroutine 57 [select]:
github.com/lesismal/nbio/timer.(*Timer).loop(0xc000090100)
F:/netsvr/vendor/github.com/lesismal/nbio/timer/timer.go:154 +0x345
created by github.com/lesismal/nbio/timer.(*Timer).Start
F:/netsvr/vendor/github.com/lesismal/nbio/timer/timer.go:57 +0xa7
goroutine 58 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0d28, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c438, 0x114c060?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c298, 0xc000a23bb0)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).acceptOne(0xc000a1c280, 0x1f0, {0xc000a200f0, 0x2, 0x2}, 0xc000a1c298)
D:/Go/src/internal/poll/fd_windows.go:936 +0xda
internal/poll.(*FD).Accept(0xc000a1c280, 0xc000a23e18)
D:/Go/src/internal/poll/fd_windows.go:970 +0x308
net.(*netFD).accept(0xc000a1c280)
D:/Go/src/net/fd_windows.go:139 +0x75
net.(*TCPListener).accept(0xc000a04168)
D:/Go/src/net/tcpsock_posix.go:148 +0x45
net.(*TCPListener).Accept(0xc000a04168)
D:/Go/src/net/tcpsock.go:297 +0x68
github.com/lesismal/nbio/nbhttp.(*Engine).listen.func1()
F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:271 +0x11d
created by github.com/lesismal/nbio/nbhttp.(*Engine).listen
F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:265 +0x1cf
goroutine 60 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0b48, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c938, 0x0?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c798, 0xcbbb08)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).Read(0xc000a1c780, {0xc000a3c000, 0x1000, 0x1000})
D:/Go/src/internal/poll/fd_windows.go:436 +0x445
net.(*netFD).Read(0xc000a1c780, {0xc000a3c000, 0x1000, 0x1000})
D:/Go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc00009a048, {0xc000a3c000, 0x1000, 0x1000})
D:/Go/src/net/net.go:183 +0xb1
net/http.(*connReader).Read(0xc000031590, {0xc000a3c000, 0x1000, 0x1000})
D:/Go/src/net/http/server.go:782 +0x23b
bufio.(*Reader).fill(0xc000a066c0)
D:/Go/src/bufio/bufio.go:106 +0x2ab
bufio.(*Reader).ReadSlice(0xc000a066c0, 0xd0?)
D:/Go/src/bufio/bufio.go:372 +0x89
bufio.(*Reader).ReadLine(0xc000a066c0)
D:/Go/src/bufio/bufio.go:401 +0x36
net/textproto.(*Reader).readLineSlice(0xc0000315c0)
D:/Go/src/net/textproto/reader.go:54 +0xe5
net/textproto.(*Reader).ReadLine(...)
D:/Go/src/net/textproto/reader.go:37
net/http.readRequest(0xc00009a048?)
D:/Go/src/net/http/request.go:1042 +0xd3
net/http.(*conn).readRequest(0xc000150240, {0xd7da90, 0xc000060320})
D:/Go/src/net/http/server.go:990 +0x5aa
net/http.(*conn).serve(0xc000150240, {0xd7db38, 0xc001ad8150})
D:/Go/src/net/http/server.go:1920 +0x5d4
created by net/http.(*Server).Serve
D:/Go/src/net/http/server.go:3089 +0x818
goroutine 49 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0c38, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c6b8, 0x2?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c518, 0xcbbb08)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).Read(0xc000a1c500, {0xc0000314b1, 0x1, 0x1})
D:/Go/src/internal/poll/fd_windows.go:436 +0x445
net.(*netFD).Read(0xc000a1c500, {0xc0000314b1, 0x1, 0x1})
D:/Go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc00009a040, {0xc0000314b1, 0x1, 0x1})
D:/Go/src/net/net.go:183 +0xb1
net/http.(*connReader).backgroundRead(0xc0000314a0)
D:/Go/src/net/http/server.go:674 +0x79
created by net/http.(*connReader).startBackgroundRead
D:/Go/src/net/http/server.go:670 +0x1a7大量连接并发进来后，再并发退出后的服务端协程堆栈
goroutine 59 [running]:
runtime/pprof.writeGoroutineStacks({0xd7a240, 0xc003da6700})
D:/Go/src/runtime/pprof/pprof.go:703 +0x85
runtime/pprof.writeGoroutine({0xd7a240, 0xc003da6700}, 0x2)
D:/Go/src/runtime/pprof/pprof.go:692 +0x4b
runtime/pprof.(*Profile).WriteTo(0x10e0f60, {0xd7a240, 0xc003da6700}, 0xc?)
D:/Go/src/runtime/pprof/pprof.go:329 +0x1bb
net/http/pprof.handler.ServeHTTP({0xc0055c8521, 0x9}, {0xd7d660, 0xc003da6700}, 0x0?)
D:/Go/src/net/http/pprof/pprof.go:259 +0x57f
net/http/pprof.Index({0xd7d660?, 0xc003da6700}, 0xc005891e00)
D:/Go/src/net/http/pprof/pprof.go:376 +0x12c
net/http.HandlerFunc.ServeHTTP(0xcbbfb8, {0xd7d660, 0xc003da6700}, 0x0?)
D:/Go/src/net/http/server.go:2122 +0x4e
net/http.(*ServeMux).ServeHTTP(0xc0055c852b?, {0xd7d660, 0xc003da6700}, 0xc005891e00)
D:/Go/src/net/http/server.go:2500 +0xc6
net/http.serverHandler.ServeHTTP({0xc0000314a0?}, {0xd7d660, 0xc003da6700}, 0xc005891e00)
D:/Go/src/net/http/server.go:2936 +0x683
net/http.(*conn).serve(0xc0001501b0, {0xd7db38, 0xc001ad8150})
D:/Go/src/net/http/server.go:1995 +0xbd5
created by net/http.(*Server).Serve
D:/Go/src/net/http/server.go:3089 +0x818
goroutine 1 [chan receive, 2 minutes]:
main.main()
F:/netsvr/cmd/netsvr.go:38 +0x93
goroutine 50 [chan receive, 2 minutes]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a000)
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91
goroutine 7 [chan receive, 2 minutes]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a008)
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91
goroutine 35 [chan receive]:
github.com/rcrowley/go-metrics.(*meterArbiter).tick(0x10ed6a0)
F:/netsvr/vendor/github.com/rcrowley/go-metrics/meter.go:239 +0x5c
created by github.com/rcrowley/go-metrics.NewMeter
F:/netsvr/vendor/github.com/rcrowley/go-metrics/meter.go:46 +0x171
goroutine 36 [chan receive]:
netsvr/internal/metrics.init.0.func2()
F:/netsvr/internal/metrics/metrics.go:83 +0x10f
created by netsvr/internal/metrics.init.0
F:/netsvr/internal/metrics/metrics.go:78 +0x726
goroutine 37 [syscall, 2 minutes]:
os/signal.signal_recv()
D:/Go/src/runtime/sigqueue.go:152 +0x2f
os/signal.loop()
D:/Go/src/os/signal/signal_unix.go:23 +0x25
created by os/signal.Notify.func1.1
D:/Go/src/os/signal/signal.go:151 +0x51
goroutine 38 [chan receive, 2 minutes]:
netsvr/pkg/quit.init.1.func1()
F:/netsvr/pkg/quit/quit.go:62 +0x45
created by netsvr/pkg/quit.init.1
F:/netsvr/pkg/quit/quit.go:61 +0x118
goroutine 39 [select]:
github.com/antlabs/timer.(*timeWheel).Run(0xc0000c3300)
F:/netsvr/vendor/github.com/antlabs/timer/time_wheel.go:293 +0x185
created by netsvr/internal/timer.init.0
F:/netsvr/internal/timer/timer.go:30 +0x10a
goroutine 40 [chan receive, 2 minutes]:
netsvr/internal/timer.init.0.func1()
F:/netsvr/internal/timer/timer.go:36 +0x59
created by netsvr/internal/timer.init.0
F:/netsvr/internal/timer/timer.go:32 +0x116
goroutine 41 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0e18, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc0001fe438, 0x10ddb80?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc0001fe298, 0xc002983960)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).acceptOne(0xc0001fe280, 0x984, {0xc004af0690, 0x2, 0x2}, 0xc0001fe298)
D:/Go/src/internal/poll/fd_windows.go:936 +0xda
internal/poll.(*FD).Accept(0xc0001fe280, 0xc003b95bc8)
D:/Go/src/internal/poll/fd_windows.go:970 +0x308
net.(*netFD).accept(0xc0001fe280)
D:/Go/src/net/fd_windows.go:139 +0x75
net.(*TCPListener).accept(0xc000198078)
D:/Go/src/net/tcpsock_posix.go:148 +0x45
net.(*TCPListener).Accept(0xc000198078)
D:/Go/src/net/tcpsock.go:297 +0x68
net/http.(*Server).Serve(0xc0001b80f0, {0xd7d480, 0xc000198078})
D:/Go/src/net/http/server.go:3059 +0x5a7
net/http.(*Server).ListenAndServe(0xc0001b80f0)
D:/Go/src/net/http/server.go:2988 +0xc5
net/http.ListenAndServe(...)
D:/Go/src/net/http/server.go:3242
main.pprof.func1()
F:/netsvr/cmd/netsvr.go:67 +0x20c
created by main.pprof
F:/netsvr/cmd/netsvr.go:62 +0x2a
goroutine 42 [IO wait, 2 minutes]:
internal/poll.runtime_pollWait(0x2a2a0f08, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c1b8, 0x114c060?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c018, 0xc000067a60)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).acceptOne(0xc000a1c000, 0x21c, {0xc001ae00f0, 0x2, 0x2}, 0xc000a1c018)
D:/Go/src/internal/poll/fd_windows.go:936 +0xda
internal/poll.(*FD).Accept(0xc000a1c000, 0xc002985cc8)
D:/Go/src/internal/poll/fd_windows.go:970 +0x308
net.(*netFD).accept(0xc000a1c000)
D:/Go/src/net/fd_windows.go:139 +0x75
net.(*TCPListener).accept(0xc000a0a030)
D:/Go/src/net/tcpsock_posix.go:148 +0x45
net.(*TCPListener).Accept(0xc000a0a030)
D:/Go/src/net/tcpsock.go:297 +0x68
netsvr/internal/worker.(*Server).Start(0xc000a16010)
F:/netsvr/internal/worker/tcp.go:43 +0xa6
netsvr/internal/worker.Start()
F:/netsvr/internal/worker/tcp.go:125 +0x13d
created by main.main
F:/netsvr/cmd/netsvr.go:35 +0x68
goroutine 20 [chan receive, 2 minutes]:
gopkg.in/natefinch/lumberjack%2ev2.(*Logger).millRun(0xc00005e240)
F:/netsvr/vendor/gopkg.in/natefinch/lumberjack.v2/lumberjack.go:379 +0x66
created by gopkg.in/natefinch/lumberjack%2ev2.(*Logger).mill.func1
F:/netsvr/vendor/gopkg.in/natefinch/lumberjack.v2/lumberjack.go:390 +0xef
goroutine 51 [chan receive, 2 minutes]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a018)
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91
goroutine 52 [chan receive, 2 minutes]:
github.com/lesismal/nbio/taskpool.(*FixedNoOrderPool).taskLoop(0xc00009a030)
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:13 +0x6b
created by github.com/lesismal/nbio/taskpool.NewFixedNoOrderPool
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/fixednoorderpool.go:40 +0x91
goroutine 53 [chan receive, 2 minutes]:
github.com/lesismal/nbio.(*poller).start(0xc000a06300)
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2
goroutine 54 [chan receive, 2 minutes]:
github.com/lesismal/nbio.(*poller).start(0xc000a06360)
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2
goroutine 55 [chan receive, 2 minutes]:
github.com/lesismal/nbio.(*poller).start(0xc000a063c0)
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2
goroutine 56 [chan receive, 2 minutes]:
github.com/lesismal/nbio.(*poller).start(0xc000a06420)
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:122 +0xa25
created by github.com/lesismal/nbio.(*Engine).Start
F:/netsvr/vendor/github.com/lesismal/nbio/engine_std.go:73 +0x2f2
goroutine 57 [select, 2 minutes]:
github.com/lesismal/nbio/timer.(*Timer).loop(0xc000090100)
F:/netsvr/vendor/github.com/lesismal/nbio/timer/timer.go:154 +0x345
created by github.com/lesismal/nbio/timer.(*Timer).Start
F:/netsvr/vendor/github.com/lesismal/nbio/timer/timer.go:57 +0xa7
goroutine 58 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0d28, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c438, 0xc000a23b20?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c298, 0xc000a23bb0)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).acceptOne(0xc000a1c280, 0xa08, {0xc003f6e3c0, 0x2, 0x2}, 0xc000a1c298)
D:/Go/src/internal/poll/fd_windows.go:936 +0xda
internal/poll.(*FD).Accept(0xc000a1c280, 0xc000a23e18)
D:/Go/src/internal/poll/fd_windows.go:970 +0x308
net.(*netFD).accept(0xc000a1c280)
D:/Go/src/net/fd_windows.go:139 +0x75
net.(*TCPListener).accept(0xc000a04168)
D:/Go/src/net/tcpsock_posix.go:148 +0x45
net.(*TCPListener).Accept(0xc000a04168)
D:/Go/src/net/tcpsock.go:297 +0x68
github.com/lesismal/nbio/nbhttp.(*Engine).listen.func1()
F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:271 +0x11d
created by github.com/lesismal/nbio/nbhttp.(*Engine).listen
F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:265 +0x1cf
goroutine 7665 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0a58, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc00417ce38, 0xc000092ea0?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc00417cc98, 0xcbbb08)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).Read(0xc00417cc80, {0xc0023f0000, 0x1000, 0x1000})
D:/Go/src/internal/poll/fd_windows.go:436 +0x445
net.(*netFD).Read(0xc00417cc80, {0xc0023f0000, 0x1000, 0x1000})
D:/Go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc000a6c398, {0xc0023f0000, 0x1000, 0x1000})
D:/Go/src/net/net.go:183 +0xb1
net/http.(*connReader).Read(0xc005a4b230, {0xc0023f0000, 0x1000, 0x1000})
D:/Go/src/net/http/server.go:782 +0x23b
bufio.(*Reader).fill(0xc0058ce960)
D:/Go/src/bufio/bufio.go:106 +0x2ab
bufio.(*Reader).ReadSlice(0xc0058ce960, 0xd0?)
D:/Go/src/bufio/bufio.go:372 +0x89
bufio.(*Reader).ReadLine(0xc0058ce960)
D:/Go/src/bufio/bufio.go:401 +0x36
net/textproto.(*Reader).readLineSlice(0xc005a4b260)
D:/Go/src/net/textproto/reader.go:54 +0xe5
net/textproto.(*Reader).ReadLine(...)
D:/Go/src/net/textproto/reader.go:37
net/http.readRequest(0xc000a6c398?)
D:/Go/src/net/http/request.go:1042 +0xd3
net/http.(*conn).readRequest(0xc0056f3b90, {0xd7da90, 0xc000a70b40})
D:/Go/src/net/http/server.go:990 +0x5aa
net/http.(*conn).serve(0xc0056f3b90, {0xd7db38, 0xc001ad8150})
D:/Go/src/net/http/server.go:1920 +0x5d4
created by net/http.(*Server).Serve
D:/Go/src/net/http/server.go:3089 +0x818
goroutine 7738 [IO wait]:
internal/poll.runtime_pollWait(0x2a2a0c38, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc000a1c6b8, 0x2?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc000a1c518, 0xcbbb08)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).Read(0xc000a1c500, {0xc0000314b1, 0x1, 0x1})
D:/Go/src/internal/poll/fd_windows.go:436 +0x445
net.(*netFD).Read(0xc000a1c500, {0xc0000314b1, 0x1, 0x1})
D:/Go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc00009a040, {0xc0000314b1, 0x1, 0x1})
D:/Go/src/net/net.go:183 +0xb1
net/http.(*connReader).backgroundRead(0xc0000314a0)
D:/Go/src/net/http/server.go:674 +0x79
created by net/http.(*connReader).startBackgroundRead
D:/Go/src/net/http/server.go:670 +0x1a7
//就是这条协程，我感觉它是泄漏的了。
goroutine 6173 [IO wait]:
internal/poll.runtime_pollWait(0x2a2d21f8, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc003fbebb8, 0xa34ab3?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc003fbea18, 0xcbbb08)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).Read(0xc003fbea00, {0xc00526a000, 0x8000, 0x8000})
D:/Go/src/internal/poll/fd_windows.go:436 +0x445
net.(*netFD).Read(0xc003fbea00, {0xc00526a000, 0x8000, 0x8000})
D:/Go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc0051f8178, {0xc00526a000, 0x8000, 0x8000})
D:/Go/src/net/net.go:183 +0xb1
github.com/lesismal/nbio.(*Conn).readTCP(0xc0051e2460, {0xc00526a000, 0x8000, 0x8000})
F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:108 +0xd0
github.com/lesismal/nbio.(*Conn).read(0xc0051e2460, {0xc00526a000, 0x8000, 0x8000})
F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:95 +0x7b
github.com/lesismal/nbio.(*poller).readConn(0xc000a06360, 0xc0050afe78?)
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:57 +0x8e
created by github.com/lesismal/nbio.(*poller).addConn
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:77 +0x205竞争警告
==================
WARNING: DATA RACE
Read at 0x00c0051e24a0 by goroutine 328:
github.com/lesismal/nbio.(*poller).deleteConn()
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:89 +0xdb
github.com/lesismal/nbio.(*Conn).Close()
F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:275 +0x1c7
github.com/lesismal/nbio/nbhttp.(*Engine).Shutdown.func2()
F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:441 +0x44
Previous write at 0x00c0051e24a0 by goroutine 763:
github.com/lesismal/nbio.(*Conn).readTCP()
F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:110 +0x10b
github.com/lesismal/nbio.(*Conn).read()
F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:95 +0x7a
github.com/lesismal/nbio.(*poller).readConn()
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:57 +0x8d
github.com/lesismal/nbio.(*poller).addConn.func1()
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:77 +0x47
Goroutine 328 (running) created at:
github.com/lesismal/nbio/nbhttp.(*Engine).Shutdown()
F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:439 +0x251
netsvr/internal/customer.Shutdown()
F:/netsvr/internal/customer/websocket.go:72 +0x74
main.main()
F:/netsvr/cmd/netsvr.go:55 +0x17a
Goroutine 763 (running) created at:
github.com/lesismal/nbio.(*poller).addConn()
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:77 +0x204
github.com/lesismal/nbio.(*Engine).AddConn()
F:/netsvr/vendor/github.com/lesismal/nbio/engine.go:211 +0x10d
github.com/lesismal/nbio/nbhttp.(*Engine).AddConnNonTLSNonBlocking()
F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:587 +0x5ad
github.com/lesismal/nbio/nbhttp.(*Engine).AddConnNonTLSNonBlocking-fm()
<autogenerated>:1 +0x6d
github.com/lesismal/nbio/nbhttp.(*Engine).listen.func1()
F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:273 +0x250
==================
==================
WARNING: DATA RACE
Read at 0x00c000031290 by main goroutine:
github.com/lesismal/nbio/nbhttp.(*Engine).Shutdown()
F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:453 +0x45d
netsvr/internal/customer.Shutdown()
F:/netsvr/internal/customer/websocket.go:72 +0x74
main.main()
F:/netsvr/cmd/netsvr.go:55 +0x17a
Previous write at 0x00c000031290 by goroutine 600:
runtime.mapdelete_fast64()
D:/Go/src/runtime/map_fast64.go:273 +0x0
github.com/lesismal/nbio/nbhttp.NewEngine.func5.1()
F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:936 +0x10f
github.com/lesismal/nbio.(*Conn).MustExecute.func1()
F:/netsvr/vendor/github.com/lesismal/nbio/conn.go:157 +0x65
github.com/lesismal/nbio/taskpool.(*MixedPool).callWitoutRecover()
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/mixedpool.go:38 +0x72
github.com/lesismal/nbio/taskpool.(*MixedPool).callWitoutRecover-fm()
<autogenerated>:1 +0x44
github.com/lesismal/nbio/taskpool.(*MixedPool).Go.func1()
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/mixedpool.go:45 +0x77
Goroutine 600 (finished) created at:
github.com/lesismal/nbio/taskpool.(*MixedPool).Go()
F:/netsvr/vendor/github.com/lesismal/nbio/taskpool/mixedpool.go:44 +0x15e
github.com/lesismal/nbio/taskpool.(*MixedPool).Go-fm()
<autogenerated>:1 +0x44
github.com/lesismal/nbio.(*Conn).MustExecute()
F:/netsvr/vendor/github.com/lesismal/nbio/conn.go:154 +0x321
github.com/lesismal/nbio/nbhttp.NewEngine.func5()
F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:929 +0x125
github.com/lesismal/nbio.(*Engine).OnClose.func1()
F:/netsvr/vendor/github.com/lesismal/nbio/engine.go:234 +0xb5
github.com/lesismal/nbio.(*poller).deleteConn()
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:89 +0x13d
github.com/lesismal/nbio.(*Conn).Close()
F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:275 +0x1c7
github.com/lesismal/nbio/nbhttp.(*Engine).Shutdown.func2()
F:/netsvr/vendor/github.com/lesismal/nbio/nbhttp/engine.go:441 +0x44
==================
Found 2 data race(s)
Process finished with the exit code 66
from nbio.
lesismal
commented on June 13, 2024
//就是这条协程，我感觉它是泄漏的了。
goroutine 6173 [IO wait]:
internal/poll.runtime_pollWait(0x2a2d21f8, 0x72)
D:/Go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc003fbebb8, 0xa34ab3?, 0x0)
D:/Go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.execIO(0xc003fbea18, 0xcbbb08)
D:/Go/src/internal/poll/fd_windows.go:175 +0x285
internal/poll.(*FD).Read(0xc003fbea00, {0xc00526a000, 0x8000, 0x8000})
D:/Go/src/internal/poll/fd_windows.go:436 +0x445
net.(*netFD).Read(0xc003fbea00, {0xc00526a000, 0x8000, 0x8000})
D:/Go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc0051f8178, {0xc00526a000, 0x8000, 0x8000})
D:/Go/src/net/net.go:183 +0xb1
github.com/lesismal/nbio.(*Conn).readTCP(0xc0051e2460, {0xc00526a000, 0x8000, 0x8000})
F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:108 +0xd0
github.com/lesismal/nbio.(*Conn).read(0xc0051e2460, {0xc00526a000, 0x8000, 0x8000})
F:/netsvr/vendor/github.com/lesismal/nbio/conn_std.go:95 +0x7b
github.com/lesismal/nbio.(*poller).readConn(0xc000a06360, 0xc0050afe78?)
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:57 +0x8e
created by github.com/lesismal/nbio.(*poller).addConn
F:/netsvr/vendor/github.com/lesismal/nbio/poller_std.go:77 +0x205
看样子像是windows的读阻塞了，如果是client side 退出通常会 TCP FIN，这里读应该就会返回。如果是server主动close，我以前好像也遇到过标准库net.TCPConn.Close并没有触发读err的问题，如果是server主动close的情况导致，以前项目里是Close时也调用了CloseRead/CloseWrite应该就可以，所以我考虑在这里加上CloseRead/CloseWrite来主动关闭了（但是这很不合理，因为是标准库的net.TCPConn，）：
https://github.com/lesismal/nbio/blob/master/conn_std.go#L266
修改：
case ConnTypeTCP:
err = c.conn.(*net.TCPConn).Close()
if err == nil {
if err1 := c.conn.(*net.TCPConn).CloseRead(); err1 != nil {
err = err1
}
if err2 := c.conn.(*net.TCPConn).CloseWrite(); err2 != nil {
err = err2
}
}有简单的完整代码可以复现这种泄露吗？我试试看什么问题。
uni*应该是没这个问题的，windows主要作为调试使用，应该不会影响linux下的server。
另外，server应该在连接建立后以及收到新的message后进行SetReadDeadline，则这个读也不会一直阻塞、超时后就会退出了。
from nbio.
Related Issues (20)
- 如何增加序列化和反序列化的函数？ HOT 3
- how to use with so reuseport option for the http example? HOT 3
- one more question. thx for the prompt reply, how well does it scale as a client? HOT 7
- just tested the performance of nbio tcp https on multicore system. it's great! but... how to do h2? HOT 1
- am i asking for a lot? pushing for tcp performance and nbios hit the mark but... HOT 18
- curious about the "total success" HOT 28
- how to increase the eventloop from the tls server example? HOT 5
- only 2,000 req/s with redis client example but using tls server example modification, i can get 25,000 req/s. what's wrong? HOT 2
- 请教大佬，在很短的时间里，向同一个conn推送多次数据 HOT 8
- 使用udp发送数据，会存在丢包！ HOT 14
- Retract v1.5.4 HOT 6
- how to set so reuseport option for server? HOT 3
- 1.5.6 仍可能存在潜在bug HOT 31
- What is the principle of realizing 10 million connections to a single server? HOT 3
- WS客户端发送超高压缩率内容，可导致服务器OOM HOT 15
- onClose 没有调用 HOT 2
- InsecureSkipVerify=false的时候，无法进行TLS连接 HOT 7
- 请教下关于二进制数据压缩的问题 HOT 3
- LetsEncrypt HOT 2
- 大佬有交流群吗 HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nbio.