I found this error in my main code , relevant log is :

2015/09/07 12:37:01 [STOP - Hammer Time] Forcefully shutting down parent
2015/09/07 12:37:01 WaitGroup at 0 sync: negative WaitGroup counter
2015/09/07 12:37:01 22203 Serve() returning...
panic: sync: WaitGroup is reused before previous Wait has returned

goroutine 1 [running]:
sync.(*WaitGroup).Wait(0xc82020aee0)
        /usr/local/go/src/sync/waitgroup.go:128 +0x114
github.com/fvbock/endless.(*endlessServer).Serve(0xc82020ae60, 0x7f9fbe8b1500, 0xc82059f130)
        /home/azmoona/app_build/vendor/src/github.com/fvbock/endless/endless.go:166 +0x32c
github.com/fvbock/endless.(*endlessServer).ListenAndServe(0xc82020ae60, 0x0, 0x0)
        /home/azmoona/app_build/vendor/src/github.com/fvbock/endless/endless.go:197 +0x42a
github.com/fvbock/endless.ListenAndServe(0xc82022a900, 0xe, 0x7f9fbe8b3740, 0xc820242300, 0x0, 0x0)
        /home/azmoona/app_build/vendor/src/github.com/fvbock/endless/endless.go:136 +0x59
main.main()
        /home/azmoona/app_build/src/webserver/azmoona.go:58 +0x605

build the simple example with -race flag, and there is always a race.

 ./simple 
2015/09/07 13:10:03 10751 localhost:4242
2015/09/07 13:10:15 10751 Received SIGHUP. forking.
2015/09/07 13:10:15 10751 Received SIGTERM.
==================
WARNING: DATA RACE
Read by goroutine 7:
  github.com/fvbock/endless.(*endlessServer).shutdown()
      /home/f0rud/gospace/src/github.com/fvbock/endless/endless.go:342 +0x57
  github.com/fvbock/endless.(*endlessServer).handleSignals()
      /home/f0rud/gospace/src/github.com/fvbock/endless/endless.go:316 +0x1272

Previous write by main goroutine:
  github.com/fvbock/endless.(*endlessServer).Serve()
      /home/f0rud/gospace/src/github.com/fvbock/endless/endless.go:163 +0x1f1
  github.com/fvbock/endless.(*endlessServer).ListenAndServe()
      /home/f0rud/gospace/src/github.com/fvbock/endless/endless.go:197 +0x526
  github.com/fvbock/endless.ListenAndServe()
      /home/f0rud/gospace/src/github.com/fvbock/endless/endless.go:136 +0x6a
  main.main()
      /home/f0rud/gospace/src/github.com/fvbock/endless/examples/simple.go:21 +0x220

Goroutine 7 (running) created at:
  github.com/fvbock/endless.(*endlessServer).ListenAndServe()
      /home/f0rud/gospace/src/github.com/fvbock/endless/endless.go:182 +0xbb
  github.com/fvbock/endless.ListenAndServe()
      /home/f0rud/gospace/src/github.com/fvbock/endless/endless.go:136 +0x6a
  main.main()
      /home/f0rud/gospace/src/github.com/fvbock/endless/examples/simple.go:21 +0x220
==================
2015/09/07 13:10:15 10751 127.0.0.1:4242 Listener closed.
2015/09/07 13:10:15 10751 Waiting for connections to finish...
2015/09/07 13:10:15 10751 Serve() returning...
2015/09/07 13:10:15 accept tcp 127.0.0.1:4242: use of closed network connection
2015/09/07 13:10:15 Server on 4242 stopped
2015/09/07 13:10:15 10930 localhost:4242
Found 1 data race(s)

../github.com/fvbock/endless/endless.go:380: srv.SetKeepAlivesEnabled undefined (type *endlessServer has no field or method SetKeepAlivesEnabled)

When I used go get to get this.

I want to use LimitListener from "golang.org/x/net/netutil" for the Serve() function, but there is no way to do that. Normally, the Serve() function has a listener param.

...
l, err := net.Listen("tcp", addr)
if err != nil {
    return
}

l = netutil.LimitListener(TcpKeepAliveListener(l), MAX_CONN_NUM)
server := &http.Server{Addr: addr, Handler: handler}
err = server.Serve(l)

Maybe we can use like this:

...
l, err := net.Listen("tcp", addr)
if err != nil {
    return
}

l = netutil.LimitListener(TcpKeepAliveListener(l), MAX_CONN_NUM)
//server := &http.Server{Addr: addr, Handler: handler}
server := endless.NewServer(addr, handler)
err = server.Serve(l)

Sorry for my broken English.

Hi,

This is a feature request to make the network="tcp" parameter used here configurable. In our case we would like to use unix as the value here.

Some Background:
I'm using endless in a couple of services and liking it very much 😄
But we are now working on a lite, offline version of our service stack, in which clients connect to a service on the same machine. Here we don't actually need TCP sockets for communication, but can instead use unix sockets directly. Unfortunately I couldn't figure out a way to do this while still using endless.

The Go stdlib supports this out of the box, the only change required was:

listener, err := net.Listen("tcp", "localhost:8080")

to

listener, err := net.Listen("unix", "some-unix-socket.sock")

However, with endless, that part seems to be not configurable.
There seems to be same additional logic for listening on a file already, but only if the ENDLESS_CONTINUE environment variable is set, which then sets isChild to true. I didn't want to mess around in that direction to much.

Is there any chance something like this could be added to endless?

i found endless uses syscall.SIGUSR1,syscall.SIGUSR2,syscall.SIGTSTP,syscall.Kill, these things are not defined on Go 1.6.2 on windows. so codes using endless can not even be compiled.
endless: 2cdc20a

would you please fix this?

if your work use this file, when you go build to make a exe, there will be

undefined: syscall.SIGUSR1
undefined: syscall.SIGUSR2
undefined: syscall.SIGTSTP
undefined: syscall.SIGUSR1
undefined: syscall.SIGUSR2
undefined: syscall.SIGTSTP
undefined: syscall.SIGUSR1
undefined: syscall.SIGUSR2
undefined: syscall.SIGTSTP
undefined: syscall.Kill
too many errors

Because these constants are not available in Windows，
For normal operation, you can modify the file in GOROOT \ src \ syscall \ types_windows.go
Anyway, these constants are useless under windows

var signals = [...]string{      //It already exists， usually in line 67 of the document
    // Omit some rows

    /** change start */
    16: "SIGUSR1",
    17: "SIGUSR2",
    18: "SIGTSTP",
    /** change end */
}


/** change start */
func Kill(...interface{}) {      //It doesn't exist, you can add it after last change
    return;
}
const (
    SIGUSR1 = Signal(0x10)
    SIGUSR2 = Signal(0x11)
    SIGTSTP = Signal(0x12)
)
/** change end */

Method from: https://learnku.com/articles/51696

I add endless to my project.
and i build it in windows env.
then it show this:

On restart, the program forks a new copy and closes the old copy, leaving the new one parentless/daemonized. How would you use this with init, upstart, systemd and others? If you can't, what do you do about crashes - restarts and reporting, and also logging?

Hey, thanks for this cool package!

Is this anyhow combinable with supervisor? I'm using it for regular restarts, log rotations, and so on. Only downside while deploying is that with supervisor I don't have graceful restarts...

Any experiences with such a stack?

Hey,
First of all thank you for this really helpful library! :-)

After having spent multiple hours testing this lib and reasoning about your code whether it is race conditions free and whether it is robust enough to survive in a high profile scenario I decided to give it a try. Since then I use endless in a mission critical production app.

All in all I'm very happy with the result. But there are situations where my Go server sporadically crashes due to a panic (sync: negative WaitGroup counter) in endless with the following trace:

goroutine 3849 [running]:
runtime.panic(0x764000, 0xc208404de0)
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/runtime/panic.c:279 +0xf5
sync.(*WaitGroup).Add(0xc208042080, 0xffffffffffffffff)
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/sync/waitgroup.go:64 +0x93
sync.(*WaitGroup).Done(0xc208042080)
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/sync/waitgroup.go:82 +0x30
bitbucket.org/justphil/glutenplan.de/vendor/endless.endlessConn.Close(0x7f0997bd9ce0, 0xc20803e018, 0xc208042000, 0x0, 0x0)
    /Users/pt/Dev/Workspaces/go/src/bitbucket.org/justphil/glutenplan.de/vendor/endless/endless.go:508 +0x48
bitbucket.org/justphil/glutenplan.de/vendor/endless.(*endlessConn).Close(0xc2083d8360, 0x0, 0x0)
    <autogenerated>:16 +0xa4
net/http.(*conn).close(0xc20836e000)
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/net/http/server.go:1047 +0x4f
net/http.func·011()
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/net/http/server.go:1104 +0x22a
net/http.(*conn).serve(0xc20836e000)
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/net/http/server.go:1187 +0x78b
created by net/http.(*Server).Serve
    /usr/local/Cellar/go/1.3.3/libexec/src/pkg/net/http/server.go:1721 +0x313

(Note: I forked your lib and extended it with the ability to write pid files. But the same problem appears in the original lib as well.)

So, I again started to reason about the endless code in order to get rid of this problem. There are basically two possibilities how the WaitGroup counter can become negative. The first one is when hammerTime() tries to forcefully shut down the parent process. And the second one is the one that is panicking when a connection is closed. To me this is really weird because according to your code this panicking implies that endless wants to close connections that have neven been established.

Any hints on this one? :-)

Bye,
Phil

Hi Folks,

Is there a way to configure a new logger? I am thinking of using logrus to structure the logging in json format for a project, so if you have any suggestion, please recommend.

-Kartlee

Solution shared by one of our devs

https://developpaper.com/using-endless-in-windows-undefined-syscall-sigusr1/

Inside: C:\Program Files\Go\src\syscall\types_windows.go

var signals = [...]string{
    //Omit line n here....
    /**Compatible with Windows start*/
    16: "SIGUSR1",
    17: "SIGUSR2",
    18: "SIGTSTP",
    /**Compatible with windows end*/
}
/**Compatible with Windows start*/
func Kill(...interface{}) {
    return;
}
const (
    SIGUSR1 = Signal(0x10)
    SIGUSR2 = Signal(0x11)
    SIGTSTP = Signal(0x12)
)
/**Compatible with windows end*/

Originally posted by @kevincobain2000 in #35 (comment)

For example on go1.5.1, the C:\Go\src\syscall\types_windows.go file is missing, but
After you edit the C:\Go\src\syscall\ztypes_windows.go file you receive 10 warnings:
dll_windows.go 1 warning in func (p *Proc) Call(...) (...) {...},
exec_windows.go 1 warning in func joinExeDirAndFName(...) (...) {...},
zsyscall_windows.go 8 warnings possible misuse of unsafe.Pointer.
If it is enough to comment out the return keyword in the first two files, then it is not clear how to fix the last one.
Ideas?

https://cs.opensource.google/go/go/+/refs/tags/go1.5.1:src/syscall/
https://cs.opensource.google/go/go/+/refs/tags/go1.10:src/syscall/

go get -u github.com/fvbock/endless

github.com/fvbock/endless

..\github.com\fvbock\endless\endless.go:99: undefined: syscall.SIGUSR1
..\github.com\fvbock\endless\endless.go:100: undefined: syscall.SIGUSR2
..\github.com\fvbock\endless\endless.go:103: undefined: syscall.SIGTSTP
..\github.com\fvbock\endless\endless.go:107: undefined: syscall.SIGUSR1
..\github.com\fvbock\endless\endless.go:108: undefined: syscall.SIGUSR2
..\github.com\fvbock\endless\endless.go:111: undefined: syscall.SIGTSTP
..\github.com\fvbock\endless\endless.go:193: undefined: syscall.Kill
..\github.com\fvbock\endless\endless.go:243: undefined: syscall.Kill
..\github.com\fvbock\endless\endless.go:288: undefined: syscall.SIGUSR1
..\github.com\fvbock\endless\endless.go:289: undefined: syscall.SIGUSR2
..\github.com\fvbock\endless\endless.go:289: too many errors

I was wondering about the comment in line 529:
"// returns a dup(2) - FD_CLOEXEC flag not set"

Is that strictly correct? If I call syscall.Syscall to GETFD , the value is 1

Isn't what happens that .File() actually returns a os.File with an FD with FD_CLOEXEC set (using dupCloseOnExec() in net/fd_unix.go )
And then... when you call
cmd := exec.Command(path, args...)
err = cmd.Start()
... the Go stdlib clears CLOEXEC on the files you put in cmd.ExtraFiles as a part of forkAndExecInChild() when it shuffles file descriptors around ?

I use systemd to manage go app.
config:
ExecReload=/bin/kill -s HUP $MAINPID

Because of pid change, systemd think process failed, and use ExecStart to start process.But process was started by endless, systemd think process start failed.

How can I use endless with systemd?

I want to make a struct like this:

type App struct {
	Server *endless.endlessServer
	Router *gin.Engine
}

func (a *App) Initialize(config *config.Config) {
	a.Server = endless.NewServer(fmt.Sprintf(":%d", a.Config.Server.Port), a.Router)
}

Using this struct: https://github.com/fvbock/endless/blob/master/endless.go#L72

But it's give me error:

cannot refer to unexported name *endless.endlessServer

How can i that endless reference in my own App struct?

This signal prevents Unix shells from sending the process to the background via CTRL-Z

It would be a good idea if SIGTSTP is not intercepted by default.

func (c *Cmd) Run() error {
if err := c.Start(); err != nil {
return err
}
return c.Wait() // if execute the command fails，return error
}
I use endless in my project. In a production environment, we need to modify the configuration file frequently, and then restart the program for the new configuration to take effect.
But the configuration file often has syntax errors, making the program unable to restart successfully. We need to modify the configuration again and restart again, but endless output "Fork err: Another process already forked. Ignoring this one."

https://github.com/fvbock/endless/blob/master/examples/multi_port.go

it sleeps 1 second before endless.ListenAndServe in the first goroutine. why ?

eg ./simple_server -test
after forking would be
./simple_server -continue - dropping the original args

kill -SIGINT pid

PS C:\Users\xxx\Workspace\xxx-xxx> go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\xxx\AppData\Local\go-build
set GOENV=C:\Users\xxx\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\xxx\go\pkg\mod
set GONOPROXY=.corp.example.com
set GONOSUMDB=.corp.example.com
set GOOS=windows
set GOPATH=C:\Users\xxx\go
set GOPRIVATE=*.corp.example.com
set GOPROXY=https://proxy.golang.com.cn,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.18.1
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\xxx\Workspace\xxx-xxx\go.mod
set GOWORK=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\xxx\AppData\Local\Temp\go-build219240513=/tmp/go-build -gno-record-gcc-switches

I want to know how to use this with websocket long connection，no connection will finish~

server.SignalHooks[endless.PRE_SIGNAL][syscall.SIGUSR1],
    a.Logger.Rotate)

func Rotate(conf *toml.TomlTree) {
.....
}

How i do?

Hi, how to create a systemd init script for application using endless package ?

Thank you

how to use with r.Run(":8080") and r.RunTLS(":8443", "server.pem", "server.key")

Hi,

When I use endless, I found the following code cause the memory leak after restart.

func main() {
    fd, _ := os.Create("access.log")
    defer fd.Close()
    // start http server
    ...
}

After restarting, use lsof to list the open FDs, there are 2 FDs for access.log

Am I using it incorrectly?

Thanks!
Kevin

The standard flag library is not a part of Go that everyone love and use. Personally I (and many other) like another flag library.
The problem is, with this library, we should use the standard flag library (LAME IMHO), or else, this won't work.
I think, using the flag library is not required at all, since the parent process, can simply change env for the child library.

And the other problem is, I want to change it to system fork (not execute with os.exec) and in that case I can not use the flag library, since the child is exactly like parent and there is no way to change child Arguments.

2020/11/24 15:28:49 net.FileListener error: file file+net : getsockopt: socket operation on non-socket

This repo hasn't been updated since 2016. Is it officially abandoned? If so, can you mark it as archive, or put out a call for maintainers as a new issue and in the README?

Support Http2.0

env:
Linux l-qfy0.dba.dev.cn0 3.10.0-327.28.3.el7.x86_64 #1 SMP Thu Aug 18 19:05:49 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

go version go1.17 linux/amd64

test.go

import (
	"context"
	"flag"
	"fmt"
	"os/exec"
	"strconv"
	"syscall"

	"github.com/fvbock/endless"
	"github.com/gin-gonic/gin"
)

var ctx context.Context
var cancel context.CancelFunc
var reload bool

var port = flag.Int("port", 8080, "")

func main() {
	flag.Parse()
	ListenHTTP(setReload)
}

func setReload() {
	reload = true
}
func ListenHTTP(reloadFunc func()) error {
	var handler = gin.Default()
	handler.GET("/", startDaemon())
	handler.GET("/hello", hello())

	ctx, cancel = context.WithCancel(context.Background())
	defer cancel()
	s := endless.NewServer(":"+strconv.Itoa(*port), handler)
	go func() {
		select {
		case <-ctx.Done():
		}
		if s != nil {
			if err := s.Shutdown(ctx); err != nil {
				fmt.Println(err)
			}
		}
	}()
	s.SignalHooks[endless.PRE_SIGNAL][syscall.SIGHUP] = append(
		s.SignalHooks[endless.PRE_SIGNAL][syscall.SIGHUP],
		reloadFunc)
	if err := s.ListenAndServe(); err != nil {
		return err
	}
	return nil
}
func hello() gin.HandlerFunc {
	return func(c *gin.Context) {
		fmt.Println("hello ", *port)
	}
}
func startDaemon() gin.HandlerFunc {
	return func(c *gin.Context) {
		start(c)
	}
}

func start(c *gin.Context) (err error) {
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()
	cmd := exec.CommandContext(ctx, "sh", []string{"start.sh"}...)
	err = cmd.Run()
	if err != nil {
		fmt.Println(err)
		return
	}
	return
}

start.sh

nohup /home/qfy/tmp/test --port=8081 > /dev/null 2>&1 &

basedir:/root/tmp

steps:

1. run test

2. start with new shell, then send http requst，now 8081 is listening

3. kill 8081 ，and reload test

4.then send http requst，but 8081 is not listening

Is that a bug need to fix ?
And how can I resolve it.

a process is running , then i modified some code ,then, i try to build it again. and it shows the error msg above.

i know the reason, but, i mean , how to use the endless with the right way.

when starting more than one listener (most likely through goroutines) the initialization order is not determined. on restart this can result in the wrong socket being assigned to a server.

solution: pass original initialization order (by address) to the child.

this only affects users that start more than one server in one binary.

Expected behavior: endless to append whatever runtime args it needs, but preserve the application command line params.

Actual behavior: endless takes over the command line params, and treats all application params as illegal

What's most curious is that args required by the main app are still considered required, even though providing them is disallowed by endless.

1074)internationalsos/riskratings % ./rr -h
Usage of ./rr:
  -continue=false: Dummy -- needed by endless
  -debug=false: Enable debugging
  -pass="": Password
  -port=":8080": Listen port
  -refresh=10: Cache refresh time, in minutes
  -uri="******************": URI of data source
  -user="": Username
1075)internationalsos/riskratings % ./rr
Password is required
Usage of ./rr:
  -continue=false: Dummy -- needed by endless
  -debug=false: Enable debugging
  -pass="": Password
  -port=":8080": Listen port
  -refresh=10: Cache refresh time, in minutes
  -uri="******************": URI of data source
  -user="": Username
1076)internationalsos/riskratings % ./rr -pass xyz -user abc -port :8123
2015/06/29 22:33:58 start listening on localhost:8123
flag provided but not defined: -pass
Usage of ./rr:
  -continue=false: listen on open fd (after forking)
  -socketorder="": previous initialization order - used when more than one listener was started

This library only provides public API for http.Handlers and not net.Listeners :-(
endlessListener is private for some reason.
Please make the TCP-level API available!

This error is printed whenever a SIGHUP is sent to an endless server. The server continues to run but this error is causing my program which calls endless to panic. Where is the error coming from?