microsoft / go-winio

Win32 IO-related utilities for Go

go-winio

This repository contains utilities for efficiently performing Win32 IO operations in Go. Currently, this is focused on accessing named pipes and other file handles, and for using named pipes as a net transport.

This code relies on IO completion ports to avoid blocking IO on system threads, allowing Go to reuse the thread to schedule another goroutine. This limits support to Windows Vista and newer operating systems. This is similar to the implementation of network sockets in Go's net package.

Please see the LICENSE file for licensing information.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit Microsoft CLA.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

Additionally, the pull request pipeline requires the following steps to be performed before mergining.

Code Sign-Off

We require that contributors sign their commits using git commit --signoff to certify they either authored the work themselves or otherwise have permission to use it in this project.

A range of commits can be signed off using git rebase --signoff.

Please see the developer certificate for more info, as well as to make sure that you can attest to the rules listed. Our CI uses the DCO Github app to ensure that all commits in a given PR are signed-off.

Linting

Code must pass a linting stage, which uses golangci-lint. The linting settings are stored in .golangci.yaml, and can be run automatically with VSCode by adding the following to your workspace or folder settings:

    "go.lintTool": "golangci-lint",
    "go.lintOnSave": "package",

Additional editor integrations options are also available.

Alternatively, golangci-lint can be installed locally and run from the repo root:

# use . or specify a path to only lint a package
# to show all lint errors, use flags "--max-issues-per-linter=0 --max-same-issues=0"
> golangci-lint run ./...

Go Generate

The pipeline checks that auto-generated code, via go generate, are up to date.

This can be done for the entire repo:

> go generate ./...

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Special Thanks

Thanks to natefinch for the inspiration for this library. See npipe for another named pipe implementation.

go-winio's Issues

external dependencies introduced

Prior this library only used libraries from golang.org. Now it's introduced some third-party dependency called "github.com/sirupsen/logrus", which itself depends on a whole nasty chain of things.

Can you remove this dependency? Or move its entire dependency chain to github.com/Microsoft/*?

Need a GrantVmGroupAccess.exe utility

Note to self - will be needed for @kevpar in the SF deployment stuff to ACL initrd.img and rootfs.vhd.

EOF message not received with message mode named pipes

In some circumstances, the 0-length message sent when CloseWrite is called on a message-mode named pipe is not received separately from the previous message. It seems to happen on CloseWrite is called while the previous message has not yet been read by the remote party.

This issue has an impact on Docker CLI and Docker for Windows, when running scripts like:
echo "hello world" | docker run ...

It would be nice if this used some CI/CD service

I use AppVeyor for my CI/CD integration with GitHub. I'm sure MSFT has their own (with Azure) -- it would be super cool if this repo included automatic validation of PRs just running the go test suite that already exists here.

Can I use it to write data to keyboard port?

Hi, guys , Thanks for your great work 👍 Can I use this library to write data to keyboard port directly?
And can you give me an example? Thanks a lot ! :)

GPIO/PWM/I2C interfaces in Golang?

I see there are many GPIO/PWM/I2C Windows.Devices interfaces now available as part of WinRT.

Anyone know how can I call these from Golang?

Closing on one gofunc while another is waiting on Read() results in ErrFileClosed

See the call stack at:
keybase/client#13745
It seems this should be considered a regular EOF instead of a scary error.
Pull request with proposed fix at:
#94

Bad permission on pipe (supposed)

Hello,

First of all thank you for this implementation :)

In my case there are two computers : one client and one server. Both are on the same local network.

I modified the plugin to accept remote connection:

pip.go line 84

	cFILE_PIPE_REJECT_REMOTE_CLIENTS = 0

However the client tells me that the password or the user is wrong. Do you know where this can come from?

Case-conflict between canonical repo address and go imports

The github.com/Microsoft org was just renamed to github.com/microsoft

This means that the canonical github repo address now conflicts with canonical go import packages (for example, "github.com/Microsoft/go-winio/pkg/guid", as referenced from https://github.com/microsoft/go-winio/blob/master/hvsock.go#L12)

It is now very easy for downstream consumers that depend on leaf packages only (which don't import any other upper-case "github.com/Microsoft/..." packages) to accidentally clone and add imports to lower-case "github.com/microsoft/..." packages. This will work in isolation, but as soon as their library has to coexist in a build with any other component which used the canonical "github.com/Microsoft/..." imports, builds will start failing.

(This was possible before, but way harder to do accidentally, since all the links and clone commands from github agreed with the canonical go import packages)

Context:

• https://twitter.com/ehotinger/status/1125792922463424512
• https://twitter.com/liggitt/status/1125843936864612353

Cautionary tale:

• sirupsen/logrus#570 (comment)

Please consider reverting the Microsoft -> microsoft rename.

Benchmarks for os.File vs win32File?

I'm curious what kind of performance improvements can be seen when switching from *os.File to win32File for read/write operations. Are there any benchmarks available, or does anyone have anecdotal experience to share?

As far as I understand it, the advantage of win32File is that it blocks the calling goroutine on a channel instead of in a syscall, which means the go runtime needs fewer OS threads and can schedule goroutines more efficiently.

Check out my named pipes.library

HTTPS://github.com/natefinch/npipe

Feel free to crib anything from there if you want (obviously, since its MIT licensed).

I haven't had a chance to look too deeply at yours to know the difference, but they looked pretty similar.

Misuse of sync.WaitGroup

With the test below, the following crash occurs instantly. The test is a bit
unwieldy, sorry about that. I am using Go 1.6 on Windows 10.
The documentation of package sync states that

If a WaitGroup is reused to wait for several independent sets of events, new Add calls must happen after all previous Wait calls have returned.

The runtime check for this invariant was added in Go 1.5.
I wonder why this issue hasn't come up yet.

=== RUN   TestCloseRace
panic: sync: WaitGroup is reused before previous Wait has returned

goroutine 369 [running]:
panic(0x572cc0, 0xc08209b460)
    C:/tools/go/src/runtime/panic.go:464 +0x3f4
sync.(*WaitGroup).Wait(0xc0820e3638)
    C:/tools/go/src/sync/waitgroup.go:129 +0x11b
github.com/microsoft/go-winio.(*win32File).closeHandle(0xc0820e3630)
    c:/Users/fjl/src/github.com/microsoft/go-winio/file.go:94 +0x62
github.com/microsoft/go-winio.(*win32File).Close(0xc0820e3630, 0x0, 0x0)
    c:/Users/fjl/src/github.com/microsoft/go-winio/file.go:103 +0x34
github.com/microsoft/go-winio.TestCloseRace.func1.1()
    c:/Users/fjl/src/github.com/microsoft/go-winio/pipe_test.go:293 +0x12d
created by time.goFunc
    C:/tools/go/src/time/sleep.go:129 +0x41
exit status 2

The test:

func TestCloseRace(t *testing.T) {
    l, err := ListenPipe(testPipeName, &PipeConfig{})
    if err != nil {
        t.Fatal(err)
    }

    var (
        maxCloseTimeout   = 100 * time.Millisecond
        numClients        = 10
        numConnsPerClient = 10
        writeDelay        = 30 * time.Millisecond
        writeData         = make([]byte, 100)
        wg                sync.WaitGroup
    )
    server := func() {
        defer wg.Done()
        defer l.Close()
        for {
            conn, err := l.Accept()
            if err != nil {
                t.Log("accept error:", err)
                return
            }
            // Kill the connection after a random delay.
            timeout := time.Duration(rand.Int63n(int64(maxCloseTimeout)))
            time.AfterFunc(timeout, func() {
                t.Logf("killing connection after %v", timeout)
                conn.Close()
            })
            go io.Copy(ioutil.Discard, conn)
        }
    }
    client := func() {
        defer wg.Done()
        for i := 0; i < numConnsPerClient; i++ {
            conn, err := DialPipe(testPipeName, nil)
            if err != nil {
                t.Log("dial error:", err)
                return
            }
            defer conn.Close()
            // Kill the connection after a random delay.
            timeout := time.Duration(rand.Int63n(int64(maxCloseTimeout)))
            time.AfterFunc(timeout, func() {
                t.Logf("killing client connection after %v", timeout)
                conn.Close()
            })
            // Write some data until the connection goes away.
            for {
                if _, err := conn.Write(writeData); err != nil {
                    t.Log("write error:", err)
                    break
                }
                time.Sleep(writeDelay)
            }
        }
    }

    wg.Add(1)
    go server()
    wg.Add(numClients)
    for i := 0; i < numClients; i++ {
        go client()
    }
    wg.Wait()
}

The first *win32PipeListener.Accept will cause the client hanged in Windows 7

I was trying go-winio in my project, but I noticed that the same code had different results in Win 7 and Win 10. In Win 7, the client was hanged on first connecting after server restarted, and worked correctly after the second time. After debugged and found that server's syscall.getQueuedCompletionStatus couldn't be triggered when the client first connected. Then I tried to find the reason and noticed that win32PipeListener.firstHandle's access was syscall.SYNCHRONIZE. I found they created named pipe with full access in the loop in this example, so I tried to modify code and the bellow is the diff block.

diff --git a/pipe.go b/pipe.go                                                                      
index d6a46f6..ac59ca0 100644                                                                       
--- a/pipe.go                                                                                       
+++ b/pipe.go                                                                                       
@@ -259,6 +259,7 @@ type acceptResponse struct {                                                    
                                                                                                    
 type win32PipeListener struct {                                                                    
        firstHandle syscall.Handle                                                                  
+       handle      syscall.Handle                                                                  
        path        string                                                                          
        config      PipeConfig                                                                      
        acceptCh    chan (chan acceptResponse)                                                      
@@ -319,7 +320,7 @@ func makeServerPipeHandle(path string, sd []byte, c *PipeConfig, first bool) (sy
                // By not asking for read or write access, the named pipe file system               
                // will put this pipe into an initially disconnected state, blocking                
                // client connections until the next call with first == false.                      
-               access = syscall.SYNCHRONIZE                                                        
+               // access = syscall.SYNCHRONIZE                                                     
        }                                                                                           
                                                                                                    
        timeout := int64(-50 * 10000) // 50ms
@@ -338,13 +339,19 @@ func makeServerPipeHandle(path string, sd []byte, c *PipeConfig, first bool) (sy
 }                                                                                                    
                                                                                                      
 func (l *win32PipeListener) makeServerPipe() (*win32File, error) {                                   
-       h, err := makeServerPipeHandle(l.path, nil, &l.config, false)                                 
-       if err != nil {                                                                               
-               return nil, err                                                                       
+       handle := l.handle                                                                            
+       if handle == 0 {                                                                              
+               h, err := makeServerPipeHandle(l.path, nil, &l.config, false)                         
+               if err != nil {                                                                       
+                       return nil, err                                                               
+               }                                                                                     
+               handle = h                                                                            
+       } else {                                                                                      
+               l.handle = 0                                                                          
        }                                                                                             
-       f, err := makeWin32File(h)                                                                    
+       f, err := makeWin32File(handle)                                                               
        if err != nil {                                                                               
-               syscall.Close(h)                                                                      
+               syscall.Close(handle)                                                                 
                return nil, err                                                                       
        }                                                                                             
        return f, nil                                                                                 
@@ -451,6 +458,7 @@ func ListenPipe(path string, c *PipeConfig) (net.Listener, error) {               
        }                                                                                             
        l := &win32PipeListener{                                                                      
                firstHandle: h,                                                                       
+               handle:      h,                                                                       
                path:        path,                                                                    
                config:      *c,                                                                      
                acceptCh:    make(chan (chan acceptResponse)),                                        

Then it worked on both Win 7 and Win 10!

Here is my test code. link

Actually, I think the current implementation is correct, and I still don't know why it fails on Win 7.

Sorry for poor English...

Test Environment:
OS: Windows 7 ( 7601: service pack 1 )

Build Environment:
OS: Windows 10 (1909)
GO version: 1.14 windows/amd64

Possible race

Found running CI on a daemon instrumented with go race detector. @jstarks

WARNING: DATA RACE
Write at 0x00c042296dd8 by goroutine 14:
github.com/Microsoft/go-winio.(_win32File).closeHandle()
e:/go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/file.go:92 +0x7b
github.com/Microsoft/go-winio.(_win32File).Close()
e:/go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/file.go:103 +0x55
github.com/Microsoft/go-winio.(*win32PipeListener).listenerRoutine()
e:/go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/pipe.go:275 +0x47c

Previous read at 0x00c042296dd8 by goroutine 94:
github.com/Microsoft/go-winio.(_win32File).asyncIo()
e:/go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/file.go:145 +0x167
github.com/Microsoft/go-winio.connectPipe()
e:/go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/pipe.go:360 +0x1b0
github.com/Microsoft/go-winio.(_win32PipeListener).listenerRoutine.func1()
e:/go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/pipe.go:265 +0x56

Goroutine 14 (running) created at:
github.com/Microsoft/go-winio.ListenPipe()
e:/go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/pipe.go:350 +0x6ce
github.com/docker/docker/pkg/listeners.Init()
e:/go/src/github.com/docker/docker/pkg/listeners/listeners_windows.go:43 +0x8e1
main.(_DaemonCli).start()
e:/go/src/github.com/docker/docker/cmd/dockerd/daemon.go:215 +0x1f80
main.runDaemon()
e:/go/src/github.com/docker/docker/cmd/dockerd/docker.go:75 +0x25a
main.newDaemonCommand.func1()
e:/go/src/github.com/docker/docker/cmd/dockerd/docker.go:41 +0xe9
github.com/spf13/cobra.(_Command).execute()
e:/go/src/github.com/docker/docker/vendor/src/github.com/spf13/cobra/command.go:643 +0xa03
github.com/spf13/cobra.(_Command).ExecuteC()
e:/go/src/github.com/docker/docker/vendor/src/github.com/spf13/cobra/command.go:739 +0x7ba
github.com/spf13/cobra.(_Command).Execute()
e:/go/src/github.com/docker/docker/vendor/src/github.com/spf13/cobra/command.go:692 +0x51
main.main()
e:/go/src/github.com/docker/docker/cmd/dockerd/docker.go:99 +0x152

Goroutine 94 (running) created at:
github.com/Microsoft/go-winio.(*win32PipeListener).listenerRoutine()
e:/go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/pipe.go:266 +0x305

listing directory contents for directory opened with FILE_FLAG_BACKUP_SEMANTICS

The OpenForBackup function uses FILE_FLAG_BACKUP_SEMANTICS to get an os.File.

There is this comment:

https://github.com/Microsoft/go-winio/blob/7da180ee92d8bd8bb8c37fc560e673e6557c392f/backup.go#L268

I can confirm Readdir() doesn't work (result is The system cannot find the path specified (unless ACLs permit reading this directory anyway)).

How can I list the contents for such a directory?

panic:Failed to find SetFileCompletionNotificationModes procedure in kernel32.dll: The specified procedure could not b e found.

go-winio.MakeOpenFile Prompt for this error.

win 2008 before the version does not have this function.
Hope to solve this compatibility problem.

Remote Named Pipe Connections

Will remote named pipe connections be implemented? The only thing preventing their usage is a pipe mode parameter to CreateNamedPipe.

Nil pointer dereference in prepareIo

Sometimes, we get nil pointer derefence in this line in file.go:

func (f *win32File) prepareIo() (*ioOperation, error) {
	f.wg.Add(1)			// <---------- HERE
	if f.closing {
		return nil, ErrFileClosed
	}
	c := &ioOperation{}
	c.ch = make(chan ioResult)
	return c, nil
}

because f is nil. This is caused by a race condition in listenerRoutine(), when someone calls pipe.Close() before connectPipe() has a chance to start:

p, err := l.makeServerPipe()
if err == nil {
	// Wait for the client to connect.
	ch := make(chan error)
	go func() {
		ch <- connectPipe(p)		// <------ 2. we use niled pointer
	}()
	select {
	case err = <-ch:
		if err != nil {
			p.Close()
			p = nil
		}
	case <-l.closeCh:
		// Abort the connect request by closing the handle.
		p.Close()
		p = nil			// <----- 1. we nil the file pointer before connectPipe()
		err = <-ch
		if err == nil || err == ErrFileClosed {
			err = ErrPipeListenerClosed
		}
		closed = true
	}

This seems related to #31

fatal error: unexpected signal during runtime execution

Hi,

My go binary listens on a named pipe. All works fine normally, but I sometimes get this panic if the named pipe receives a connection:

fatal error: unexpected signal during runtime execution
[signal 0xc0000005 code=0x0 addr=0xffffffffffffffff pc=0x40f115]

goroutine 18 [running]:
runtime.throw(0xa53f6f, 0x2a)
	C:/Go/src/runtime/panic.go:596 +0x9c fp=0xc042151e08 sp=0xc042151de8
runtime.sigpanic()
	C:/Go/src/runtime/signal_windows.go:155 +0x18b fp=0xc042151e38 sp=0xc042151e08
runtime.lock(0x5105fcc3b06b83db)
	C:/Go/src/runtime/lock_sema.go:43 +0x65 fp=0xc042151e70 sp=0xc042151e38
runtime.chansend(0x970300, 0x5105fcc3b06b8383, 0xc042151fb0, 0xc042151f01, 0x5d013a, 0xc042151f9c)
	C:/Go/src/runtime/chan.go:176 +0xba fp=0xc042151f20 sp=0xc042151e70
runtime.chansend1(0x970300, 0x5105fcc3b06b8383, 0xc042151fb0)
	C:/Go/src/runtime/chan.go:113 +0x4d fp=0xc042151f60 sp=0xc042151f20
vendor/github.com/Microsoft/go-winio.ioCompletionProcessor(0x1f8)
	src/vendor/github.com/Microsoft/go-winio/file.go:133 +0xea fp=0xc042151fd8 sp=0xc042151f60
runtime.goexit()
	C:/Go/src/runtime/asm_amd64.s:2197 +0x1 fp=0xc042151fe0 sp=0xc042151fd8
created by vendor/github.com/Microsoft/go-winio.initIo
	src/vendor/github.com/Microsoft/go-winio/file.go:57 +0x87

Other google results for this error were submitted directly to golang/go, so i don't know if this is an issue with Golang's runtime or an issue with this library.

It seems to happen randomly, based on synchronisation state of other parts of the program.

go version go1.8 windows/amd64

EDIT: Simplified report using the latest information.

MailSlot support?

Is there any support for MailSlot? Using ListenPipe with a mailslot path gives error Incorrect function, I'm wondering if MailSlot support is possible, or is there any other library that I can use?

github.com/Microsoft/go-winio/pkg/guid v0.4.14 does not build on linux

$ go build github.com/Microsoft/go-winio/pkg/guid
../../../go/pkg/mod/github.com/!microsoft/[email protected]/pkg/guid/guid.go:16:2: build constraints exclude all Go files in /usr/local/google/home/fejta/go/pkg/mod/golang.org/x/[email protected]/windows

Can we flag packages that we only expect to build on linux? AKA add a // +build windows line etc

$ grep -R "+build" ~/go/pkg/mod/golang.org/x/[email protected]/windows/*
/usr/local/google/home/fejta/go/pkg/mod/golang.org/x/[email protected]/windows/aliases.go:// +build windows
/usr/local/google/home/fejta/go/pkg/mod/golang.org/x/[email protected]/windows/aliases.go:// +build go1.9
/usr/local/google/home/fejta/go/pkg/mod/golang.org/x/[email protected]/windows/eventlog.go:// +build windows

Race condition in construction of `win32PipeListener`

During the construction of win32PipeListener, an unused named pipe instance gets created. The ListenPipe constructor attempts to block this unused instance with a CreateFile call on line 354, but this cannot happen transactionally during the creation of the named pipe: between line 349, just after the instance is created, and line 354, an optimistic named pipe client can connect to the pipe instance before the listener itself has a chance to block it. This causes listener construction to fail, despite the existence of a valid client on the named pipe handle.

I have a general use case where this race condition cannot be worked around: the named pipe client starts the named pipe server, and uses the existence of the pipe name to assume that the server is ready to accept connections. I suspect that #67 may be caused by this race condition as well.

Cross-compile Linux->Windows fails on latest commit: "undefined: securityAttributes"

On the current HEAD I see this on a cross-compile hosted on Linux (golang 1.8.1) and targeted at Windows (CGO_ENABLED=0; GOOS=windows)

go install -installsuffix static -ldflags ' -extldflags -static'  ./cmd/... ./pkg/...
# github.com/drud/ddev/vendor/github.com/Microsoft/go-winio
vendor/github.com/Microsoft/go-winio/zsyscall_windows.go:142: undefined: securityAttributes
vendor/github.com/Microsoft/go-winio/zsyscall_windows.go:151: undefined: securityAttributes
vendor/github.com/Microsoft/go-winio/zsyscall_windows.go:164: undefined: securityAttributes
vendor/github.com/Microsoft/go-winio/zsyscall_windows.go:173: undefined: securityAttributes

This problem appears to have been introduced in #48, as it happens on 13736c3 and does not happen before that PR (compiles fine with f3b1913)

Exception when executing go code (e.g. docker daemon)

Every time I call a command from the docker daemon, I get an Exception as shows below (in Powershell, cmd.exe is the same):

$ docker version
Exception 0xc000001d 0x0 0x0 0x18000890c
PC=0x18000890c

syscall.Syscall(0x7ffb27dadc40, 0x2, 0x274, 0x3, 0x0, 0x264, 0x0, 0x0)
        /usr/local/go/src/runtime/syscall_windows.go:163 +0x5c
github.com/Microsoft/go-winio.setFileCompletionNotificationModes(0x274, 0x203, 0x0, 0x0)
        /go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/zsyscall.go:88 +0x70
github.com/Microsoft/go-winio.makeWin32File(0x274, 0xc08202f6e8, 0x0, 0x0)
        /go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/file.go:76 +0xf8
github.com/Microsoft/go-winio.DialPipe(0xc0820a9148, 0x16, 0xc08202f870, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/pipe.go:186 +0x81d
github.com/docker/go-connections/sockets.DialPipe(0xc0820a9148, 0x16, 0x773594000, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/go-connections/sockets/sockets_windows.go:12 +0x54
github.com/docker/go-connections/sockets.ConfigureTransport.func2(0xced7b0, 0x3, 0xc0822fd060, 0x19, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/go-connections/sockets/sockets.go:29 +0x6e
net/http.(*Transport).dial(0xc0820b06c0, 0xced7b0, 0x3, 0xc0822fd060, 0x19, 0x0, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/http/transport.go:662 +0x9d
net/http.(*Transport).dialConn(0xc0820b06c0, 0x0, 0xcea578, 0x4, 0xc0822fd060, 0x19, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/http/transport.go:765 +0x1e44
net/http.(*Transport).getConn.func4(0xc0820b06c0, 0x0, 0xcea578, 0x4, 0xc0822fd060, 0x19, 0xc08230c6c0)
        /usr/local/go/src/net/http/transport.go:709 +0x6d
created by net/http.(*Transport).getConn
        /usr/local/go/src/net/http/transport.go:711 +0x269

goroutine 1 [select]:
github.com/docker/engine-api/client/transport/cancellable.Do(0x2be86f0, 0xc082008d88, 0x2bed900, 0xc0822fc880, 0xc082056700, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/engine-api/client/transport/cancellable/cancellable.go:56 +0x4a4
github.com/docker/engine-api/client.(*Client).sendClientRequest(0xc08230c420, 0x2be86f0, 0xc082008d88, 0xce5800, 0x3, 0xcf2180, 0x8, 0x0, 0x0, 0x0, ...)
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/engine-api/client/request.go:106 +0x510
github.com/docker/engine-api/client.(*Client).sendRequest(0xc08230c420, 0x2be86f0, 0xc082008d88, 0xce5800, 0x3, 0xcf2180, 0x8, 0x0, 0x0, 0x0, ...)
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/engine-api/client/request.go:75 +0x2e3
github.com/docker/engine-api/client.(*Client).get(0xc08230c420, 0x2be86f0, 0xc082008d88, 0xcf2180, 0x8, 0x0, 0x0, 0xc08230a900, 0x0, 0x0)
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/engine-api/client/request.go:33 +0xad
github.com/docker/engine-api/client.(*Client).ServerVersion(0xc08230c420, 0x2be86f0, 0xc082008d88, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/engine-api/client/version.go:12 +0xa1
github.com/docker/docker/api/client/system.runVersion(0xc08206a510, 0xc0823005a0, 0x0, 0x0)
        /go/src/github.com/docker/docker/api/client/system/version.go:87 +0x41c
github.com/docker/docker/api/client/system.NewVersionCommand.func1(0xc082307200, 0x113d5e8, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/api/client/system/version.go:49 +0x3c
github.com/spf13/cobra.(*Command).execute(0xc082307200, 0x113d5e8, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/vendor/src/github.com/spf13/cobra/command.go:593 +0x705
github.com/spf13/cobra.(*Command).ExecuteC(0xc08206d8c0, 0xc082307200, 0x0, 0x0)
        /go/src/github.com/docker/docker/vendor/src/github.com/spf13/cobra/command.go:683 +0x563
github.com/spf13/cobra.(*Command).Execute(0xc08206d8c0, 0x0, 0x0)
        /go/src/github.com/docker/docker/vendor/src/github.com/spf13/cobra/command.go:642 +0x34
github.com/docker/docker/cli/cobraadaptor.CobraAdaptor.run(0xc08206d8c0, 0xc08206a510, 0xc0820081e0, 0x7, 0xc082004690, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/cli/cobraadaptor/adaptor.go:118 +0x266
github.com/docker/docker/cli/cobraadaptor.CobraAdaptor.Command.func1(0xc082004690, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/docker/docker/cli/cobraadaptor/adaptor.go:126 +0x93
github.com/docker/docker/cli.(*Cli).Run(0xc0822fab10, 0xc082004690, 0x1, 0x1, 0x0, 0x0)
        /go/src/github.com/docker/docker/cli/cli.go:81 +0x352
main.main()
        /go/src/github.com/docker/docker/cmd/docker/docker.go:72 +0x4b1

goroutine 5 [syscall]:
os/signal.signal_recv(0x0)
        /usr/local/go/src/runtime/sigqueue.go:116 +0x139
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:22 +0x1f
created by os/signal.init.1
        /usr/local/go/src/os/signal/signal_unix.go:28 +0x3e

goroutine 7 [select]:
net/http.(*Transport).getConn(0xc0820b06c0, 0xc082056700, 0x0, 0xcea578, 0x4, 0xc0822fd060, 0x19, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/http/transport.go:714 +0x4f6
net/http.(*Transport).RoundTrip(0xc0820b06c0, 0xc082056700, 0xc0820b06c0, 0x0, 0x0)
        /usr/local/go/src/net/http/transport.go:314 +0x7f0
net/http.send(0xc082056700, 0x2be44e0, 0xc0820b06c0, 0x0, 0x0, 0x0, 0xc08230e740, 0x0, 0x0)
        /usr/local/go/src/net/http/client.go:260 +0x6be
net/http.(*Client).send(0xc0822faf30, 0xc082056700, 0x0, 0x0, 0x0, 0xc839a0, 0x0, 0x0)
        /usr/local/go/src/net/http/client.go:155 +0x18c
net/http.(*Client).doFollowingRedirects(0xc0822faf30, 0xc082056700, 0xe69930, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/http/client.go:475 +0x8ab
net/http.(*Client).Do(0xc0822faf30, 0xc082056700, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/http/client.go:188 +0x106
github.com/docker/engine-api/client/transport/cancellable.Do.func1(0x2bed900, 0xc0822fc880, 0xc082056700, 0xc08230c660)
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/engine-api/client/transport/cancellable/cancellable.go:49 +0x3c
created by github.com/docker/engine-api/client/transport/cancellable.Do
        /go/src/github.com/docker/docker/vendor/src/github.com/docker/engine-api/client/transport/cancellable/cancellable.go:52 +0x106

goroutine 9 [syscall, locked to thread]:
syscall.Syscall6(0x7ffb27da6bf0, 0x5, 0x264, 0xc08231bf54, 0xc08231bf58, 0xc08231bf60, 0xffffffff, 0x0, 0x1, 0x1, ...)
        /usr/local/go/src/runtime/syscall_windows.go:174 +0x5c
github.com/Microsoft/go-winio.getQueuedCompletionStatus(0x264, 0xc08231bf54, 0xc08231bf58, 0xc08231bf60, 0xffffffff, 0x0, 0x0)
        /go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/zsyscall.go:76 +0xb1
github.com/Microsoft/go-winio.ioCompletionProcessor(0x264)
        /go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/file.go:127 +0x7c
created by github.com/Microsoft/go-winio.initIo
        /go/src/github.com/docker/docker/vendor/src/github.com/Microsoft/go-winio/file.go:55 +0xb3
rax     0x0
rbx     0x111fe08
rcx     0x274
rdi     0x256000
rsi     0xc08202f5f8
rbp     0x0
rsp     0x8fd88
r8      0x1
r9      0x0
r10     0x274
r11     0x13c4d48
r12     0x2
r13     0xe66316
r14     0x3
r15     0x8
rip     0x18000890c
rflags  0x10246
cs      0x33
fs      0x53
gs      0x2b

Apparently this is a Go - Windows issue, and it's actually reported in a few places like syncthing/syncthing#3432 golang/go#13541.

Looks like it's originating in Microsoft/go-winio:zsyscall.go#L88 and is related to kernel32.dll.

I encountered a similar Exception when I ran one of my own tools written in Go (using Cgo), but it only happened the first time. Second time (and then on) it worked without problems, unlike docker which fails every time the daemon is called by a client command.

I originally reported this issue here docker/for-win#140 but, as noted, I added it here as well, maybe someone here can help.

Race condition in `makeConnectedServerPipe` causes `win32PipeListener.Close()` deadlock

On line 273 in pipe.go, makeConnectedServerPipe signals to the connectPipe goroutine that the listener has been closed by closing the underlying pipe it is trying to connect. However, if the connectPipe call succeeds before the call to p.Close(), then err may also be one of cERROR_PIPE_CONNECTED or cERROR_NO_DATA. This causes makeConnectedServerPipe to effectively swallow the control signal sent over l.closeCh, preventing listenerRoutine from accepting it, causing win32PipeListener.Close() to deadlock without ever closing the listener.

I have been able to reproduce this behavior while developing #84, but only by running go test in a loop until the deadlock occurs.

Security Descriptors should be preserved in binary form instead of using SDDL

Since we recently found some issues with translation of SDDL to binary Security Descriptor when the SDDL includes any domain info, we should skip translating to and from the SDDL and encode the binary directly into the tar streams instead.

New release of go-winio to support the new pkg/process

pkg/process that was introduced in f272bf3 and is being used already in:
microsoft/hcsshim@1e4afe9#diff-557d18187d922c13ba3a0277fac10fa3R9

Can we please cut a release to support the new code and then update microsoft/hcsshim?

Context - trying to rationalize the dependencies and bring in newer versions of stuff into kubernetes ( WIP in kubernetes/kubernetes#86975 )

DialPipe problem with multiple calls / waiting for busy pipe

Currently there is some work in progress (portainer/portainer#1186) to add mapped named pipe support into Portainer so it will be easier to run it in a Windows container ( docker run -u ContainerAdministrator -it -v //./pipe/docker_engine://./pipe/docker_engine -p 9000:9000 stefanscherer/portainer:insider -H npipe:////./pipe/docker_engine )

We're using winio.DialPipe() to connect to the Docker engine from inside a microsoft/nanoserver-insider container.

At the moment we see problems in Portainer as it sometimes makes multiple requests requested by the web frontend. Here's a WireShark screenshot with the problem.

The two /api/endpoints/1/docker/xx GET requests use separate connections and run in parallel. One of the calls aborts with "open //./pipe/docker_engine: The system cannot find the file specified."

I have seen the check for cERROR_PIPE_BUSY and first thought that createFile() returns cERROR_FILE_NOT_FOUND and aborts immediatelly.

We are using the default timeout right now

    return winio.DialPipe(namedPipePath, nil)

but I also tried giving a 5 second timeout

    timeout := 5000 * time.Millisecond
    return winio.DialPipe(namedPipePath, &timeout)

Digging deeper it seems like the waitNamedPipe() just does not wait long enough or cannot start wait for the pipe.

Adding some Println in DialPipe I now see this running Portainer in the second DialPipe call:

DialPipe createFile err All pipe instances are busy.
DialPipe waitNamedPipe ms 4998
DialPipe waitNamedPipe returns err The system cannot find the file specified.
DialPipe returns err The system cannot find the file specified.

The problem is that waitNamedPipe returns immediatelly and does not wait the given 5000 milliseconds.

How can we make DialPipe more reliable?

[v0.4.14] GetCurrentProcess mismatch between golang.org/x/sys/windows

When using latest golang.org/x/sys/windows (golang.org/x/sys - version: 2aa67d56cdd77167a5fb51358e8c3ce04bad68bb) we are hitting following mismatch on GetCurrentProcess().

• vendor/github.com/Microsoft/go-winio
  vendor/github.com/Microsoft/go-winio/privilege.go:129:7: assignment mismatch: 2 variables but "golang.org/x/sys/windows".GetCurrentProcess returns 1 values

Looks like the new def in golang.org/x/sys/windows is returns only 1 argument (no error)

import: golang.org/x/sys/windows
Old:
syscall_windows.go://sys GetCurrentProcess() (pseudoHandle Handle, err error)

New:
syscall_windows.go://sys GetCurrentProcess() (pseudoHandle Handle)

--
Thanks in advance for taking a look at this issue. Please let me know if this is not the right place for this issue.

Close on named pipe server does not allow re-listening on the same address

Hi,

I want to reuse the same name for a pipe created with winio.ListenPipe.

I close the original one, but when creating a new one with the same pipe address, I get an "Access is denied" error.

How can i completely close the original pipe server so that the name can be reused?

open \\.\pipe\testNamedPipe: The parameter is incorrect

I was trying to create a named pipe in the windows 2003 server or XP, this gives me following error

C:\Documents and Settings\Administrator\Desktop\Lokesh>namedPipe.exe \.\pipe\testNamedPipe
Error in Pipe creation open \.\pipe\testNamedPipe: The parameter is incorrect.panic: runtime error: invalid memory addr
ess or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x14 pc=0x49a601]

goroutine 1 [running]:
main.main()
/home/lokesh/JunoSource/src/github.com/ContinuumLLC/platform-common-lib/src/testApps/namedPipe/main.go:20 +0x101

C:\Documents and Settings\Administrator\Desktop\Lokesh>

Code

func main() {
	l, err := CreatePipe(os.Args[1], &namedpipes.PipeConfig{
		InputBufferSize:  2000,
		OutputBufferSize: 2000,
	})
	if err != nil {
		fmt.Printf("Error in Pipe creation %+v", err)
	}
	for {
		conn, err := l.Accept()
		if err != nil {
			fmt.Printf("LRP: %s, Pipe accept failed: %+v", os.Args[1], err)
			continue
		}
		fmt.Printf("Connection %+v", conn)
	}
}

func CreatePipe(pipeName string, config *PipeConfig) (net.Listener, error) {
	var cfg = &winio.PipeConfig{}
	if config != nil {
		(*cfg).InputBufferSize = (*config).InputBufferSize
		(*cfg).MessageMode = (*config).MessageMode
		(*cfg).OutputBufferSize = (*config).OutputBufferSize
		(*cfg).SecurityDescriptor = (*config).SecurityDescriptor
	}
	return winio.ListenPipe(pipeName, cfg)
}

Pipe tests fail when running in Windows 7

I was testing mangos in Windows 7 and I notice something is off with IPC pubsub. After debugging for a while I realize some tests for go-winio/pipe don't pass, they just hang infinitely.

E.g.

go test ./. -run TestEchoWithMessaging

Environment

• OS: Windows 7 Professional x64
• Go: 1.12.7 windows/amd64

Consider: Committing vendor folder

Consider commiting the vendor folder

Integrate with the Go runtime’s event loop

This is a feature request for integration with the Go runtime’s event loop. This would mean that operations appeared to block, but in fact did not, just like with the rest of Go.

DialPipe doesn't retry on ERROR_FILE_NOT_FOUND

I am seeing an error where DialPipe is returning ERROR_FILE_NOT_FOUND (2). If I retry a few times the function succeeds. I see DialPipe only retries if it received ERROR_PIPE_BUSY. However I took a look at the npipe library and it seems like it retries on either ERROR_PIPE_BUSY or ERROR_FILE_NOT_FOUND.

I have not tested npipe but I am using a work around of retrying DialPipe every 10ms if it returns this errno which seems to resolve my issue. I think DialPipe should do this internally the same way it handles ERROR_PIPE_BUSY, retrying until it exceeds the overall timeout. Although I am not familiar with windows APIs and it seems counter intuitive that "file not found" would be a temporary error.

This library doesn't build on Windows 386

Trying to use this on docker's client, our build fails on win 386:

18:27:44 Building: bundles/1.10.0-dev/cross/windows/386/docker-1.10.0-dev.exe
18:27:54 # github.com/Microsoft/go-winio
18:27:54 vendor/src/github.com/Microsoft/go-winio/sd.go:68: type [1073741824]uint16 too large

/cc @jhowardmsft, @jstarks

go-winio module problem

I can't install go-winio module.
go version 1.12..7 on Windows, without go-winio on disc.
I create new project and execute:
go mod init test
go get github.com/microsoft/go-winio

go: github.com/microsoft/[email protected]: parsing go.mod: unexpected module path "github.com/Microsoft/go-winio"
go: error loading module requirements

Possible data race on pipe connection

The race detector caught the following data race:

==================
WARNING: DATA RACE
Read at 0x00c0420042c0 by goroutine 354:
  github.com/Microsoft/go-winio.(*win32PipeListener).listenerRoutine.func1()
      c:/gopath/src/github.com/Microsoft/go-winio/pipe.go:269 +0x42
Previous write at 0x00c0420042c0 by goroutine 311:
  github.com/Microsoft/go-winio.(*win32PipeListener).listenerRoutine()
      c:/gopath/src/github.com/Microsoft/go-winio/pipe.go:280 +0x4e7
Goroutine 354 (running) created at:
  github.com/Microsoft/go-winio.(*win32PipeListener).listenerRoutine()
      c:/gopath/src/github.com/Microsoft/go-winio/pipe.go:268 +0x2ee
Goroutine 311 (running) created at:
  github.com/Microsoft/go-winio.ListenPipe()
      c:/gopath/src/github.com/Microsoft/go-winio/pipe.go:354 +0x3c5
  github.com/fsouza/go-dockerclient.newNativeServer()
      c:/gopath/src/github.com/fsouza/go-dockerclient/client_windows_test.go:37 +0x256
  github.com/fsouza/go-dockerclient.TestPingErrorWithNativeClient()
      c:/gopath/src/github.com/fsouza/go-dockerclient/client_test.go:462 +0x73
  testing.tRunner()
      C:/go/src/testing/testing.go:747 +0x173
==================
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0xd9d0bb]

This is the build failure: https://ci.appveyor.com/project/fsouza/go-dockerclient/build/job/xeqy446run4bfeu8

I can see two possible fixes:

1. Don't set p = nil (why is that needed?)
2. Pass a copy of p to the goroutine that invokes connectPipe

CancelIO as timeout might give dataloss

https://github.com/Microsoft/go-winio/blob/97e4973ce50b2ff5f09635a57e2b88a037aae829/file.go#L186:L201

It might just be me, but if this is used with COM ports, wouldn't you lose the data when a timeout occurs ?

I've not managed to have a ReadFile op canceled on a COM port without loosing all data already received.

Forcing of Windows timer resolution to 1ms no longer necessary, degrades performance under go1.9

It is my belief that changes introduced by #12 and #13 should be reverted. Reasons for doing so:

• go-winio forces the Windows timer to 1ms resolution, which has a significant impact on battery life, as well as causes noticeable CPU utilization for virtual machines.
• This forcing of the Windows timer to 1ms resolution was done to work around an issue pertaining only to Go 1.6 (#12 and #13), however go-winio no longer builds on Go 1.6.
• go-winio's forcing of the high-resolution timer defeats improvements to Windows timer handling introduced in Go 1.9.

Background:

• Go 1.6 removed a call that set the Windows timer to 1ms resolution (timeBeginPeriod(1)). https://golang.org/doc/go1.6 :

On Windows, Go programs in Go 1.5 and earlier forced the global Windows timer resolution to 1ms at startup by calling timeBeginPeriod(1). Go no longer needs this for good scheduler performance, and changing the global timer resolution caused problems on some systems, so the call has been removed.

• This negatively affected performance of go-winio, so #12 and #13 were introduced as a workaround to set the timer resolution to 1ms.
• Go 1.7 returned to the behavior of Go 1.5, and once again set the timer to 1ms resolution. https://golang.org/doc/go1.7 :

On Windows, Go programs in Go 1.5 and earlier forced the global Windows timer resolution to 1ms at startup by calling timeBeginPeriod(1). Changing the global timer resolution caused problems on some systems, and testing suggested that the call was not needed for good scheduler performance, so Go 1.6 removed the call. Go 1.7 brings the call back: under some workloads the call is still needed for good scheduler performance.

• Go 1.9 further refined the behavior https://golang.org/doc/go1.9:

On Windows, Go no longer forces the system timer to run at high resolution when the program is idle. This should reduce the impact of Go programs on battery life.

• By always forcing the timer to 1ms, go-winio effectively defeats the improvements in timer handling made with go1.9. This leads to a significant impact on battery life, as well as VM performance.

I have found that, under go1.9, commenting out the line that sets the Windows timer to 1ms (https://github.com/Microsoft/go-winio/blob/master/file.go#L157) reduces VM host CPU utilization by ~25%, while having no detectable impact on named-pipe performance.

Note: I would contribute a pull request, myself, but I am currently unable to sign the CLA.

Mentioning @darrenstahlmsft @jstarks as they were involved with the original PR's #12 and #13.

(*win32File).Read() does not use file position

The Offset/OffsetHigh fields of syscall.Overlapped are never set, so the same bytes are returned from (*win32File).Read() over and over again.

Windows Server 2003 listen named pipe error

lis, err := winio.ListenPipe("\\.\pipe\example", nil) in Windows Server 2003
returns err: "open \.\pipe\example: The parameter is incorrect."
What the hack?

好期待有 nodejs winio

好期待有 nodejs winio，
来封装调用 winio.dll

Add build constraints for non-Windows platforms

winio should not try to compile on non-Windows platforms.

This package is a dependency for many other packages and programs that are intended to build on non-Windows systems. Because this package does not include any build constraints delcaring which OS it is intended to build on, non-Windows users will receive syscall errors, since the syscall packages are OS specific.

There are workarounds, but it would be highly preferred for this package to provide build constraints.

moby/swarmkit#1067
https://forums.docker.com/t/problem-while-getting-swarmkit-source/15168

Make a new release (and keep making them)

Last release was on Feb 21 and there's been a lot of work since then.

Unfortunately, the behavior of go too,l when used with modules, is that it considers latest release as the latest version so people using go-winio are getting a pretty old version.

Please make a new release (and keep making them with some regularity).

Re-evaluate "cgo" dependency?

As of 23ad5a714ce953959ea79723df23a913ff886ddd, the go-winio package depends on cgo. Some projects, mine included, try to avoid cgo dependencies because of the can of worms it opens, including adding complexity to cross-compiling and, in my case, the addition of the gcc toolchain to builder systems.

If cgo is required, so be it, but I figured I'd at least raise the issue for discussion. Is there any way to achieve a similar end without it? Thanks!

What's the rationale behind this mod17 function?

Hi there!

Reddit is discussing your interesting mod17() function. Discussion here:
https://redd.it/5nksq5

What's the rationale behind this "interesting" implementation?

Cheers,
Lukas

Crash in SetFileCompletionNotificationModes

According to Microsoft's own documentation, there is a known bug where SetFileCompletionNotificationModes crashes on some systems (see https://support.microsoft.com/kb/2568167 for details)
PR: #101

Windows Kubelet crashes with the below stacktrace

goroutine 26 [running]:
runtime.throw(0x33d1a16, 0x2a)
/home/madhanm/go/src/runtime/panic.go:596 +0x9c fp=0xc042425e08 sp=0xc042425de8
runtime.sigpanic()
/home/madhanm/go/src/runtime/signal_windows.go:155 +0x18b fp=0xc042425e38 sp=0xc042425e08
runtime.lock(0xc05)
/home/madhanm/go/src/runtime/lock_sema.go:43 +0x65 fp=0xc042425e70 sp=0xc042425e38
runtime.chansend(0x2c24f20, 0xbad, 0xc042425fb0, 0xc042425f01, 0x27e4daa, 0xc042425f9c)
/home/madhanm/go/src/runtime/chan.go:176 +0xba fp=0xc042425f20 sp=0xc042425e70
runtime.chansend1(0x2c24f20, 0xbad, 0xc042425fb0)
/home/madhanm/go/src/runtime/chan.go:113 +0x4d fp=0xc042425f60 sp=0xc042425f20
k8s.io/kubernetes/vendor/github.com/Microsoft/go-winio.ioCompletionProcessor(0x304)
/home/madhanm/repo/gopath/src/k8s.io/k8swin166/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/Microsoft/go-winio/file.go:131 +0xea fp=0xc042425fd8 sp=0xc042425f60
runtime.goexit()
/home/madhanm/go/src/runtime/asm_amd64.s:2197 +0x1 fp=0xc042425fe0 sp=0xc042425fd8
created by k8s.io/kubernetes/vendor/github.com/Microsoft/go-winio.initIo
/home/madhanm/repo/gopath/src/k8s.io/k8swin166/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/Microsoft/go-winio/file.go:55 +0x87

Immediately open and then close a client handle - why?

Hi,

could someone explain why this is there?

https://github.com/Microsoft/go-winio/blob/master/pipe.go#L337-L339

	// Immediately open and then close a client handle so that the named pipe is
	// created but not currently accepting connections.
	h2, err := createFile(path, 0, 0, nil, syscall.OPEN_EXISTING, cSECURITY_SQOS_PRESENT|cSECURITY_ANONYMOUS, 0)