cannot convert file.Fd() (value of type uintptr) to type _Ctype_HANDLE about go-yara

[hillu]

Thanks. You are right about the above question and after I updated to go-yara v4.2.4 it works fine.

Great, I am closing this issue then. For the c-shared problem, please create a new issue.

[hillu]

_yr_compiler_add_fd is supposed to take an int as its second argument, not a HANDLE. It looks like the wrong _yr_compiler_add_fd variant was selected (compat.h). Perhaps your C compiler/preprocessor does not define _WIN32?

What C compiler are you using?

[czqgithub2017]

My compiler is "i686-w64-mingw32-gcc“，and the gcc version information:

i686-w64-mingw32-gcc (i686-win32-dwarf-rev0, Built by MinGW-W64 project) 8.1.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE._

build script:

set GOOS=windows
set GOARCH=386
set CGO_ENABLED=1
set CC=i686-w64-mingw32-gcc
set PKG_CONFIG_PATH=d:\setup_yara\4.2.0\i686\lib\pkgconfig
go build -tags "yara_static dev" -ldflags "-extldflags "-static" -X main.appVer=2022.05.01" -o ../bin/scanner.exe

[czqgithub2017]

My compiler is "i686-w64-mingw32-gcc“，and the gcc version information:

i686-w64-mingw32-gcc (i686-win32-dwarf-rev0, Built by MinGW-W64 project) 8.1.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE._

build script:

set GOOS=windows
set GOARCH=386
set CGO_ENABLED=1
set CC=i686-w64-mingw32-gcc
set PKG_CONFIG_PATH=d:\setup_yara\4.2.0\i686\lib\pkgconfig
go build -tags "yara_static dev" -ldflags "-extldflags "-static" -X main.appVer=2022.05.01" -o ../bin/scanner.exe

[hillu]

For diagnosis, please try the following things:

1. Create a fresh checkout of go-yara and verify that running go build in that directory fails with the same error message.
2. If it fails, try editing compat.h: Remove the #ifdef _WIN32 and #else … #endif branches around the three function definitions. Does running go build work now?
3. Create an empty file empty.h, run gcc -E -dM empty.h, and paste the entire output.

Thanks!

[hillu]

@czqgithub2017 ping?

[ca110us]

For diagnosis, please try the following things:

1. Create a fresh checkout of go-yara and verify that running go build in that directory fails with the same error message.
2. If it fails, try editing compat.h: Remove the #ifdef _WIN32 and #else … #endif branches around the three function definitions. Does running go build work now?
3. Create an empty file empty.h, run gcc -E -dM empty.h, and paste the entire output.

Thanks!

Hi, I encountered the same problem, and it seems that version 4 doesn't have compat.h file, so how to fix it?

[ca110us]

And i just try the version 4.2.0:
# github.com/hillu/go-yara/v4 C:\Users\zhangboyi\go\pkg\mod\github.com\hillu\go-yara\[email protected]\compiler.go:124:71: cannot convert file.Fd() (value of type uintptr) to type _Ctype_HANDLE

[hillu]

@ca110us Please update to a newer 4.2.x-based release, e.g. by running

go get github.com/hillu/go-yara/[email protected]

[ca110us]

It seems that the second parameter of the yr_compiler_add_fd function in Windows is a Handle type. Should we convert the file descriptor to a Handle first?

[hillu]

It seems that the second parameter of the yr_compiler_add_fd function in Windows is a Handle type. Should we convert the file descriptor to a Handle first?

No, we should not let the Go compiler know that we are passing C HANDLEs (which are defined as void* but which are not real pointers) across the CGo boundary..The conversion to a HANDLE needs to happen in the C wrapper code. From the commit message of 885b44f:

win32 fix: Avoid passing HANDLE (void*) into C function

On the Go side, CGO treats void* as unsafe.Pointer. Invalid values cause
the garbage collector to panic, e.g.:

runtime: garbage collector found invalid heap pointer *(0x5b80ff14+0x4)=0xa0 s=nil

-- where 0xa0 is the value of the filehandle passed into yr_rules_scan_fd()

Can you please try and see if updating go-yara to something newer than 4.2.0 fixes your problem? Thank you.

[ca110us]

Thanks. You are right about the above question and after I updated to go-yara v4.2.4 it works fine.
But I encountered a new problem, if I compiled as a static DLL for other golang projects, it will crash when calling this DLL, even if the DLL is just a simple compilation of Yara text.

[ca110us]

fatal error: unexpected signal during runtime execution
[signal 0xc0000005 code=0x1 addr=0x0 pc=0x7ffe18cd0ac2]

goroutine 1 [running, locked to thread]:
runtime.throw({0x7ffe18d9e406?, 0xc00007d8a8?})
        C:/Program Files/Go/src/runtime/panic.go:1047 +0x65 fp=0xc00007d868 sp=0xc00007d838 pc=0x7ffe18ca63e5
runtime.sigpanic()
        C:/Program Files/Go/src/runtime/signal_windows.go:249 +0x213 fp=0xc00007d8b0 sp=0xc00007d868 pc=0x7ffe18cb7fb3
runtime.memmove()
        C:/Program Files/Go/src/runtime/memmove_amd64.s:151 +0x102 fp=0xc00007d8b8 sp=0xc00007d8b0 pc=0x7ffe18cd0ac2  
runtime.heapBitsSetType(0x8?, 0x300, 0x300?, 0x0?)
        C:/Program Files/Go/src/runtime/mbitmap.go:1381 +0x592 fp=0xc00007d9e0 sp=0xc00007d8b8 pc=0x7ffe18c84612      
runtime.mallocgc(0x300, 0x7ffe18d89780, 0x1)
        C:/Program Files/Go/src/runtime/malloc.go:1050 +0x64d fp=0xc00007da58 sp=0xc00007d9e0 pc=0x7ffe18c7d34d       
runtime.makeslice(0x0?, 0x0?, 0x0?)
        C:/Program Files/Go/src/runtime/slice.go:103 +0x52 fp=0xc00007da80 sp=0xc00007da58 pc=0x7ffe18cb82f2
sync.(*Pool).pinSlow(0x7ffe18d6e560)
        C:/Program Files/Go/src/sync/pool.go:230 +0x185 fp=0xc00007db18 sp=0xc00007da80 pc=0x7ffe18cd77e5
sync.(*Pool).pin(0x7ffe18d6e560)
        C:/Program Files/Go/src/sync/pool.go:209 +0x4e fp=0xc00007db30 sp=0xc00007db18 pc=0x7ffe18cd762e
sync.(*Pool).Get(0x7ffe18d6e560)
        C:/Program Files/Go/src/sync/pool.go:131 +0x25 fp=0xc00007db68 sp=0xc00007db30 pc=0x7ffe18cd7365
fmt.newPrinter()
        C:/Program Files/Go/src/fmt/print.go:137 +0x25 fp=0xc00007db90 sp=0xc00007db68 pc=0x7ffe18cfc205
fmt.Fprintln({0x7ffe18dbb1f8, 0x0}, {0xc00007dc18, 0x1, 0x1})
        C:/Program Files/Go/src/fmt/print.go:283 +0x36 fp=0xc00007dbe0 sp=0xc00007db90 pc=0x7ffe18cfc4d6
fmt.Println(...)
        C:/Program Files/Go/src/fmt/print.go:294
main.init_rules()
        E:/S/detector/detector.go:15 +0x56 fp=0xc00007dc38 sp=0xc00007dbe0 pc=0x7ffe18d06f56
_cgoexp_4d0f629fe01b_init_rules(0x47ea9ffd1c)
        _cgo_gotypes.go:42 +0x1c fp=0xc00007dc48 sp=0xc00007dc38 pc=0x7ffe18d0701c
runtime.cgocallbackg1(0x7ffe18d07000, 0x0?, 0x0)
        C:/Program Files/Go/src/runtime/cgocall.go:316 +0x2c2 fp=0xc00007dd18 sp=0xc00007dc48 pc=0x7ffe18c74082
runtime.cgocallbackg(0xc000038000?, 0x300000002?, 0xc000038000?)
        C:/Program Files/Go/src/runtime/cgocall.go:235 +0x105 fp=0xc00007dda8 sp=0xc00007dd18 pc=0x7ffe18c73cc5
runtime.cgocallbackg(0x7ffe18d07000, 0x47ea9ffd1c, 0x0)
        <autogenerated>:1 +0x36 fp=0xc00007ddd0 sp=0xc00007dda8 pc=0x7ffe18cd1b36
runtime: g 1: unexpected return pc for runtime.cgocallback called from 0x64bea0
stack: frame={sp:0xc00007ddd0, fp:0xc00007ddf8} stack=[0xc000076000,0xc00007e000)
0x000000c00007dcd0:  0x000000c00007dc8d  0x000000c000038000
0x000000c00007dce0:  0x0000000200000003  0x000000c000038000
0x000000c00007dcf0:  0x000000c00007dcc8  0x0000000000000000
0x000000c00007dd00:  0x00007ffe18da06b0  0x000000c00007dd98
0x000000c00007dd10:  0x00007ffe18c73cc5 <runtime.cgocallbackg+0x0000000000000105>  0x00007ffe18d07000 <_cgoexp_4d0f629fe01b_init_rules+0x0000000000000000>
0x000000c00007dd20:  0x0000000000000000  0x0000000000000000
0x000000c00007dd30:  0x00000000005f358a  0x0000000000000000
0x000000c00007dd40:  0x00007ffe18d07040  0x0000000000000000
0x000000c00007dd50:  0x000000c00007de70  0x0000000000000001
0x000000c00007dd60:  0x0000000000000000  0x0000000000000000
0x000000c00007dd70:  0x000000c00007de00  0x000000c000038000
0x000000c00007dd80:  0x0000000000733ae0  0x00007ffe18d07000 <_cgoexp_4d0f629fe01b_init_rules+0x0000000000000000>
0x000000c00007dd90:  0x00000047ea9ffd1c  0x000000c00007ddc0
0x000000c00007dda0:  0x00007ffe18cd1b36 <runtime.cgocallbackg+0x0000000000000036>  0x000000c000038000
0x000000c00007ddb0:  0x0000000300000002  0x000000c000038000
0x000000c00007ddc0:  0x00000047ea9ffbd8  0x00007ffe18ccf8f7 <runtime.cgocallback+0x00000000000000d7>
0x000000c00007ddd0: <0x00007ffe18d07000 <_cgoexp_4d0f629fe01b_init_rules+0x0000000000000000>  0x00000047ea9ffd1c
0x000000c00007dde0:  0x0000000000000000  0x000000c00007de00
0x000000c00007ddf0: !0x000000000064bea0 >0x00000000005f35eb
0x000000c00007de00:  0x000000000064faa0  0x0000000000733da0
0x000000c00007de10:  0x000000c0000046c0  0x0000000000674e20
0x000000c00007de20:  0x0000000000733ae0  0x000000c00007dea0
0x000000c00007de30:  0x000000000064af69  0x000000000064faa0
0x000000c00007de40:  0x0000000000733da0  0x000000000065b885
0x000000c00007de50:  0x010000000072c340  0x0000000000000000
0x000000c00007de60:  0x0000000000000000  0x0000000000000000
0x000000c00007de70:  0x0000000000000000  0x0000000000000000
0x000000c00007de80:  0x0000000000000000  0x0000000000000000
0x000000c00007de90:  0x0000000000733ae0  0x00000000006a1fd8
0x000000c00007dea0:  0x000000c00007dee0  0x000000000066a885
0x000000c00007deb0:  0x00007ffe18d07040  0x0000000000000000
0x000000c00007dec0:  0x0000000000000000  0x000000c000026340
0x000000c00007ded0:  0x0000000000000000  0x0000000000000000
0x000000c00007dee0:  0x000000c00007df70  0x000000000067e756
0x000000c00007def0:  0x00000000006bbda8
runtime.cgocallback(0x5f35eb, 0x64faa0, 0x733da0)
        C:/Program Files/Go/src/runtime/asm_amd64.s:994 +0xd7 fp=0xc00007ddf8 sp=0xc00007ddd0 pc=0x7ffe18ccf8f7

goroutine 1 [runnable, locked to thread]:
syscall.WSAEnumProtocols(0x1c0000c2a70?, 0x1c0000c2a78?, 0x1c0000c2a6c?)
        C:/Program Files/Go/src/syscall/zsyscall_windows.go:1277 +0xfa fp=0x1c0000c2a50 sp=0x1c0000c2a48 pc=0x7ffe18cdb27a
internal/poll.checkSetFileCompletionNotificationModes()
        C:/Program Files/Go/src/internal/poll/fd_windows.go:44 +0x79 fp=0x1c0000c7908 sp=0x1c0000c2a50 pc=0x7ffe18cf6319
internal/poll.init.0()
        C:/Program Files/Go/src/internal/poll/fd_windows.go:62 +0x94 fp=0x1c0000c7ac0 sp=0x1c0000c7908 pc=0x7ffe18cf6454
runtime.doInit(0x7ffe18d5fc00)
        C:/Program Files/Go/src/runtime/proc.go:6321 +0x12d fp=0x1c0000c7bf0 sp=0x1c0000c7ac0 pc=0x7ffe18cb55ad
runtime.doInit(0x7ffe18d60680)
        C:/Program Files/Go/src/runtime/proc.go:6298 +0x71 fp=0x1c0000c7d20 sp=0x1c0000c7bf0 pc=0x7ffe18cb54f1
runtime.doInit(0x7ffe18d5fde0)
        C:/Program Files/Go/src/runtime/proc.go:6298 +0x71 fp=0x1c0000c7e50 sp=0x1c0000c7d20 pc=0x7ffe18cb54f1
runtime.doInit(0x7ffe18d5ede0)
        C:/Program Files/Go/src/runtime/proc.go:6298 +0x71 fp=0x1c0000c7f80 sp=0x1c0000c7e50 pc=0x7ffe18cb54f1
runtime.main()
        C:/Program Files/Go/src/runtime/proc.go:233 +0x1bf fp=0x1c0000c7fe0 sp=0x1c0000c7f80 pc=0x7ffe18ca8adf
runtime.goexit()
        C:/Program Files/Go/src/runtime/asm_amd64.s:1594 +0x1 fp=0x1c0000c7fe8 sp=0x1c0000c7fe0 pc=0x7ffe18ccfb61

goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        C:/Program Files/Go/src/runtime/proc.go:363 +0xd6 fp=0x1c00003bfb0 sp=0x1c00003bf90 pc=0x7ffe18ca8eb6
runtime.goparkunlock(...)
        C:/Program Files/Go/src/runtime/proc.go:369
runtime.forcegchelper()
        C:/Program Files/Go/src/runtime/proc.go:302 +0xb1 fp=0x1c00003bfe0 sp=0x1c00003bfb0 pc=0x7ffe18ca8d51
runtime.goexit()
        C:/Program Files/Go/src/runtime/asm_amd64.s:1594 +0x1 fp=0x1c00003bfe8 sp=0x1c00003bfe0 pc=0x7ffe18ccfb61
created by runtime.init.6
        C:/Program Files/Go/src/runtime/proc.go:290 +0x25

goroutine 3 [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        C:/Program Files/Go/src/runtime/proc.go:363 +0xd6 fp=0x1c00003df90 sp=0x1c00003df70 pc=0x7ffe18ca8eb6
runtime.goparkunlock(...)
        C:/Program Files/Go/src/runtime/proc.go:369
runtime.bgsweep(0x0?)
        C:/Program Files/Go/src/runtime/mgcsweep.go:278 +0x8e fp=0x1c00003dfc8 sp=0x1c00003df90 pc=0x7ffe18c93aee
runtime.gcenable.func1()
        C:/Program Files/Go/src/runtime/mgc.go:178 +0x26 fp=0x1c00003dfe0 sp=0x1c00003dfc8 pc=0x7ffe18c88886
runtime.goexit()
        C:/Program Files/Go/src/runtime/asm_amd64.s:1594 +0x1 fp=0x1c00003dfe8 sp=0x1c00003dfe0 pc=0x7ffe18ccfb61
created by runtime.gcenable
        C:/Program Files/Go/src/runtime/mgc.go:178 +0x6b

goroutine 4 [GC scavenge wait]:
runtime.gopark(0x1c000020070?, 0x7ffe18dbac20?, 0x1?, 0x0?, 0x0?)
        C:/Program Files/Go/src/runtime/proc.go:363 +0xd6 fp=0x1c00004df70 sp=0x1c00004df50 pc=0x7ffe18ca8eb6
runtime.goparkunlock(...)
        C:/Program Files/Go/src/runtime/proc.go:369
runtime.(*scavengerState).park(0x7ffe18e37e00)
        C:/Program Files/Go/src/runtime/mgcscavenge.go:389 +0x53 fp=0x1c00004dfa0 sp=0x1c00004df70 pc=0x7ffe18c91b73
runtime.bgscavenge(0x0?)
        C:/Program Files/Go/src/runtime/mgcscavenge.go:617 +0x45 fp=0x1c00004dfc8 sp=0x1c00004dfa0 pc=0x7ffe18c92165
runtime.gcenable.func2()
        C:/Program Files/Go/src/runtime/mgc.go:179 +0x26 fp=0x1c00004dfe0 sp=0x1c00004dfc8 pc=0x7ffe18c88826
runtime.goexit()
        C:/Program Files/Go/src/runtime/asm_amd64.s:1594 +0x1 fp=0x1c00004dfe8 sp=0x1c00004dfe0 pc=0x7ffe18ccfb61
created by runtime.gcenable
        C:/Program Files/Go/src/runtime/mgc.go:179 +0xaa
exit status 2

[ca110us]

Code:

package main

import "C"
import (
	"github.com/hillu/go-yara/v4"
)

// var rules []byte
var YaraInstance *yara.Rules

//export init_rules
func init_rules() {
	YaraInstance, _ = yara.Compile(`rule test : tag1 { meta: author = "Hilko Bengen" strings: $a = "abc" fullword condition: $a }`, nil)
}

func main() {}

Compile command:

The same command as compiling an exe but with added flag -buildmode=c-shared