timandy / routine Goto Github PK

想问一下可靠性如何？作者有把此工具用在线上项目吗？go1.18.1可以使用吗？

Is there an existing issue for this?

• I have searched the existing issues

Question

go1.20版本编译异常：github.com/timandy/routine/g.getg0: relocation target type.runtime.g not defined

Code of Conduct

• I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• I have searched the existing issues

Question

Excuse me, can I directly modify the golang runtime?
Because this information is available in the runtime, such as goid or more information about GMP

Code of Conduct

• I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• I have searched the existing issues

Question

Do we need to call the remove function on the thread local ?

I dont want to loose created variable in memory .

Thanks.

Code of Conduct

• I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• I have searched the existing issues

Question

threadlocalmap不用id池吗，这样remove以后id一直递增了

Code of Conduct

• I agree to follow this project's Code of Conduct

是否我的使用方法不对？

这一段代码的作用是什么，不太理解，希望可以解答一下

Is there an existing issue for this?

• I have searched the existing issues

Question

var (
	threadLocal1 = routine.NewThreadLocal[string]()
)

func main() {
	var m1 runtime.MemStats
	runtime.ReadMemStats(&m1)
	fmt.Printf("1 - %d Kb\n", m1.Alloc/1024)
        for i := 0; i < 1000; i++ {
          go func(){
             threadLocal1.Set("bigFile")
             time.Sleep(time.Second)

             threadLocal1.Get()
	     time.Sleep(time.Second)
	     
             var sub runtime.MemStats
	     runtime.ReadMemStats(&sub)
	     fmt.Printf("sub - %d Kb\n", sub.Alloc/1024)
           }
        }
        
        time.Sleep(time.Second * 5)

	var m2 runtime.MemStats
	runtime.ReadMemStats(&m2)
	fmt.Printf("2 - %d Kb\n", m2.Alloc/1024)
}

输出打印的内容使用量并没有下来。

Code of Conduct

• I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• I have searched the existing issues

Question

cgoCtxt 只在 cgo 调用时记录上下文地址, 且 cgo 调用后, cgoCtxt 数组会自动清空
相比 labels 字段, cgoCtxt 的使用频率更低, 发生冲突的概率也会更低

下面这段代码是我借用 g.cgoCtxt 来存储 GoroutineKiller 来做协程的超时退出:

type GoroutineKiller struct {
	deadlineStackSize uintptr
	deadlineTime      time.Time
	done              bool
}

func SetKiller(timeout time.Duration, maxStack uintptr) (deferdo func(fn func(userPanicked any, killed Killed))) {
	var killer = new(GoroutineKiller)
	stack_hi, sp, cgoCtxt := runtime.GetG()
	if cgoCtxt.Len != 0 { // 当前协程 cgoCtxt 已被使用

	} else {
		cgoCtxt.Data = unsafe.Pointer(killer)
		killer.deadlineStackSize = maxStack + (stack_hi - sp /*已占用栈大小*/)
		killer.deadlineTime = time.Now().Add(timeout)
	}
	return func(fn func(panicked any, killed Killed)) {
		_, _, cgoCtxt := runtime.GetG()
		if cgoCtxt.Len == 0 {
			cgoCtxt.Data = nil // goexit0 不会清理这个字段, 得主动清理
		}
		var panicked any
		var killed_ Killed
		if e := recover(); e != nil {
			if k, ok := e.(Killed); ok {
				killed_ = k
			} else {
				panicked = e
			}
		}
		fn(panicked, killed_)
	}
}

var time_now time.Time
var checker_count int
var checker_close = true

func init() {
	time_now = time.Now()
	go func() {
		for range time.NewTicker(1 * time.Microsecond).C {
			time_now = time.Now()
			if checker_count == 100-1 { // 抽检 1%
				checker_count++
				checker_close = false
			} else if checker_count == 100 {
				checker_count = 0
				checker_close = true
			} else {
				checker_count++
			}
		}
	}()
}

type Killed int

const KilledTimeout Killed = 1234
const KilledStackTooLarge Killed = 5678

func CheckKiller() {
	if checker_close {
		return
	}
	stack_hi, sp, cgoCtxt := runtime.GetG()
	if cgoCtxt.Data == nil || cgoCtxt.Len != 0 {
		return
	}
	var killer = (*GoroutineKiller)(cgoCtxt.Data)
	if killer.done {
		return
	}
	if time_now.After(killer.deadlineTime) {
		killer.done = true
		panic(KilledTimeout)
	}
	if size := stack_hi - sp; size > killer.deadlineStackSize {
		killer.done = true
		panic(KilledStackTooLarge)
	}
}

Code of Conduct

• I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• I have searched the existing issues

Question

thread 结构体的数据存在 g.labels 里

1. golang 的设计中，哪些golang自带的基础组件会占用到 g.labels 成员？
2. 怎么检测出 g.labels 成员被哪个模块占用？

Code of Conduct

• I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• I have searched the existing issues

Question

The example "Use ThreadLocal" does not work, it reports "invalid operation: cannot index routine.NewThreadLocal (value of type func() routine.ThreadLocal)", "invalid operation: cannot index routine.NewInheritableThreadLocal (value of type func() routine.ThreadLocal)
"

golang version 1.21

Code of Conduct

• I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• I have searched the existing issues

Question

As Title, When will the Golang 1.19 compatible version be released ?

Code of Conduct

• I agree to follow this project's Code of Conduct

func TestPProf(t *testing.T) {
	const concurrency = 100
	const loopTimes = 10000
	tls := NewThreadLocal()
	tls.Set("你好")
	wg := &sync.WaitGroup{}
	wg.Add(concurrency)
	for i := 0; i < concurrency; i++ {
		tmp := i
		go func() {
			for j := 0; j < loopTimes; j++ {
				time.Sleep(10 * time.Millisecond)
				tls.Set(tmp)
				assert.Equal(t, tmp, tls.Get())
				pprof.Do(context.Background(), pprof.Labels("key", "value"), func(ctx context.Context) {
					tls.Set("hi")
					label, find := pprof.Label(ctx, "key")
					assert.True(t, find)
					assert.Equal(t, "value", label)
					//
					//assert.Nil(t, currentThread(false))
					//assert.Nil(t, tls.Get())
					//tls.Set("hi")
					assert.Equal(t, "hi", tls.Get())
					//
					label2, find2 := pprof.Label(ctx, "key")
					assert.True(t, find2)
					assert.Equal(t, "value", label2)
				})
				assert.Nil(t, tls.Get())
			}
			wg.Done()
		}()
	}
	assert.Nil(t, pprof.StartCPUProfile(&bytes.Buffer{}))
	wg.Wait()
	pprof.StopCPUProfile()
	assert.Equal(t, "你好", tls.Get())
}

使用上述测试代码，发现偶发如下报错

[root@ routine]# go test -run TestPProf
unexpected fault address 0x5f87cb
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0x5f87cb pc=0x598424]

goroutine 27 [running]:
runtime.throw({0x5f8c17?, 0xc00017f828?})
        /usr/local/go/src/runtime/panic.go:992 +0x71 fp=0xc000328d58 sp=0xc000328d28 pc=0x436dd1
runtime.sigpanic()
        /usr/local/go/src/runtime/signal_unix.go:825 +0x305 fp=0xc000328da8 sp=0xc000328d58 pc=0x44cdc5
github.com/timandy/routine.(*threadLocalMap).set(...)
        /mnt/hgfs/go/routine/thread_local_map.go:24
github.com/timandy/routine.(*threadLocal).Set(0xc00005d400, {0x5bf720?, 0x6400b0})
        /mnt/hgfs/go/routine/thread_local.go:36 +0x64 fp=0xc000328de0 sp=0xc000328da8 pc=0x598424
github.com/timandy/routine.TestPProf.func1.1({0x641c40, 0xc000351ce0})
        /mnt/hgfs/go/routine/thread_test.go:34 +0x69 fp=0xc000328e70 sp=0xc000328de0 pc=0x5a98e9
runtime/pprof.Do({0x641bd0?, 0xc0000160c0?}, {{0xc00032ea40?, 0x742a98?, 0x5bee20?}}, 0xc000328fb0)
        /usr/local/go/src/runtime/pprof/runtime.go:40 +0xa3 fp=0xc000328ee0 sp=0xc000328e70 pc=0x5092e3
github.com/timandy/routine.TestPProf.func1()
        /mnt/hgfs/go/routine/thread_test.go:33 +0xe7 fp=0xc000328fe0 sp=0xc000328ee0 pc=0x5a95a7
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc000328fe8 sp=0xc000328fe0 pc=0x4681e1
created by github.com/timandy/routine.TestPProf
        /mnt/hgfs/go/routine/thread_test.go:28 +0x8e

在windows平台不会发生

go1.18.1

目前可以猜测的是，thread经过gc后，是没有重置数据的

我提了一个pr，你可以看看这样处理是否可以解决

Is there an existing issue for this?

• I have searched the existing issues

Does this issue reproduce with the latest release?

• I have upgrade to the latest version

Steps To Reproduce

Library is not working as expected on 1.21.
Tests from this library are failing:

go test ./...
*** GOOS: darwin ***
*** GOARCH: arm64 ***
#numField: 50
#offsetGoid: 152
#offsetPaniconfault: 181
#offsetGopc: 280
#offsetLabels: 344
--- FAIL: TestObject (0.00s)
thread_local_map_test.go:13:
Error Trace:	./thread_local_map_test.go:13
Error:      	Expected and actual point to the same object: 0x10503ef00 &routine.object{}
Test:       	TestObject
FAIL
FAIL	github.com/timandy/routine	3.298s
ok  	github.com/timandy/routine/g	(cached)
FAIL 

Also output of tests on Linux, AMD64:

go test ./...


--- FAIL: TestFutureTask_Routine_Complete (0.87s)
panic: RuntimeError: Task execution timeout after 100ms.
   at github.com/timandy/routine.(*futureTask).timeout() in /home/dovechkin/workspace/routine/future_task.go:116
   at github.com/timandy/routine.(*futureTask).GetWithTimeout() in /home/dovechkin/workspace/routine/future_task.go:92
   at github.com/timandy/routine.TestFutureTask_Routine_Complete() in /home/dovechkin/workspace/routine/future_task_test.go:497
   at testing.tRunner() in /usr/local/go/src/testing/testing.go:1595
   --- End of error stack trace ---
   created by testing.(*T).Run() in /usr/local/go/src/testing/testing.go:1648 [recovered]
        panic: RuntimeError: Task execution timeout after 100ms.
   at github.com/timandy/routine.(*futureTask).timeout() in /home/dovechkin/workspace/routine/future_task.go:116
   at github.com/timandy/routine.(*futureTask).GetWithTimeout() in /home/dovechkin/workspace/routine/future_task.go:92
   at github.com/timandy/routine.TestFutureTask_Routine_Complete() in /home/dovechkin/workspace/routine/future_task_test.go:497
   at testing.tRunner() in /usr/local/go/src/testing/testing.go:1595
   --- End of error stack trace ---
   created by testing.(*T).Run() in /usr/local/go/src/testing/testing.go:1648

goroutine 4349 [running]:
testing.tRunner.func1.2({0x5e8b40, 0xc000100190})
        /usr/local/go/src/testing/testing.go:1545 +0x238
testing.tRunner.func1()
        /usr/local/go/src/testing/testing.go:1548 +0x397
panic({0x5e8b40?, 0xc000100190?})
        /usr/local/go/src/runtime/panic.go:914 +0x21f
github.com/timandy/routine.(*futureTask).GetWithTimeout(0xc000a26200, 0x4d5000?)
        /home/dovechkin/workspace/routine/future_task.go:97 +0x1f5
github.com/timandy/routine.TestFutureTask_Routine_Complete(0xc000a0eb60?)
        /home/dovechkin/workspace/routine/future_task_test.go:497 +0x84
testing.tRunner(0xc000a0ed00, 0x635ab8)
        /usr/local/go/src/testing/testing.go:1595 +0xff
created by testing.(*T).Run in goroutine 1
        /usr/local/go/src/testing/testing.go:1648 +0x3ad
FAIL    github.com/timandy/routine      25.452s
ok      github.com/timandy/routine/g    1.737s
FAIL

Expected Behavior

Library tests should pass

Current Behavior

Library tests are failing

Environment

go 1.21

Code of Conduct

• I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• I have searched the existing issues

Question

Go 里面并没有直接管控 “Thread” 的能力，都是 GoRoutine，所以，这个 “Local” 其实是对 GoRoutine 的。
虽然 “ThreadLocal” 能很好对接从 Java 过来的开发人员，但实际上这个命名并不是很妥当。

Code of Conduct

• I agree to follow this project's Code of Conduct

如何使用泛型版本

1. 编辑你项目的 go.mod 文件，把 go 版本修改为 1.18。
2. 复制 feature/generic 分支最新提交的 SHA 值。
3. 到你项目下执行 go get github.com/timandy/routine@SHA，替换 SHA 为复制的值。

How to use the generic version

1. Edit your the project's go.mod file and change the go version to 1.18.
2. Copy the SHA value of the latest commit of the feature/generic branch.
3. Go to your project and execute go get github.com/timandy/routine@SHA, replace SHA with the copied value.