Comments (3)
mudler
commented on May 22, 2024
that should be fixed in master now, thanks for the issue!
from localai.
mudler
commented on May 22, 2024
Closing as this is fixed in the latest release. For concurrent calls there will be an update soon using latest llama.cpp code which should allow simultaneous requests as well.
from localai.
chengjia604
commented on May 22, 2024
每当我在另一个 api 请求完成之前发送一个 api 请求时，我都会收到以下错误：
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8b1421] runtime stack: runtime.throw({0x9c56fb?, 0x4?}) /usr/local/go/src/runtime/panic.go:1047 +0x5d fp=0x7f33477e41d0 sp=0x7f33477e41a0 pc=0x445bfd runtime.sigpanic() /usr/local/go/src/runtime/signal_unix.go:821 +0x3e9 fp=0x7f33477e4230 sp=0x7f33477e41d0 pc=0x45c049 goroutine 34 [syscall]: runtime.cgocall(0x8a3420, 0xc000169888) /usr/local/go/src/runtime/cgocall.go:157 +0x5c fp=0xc000169860 sp=0xc000169828 pc=0x414f1c github.com/go-skynet/llama/go._Cfunc_llama_predict(0x7f3334000c30, 0x15e85f0, 0xc000318080) _cgo_gotypes.go:154 +0x4c fp=0xc000169888 sp=0xc000169860 pc=0x511f0c github.com/go-skynet/llama/go.(*LLama).Predict.func1(0x7f3334000b60?, 0x8ffffffff?, {0xc000318080, 0x3f0000003f666666?, 0x403f800000?}) /go/pkg/mod/github.com/go-skynet/[email protected]/go/llama.go:39 +0x7e fp=0xc0001698c8 sp=0xc000169888 pc=0x5127de github.com/go-skynet/llama/go.(*LLama).Predict(0x9a7f62?, {0xc00033c000, 0xc5}, {0xc000169af8, 0x5, 0x1?}) /go/pkg/mod/github.com/go-skynet/[email protected]/go/llama.go:39 +0x285 fp=0xc000169a20 sp=0xc0001698c8 pc=0x512545 main.api.func1(0x939a80?) /build/api.go:55 +0x359 fp=0xc000169b30 sp=0xc000169a20 pc=0x89fe99 github.com/gofiber/fiber/v2.(*App).next(0xc000132900, 0xc00033a000) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:134 +0x1b6 fp=0xc000169bd8 sp=0xc000169b30 pc=0x81d996 github.com/gofiber/fiber/v2.(*App).handler(0xc000132900, 0x4d8f37?) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:160 +0x87 fp=0xc000169c38 sp=0xc000169bd8 pc=0x81dbc7 github.com/gofiber/fiber/v2.(*App).handler-fm(0xc00032e000?) <autogenerated>:1 +0x2c fp=0xc000169c58 sp=0xc000169c38 pc=0x822d4c github.com/valyala/fasthttp.(*Server).serveConn(0xc0002a2000, {0xa74040?, 0xc000306018}) /go/pkg/mod/github.com/valyala/[email protected]/server.go:2372 +0x11d3 fp=0xc000169ec8 sp=0xc000169c58 pc=0x7815f3 github.com/valyala/fasthttp.(*Server).serveConn-fm({0xa74040?, 0xc000306018?}) <autogenerated>:1 +0x39 fp=0xc000169ef0 sp=0xc000169ec8 pc=0x790cb9 github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc0001234a0, 0xc00031a000) /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:224 +0xa9 fp=0xc000169fa0 sp=0xc000169ef0 pc=0x78cf89 github.com/valyala/fasthttp.(*workerPool).getCh.func1() /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:196 +0x38 fp=0xc000169fe0 sp=0xc000169fa0 pc=0x78ccf8 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000169fe8 sp=0xc000169fe0 pc=0x477d61 created by github.com/valyala/fasthttp.(*workerPool).getCh /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:195 +0x1b0 goroutine 1 [IO wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000265368 sp=0xc000265348 pc=0x448916 runtime.netpollblock(0x7f334c2b87f8?, 0x4145af?, 0x0?) /usr/local/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc0002653a0 sp=0xc000265368 pc=0x441317 internal/poll.runtime_pollWait(0x7f334c2c1518, 0x72) /usr/local/go/src/runtime/netpoll.go:306 +0x89 fp=0xc0002653c0 sp=0xc0002653a0 pc=0x472729 internal/poll.(*pollDesc).wait(0xc00015eb00?, 0x437d60?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc0002653e8 sp=0xc0002653c0 pc=0x4e6952 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Accept(0xc00015eb00) /usr/local/go/src/internal/poll/fd_unix.go:614 +0x2bd fp=0xc000265490 sp=0xc0002653e8 pc=0x4ec25d net.(*netFD).accept(0xc00015eb00) /usr/local/go/src/net/fd_unix.go:172 +0x35 fp=0xc000265548 sp=0xc000265490 pc=0x56cb75 net.(*TCPListener).accept(0xc000012888) /usr/local/go/src/net/tcpsock_posix.go:148 +0x25 fp=0xc000265570 sp=0xc000265548 pc=0x582c05 net.(*TCPListener).Accept(0xc000012888) /usr/local/go/src/net/tcpsock.go:297 +0x3d fp=0xc0002655a0 sp=0xc000265570 pc=0x581cfd github.com/valyala/fasthttp.acceptConn(0xc0002a2000, {0xa71790, 0xc000012888}, 0xc000265798) /go/pkg/mod/github.com/valyala/[email protected]/server.go:1931 +0x62 fp=0xc000265680 sp=0xc0002655a0 pc=0x77fa82 github.com/valyala/fasthttp.(*Server).Serve(0xc0002a2000, {0xa71790?, 0xc000012888}) /go/pkg/mod/github.com/valyala/[email protected]/server.go:1824 +0x4f4 fp=0xc0002657c8 sp=0xc000265680 pc=0x77f094 github.com/gofiber/fiber/v2.(*App).Listen(0xc000132900, {0x9a9310?, 0x4?}) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/listen.go:82 +0x110 fp=0xc000265828 sp=0xc0002657c8 pc=0x815190 main.api(0xc000014338, {0x9adfd0?, 0x7?}, 0x8) /build/api.go:76 +0xfb fp=0xc000265890 sp=0xc000265828 pc=0x89fafb main.main.func2(0xc000290160?) /build/main.go:184 +0xf2 fp=0xc000265908 sp=0xc000265890 pc=0x8a2692 github.com/urfave/cli/v2.(*Command).Run(0xc000290160, 0xc0000250c0, {0xc0001728d0, 0x3, 0x3}) /go/pkg/mod/github.com/urfave/cli/[email protected]/command.go:273 +0x9eb fp=0xc000265ba8 sp=0xc000265908 pc=0x88e1ab github.com/urfave/cli/v2.(*Command).Run(0xc000290420, 0xc000024cc0, {0xc000024080, 0x4, 0x4}) /go/pkg/mod/github.com/urfave/cli/[email protected]/command.go:266 +0xc4d fp=0xc000265e48 sp=0xc000265ba8 pc=0x88e40d github.com/urfave/cli/v2.(*App).RunContext(0xc0002401e0, {0xa71a98?, 0xc00002c040}, {0xc000024080, 0x4, 0x4}) /go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:332 +0x616 fp=0xc000265eb8 sp=0xc000265e48 pc=0x88b016 github.com/urfave/cli/v2.(*App).Run(...) /go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:309 main.main() /build/main.go:262 +0x931 fp=0xc000265f80 sp=0xc000265eb8 pc=0x8a2191 runtime.main() /usr/local/go/src/runtime/proc.go:250 +0x207 fp=0xc000265fe0 sp=0xc000265f80 pc=0x4484e7 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000265fe8 sp=0xc000265fe0 pc=0x477d61 goroutine 2 [force gc (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00004efb0 sp=0xc00004ef90 pc=0x448916 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:387 runtime.forcegchelper() /usr/local/go/src/runtime/proc.go:305 +0xb0 fp=0xc00004efe0 sp=0xc00004efb0 pc=0x448750 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00004efe8 sp=0xc00004efe0 pc=0x477d61 created by runtime.init.6 /usr/local/go/src/runtime/proc.go:293 +0x25 goroutine 3 [GC sweep wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00004f780 sp=0xc00004f760 pc=0x448916 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:387 runtime.bgsweep(0x0?) /usr/local/go/src/runtime/mgcsweep.go:278 +0x8e fp=0xc00004f7c8 sp=0xc00004f780 pc=0x434bee runtime.gcenable.func1() /usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc00004f7e0 sp=0xc00004f7c8 pc=0x429ec6 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00004f7e8 sp=0xc00004f7e0 pc=0x477d61 created by runtime.gcenable /usr/local/go/src/runtime/mgc.go:178 +0x6b goroutine 4 [GC scavenge wait]: runtime.gopark(0xc000078000?, 0xa6a010?, 0x1?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00004ff70 sp=0xc00004ff50 pc=0x448916 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:387 runtime.(*scavengerState).park(0xe03d60) /usr/local/go/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc00004ffa0 sp=0xc00004ff70 pc=0x432b33 runtime.bgscavenge(0x0?) /usr/local/go/src/runtime/mgcscavenge.go:628 +0x45 fp=0xc00004ffc8 sp=0xc00004ffa0 pc=0x433105 runtime.gcenable.func2() /usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc00004ffe0 sp=0xc00004ffc8 pc=0x429e66 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00004ffe8 sp=0xc00004ffe0 pc=0x477d61 created by runtime.gcenable /usr/local/go/src/runtime/mgc.go:179 +0xaa goroutine 5 [finalizer wait]: runtime.gopark(0x1a0?, 0xe04c80?, 0x60?, 0x78?, 0xc00004e770?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00004e628 sp=0xc00004e608 pc=0x448916 runtime.runfinq() /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc00004e7e0 sp=0xc00004e628 pc=0x428f07 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00004e7e8 sp=0xc00004e7e0 pc=0x477d61 created by runtime.createfing /usr/local/go/src/runtime/mfinal.go:163 +0x45 goroutine 6 [sleep]: runtime.gopark(0x63b6c190e8fcd?, 0xc000050788?, 0x25?, 0x82?, 0xc0001234d0?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000050758 sp=0xc000050738 pc=0x448916 time.Sleep(0x2540be400) /usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc000050798 sp=0xc000050758 pc=0x474b55 github.com/valyala/fasthttp.(*workerPool).Start.func2() /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:67 +0x56 fp=0xc0000507e0 sp=0xc000050798 pc=0x78c456 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000507e8 sp=0xc0000507e0 pc=0x477d61 created by github.com/valyala/fasthttp.(*workerPool).Start /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:59 +0xdd goroutine 7 [syscall]: runtime.cgocall(0x8a3420, 0xc00016d888) /usr/local/go/src/runtime/cgocall.go:157 +0x5c fp=0xc00016d860 sp=0xc00016d828 pc=0x414f1c github.com/go-skynet/llama/go._Cfunc_llama_predict(0x7f333c000c40, 0x15e85f0, 0xc000318000) _cgo_gotypes.go:154 +0x4c fp=0xc00016d888 sp=0xc00016d860 pc=0x511f0c github.com/go-skynet/llama/go.(*LLama).Predict.func1(0x7f333c000b60?, 0x8ffffffff?, {0xc000318000, 0x3f0000003f666666?, 0x403f800000?}) /go/pkg/mod/github.com/go-skynet/[email protected]/go/llama.go:39 +0x7e fp=0xc00016d8c8 sp=0xc00016d888 pc=0x5127de github.com/go-skynet/llama/go.(*LLama).Predict(0x9a7f62?, {0xc000314000, 0xd4}, {0xc00016daf8, 0x5, 0xe8?}) /go/pkg/mod/github.com/go-skynet/[email protected]/go/llama.go:39 +0x285 fp=0xc00016da20 sp=0xc00016d8c8 pc=0x512545 main.api.func1(0x939a80?) /build/api.go:55 +0x359 fp=0xc00016db30 sp=0xc00016da20 pc=0x89fe99 github.com/gofiber/fiber/v2.(*App).next(0xc000132900, 0xc00012e840) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:134 +0x1b6 fp=0xc00016dbd8 sp=0xc00016db30 pc=0x81d996 github.com/gofiber/fiber/v2.(*App).handler(0xc000132900, 0x4d8f37?) /go/pkg/mod/github.com/gofiber/fiber/[email protected]/router.go:160 +0x87 fp=0xc00016dc38 sp=0xc00016dbd8 pc=0x81dbc7 github.com/gofiber/fiber/v2.(*App).handler-fm(0xc000179800?) <autogenerated>:1 +0x2c fp=0xc00016dc58 sp=0xc00016dc38 pc=0x822d4c github.com/valyala/fasthttp.(*Server).serveConn(0xc0002a2000, {0xa74040?, 0xc000306010}) /go/pkg/mod/github.com/valyala/[email protected]/server.go:2372 +0x11d3 fp=0xc00016dec8 sp=0xc00016dc58 pc=0x7815f3 github.com/valyala/fasthttp.(*Server).serveConn-fm({0xa74040?, 0xc000306010?}) <autogenerated>:1 +0x39 fp=0xc00016def0 sp=0xc00016dec8 pc=0x790cb9 github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc0001234a0, 0xc00006f0a0) /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:224 +0xa9 fp=0xc00016dfa0 sp=0xc00016def0 pc=0x78cf89 github.com/valyala/fasthttp.(*workerPool).getCh.func1() /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:196 +0x38 fp=0xc00016dfe0 sp=0xc00016dfa0 pc=0x78ccf8 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00016dfe8 sp=0xc00016dfe0 pc=0x477d61 created by github.com/valyala/fasthttp.(*workerPool).getCh /go/pkg/mod/github.com/valyala/[email protected]/workerpool.go:195 +0x1b0 goroutine 18 [sleep]: runtime.gopark(0x63b6e3cc0ee1e?, 0x9159a0?, 0x68?, 0x44?, 0x1?) /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00005cf88 sp=0xc00005cf68 pc=0x448916 time.Sleep(0x3b9aca00) /usr/local/go/src/runtime/time.go:195 +0x135 fp=0xc00005cfc8 sp=0xc00005cf88 pc=0x474b55 github.com/valyala/fasthttp.updateServerDate.func1() /go/pkg/mod/github.com/valyala/[email protected]/header.go:2246 +0x1e fp=0xc00005cfe0 sp=0xc00005cfc8 pc=0x78d3de runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00005cfe8 sp=0xc00005cfe0 pc=0x477d61 created by github.com/valyala/fasthttp.updateServerDate /go/pkg/mod/github.com/valyala/[email protected]/header.go:2244 +0x25```
I would like to ask you whether you can request the API concurrently now. I have not been able to do so here. I want to confirm whether concurrent calls can be used.
from localai.
Related Issues (20)
- limit models to api keys HOT 1
- After several exchanges, llama-cpp hangs up and no longer returns any response. HOT 7
- Very difficult to use, there are a lot of bugs HOT 2
- llama-cpp CPU 1500%，Very slow...What to do? HOT 1
- tts coqui xtts_v2 not working without speaker_idx HOT 2
- ERROR [api builder 3/12] COPY .git . HOT 2
- LOCALAI_LOG_LEVEL seemingly does nothing HOT 3
- I would like the --tensor-parallel-size setting of the vllm backend to be added to yaml. HOT 1
- embeddings curl example on Mac "could not load model: rpc error" HOT 1
- lamma3 8B and 70B gguf support? Can support function calling? HOT 2
- transport: Error while dialing: dial tcp 127.0.0.1:40825: connect: connection refused HOT 4
- [SSE] The chat completion chunk object from the /chat/completions API endpoint contains a single character instead of a complete token HOT 2
- Fine grained authentication system HOT 4
- All-in-One (AIO) Image - Models are not loading due to "grpc service not ready" HOT 3
- txt2vid requires opencv + memory attn. HOT 6
- [Feature Request] - GPT-Researcher HOT 2
- missing nvidia-smi disable gpu support HOT 5
- [question,documentation]How to load API keys list from json file? HOT 2
- Embeddings with langchain fail: "rpc error: code = Unavailable desc = error reading from server: EOF" HOT 2
- AIO - memory issue - embedding HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from localai.