OOM - Segmentation fault (core dumped) about turbopilot

[ravenscroftj]

ok thanks! How much memory is available on the system? Is it possible that it is just running out of RAM?

Also how long is the file that you're trying to autocomplete? Is it hundreds of lines long? Does the system work on a short or new file? There might be an issue with how much context is being fed through via vscode-fauxpilot

[juzkev]

I believe it is highly unlikely that it was running out of memory. The system ahs 24GB ram and the test was simply ran on single line python file with def hello_world():

[quanha-0878]

i'm using MODEL="/models/santacoder-q4_0.bin" and have same issue

[2023-08-21 10:01:54.792] [info] Initializing Starcoder/Wizardcoder type model for 'starcoder' model type
[2023-08-21 10:01:54.792] [info] Attempt to load model from starcoder
load_model: loading model from '/models/santacoder-q4_0.bin'
load_model: n_vocab = 49280
load_model: n_ctx   = 2048
load_model: n_embd  = 2048
load_model: n_head  = 16
load_model: n_layer = 24
load_model: ftype   = 2002
load_model: qntvr   = 2
load_model: ggml ctx size = 1542.88 MB
load_model: memory size =   768.00 MB, n_mem = 49152

load_model: model size  =   774.73 MB
[2023-08-21 10:01:56.353] [info] Loaded model in 1561.73ms
(2023-08-21 10:01:56) [INFO    ] Crow/1.0 server is running at http://0.0.0.0:18080 using 2 threads
(2023-08-21 10:01:56) [INFO    ] Call `app.loglevel(crow::LogLevel::Warning)` to hide Info level logs.
(2023-08-21 10:03:55) [INFO    ] Request: 192.168.0.1:53945 0x7f27b5a5f040 HTTP/1.1 POST /v1/engines/codegen/completions
Segmentation fault (core dumped)