Coder Social home page Coder Social logo

Comments (5)

hitendra-galaxyweblinks avatar hitendra-galaxyweblinks commented on May 27, 2024 
2022-09-09T03:08:38.283+0530 [DEBUG] client.driver_mgr: plugin process exited: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver pid=312468 error="exit status 2"
2022-09-09T03:08:38.283+0530 [WARN]  client.driver_mgr: failed to reattach to plugin, starting new instance: driver=firecracker-task-driver err="singleton plugin exited"
2022-09-09T03:08:38.283+0530 [ERROR] client.alloc_runner.task_runner: failed to kill task: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy backoff=5s error="plugin is shut down"
2022-09-09T03:08:38.283+0530 [DEBUG] client.alloc_runner.task_runner: driver plugin has shutdown; attempting to recover task: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy driver=firecracker-task-driver
2022-09-09T03:08:38.283+0530 [DEBUG] client.driver_mgr: starting plugin: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver args=["/root/plugins/firecracker-task-driver"]
2022-09-09T03:08:38.283+0530 [WARN]  client.driver_mgr: received fingerprint error from driver: driver=firecracker-task-driver error="plugin is shut down"
2022-09-09T03:08:38.283+0530 [WARN]  client: dropping node event: node_event="Message: ready, Subsystem: Driver, Details: driver: firecracker-task-driver, Timestamp: 2022-09-09 03:08:36.384981362 +0530 IST m=+231.761923755"
2022-09-09T03:08:38.283+0530 [DEBUG] client.driver_mgr: plugin started: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver pid=312569
2022-09-09T03:08:38.283+0530 [DEBUG] client.driver_mgr: waiting for RPC address: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver
2022-09-09T03:08:38.300+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: plugin address: driver=firecracker-task-driver address=/tmp/plugin666233150 network=unix timestamp="2022-09-09T03:08:38.300+0530"
2022-09-09T03:08:38.300+0530 [DEBUG] client.driver_mgr: using plugin: driver=firecracker-task-driver version=2
2022-09-09T03:08:38.301+0530 [DEBUG] client.driver_mgr.stdio: received EOF, stopping recv loop: driver=firecracker-task-driver err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio"
2022-09-09T03:08:38.302+0530 [INFO]  client.driver_mgr.firecracker-task-driver: buildFingerprint(): driver=firecracker-task-driver @module=firecracker-task-driver driver.FingerPrint=healthy timestamp="2022-09-09T03:08:38.301+0530"
2022-09-09T03:08:38.302+0530 [WARN]  client: dropping node event: node_event="Message: failed to fingerprint driver, Subsystem: Driver, Details: driver: firecracker-task-driver, Timestamp: 2022-09-09 03:08:36.701565273 +0530 IST m=+232.078507666"
2022-09-09T03:08:38.316+0530 [INFO]  client.driver_mgr.firecracker-task-driver: Starting firecracker: driver=firecracker-task-driver @module=firecracker-task-driver driver_initialize_container="&{/usr/local/bin/firecracker /tmp/hello-vmlinux.bin  console=ttyS0 reboot=k panic=1 pci=off nomodules /tmp/nginx_v1.ext4  [] galaxy {   []} []    false 1  300    false false [] <nil> 0xd248c0}+" timestamp="2022-09-09T03:08:38.316+0530"

2022-09-09T03:08:38.650+0530 [DEBUG] http: request complete: method=GET path=/v1/client/allocation/de27a9ad-ea65-9ee5-79f3-dc02f224bab6/stats duration="546.022µs"
2022-09-09T03:08:38.659+0530 [DEBUG] client: state changed, updating node and re-registering
2022-09-09T03:08:38.674+0530 [INFO]  client: node registration complete
2022-09-09T03:08:38.674+0530 [DEBUG] client: evaluations triggered by node registration: num_evals=1
2022-09-09T03:08:38.674+0530 [DEBUG] worker: dequeued evaluation: worker_id=3c7c0d84-8f8d-cdec-9d23-f0b43b1fb604 eval_id=d5dc9451-1843-b63d-20fb-f53e86ddda7d type=service namespace=default job_id=Web node_id=baa9672e-99e1-31ec-02b0-7d44e7c945a5 triggered_by=node-update
2022-09-09T03:08:38.674+0530 [DEBUG] worker.service_sched: reconciled current state with desired state: eval_id=d5dc9451-1843-b63d-20fb-f53e86ddda7d job_id=Web namespace=default worker_id=3c7c0d84-8f8d-cdec-9d23-f0b43b1fb604

results=
| Total changes: (place 0) (destructive 0) (inplace 0) (stop 0) (disconnect 0) (reconnect 0)
| Desired Changes for "Nginx": (place 0) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0)

2022-09-09T03:08:38.674+0530 [DEBUG] worker.service_sched: setting eval status: eval_id=d5dc9451-1843-b63d-20fb-f53e86ddda7d job_id=Web namespace=default worker_id=3c7c0d84-8f8d-cdec-9d23-f0b43b1fb604 status=complete
2022-09-09T03:08:38.676+0530 [DEBUG] worker: updated evaluation: worker_id=3c7c0d84-8f8d-cdec-9d23-f0b43b1fb604 eval="<Eval \"d5dc9451-1843-b63d-20fb-f53e86ddda7d\" JobID: \"Web\" Namespace: \"default\">"
2022-09-09T03:08:38.676+0530 [DEBUG] worker: ack evaluation: worker_id=3c7c0d84-8f8d-cdec-9d23-f0b43b1fb604 eval_id=d5dc9451-1843-b63d-20fb-f53e86ddda7d type=service namespace=default job_id=Web node_id=baa9672e-99e1-31ec-02b0-7d44e7c945a5 triggered_by=node-update
2022-09-09T03:08:38.694+0530 [INFO]  client.driver_mgr.firecracker-task-driver: Writing to: driver=firecracker-task-driver @module=firecracker-task-driver driver_initialize_container="/tmp/Galaxy-de27a9ad-ea65-9ee5-79f3-dc02f224bab6+" timestamp="2022-09-09T03:08:38.694+0530"
2022-09-09T03:08:38.695+0530 [DEBUG] client.alloc_runner.task_runner: task successfully recovered on driver: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy driver=firecracker-task-driver
2022-09-09T03:08:38.698+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: panic: runtime error: invalid memory address or nil pointer dereference: driver=firecracker-task-driver
2022-09-09T03:08:38.698+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xca7967]: driver=firecracker-task-driver
2022-09-09T03:08:38.698+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: driver=firecracker-task-driver
2022-09-09T03:08:38.698+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: goroutine 70 [running]:: driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink.(*Handle).LinkDel(0x18a3cc0, 0x0, 0x0, 0x0, 0x0): driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink/link_linux.go:1351 +0x37: driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink.LinkDel(...): driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink/link_linux.go:1344: driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/driver.(*taskHandle).shutdown(0xc0003140f0, 0x12a05f200, 0x34, 0xc0003140f0): driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/driver/handle.go:224 +0x73: driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/driver.(*Driver).StopTask(0xc00053e640, 0xc00025a8c0, 0x34, 0x12a05f200, 0x0, 0x0, 0xc00052d500, 0x1143e60): driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/driver/driver.go:370 +0x6a: driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers.(*driverPluginServer).StopTask(0xc0005c0d20, 0x1130960, 0xc000306d20, 0xc00053ecd0, 0xc0005c0d20, 0xc000306d20, 0xc000074b30): driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers/server.go:181 +0x8f: driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers/proto._Driver_StopTask_Handler(0xf3c3a0, 0xc0005c0d20, 0x1130960, 0xc000306d20, 0xc00052d5c0, 0x0, 0x1130960, 0xc000306d20, 0xc00025a840, 0x3a): driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers/proto/driver.pb.go:4375 +0x217: driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).processUnaryRPC(0xc0004fcc00, 0x11401e0, 0xc00050a900, 0xc000556000, 0xc00020c9f0, 0x1894718, 0x0, 0x0, 0x0): driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:1024 +0x4f4: driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).handleStream(0xc0004fcc00, 0x11401e0, 0xc00050a900, 0xc000556000, 0x0): driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:1313 +0xd97: driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc000402060, 0xc0004fcc00, 0x11401e0, 0xc00050a900, 0xc000556000): driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:722 +0xbb: driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: created by github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).serveStreams.func1: driver=firecracker-task-driver
2022-09-09T03:08:38.699+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:720 +0xa1: driver=firecracker-task-driver
2022-09-09T03:08:38.703+0530 [DEBUG] client.driver_mgr: plugin process exited: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver pid=312569 error="exit status 2"
2022-09-09T03:08:38.703+0530 [ERROR] client.alloc_runner.task_runner: failed to kill task: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy backoff=5s error="plugin is shut down"
2022-09-09T03:08:38.703+0530 [DEBUG] client.alloc_runner.task_runner: driver plugin has shutdown; attempting to recover task: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy driver=firecracker-task-driver
2022-09-09T03:08:38.703+0530 [WARN]  client.driver_mgr: received fingerprint error from driver: driver=firecracker-task-driver error="plugin is shut down"
2022-09-09T03:08:38.703+0530 [WARN]  client.driver_mgr: failed to reattach to plugin, starting new instance: driver=firecracker-task-driver err="singleton plugin exited"
2022-09-09T03:08:38.704+0530 [DEBUG] client.driver_mgr: starting plugin: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver args=["/root/plugins/firecracker-task-driver"]
2022-09-09T03:08:38.703+0530 [WARN]  client: dropping node event: node_event="Message: ready, Subsystem: Driver, Details: driver: firecracker-task-driver, Timestamp: 2022-09-09 03:08:36.728176949 +0530 IST m=+232.105119340"
2022-09-09T03:08:38.704+0530 [DEBUG] client.driver_mgr: plugin started: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver pid=312666
2022-09-09T03:08:38.704+0530 [DEBUG] client.driver_mgr: waiting for RPC address: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver
2022-09-09T03:08:38.741+0530 [DEBUG] client.driver_mgr: using plugin: driver=firecracker-task-driver version=2
2022-09-09T03:08:38.741+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: plugin address: driver=firecracker-task-driver address=/tmp/plugin339428083 network=unix timestamp="2022-09-09T03:08:38.741+0530"
2022-09-09T03:08:38.742+0530 [DEBUG] client.driver_mgr.stdio: received EOF, stopping recv loop: driver=firecracker-task-driver err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio"
2022-09-09T03:08:38.742+0530 [INFO]  client.driver_mgr.firecracker-task-driver: buildFingerprint(): driver=firecracker-task-driver @module=firecracker-task-driver driver.FingerPrint=healthy timestamp="2022-09-09T03:08:38.742+0530"
2022-09-09T03:08:38.742+0530 [WARN]  client: dropping node event: node_event="Message: failed to fingerprint driver, Subsystem: Driver, Details: driver: firecracker-task-driver, Timestamp: 2022-09-09 03:08:37.021332688 +0530 IST m=+232.398275099"
2022-09-09T03:08:38.744+0530 [INFO]  client.driver_mgr.firecracker-task-driver: Starting firecracker: driver=firecracker-task-driver @module=firecracker-task-driver driver_initialize_container="&{/usr/local/bin/firecracker /tmp/hello-vmlinux.bin  console=ttyS0 reboot=k panic=1 pci=off nomodules /tmp/nginx_v1.ext4  [] galaxy {   []} []    false 1  300    false false [] <nil> 0xd248c0}+" timestamp="2022-09-09T03:08:38.744+0530"

2022-09-09T03:08:39.062+0530 [INFO]  client.driver_mgr.firecracker-task-driver: Writing to: driver=firecracker-task-driver @module=firecracker-task-driver driver_initialize_container="/tmp/Galaxy-de27a9ad-ea65-9ee5-79f3-dc02f224bab6+" timestamp="2022-09-09T03:08:39.062+0530"
2022-09-09T03:08:39.062+0530 [DEBUG] client.alloc_runner.task_runner: task successfully recovered on driver: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy driver=firecracker-task-driver
2022-09-09T03:08:39.064+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: panic: runtime error: invalid memory address or nil pointer dereference: driver=firecracker-task-driver
2022-09-09T03:08:39.064+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xca7967]: driver=firecracker-task-driver
2022-09-09T03:08:39.064+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: driver=firecracker-task-driver
2022-09-09T03:08:39.064+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: goroutine 144 [running]:: driver=firecracker-task-driver
2022-09-09T03:08:39.064+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink.(*Handle).LinkDel(0x18a3cc0, 0x0, 0x0, 0x0, 0x0): driver=firecracker-task-driver
2022-09-09T03:08:39.064+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink/link_linux.go:1351 +0x37: driver=firecracker-task-driver
2022-09-09T03:08:39.064+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink.LinkDel(...): driver=firecracker-task-driver
2022-09-09T03:08:39.064+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink/link_linux.go:1344: driver=firecracker-task-driver
2022-09-09T03:08:39.064+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/driver.(*taskHandle).shutdown(0xc0003ac0f0, 0x12a05f200, 0x34, 0xc0003ac0f0): driver=firecracker-task-driver
2022-09-09T03:08:39.064+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/driver/handle.go:224 +0x73: driver=firecracker-task-driver
2022-09-09T03:08:39.064+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/driver.(*Driver).StopTask(0xc00052a6e0, 0xc0000e4d00, 0x34, 0x12a05f200, 0x0, 0x0, 0xc00054f200, 0x1143e60): driver=firecracker-task-driver
2022-09-09T03:08:39.064+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/driver/driver.go:370 +0x6a: driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers.(*driverPluginServer).StopTask(0xc0005a6ec0, 0x1130960, 0xc00063a2d0, 0xc00052b540, 0xc0005a6ec0, 0xc00063a2d0, 0xc00007bb30): driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers/server.go:181 +0x8f: driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers/proto._Driver_StopTask_Handler(0xf3c3a0, 0xc0005a6ec0, 0x1130960, 0xc00063a2d0, 0xc00054f2c0, 0x0, 0x1130960, 0xc00063a2d0, 0xc0000e4cc0, 0x3a): driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers/proto/driver.pb.go:4375 +0x217: driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).processUnaryRPC(0xc00009b200, 0x11401e0, 0xc0004c0a80, 0xc0004ac000, 0xc000525a10, 0x1894718, 0x0, 0x0, 0x0): driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:1024 +0x4f4: driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).handleStream(0xc00009b200, 0x11401e0, 0xc0004c0a80, 0xc0004ac000, 0x0): driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:1313 +0xd97: driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0003c0240, 0xc00009b200, 0x11401e0, 0xc0004c0a80, 0xc0004ac000): driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:722 +0xbb: driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: created by github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).serveStreams.func1: driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:720 +0xa1: driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr: plugin process exited: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver pid=312666 error="exit status 2"
2022-09-09T03:08:39.065+0530 [WARN]  client.driver_mgr: received fingerprint error from driver: driver=firecracker-task-driver error="plugin is shut down"
2022-09-09T03:08:39.065+0530 [WARN]  client.driver_mgr: failed to reattach to plugin, starting new instance: driver=firecracker-task-driver err="singleton plugin exited"
2022-09-09T03:08:39.065+0530 [ERROR] client.alloc_runner.task_runner: failed to kill task: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy backoff=5s error="plugin is shut down"
2022-09-09T03:08:39.065+0530 [DEBUG] client.alloc_runner.task_runner: driver plugin has shutdown; attempting to recover task: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy driver=firecracker-task-driver
2022-09-09T03:08:39.065+0530 [DEBUG] client.driver_mgr: starting plugin: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver args=["/root/plugins/firecracker-task-driver"]
2022-09-09T03:08:39.066+0530 [WARN]  client: dropping node event: node_event="Message: ready, Subsystem: Driver, Details: driver: firecracker-task-driver, Timestamp: 2022-09-09 03:08:37.051011199 +0530 IST m=+232.427953591"
2022-09-09T03:08:39.066+0530 [DEBUG] client.driver_mgr: plugin started: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver pid=312767
2022-09-09T03:08:39.066+0530 [DEBUG] client.driver_mgr: waiting for RPC address: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver
2022-09-09T03:08:39.080+0530 [DEBUG] client.driver_mgr: using plugin: driver=firecracker-task-driver version=2
2022-09-09T03:08:39.080+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: plugin address: driver=firecracker-task-driver address=/tmp/plugin327846829 network=unix timestamp="2022-09-09T03:08:39.080+0530"
2022-09-09T03:08:39.081+0530 [DEBUG] client.driver_mgr.stdio: received EOF, stopping recv loop: driver=firecracker-task-driver err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio"
2022-09-09T03:08:39.081+0530 [INFO]  client.driver_mgr.firecracker-task-driver: buildFingerprint(): driver=firecracker-task-driver @module=firecracker-task-driver driver.FingerPrint=healthy timestamp="2022-09-09T03:08:39.081+0530"
2022-09-09T03:08:39.082+0530 [WARN]  client: dropping node event: node_event="Message: failed to fingerprint driver, Subsystem: Driver, Details: driver: firecracker-task-driver, Timestamp: 2022-09-09 03:08:37.303751974 +0530 IST m=+232.680694369"
2022-09-09T03:08:39.082+0530 [INFO]  client.driver_mgr.firecracker-task-driver: Starting firecracker: driver=firecracker-task-driver driver_initialize_container="&{/usr/local/bin/firecracker /tmp/hello-vmlinux.bin  console=ttyS0 reboot=k panic=1 pci=off nomodules /tmp/nginx_v1.ext4  [] galaxy {   []} []    false 1  300    false false [] <nil> 0xd248c0}+" @module=firecracker-task-driver timestamp="2022-09-09T03:08:39.082+0530"
2022-09-09T03:08:39.441+0530 [INFO]  client.driver_mgr.firecracker-task-driver: Writing to: driver=firecracker-task-driver @module=firecracker-task-driver driver_initialize_container="/tmp/Galaxy-de27a9ad-ea65-9ee5-79f3-dc02f224bab6+" timestamp="2022-09-09T03:08:39.440+0530"
2022-09-09T03:08:39.441+0530 [DEBUG] client.alloc_runner.task_runner: task successfully recovered on driver: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: panic: runtime error: invalid memory address or nil pointer dereference: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xca7967]: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: goroutine 129 [running]:: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink.(*Handle).LinkDel(0x18a3cc0, 0x0, 0x0, 0x0, 0x0): driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink/link_linux.go:1351 +0x37: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink.LinkDel(...): driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink/link_linux.go:1344: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/driver.(*taskHandle).shutdown(0xc00064e000, 0x12a05f200, 0x34, 0xc00064e000): driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/driver/handle.go:224 +0x73: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/driver.(*Driver).StopTask(0xc0001f0780, 0xc00003ed40, 0x34, 0x12a05f200, 0x0, 0x0, 0xc00038a900, 0x1143e60): driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/driver/driver.go:370 +0x6a: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers.(*driverPluginServer).StopTask(0xc0001e0460, 0x1130960, 0xc000367740, 0xc000584190, 0xc0001e0460, 0xc000367740, 0xc000621b30): driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers/server.go:181 +0x8f: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers/proto._Driver_StopTask_Handler(0xf3c3a0, 0xc0001e0460, 0x1130960, 0xc000367740, 0xc00038a9c0, 0x0, 0x1130960, 0xc000367740, 0xc00003ecc0, 0x3a): driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers/proto/driver.pb.go:4375 +0x217: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).processUnaryRPC(0xc00009ac00, 0x11401e0, 0xc00009b680, 0xc0001eca00, 0xc00017fe30, 0x1894718, 0x0, 0x0, 0x0): driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:1024 +0x4f4: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).handleStream(0xc00009ac00, 0x11401e0, 0xc00009b680, 0xc0001eca00, 0x0): driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:1313 +0xd97: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0003bb030, 0xc00009ac00, 0x11401e0, 0xc00009b680, 0xc0001eca00): driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:722 +0xbb: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: created by github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).serveStreams.func1: driver=firecracker-task-driver
2022-09-09T03:08:39.445+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:720 +0xa1: driver=firecracker-task-driver
2022-09-09T03:08:39.459+0530 [DEBUG] client.driver_mgr: plugin process exited: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver pid=312767 error="exit status 2"
2022-09-09T03:08:39.459+0530 [ERROR] client.alloc_runner.task_runner: failed to kill task: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy backoff=5s error="plugin is shut down"
2022-09-09T03:08:39.459+0530 [WARN]  client.driver_mgr: failed to reattach to plugin, starting new instance: driver=firecracker-task-driver err="singleton plugin exited"
2022-09-09T03:08:39.459+0530 [DEBUG] client.driver_mgr: starting plugin: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver args=["/root/plugins/firecracker-task-driver"]
2022-09-09T03:08:39.459+0530 [WARN]  client.driver_mgr: received fingerprint error from driver: driver=firecracker-task-driver error="plugin is shut down"
2022-09-09T03:08:39.459+0530 [DEBUG] client.alloc_runner.task_runner: driver plugin has shutdown; attempting to recover task: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy driver=firecracker-task-driver
2022-09-09T03:08:39.459+0530 [WARN]  client: dropping node event: node_event="Message: ready, Subsystem: Driver, Details: driver: firecracker-task-driver, Timestamp: 2022-09-09 03:08:37.31985225 +0530 IST m=+232.696794645"
2022-09-09T03:08:39.460+0530 [DEBUG] client.driver_mgr: plugin started: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver pid=312866
2022-09-09T03:08:39.460+0530 [DEBUG] client.driver_mgr: waiting for RPC address: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver
2022-09-09T03:08:39.497+0530 [DEBUG] client.driver_mgr: using plugin: driver=firecracker-task-driver version=2
2022-09-09T03:08:39.497+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: plugin address: driver=firecracker-task-driver address=/tmp/plugin062134114 network=unix timestamp="2022-09-09T03:08:39.496+0530"
2022-09-09T03:08:39.497+0530 [DEBUG] client.driver_mgr.stdio: received EOF, stopping recv loop: driver=firecracker-task-driver err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio"
2022-09-09T03:08:39.498+0530 [INFO]  client.driver_mgr.firecracker-task-driver: buildFingerprint(): driver=firecracker-task-driver @module=firecracker-task-driver driver.FingerPrint=healthy timestamp="2022-09-09T03:08:39.498+0530"
2022-09-09T03:08:39.498+0530 [WARN]  client: dropping node event: node_event="Message: failed to fingerprint driver, Subsystem: Driver, Details: driver: firecracker-task-driver, Timestamp: 2022-09-09 03:08:37.659460858 +0530 IST m=+233.036403268"
2022-09-09T03:08:39.499+0530 [INFO]  client.driver_mgr.firecracker-task-driver: Starting firecracker: driver=firecracker-task-driver @module=firecracker-task-driver driver_initialize_container="&{/usr/local/bin/firecracker /tmp/hello-vmlinux.bin  console=ttyS0 reboot=k panic=1 pci=off nomodules /tmp/nginx_v1.ext4  [] galaxy {   []} []    false 1  300    false false [] <nil> 0xd248c0}+" timestamp="2022-09-09T03:08:39.499+0530"
2022-09-09T03:08:39.704+0530 [DEBUG] client: state changed, updating node and re-registering
2022-09-09T03:08:39.705+0530 [INFO]  client: node registration complete
2022-09-09T03:08:39.705+0530 [DEBUG] client: evaluations triggered by node registration: num_evals=1
2022-09-09T03:08:39.705+0530 [DEBUG] worker: dequeued evaluation: worker_id=60296042-594e-0269-8f0e-0ae405013d1d eval_id=cdd595a5-fc10-69b8-a0be-51138c23a244 type=service namespace=default job_id=Web node_id=baa9672e-99e1-31ec-02b0-7d44e7c945a5 triggered_by=node-update
2022-09-09T03:08:39.705+0530 [DEBUG] worker.service_sched: reconciled current state with desired state: eval_id=cdd595a5-fc10-69b8-a0be-51138c23a244 job_id=Web namespace=default worker_id=60296042-594e-0269-8f0e-0ae405013d1d

results=
| Total changes: (place 0) (destructive 0) (inplace 0) (stop 0) (disconnect 0) (reconnect 0)
| Desired Changes for "Nginx": (place 0) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0)

2022-09-09T03:08:39.706+0530 [DEBUG] worker.service_sched: setting eval status: eval_id=cdd595a5-fc10-69b8-a0be-51138c23a244 job_id=Web namespace=default worker_id=60296042-594e-0269-8f0e-0ae405013d1d status=complete
2022-09-09T03:08:39.706+0530 [DEBUG] worker: updated evaluation: worker_id=60296042-594e-0269-8f0e-0ae405013d1d eval="<Eval \"cdd595a5-fc10-69b8-a0be-51138c23a244\" JobID: \"Web\" Namespace: \"default\">"
2022-09-09T03:08:39.706+0530 [DEBUG] worker: ack evaluation: worker_id=60296042-594e-0269-8f0e-0ae405013d1d eval_id=cdd595a5-fc10-69b8-a0be-51138c23a244 type=service namespace=default job_id=Web node_id=baa9672e-99e1-31ec-02b0-7d44e7c945a5 triggered_by=node-update
2022-09-09T03:08:39.909+0530 [INFO]  client.driver_mgr.firecracker-task-driver: Writing to: driver=firecracker-task-driver driver_initialize_container="/tmp/Galaxy-de27a9ad-ea65-9ee5-79f3-dc02f224bab6+" @module=firecracker-task-driver timestamp="2022-09-09T03:08:39.909+0530"
2022-09-09T03:08:39.909+0530 [DEBUG] client.alloc_runner.task_runner: task successfully recovered on driver: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy driver=firecracker-task-driver
2022-09-09T03:08:39.912+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: panic: runtime error: invalid memory address or nil pointer dereference: driver=firecracker-task-driver
2022-09-09T03:08:39.912+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xca7967]: driver=firecracker-task-driver
2022-09-09T03:08:39.912+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: goroutine 57 [running]:: driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink.(*Handle).LinkDel(0x18a3cc0, 0x0, 0x0, 0x0, 0x0): driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink/link_linux.go:1351 +0x37: driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink.LinkDel(...): driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/vishvananda/netlink/link_linux.go:1344: driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/driver.(*taskHandle).shutdown(0xc0003a40f0, 0x12a05f200, 0x34, 0xc0003a40f0): driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/driver/handle.go:224 +0x73: driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/driver.(*Driver).StopTask(0xc0000d29b0, 0xc0003a28c0, 0x34, 0x12a05f200, 0x0, 0x0, 0xc0005d4000, 0x1143e60): driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/driver/driver.go:370 +0x6a: driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers.(*driverPluginServer).StopTask(0xc00065ba20, 0x1130960, 0xc00050e180, 0xc0002cc140, 0xc00065ba20, 0xc00050e180, 0xc000231b30): driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers/server.go:181 +0x8f: driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers/proto._Driver_StopTask_Handler(0xf3c3a0, 0xc00065ba20, 0x1130960, 0xc00050e180, 0xc0005d40c0, 0x0, 0x1130960, 0xc00050e180, 0xc0003a2880, 0x3a): driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/github.com/hashicorp/nomad/plugins/drivers/proto/driver.pb.go:4375 +0x217: driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).processUnaryRPC(0xc0004e5380, 0x11401e0, 0xc0004e5e00, 0xc0004b4000, 0xc0003d1080, 0x1894718, 0x0, 0x0, 0x0): driver=firecracker-task-driver
2022-09-09T03:08:39.913+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:1024 +0x4f4: driver=firecracker-task-driver
2022-09-09T03:08:39.914+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).handleStream(0xc0004e5380, 0x11401e0, 0xc0004e5e00, 0xc0004b4000, 0x0): driver=firecracker-task-driver
2022-09-09T03:08:39.914+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:1313 +0xd97: driver=firecracker-task-driver
2022-09-09T03:08:39.914+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0004c9450, 0xc0004e5380, 0x11401e0, 0xc0004e5e00, 0xc0004b4000): driver=firecracker-task-driver
2022-09-09T03:08:39.914+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:722 +0xbb: driver=firecracker-task-driver
2022-09-09T03:08:39.914+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: created by github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc.(*Server).serveStreams.func1: driver=firecracker-task-driver
2022-09-09T03:08:39.914+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: 	/root/go/src/github.com/cneira/firecracker-task-driver/vendor/google.golang.org/grpc/server.go:720 +0xa1: driver=firecracker-task-driver
2022-09-09T03:08:39.916+0530 [DEBUG] client.driver_mgr: plugin process exited: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver pid=312866 error="exit status 2"
2022-09-09T03:08:39.916+0530 [WARN]  client.driver_mgr: received fingerprint error from driver: driver=firecracker-task-driver error="plugin is shut down"
2022-09-09T03:08:39.916+0530 [WARN]  client.driver_mgr: failed to reattach to plugin, starting new instance: driver=firecracker-task-driver err="singleton plugin exited"
2022-09-09T03:08:39.917+0530 [WARN]  client: dropping node event: node_event="Message: ready, Subsystem: Driver, Details: driver: firecracker-task-driver, Timestamp: 2022-09-09 03:08:37.688502153 +0530 IST m=+233.065444546"
2022-09-09T03:08:39.916+0530 [ERROR] client.alloc_runner.task_runner: failed to kill task: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy backoff=5s error="plugin is shut down"
2022-09-09T03:08:39.917+0530 [DEBUG] client.alloc_runner.task_runner: driver plugin has shutdown; attempting to recover task: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy driver=firecracker-task-driver
2022-09-09T03:08:39.917+0530 [DEBUG] client.driver_mgr: starting plugin: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver args=["/root/plugins/firecracker-task-driver"]
2022-09-09T03:08:39.917+0530 [DEBUG] client.driver_mgr: plugin started: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver pid=312965
2022-09-09T03:08:39.917+0530 [DEBUG] client.driver_mgr: waiting for RPC address: driver=firecracker-task-driver path=/root/plugins/firecracker-task-driver
2022-09-09T03:08:39.946+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: plugin address: driver=firecracker-task-driver address=/tmp/plugin998461408 network=unix timestamp="2022-09-09T03:08:39.946+0530"
2022-09-09T03:08:39.946+0530 [DEBUG] client.driver_mgr: using plugin: driver=firecracker-task-driver version=2
2022-09-09T03:08:39.947+0530 [DEBUG] client.driver_mgr.stdio: received EOF, stopping recv loop: driver=firecracker-task-driver err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio"
2022-09-09T03:08:39.948+0530 [INFO]  client.driver_mgr.firecracker-task-driver: buildFingerprint(): driver=firecracker-task-driver @module=firecracker-task-driver driver.FingerPrint=healthy timestamp="2022-09-09T03:08:39.948+0530"
2022-09-09T03:08:39.948+0530 [WARN]  client: dropping node event: node_event="Message: failed to fingerprint driver, Subsystem: Driver, Details: driver: firecracker-task-driver, Timestamp: 2022-09-09 03:08:37.98126072 +0530 IST m=+233.358203111"
2022-09-09T03:08:39.948+0530 [INFO]  client.driver_mgr.firecracker-task-driver: Starting firecracker: driver=firecracker-task-driver @module=firecracker-task-driver driver_initialize_container="&{/usr/local/bin/firecracker /tmp/hello-vmlinux.bin  console=ttyS0 reboot=k panic=1 pci=off nomodules /tmp/nginx_v1.ext4  [] galaxy {   []} []    false 1  300    false false [] <nil> 0xd248c0}+" timestamp="2022-09-09T03:08:39.948+0530"
2022-09-09T03:08:40.223+0530 [INFO]  client.driver_mgr.firecracker-task-driver: Writing to: driver=firecracker-task-driver @module=firecracker-task-driver driver_initialize_container="/tmp/Galaxy-de27a9ad-ea65-9ee5-79f3-dc02f224bab6+" timestamp="2022-09-09T03:08:40.222+0530"
2022-09-09T03:08:40.223+0530 [DEBUG] client.alloc_runner.task_runner: task successfully recovered on driver: alloc_id=de27a9ad-ea65-9ee5-79f3-dc02f224bab6 task=Galaxy driver=firecracker-task-driver
2022-09-09T03:08:40.225+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: panic: runtime error: invalid memory address or nil pointer dereference: driver=firecracker-task-driver
2022-09-09T03:08:40.225+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xca7967]: driver=firecracker-task-driver
2022-09-09T03:08:40.225+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: driver=firecracker-task-driver
2022-09-09T03:08:40.225+0530 [DEBUG] client.driver_mgr.firecracker-task-driver: goroutine 57 [running]:: driver=firecracker-task-driver

from firecracker-task-driver.

cneira avatar cneira commented on May 27, 2024

Thanks for the bug report, I'll take a look when I have available time to work on this.

from firecracker-task-driver.

Joffref avatar Joffref commented on May 27, 2024

Bug is due to the following issue that has been solved in firecracker-go-sdk upstream firecracker-microvm/firecracker-go-sdk#428.

Unfortunately, this plugin is not up-to-date with the firecraker-go-sdk upstream.

from firecracker-task-driver.

alexhulbert avatar alexhulbert commented on May 27, 2024

@Joffref I tried cherry-picking that commit on top of the latest pre-1.0-compatible firecracker-go-sdk and basing the firecracker-task-driver off of it, but I'm still getting the same issue. Is there anything else I need to do to get this integrated with the task driver? My forked firecracker-go-sdk and firecracker-task-driver are available at https://github.com/radixbio/firecracker-go-sdk and https://github.com/radixbio/firecracker-task-driver respectively for context.

Thanks for linking the issue!!

from firecracker-task-driver.

cneira avatar cneira commented on May 27, 2024

@hitendra-galaxyweblinks were you using the cni-plugins from my fork? if not that the cause of the segfault. Unfortunately by mistake a erased my fork with the changes need a month ago, but luckily seems the current sdk fixed the issue. I'll try to try this during the week should be and easy fix.
This is described in #11 , the thing is I don't have the cni plugins I fixed as I delete them by mistake, but is not so bad is this is fixed on the updated sdk.

from firecracker-task-driver.

Related Issues (15)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.