Coder Social home page Coder Social logo

nebula-contrib / nebulagraph-lite Goto Github PK

View Code? Open in Web Editor NEW
10.0 1.0 0.0 776 KB

The minimal, ad-hoc way of plug and play NebulaGraph with pip install, even inside Colab Notebook!

License: Apache License 2.0

Shell 5.89% Python 75.75% Jupyter Notebook 18.35%
graphdatabase nebulagraph colab

nebulagraph-lite's People

Contributors

wey-gu avatar

Stargazers

avatar avatar avatar avatar avatar avatar

Watchers

avatar

nebulagraph-lite's Issues

feat: add CLI

We could add CLI support

  • callable with python3 -m nebulagraph_lite.cli, via #21
  • executable under bin should be placed after installation with pip

feat: stop gracefully

It seems now the stop of service will cause metad corruption for always, cleanup is needed :(

❯ tail  ~/.nebulagraph/lite/logs/meta0/*
==> /home/w/.nebulagraph/lite/logs/meta0/metad-stderr.log <==
/usr/local/nebula/bin/nebula-metad(event_base_loop+0x36e)[0x2648e8e]
/usr/local/nebula/bin/nebula-metad(_ZN5folly9EventBase8loopBodyEib+0x46d)[0x2586efd]
/usr/local/nebula/bin/nebula-metad(_ZN5folly9EventBase4loopEv+0x3d)[0x25877ed]
/usr/local/nebula/bin/nebula-metad(_ZN5folly9EventBase11loopForeverEv+0x17)[0x258a427]
/usr/local/nebula/bin/nebula-metad(_ZN5folly20IOThreadPoolExecutor9threadRunESt10shared_ptrINS_18ThreadPoolExecutor6ThreadEE+0x338)[0x2514578]
/usr/local/nebula/bin/nebula-metad(_ZN5folly6detail8function14FunctionTraitsIFvvEE7callBigISt5_BindIFMNS_18ThreadPoolExecutorEFvSt10shared_ptrINS7_6ThreadEEEPS7_SA_EEEEvRNS1_4DataE+0x46)[0x2522ae6]
/usr/local/nebula/bin/nebula-metad[0x2b4d5ef]
/lib64/libpthread.so.0(+0x7ea4)[0x7c7760807ea4]
/lib64/libc.so.6(clone+0x6c)[0x7c77604fe96c]
(safe mode, symbolizer not available)

==> /home/w/.nebulagraph/lite/logs/meta0/metad-stdout.log <==

==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.ERROR <==
Log file created at: 2024/01/14 10:52:40
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
F20240114 10:52:40.701972  6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005

==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.FATAL <==
Log file created at: 2024/01/14 10:52:40
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
F20240114 10:52:40.701972  6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005

==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.INFO <==
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
I20240114 10:52:40.028664  6557 MetaDaemon.cpp:137] localhost = "127.0.0.1":9559
I20240114 10:52:40.337798  6557 NebulaStore.cpp:48] Start the raft service...
I20240114 10:52:40.343425  6557 NebulaSnapshotManager.cpp:25] Send snapshot is rate limited to 10485760 for each part by default
I20240114 10:52:40.637010  6557 RaftexService.cpp:46] Start raft service on 9560
I20240114 10:52:40.647907  6557 NebulaStore.cpp:82] Scan the local path, and init the spaces_
I20240114 10:52:40.648157  6557 NebulaStore.cpp:90] Scan path "/data/meta/nebula/0"
I20240114 10:52:40.648191  6557 NebulaStore.cpp:292] Init data from partManager for "127.0.0.1":9559
I20240114 10:52:40.648216  6557 NebulaStore.cpp:417] Create data space 0
F20240114 10:52:40.701972  6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005

==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.ERROR.20240114-104504.3898 <==
Log file created at: 2024/01/14 10:45:04
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
E20240114 10:45:04.757242  3898 FileUtils.cpp:377] Failed to read the directory "/data/meta/nebula" (2): No such file or directory
E20240114 10:45:06.707832  3898 FileUtils.cpp:76] Failed to remove the directory "/data/meta/nebula/0/checkpoints/META_UPGRADE_SNAPSHOT_2024_01_14_10_45_06/data" (39): Directory not empty
E20240114 10:45:06.707902  3898 FileUtils.cpp:49] Failed to remove "data" in "/data/meta/nebula/0/checkpoints/META_UPGRADE_SNAPSHOT_2024_01_14_10_45_06"
E20240114 10:45:06.707931  3898 FileUtils.cpp:70] Failed to remove the content of the directory "/data/meta/nebula/0/checkpoints/META_UPGRADE_SNAPSHOT_2024_01_14_10_45_06"

==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.ERROR.20240114-105240.6557 <==
Log file created at: 2024/01/14 10:52:40
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
F20240114 10:52:40.701972  6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005

==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.FATAL.20240114-105240.6557 <==
Log file created at: 2024/01/14 10:52:40
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
F20240114 10:52:40.701972  6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005

==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.INFO.20240114-104504.3898 <==
I20240114 10:51:34.553622  3898 RaftexService.cpp:67] Stopping the raftex service on port 9560
I20240114 10:51:34.553655  3898 RaftexService.cpp:77] All partitions have stopped
I20240114 10:51:34.553689  3898 NebulaStore.cpp:346] Stop kv engine...
I20240114 10:51:34.553740  3898 NebulaStore.cpp:36] Cut off the relationship with meta client
I20240114 10:51:34.570483  3898 Part.h:59] [Port: 9560, Space: 0, Part: 0] ~Part()
I20240114 10:51:34.579130  3898 RocksEngine.h:247] Release rocksdb on /data/meta/nebula/0
I20240114 10:51:34.580722  3898 NebulaStore.cpp:44] ~NebulaStore()
I20240114 10:51:34.580850  3898 MetaDaemon.cpp:215] The meta Daemon stopped
I20240114 10:51:34.593420  3898 JobManager.cpp:138] JobManager::shutDown() begin
I20240114 10:51:34.593461  3898 JobManager.cpp:141] JobManager not running, exit

==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.INFO.20240114-105240.6557 <==
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
I20240114 10:52:40.028664  6557 MetaDaemon.cpp:137] localhost = "127.0.0.1":9559
I20240114 10:52:40.337798  6557 NebulaStore.cpp:48] Start the raft service...
I20240114 10:52:40.343425  6557 NebulaSnapshotManager.cpp:25] Send snapshot is rate limited to 10485760 for each part by default
I20240114 10:52:40.637010  6557 RaftexService.cpp:46] Start raft service on 9560
I20240114 10:52:40.647907  6557 NebulaStore.cpp:82] Scan the local path, and init the spaces_
I20240114 10:52:40.648157  6557 NebulaStore.cpp:90] Scan path "/data/meta/nebula/0"
I20240114 10:52:40.648191  6557 NebulaStore.cpp:292] Init data from partManager for "127.0.0.1":9559
I20240114 10:52:40.648216  6557 NebulaStore.cpp:417] Create data space 0
F20240114 10:52:40.701972  6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005

==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.WARNING.20240114-104504.3898 <==
Log file created at: 2024/01/14 10:45:04
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
E20240114 10:45:04.757242  3898 FileUtils.cpp:377] Failed to read the directory "/data/meta/nebula" (2): No such file or directory
E20240114 10:45:06.707832  3898 FileUtils.cpp:76] Failed to remove the directory "/data/meta/nebula/0/checkpoints/META_UPGRADE_SNAPSHOT_2024_01_14_10_45_06/data" (39): Directory not empty
E20240114 10:45:06.707902  3898 FileUtils.cpp:49] Failed to remove "data" in "/data/meta/nebula/0/checkpoints/META_UPGRADE_SNAPSHOT_2024_01_14_10_45_06"
E20240114 10:45:06.707931  3898 FileUtils.cpp:70] Failed to remove the content of the directory "/data/meta/nebula/0/checkpoints/META_UPGRADE_SNAPSHOT_2024_01_14_10_45_06"

==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.penguin.root.log.WARNING.20240114-105240.6557 <==
Log file created at: 2024/01/14 10:52:40
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
F20240114 10:52:40.701972  6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005

==> /home/w/.nebulagraph/lite/logs/meta0/nebula-metad.WARNING <==
Log file created at: 2024/01/14 10:52:40
Running on machine: penguin
Running duration (h:mm:ss): 0:00:00
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
F20240114 10:52:40.701972  6635 RocksEngine.cpp:82] Check failed: status.ok() Corruption: Corruption: IO error: No such file or directory: While open a file for random read: /data/meta/nebula/0/data/000009.ldb: No such file or directory in file /data/meta/nebula/0/data/MANIFEST-000005

design of nebulagraph-lite

Target system:

  • Linux
  • Google Colab
  • WSL2 Ubuntu
  • macOS Docker Desktop(i.e. Jupyter extension running in container)/OrbStack/Lima

Dependencies:

Usage:

pip install nebulagraph-lite
from nebulagraph_lite import nebulagraph_let as ng_let

n = ng_let()

n.start()

Design

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.