br0xen / boltbrowser Goto Github PK
View Code? Open in Web Editor NEWA CLI Browser for BoltDB Files
License: GNU General Public License v3.0
A CLI Browser for BoltDB Files
License: GNU General Public License v3.0
Line 10 in 7d7e0bf
The original project is not maintained anymore. Author recommends to use the fork at https://github.com/coreos/bbolt.
When I am using a large DB, boltbrowser hangs and fails to load.
The code below creates a series of buckets with integer-encoded names. See screenshot of what this results in. The bucket names are not decoded correctly on display. Integer-encoded keys, however, show up correctly.
package main
import (
"encoding/binary"
log "github.com/sirupsen/logrus"
bolt "github.com/etcd-io/bbolt"
)
func main() {
db, err := bolt.Open("rewards.db", 0600, nil)
if err != nil {
log.Fatal("Failed to init db:", err)
}
db.Update(func(tx *bolt.Tx) error {
rBucket, err := tx.CreateBucketIfNotExists([]byte("rewards"))
if err != nil {
log.Fatal("Cannot create rewards bucket:", err)
}
for i := 1; i < 20; i++ {
buk, err := rBucket.CreateBucketIfNotExists(inttob(i));
if err != nil {
log.Fatal("Cannot make bucket", i)
}
buk.Put(inttob(i*3), []byte("Hello"))
}
return nil
})
log.Print("Done")
}
func inttob(v int) []byte {
return itob(int64(v))
}
// itob returns an 8-byte big endian representation of v.
func itob(v int64) []byte {
b := make([]byte, 8)
binary.BigEndian.PutUint64(b, uint64(v))
return b
}
func btoi(b []byte) int64 {
return int64(binary.BigEndian.Uint64(b))
}
i tried this
$ go build boltbrowser.go
# command-line-arguments
./boltbrowser.go:18: undefined: BoltDB
./boltbrowser.go:44: undefined: defaultStyle
./boltbrowser.go:47: undefined: mainLoop
I happened to have an entry with the key="/e/210". When I browse that row, it doesn't allow me to rename/edit/delete it. It says "Not sure what to do here".
Currently boltbrowser will only save ascii values back into the DB.
It'd be nice to have it check:
My current thought is that other supported data types would just be the basic go types:
bool
int
, int8
, int16
, int32
, int64
uint
, uint8
, uint16
, uint32
, uint64
, uintptr
(doubt if we actually need that one)byte
float32
, float64
complex64
, complex128
Eventually, I would like to add support for custom data types, probably through the use of go plugins:
https://golang.org/pkg/plugin/
I believe that doing this should mostly resolve pull request #31 and alleviate issue #37
At least on a Mac, it doesn't look like 'd' works to delete items, but instead 'D' works.
The help screen should be updated.
Thank you.
The screen should probably show a message if the db it's trying to open is locked.
I'm not sure what a good amount of time to wait before giving up would be, but that can be tweaked.
When using the 'X' hotkey to export, you are prompted for a filename and the data is saved correctly.
However the file isn't closed (it cannot be opened by another process until you exit boltbrowser)
The next time a export is performed, the entered filename isn't used, and data appears to be appended to the original file (which is still open).
OS: Windows 10
Version: Most recent from readme.md link
When I make changes in a file, I need to close the browser and reopen it. I lose all my open buckets and it's uncomfortable. It may be convenient to have a 'refresh' option so there will be no need to close and reopen the browser.
Keys with long values delete from the end of the line instead of where the cursor is.
@br0xen I am using boltdb in one of my projects and I like boltbrowser. I would like to embed boltbrowser into my application binary. To do this, I need to be able to import it as a package. Current package name (main
) does not allow me to that. I propose to move all code files except boltbrowser.go
into a new package under github.com/br0xen/boltbrowser/boltbrowser
. This will not break backwards compatibility and will allow other packages to import boltbrowser as a library. I can send you a PR if it's okay. Are you happy with this change?
The value of 59629 is saved/encoded using the following:
tx.Bucket([]byte("config")).Put([]byte("lastBlockLevel"), inttob(blockLevel))
// itob returns an 8-byte big endian representation of v.
func itob(v int64) []byte {
b := make([]byte, 8)
binary.BigEndian.PutUint64(b, uint64(v))
return b
}
Displays correctly when viewing bucket/kv, but attempting to edit doesn't work.
Am I storing integers correctly or is there a better way?
The example itob
function on github.com/etcd-io/bbolt for encoding integers accepts signed values. The browser does not take this into consideration on display and you get the below overflow:
1061401654: {"settings":{"missed":false,"success":true}}
1068588820: {"settings":{"missed":false,"success":false}}
18446743072381089533: {"settings":{"missed":true,"success":true}}
The 1844... number should be -1001328462083 when decoded. This is a display issue only as keys fetched from db within our app and decoded retain their correct value.
panic when browse containerd 1.17.1‘s metadata
panic: runtime error: index out of range [18446744073709547520] with length 281474976710655
goroutine 1 [running]:
github.com/boltdb/bolt.(*DB).page(...)
/data00/code/pkg/mod/github.com/boltdb/[email protected]/db.go:796
github.com/boltdb/bolt.Open({0x7ffd572e0544, 0x3a}, 0x48690?, 0xc00008de60)
/data00/code/pkg/mod/github.com/boltdb/[email protected]/db.go:237 +0x545
main.main()
/data00/code/pkg/mod/github.com/br0xen/[email protected]/main.go:117 +0x1bc
root@n37-006-180:~# boltbrowser /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db
panic: runtime error: index out of range [18446744073709547520] with length 281474976710655
goroutine 1 [running]:
github.com/boltdb/bolt.(*DB).page(...)
/data00/code/pkg/mod/github.com/boltdb/[email protected]/db.go:796
github.com/boltdb/bolt.Open({0x7ffc1e7f2544, 0x3a}, 0x48690?, 0xc000059e60)
/data00/code/pkg/mod/github.com/boltdb/[email protected]/db.go:237 +0x545
main.main()
/data00/code/pkg/mod/github.com/br0xen/[email protected]/main.go:117 +0x1bc
Maybe we should update bolt version
Due to:
boltdb/bolt
is no longer maintained.boltdb/[email protected]
And etcd bbolt is a fork of Ben Johnson's Bolt key/value store. The purpose of this fork is to provide the Go community with an active maintenance and development target for Bolt; the goal is improved reliability and stability. bbolt includes bug fixes, performance enhancements, and features not found in Bolt while preserving backwards compatibility with the Bolt API.
see https://github.com/etcd-io/bbolt#bbolt
https://github.com/br0xen/boltbrowser/releases/tag/2.1
Line 15 in 59a4e49
Need to be 2.1 or the version of next release.
I have a trivial testing app:
package main
import (
"fmt"
"strconv"
"github.com/boltdb/bolt"
)
func main() {
db, err := bolt.Open("my.db", 0600, nil)
if err != nil {
panic(err)
}
defer db.Close()
err = db.Update(func(tx *bolt.Tx) (err error) {
bucket, err := tx.CreateBucketIfNotExists([]byte("Test"))
if err != nil {
return err
}
raw := bucket.Get([]byte("num"))
num := 0
if len(raw) != 0 {
num, err = strconv.Atoi(string(raw))
if err != nil {
return err
}
}
num++
fmt.Println(num)
bucket.Put([]byte("num"), []byte(strconv.Itoa(num)))
return nil
})
if err != nil {
panic(err)
}
}
Every time you run it, it reads a number, increments it, and puts it back.
If I have boltbrowser connected to my.db and I run my app then my app sits and waits until boltbrowser releases it's connection. I know bolt only allows 1 write transaction at a time. It'd be nice if boltbrowser had a read only flag so viewing a DB that's currently under use won't halt all interactions with the database.
I imagine if another process is changing the DB then there's no way to know for sure if a value in a bucket didn't change immediately after being accessed and shown on the screen. Personally I'm ok with that.
It looks like all you'd have to do is prevent p/P, b/B, D, e, and r key presses if an "-ro" flag is present. Maybe just show a popup saying "Cannot do that in Read Only mode."
Hi.
I have a database with a bucket of 816 large pairs but when i try to delete it boltbrowser shuts down with the resulting error:
panic: page 3278 already freed
goroutine 1 [running]:
github.com/boltdb/bolt.(*freelist).free(0xc0000ea090, 0xb999, 0x7f5b16891000)
/home/admin/bolt/src/github.com/boltdb/bolt/freelist.go:121 +0x2a6
github.com/boltdb/bolt.(*Bucket).free.func1(0x7f5b16891000, 0x0, 0x2)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:684 +0x51
github.com/boltdb/bolt.(*Bucket)._forEachPageNode(0xc00004c340, 0xcce, 0x2, 0xc00006d9e0)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:506 +0x87
github.com/boltdb/bolt.(*Bucket)._forEachPageNode(0xc00004c340, 0x5f, 0x1, 0xc00006d9e0)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:513 +0xe6
github.com/boltdb/bolt.(*Bucket)._forEachPageNode(0xc00004c340, 0x15b, 0x0, 0xc00006d9e0)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:513 +0xe6
github.com/boltdb/bolt.(*Bucket).forEachPageNode(0xc00004c340, 0xc00006d9e0)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:499 +0x72
github.com/boltdb/bolt.(*Bucket).free(0xc00004c340)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:682 +0x67
github.com/boltdb/bolt.(*Bucket).DeleteBucket(0xc0000102b8, 0xc00006dbb0, 0x12, 0x20, 0x12, 0x20)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:255 +0x27e
github.com/boltdb/bolt.(*Tx).DeleteBucket(...)
/home/admin/bolt/src/github.com/boltdb/bolt/tx.go:121
main.deleteKey.func1(0xc0000102a0, 0x540a68, 0xc0000102a0)
/home/admin/bolt/src/github.com/br0xen/boltbrowser/bolt_model.go:375 +0x444
github.com/boltdb/bolt.(*DB).Update(0xc0000ec000, 0xc00006dc60, 0x0, 0x0)
/home/admin/bolt/src/github.com/boltdb/bolt/db.go:598 +0x90
main.deleteKey(0xc005882050, 0x1, 0x1, 0x1, 0xc00587e2d0)
/home/admin/bolt/src/github.com/br0xen/boltbrowser/bolt_model.go:370 +0xd5
main.(*BrowserScreen).handleDeleteKeyEvent(0xc00585e000, 0x7900000000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x0)
/home/admin/bolt/src/github.com/br0xen/boltbrowser/screen_browser.go:263 +0x1d2
main.(*BrowserScreen).handleKeyEvent(0xc00585e000, 0x7900000000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x0)
/home/admin/bolt/src/github.com/br0xen/boltbrowser/screen_browser.go:84 +0x119
main.mainLoop(0xc00013e080, 0x3000100080001, 0xc000030001)
/home/admin/bolt/src/github.com/br0xen/boltbrowser/mainloop.go:25 +0x22d
main.main()
/home/admin/bolt/src/github.com/br0xen/boltbrowser/main.go:133 +0x30e
Not a Go person (rarely have to use it) so this is mostl likely something real simple, but I don't know how to proceed:
When I run:
go get github.com/br0xen/boltbrowser
I just get:
package github.com/rivo/uniseg: found packages uniseg (doc.go) and main (gen_breaktest.go) in /home/kacey/go/src/github.com/rivo/uniseg
and it stops. What can I do from here?
(Running Ubuntu)
thanks!
While attempting to grab boltbrowser for linux (x64) I discovered that the letsencrypt cert for this URL has expired:
wget -O boltbrowser https://git.bullercodeworks.com/attachments/29367198-79f9-4fb3-9a66-f71a0e605006 --2022-01-14 12:28:48-- https://git.bullercodeworks.com/attachments/29367198-79f9-4fb3-9a66-f71a0e605006 Resolving git.bullercodeworks.com (git.bullercodeworks.com)... 149.28.127.101 Connecting to git.bullercodeworks.com (git.bullercodeworks.com)|149.28.127.101|:443... connected. ERROR: cannot verify git.bullercodeworks.com's certificate, issued by ‘/C=US/O=Let's Encrypt/CN=R3’: Issued certificate has expired. To connect to git.bullercodeworks.com insecurely, use
--no-check-certificate'.
`
Is it possible to just release the binaries as part of a standard GitHub Release?
I got this panic while working on #18
main.(*BoltDB).buildVisiblePathSlice(0x0, 0x0, 0xc4200e0000, 0x3, 0x80, 0x0)
$GOPATH/src/github.com/br0xen/boltbrowser/bolt_model.go:130 +0x37
main.(*BoltDB).getNextVisiblePath(0x0, 0x0, 0x0, 0x0, 0xc42003dc50, 0x10e3a35, 0x11db480)
$GOPATH/src/github.com/br0xen/boltbrowser/bolt_model.go:167 +0x2f
main.(*BrowserScreen).moveCursorDown(0xc4200e8000, 0xc42003dcd0)
$GOPATH/src/github.com/br0xen/boltbrowser/screen_browser.go:474 +0x4d
main.(*BrowserScreen).handleBrowseKeyEvent(0xc4200e8000, 0xffec0000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3, 0x2)
$GOPATH/src/github.com/br0xen/boltbrowser/screen_browser.go:123 +0x9d
main.(*BrowserScreen).handleKeyEvent(0xc4200e8000, 0xffec0000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3, 0x0)
$GOPATH/src/github.com/br0xen/boltbrowser/screen_browser.go:78 +0x19c
main.mainLoop(0x0, 0x3000100080001, 0x30001)
$GOPATH/src/github.com/br0xen/boltbrowser/mainloop.go:25 +0x299
main.main()
$GOPATH/src/github.com/br0xen/boltbrowser/boltbrowser.go:56 +0x21b
To reproduce it you need to invoke boltbrowser
with more than one file and make sure that one of them produces an error while being opened (use non-db file for example). Once you get to the screen for this file, press any arrow button.
here is the panic
panic: runtime error: index out of range [1] with length 1
goroutine 1 [running]:
main.(*BrowserScreen).jumpCursorDown(0xc000244000, 0x1, 0x1)
/home/builder/boltbrowser/screen_browser.go:467 +0x35d
main.(*BrowserScreen).handleBrowseKeyEvent(0xc000244000, 0x60000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0xc00003dca0)
/home/builder/boltbrowser/screen_browser.go:121 +0x73d
main.(*BrowserScreen).handleKeyEvent(0xc000244000, 0x60000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x0)
/home/builder/boltbrowser/screen_browser.go:84 +0x1fa
main.mainLoop(0xc00000c060, 0x3000100080001, 0x30001)
/home/builder/boltbrowser/mainloop.go:25 +0x21e
main.main()
/home/builder/boltbrowser/main.go:141 +0x333
reproduced (with same database) over ssh and locally with both kitty and xfce4-terminal
linux, TERM=xterm-256color
latest master branch (commit 7f10a81cece0f6754d1902b69c03892f267f65ee
)
i will try and come up with a patch, but using /
in the meantime
Probably this looks like:
This not only helps with issue reports, but also with automated testing of builds.
boltbrowser --version
Invalid optionUsage: boltbrowser [OPTIONS] <filename(s)>
Options:
-timeout=duration
DB file open timeout (default 1s)
-ro, -readonly
Open the DB in read-only mode
It would be useful to import previously exported json data into an existing bucket
This does not work, after installing in mac. I am using Sonama 14.
zsh: command not found: boltbrowser
Hey, I am using boltbrowser trying to fix a problem I have with the lnd software, which uses a bolt DB.
I need to delete some records in a 120k long list. Scrolling takes a bit, I estimate it will take 30+ hours of scrolling to get to the record.
Being able to jump to a record starting with XX, or searching, would be the absolute best option.
Please consider, it would help a lot. Thank you!
When the value of a key is long, allow scrolling of the right pane. Hooks are already in place listening for 'J' and 'K' to scroll the right pane.
Starting in Go 1.17, installing executables with go get is deprecated. go install may be used instead.
If you are browsing a bucket with lots of keys, it seems that the terminal does not scroll or highlight properly. You can see the correct key in the right panel, but the left panel is always at the top, it does not scroll. Not sure whether this can be implemented or it is a limitation of the term library.
All pre-built binaries show Not found
Example: Linux 64-bit
https://git.bullercodeworks.com/attachments/29367198-79f9-4fb3-9a66-f71a0e605006
It was working a couple of months ago. This is the preferred method for using it on machines without go
installed.
it's probably not a good idea to store values in the root bucket, but shouldn't boltbrowser account for that ?
the following code generates a boltdb boltbrowser
can't open
func main() {
db, err := bolt.Open("crash.db", 0600, nil)
if err != nil {
panic(err)
}
defer db.Close()
db.Update(func(tx *bolt.Tx) error {
return tx.Cursor().Bucket().Put([]byte("boat"), []byte("I'm on a root!"))
})
db.View(func(tx *bolt.Tx) error {
fmt.Printf("%s\n", tx.Cursor().Bucket().Get([]byte("boat")))
return nil
})
}
Hi,
First of all, thanks for creating boltbrowser, it's the shit. 👍
When creating a database with algernon, with the following command:
algernon -t --boltdb test.db . :7777
And then closing the web server by typing exit
or pressing ctrl-d, accessing the database with boltbrowser works fine:
boltbrowser test.db
Adding keys and values (on the Algernon Lua prompt) and then examining them in boltbrowser (after exiting the webserver) also works great:
kv = KeyValue("pageviews")
kv:set("counter", 9999)
However, if accessing the database while the web server is running, there is no error message and no timeout and the console interface just goes into black and hangs (can't be closed with esc
, q
or ctrl-c
), until the web server has quit and released the lock on the database.
This is a feature request for adding a timeout and/or a way to exit boltbrowser when the database is locked. Ignoring the database lock and entering some sort of read-only mode would also be interesting, if possible.
Cheers,
Alexander F Rødseth
I'd like to have the right pane do some syntax highlighting/formatting mostly for json values.
J/K act the same as arrow up/down. They move the left pane cursor. None of the keys seem to scroll the right pane..
It could be nice in some cases to be able to run a value through an external program and view the returned value in the right pane...
I'm thinking of instances where, perhaps, hex/binary data is stored, but it should be readable in ascii if converted through xxd
or something.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.