nyaxt / otaru Goto Github PK
View Code? Open in Web Editor NEWStore 10MB-10GB-ish personal files on cloud
License: Apache License 2.0
Store 10MB-10GB-ish personal files on cloud
License: Apache License 2.0
Now that we have optimized inodedb, gcloud/datastore/dbtransactionlogio isn't able to keep up with persisting inodedb txs.
Looks like this is the root cause of #39.
We currently never explicitly GC type OpenFile struct
.
This is problematic because:
assetfs is broken somehow
The CachedBlobEntry
returned from the manager may be closed before be.OpenHandle
is called.
Would be useful for debugging #39
It should have at least minimum documentation on its architecture and basic code flow
CachedBlobEntry
shouldn't be touched casually.invalidateCacheBlob
takes an instance of CachedBlobStore
. srsly?We currently casually block on Open
, but we should avoid it by scheduling write back better.
We should avoid scheduling backend IO while holding mu.
Maybe introduce more "-ing" states.
This can lead to broken cache -> data corruption
The "Mem Allocated" count seems to increase over time.
No HTTP req is sent when loglevel is changed from webui.
inodedb.Sync() takes too long time if cachedblobstore is full and busy flushing out dirty blob caches.
This might be cause of unknown EIOs and pauses during rsyncs.
Currently we:
lock {
serialize+gzip
write blob
}
This should be:
lock {
serialize+gzip
}
write blob
make chmod +x file
work
/scripts/app.js 404
We regularly get some, so need to write retries:
https://circleci.com/gh/nyaxt/otaru/55
This indicator would be useful to diagnose otaru bottleneck.
If sync writeback is happening too often, it indicates that cachedblobstore is not keeping up with new writes.
2015/08/13 08:53:27 D [bfuse] main.go:93: short kernel write: written=16/4112 error="" stack=
goroutine 3596 [running]:
bazil.org/fuse.stack(0x0, 0x0)
/Users/kouhei/go/src/github.com/nyaxt/otaru/Godeps/_workspace/src/bazil.org/fuse/debug.go:9 +0x84
bazil.org/fuse.(*Conn).writeToKernel(0xc2083c1080, 0xc215c36000, 0x1010, 0x1010, 0x0, 0x0)
/Users/kouhei/go/src/github.com/nyaxt/otaru/Godeps/_workspace/src/bazil.org/fuse/fuse.go:1045 +0x15f
bazil.org/fuse.(*Conn).respond(0xc2083c1080, 0xc215c36000, 0x1010, 0x1010)
/Users/kouhei/go/src/github.com/nyaxt/otaru/Godeps/_workspace/src/bazil.org/fuse/fuse.go:1052 +0x55
bazil.org/fuse.(*Header).respond(0xc2160c49a0, 0xc215c36000, 0x1010, 0x1010)
/Users/kouhei/go/src/github.com/nyaxt/otaru/Godeps/_workspace/src/bazil.org/fuse/fuse.go:278 +0x5d
bazil.org/fuse.(*ReadRequest).Respond(0xc2160c49a0, 0xc216233360)
/Users/kouhei/go/src/github.com/nyaxt/otaru/Godeps/_workspace/src/bazil.org/fuse/fuse.go:1686 +0x16f
bazil.org/fuse/fs.(*Server).serve(0xc2082be000, 0xb3b238, 0xc2160c49a0)
/Users/kouhei/go/src/github.com/nyaxt/otaru/Godeps/_workspace/src/bazil.org/fuse/
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.