bnclabs / gofast Goto Github PK
View Code? Open in Web Editor NEWHigh performance transport protocol for distributed applications.
License: MIT License
High performance transport protocol for distributed applications.
License: MIT License
Brainstorm and validate rolling upgrade scenarios using PeerVersion. Add a recommendation that can be used by gofast developers.
Hello,
I have recently come across gofast and was wondering how it might compare against QUIC or other similar UDP based protocols that are, in general, much faster than TCP protocols.
Any thoughts on this would be greatly appreciated.
Thanks
This endpoint should return the list of active transport names in JSON format.
Content-Type: application/json.
TagVersion doesn't make sense. Once gofast protocol is specified and standardised, it can be extended without compromising on backward compatibility. In case we can't provide backward compatibility we should spawn gofast2.
Repeat perf/ on Raspberry-pi.
Gofast should support disabling or enabling component level logging. Proposal is to export an API, LogComponents() whose godoc should describe supported components to log.
Subsequently gofast source should use a local variant of infof, errorf, warnf, etc... to enable or disable log.
When NewTransport() is issued with nil settings, then automatically fallback to DefaultSettings().
Behind the scenes, support load balancing based on random / moving-average of latency.
This implementation assigns tags 37-39 with different meanings than in the document http://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml#tags . Using these tags should be avoided, as CBOR data that makes use of them with the meanings other than specified in that document would make them less interoperable; instead, tags starting from 266 should be used; they're currently not assigned according to that document.
Repeat perf/ on Raspberry-pi.
This can be used by those who are not keen about versioning their application or protocol. I might suggest something like this:
type Version64 uint64
than implements Version{} interface. Subsequently users can simply do:
gofast.NewTransport("myapp", conn, gofast.Version64(1), setts)
Transport structure has got: pData, pStrms, pTxcmd
This can be replaced with lock-less buffer pool.
This benchmark can be added to perf/ program that already benchmarks POST, REQUEST and STREAM messages.
Http handler for gofast mounted by importing http/ package uses stdlib's encoding/json to create response text. Instead use gson.
Socket Read() and Write() are blocking, in TCP/IP there can be scenarios of half open connections and other un-detectable network errors on the connections.
On approach is to use net/ package's feature of setting Deadline timeouts for reads and writes, but profiling shows that this can lead to performance issues when we scale connections.
Another approach is to use the heart-beat. That is both peers should set the heartbeat, and expose an API to learn when the last heartbeat was received, set a threshold and close the socket after that.
May be we need a separate monitor routine for this.
TagId and TagData keynames are now encoded as uint64, encode them as tags Major-Type-6.
There won't be any change to the memory foot-print, but this will break nodes compiled with previous version of gofast.
Calling log.Verbosef() or log.Debugf() escapes values to heap leading to GC overheads.
Since connections have to scale, we need some performance profiling on select blocks in gofast.
Also run travis for 1.7 and 1.8
Gofast is an active library package, a quick page on go-routine block diagram and the data/control flow between them will help understand the code better.
Picture can say thousand words. Create a block diagram of go-routines, how the interact with application logic and underlying socket. Mostly it is important to trace the execution path and its exceptional cases.
Opaque values can be anything >=256. Having said that there are some exceptions:
While picking a opaque value from user supplied range filter the above values automatically.
Hello,
I wanted to build your example to test gofast but ran into some errors with a repository not being found.
Can you please help?
Thanks
test> go build .\example\
go: cannot find main module, but found .git/config in D:\FarpointFS\test-002\test
to create a module there, run:
go mod init
PS D:\FarpointFS\test-002\test> go mod init gofast
go: creating new go.mod: module gofast
go: to add module requirements and sums:
go mod tidy
PS D:\FarpointFS\test-002\test> go mod tidy
go: finding module for package github.com/bnclabs/gofast/http
go: finding module for package github.com/bnclabs/gosettings
go: finding module for package github.com/bnclabs/golog
go: finding module for package github.com/bnclabs/gson
go: finding module for package github.com/bnclabs/gofast
go: downloading github.com/bnclabs/gofast v0.0.0-20210210111353-5c982d6094f0
go: downloading github.com/bnclabs/golog v0.0.0-20210210111405-2049724535b8
go: downloading github.com/bnclabs/gosettings v0.0.0-20210210111304-0d91c5ae496a
go: downloading github.com/bnclabs/gson v0.0.0-20210409011557-2e73bae93c0b
go: found github.com/bnclabs/golog in github.com/bnclabs/golog v0.0.0-20210210111405-2049724535b8
go: found github.com/bnclabs/gosettings in github.com/bnclabs/gosettings v0.0.0-20210210111304-0d91c5ae496a
go: found github.com/bnclabs/gson in github.com/bnclabs/gson v0.0.0-20210409011557-2e73bae93c0b
go: found github.com/bnclabs/gofast in github.com/bnclabs/gofast v0.0.0-20210210111353-5c982d6094f0
go: found github.com/bnclabs/gofast/http in github.com/bnclabs/gofast v0.0.0-20210210111353-5c982d6094f0
go: finding module for package golang.org/x/text/language
go: finding module for package github.com/prataprc/color
go: finding module for package golang.org/x/text/collate
go: found golang.org/x/text/collate in golang.org/x/text v0.5.0
go: found golang.org/x/text/language in golang.org/x/text v0.5.0
go: finding module for package github.com/prataprc/color
gofast imports
github.com/bnclabs/golog imports
github.com/prataprc/color: cannot find module providing package github.com/prataprc/color: module github.com/prataprc/color: git ls-remote -q origin in C:\Users\lonni\go\pkg\mod\cache\vcs\cd6867e44e63b3e2281366409487ea0ca5ceb7587e83ec3a555ba4b2b38e1358: exit status 128:
remote: Repository not found.
fatal: repository 'https://github.com/prataprc/color/' not found
So that applications can learn remote peer's name, buffersize, and tags.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.