Create FASTER store about neo-modules

[canesin]

@erikzhang you are probably aware but there is a new variant "FasterLog" that seems a even better fit for the project: https://github.com/microsoft/FASTER/blob/master/docs/cs/FasterLog.md

[shargon]

I think that it's a very good idea.

[shargon]

The problem with mongo it's that the we need to deserialize the binary to the json, because it has no sense if we don't use json format in Mongo. For debug it could be useful, but not for release, it will be slower

[shargon]

@erikzhang I can assign this task to me if you like

[vncoelho]

Considering what you already did on RocksDB this sounds a good idea, @shargon, good initiative!

@shargon do you believe MongoDB is also becoming a feasible option? Many people will like it because it is easy ported to other ones such as DynamoDB.

[vncoelho]

@shargon, I believe you are right.

Perhaps it would be good as a Publisher only, as @hal0x2328 did here: https://github.com/CityOfZion/neo-plugins-coz/tree/master/DynamoDBPublisher

[erikzhang]

I will take a look.

[cschuchardt88]

I did create this storage, but its very slow cause of the overhead for seek forward and seek backward. Also it a memory cache, like Redis, but you can take snapshots.

FasterDBStore.zip

edit: i can look into it again if needed.

[cschuchardt88]

I looked back into it and its slow at syncing offline. 2-3 per second. I think i know why it's the seek function. Maybe you can write a better one? Keep in mind that sessions are not thread safe.

FasterDBStore Source Code

[cschuchardt88]

After getting from help from the faster team, I was able to get get about 5-10 per second compared to leveldb which is hitting 50-100 per sec. Leveldb is about 10 times faster. But there is an upside to this. This is the store size of just 2,000 blocks on testnet T5.

Now Faster has so much potential but we would need to rip out the current caching system and storage system we have in neo. neo is very limited on performance, currently. But once synced you don't notice it because a new block comes every 15 seconds.

Some features that Faster has is

• Devices Types
  • Azure storage
  • Custom Storage Devices (like you can make AWS, or anything you want)
  • Clustered Remote Server Devices
  • Shared Storage Devices
  • In Memory Devices
• Everything is customizable from
  • Serialization
  • Key Value Types
  • and more
• Downsides
  • No Range Index (seeking)