joincolony / pinion Goto Github PK
View Code? Open in Web Editor NEW📌Pin orbit-db stores and ipfs hashes
License: MIT License
pinion's Issues
Check store sync issue
Currently, we have seeing this warning on the console: "WARNING! Head hash didn't match the contents. We need to investigate and fix that
Q: Will this work in the browser via pubsub?
Perhaps a wild question, but I'm definitely curious!
Make sure the service is resilient
Add request validation, timeout, caching wherever is necessary to make sure the service doesn't crash
Deploy pinion
Close open stores on timeout
After opening a store, we wanna make sure we close them after a period of time so we make sure we don't have an open store we are not using.
We should debounce the store close call using two timeout levels, one for the store being opened and one more for the last message received. So if a store is opened, we will close it after some X time except if a message is received for that store then we extend that timeout a little bit longer
Add IPFS hash pinning method
As a user, I wanna be able to pin IPFS content by posting a hash of it
Validate action messages for sanity
The service should be able to validate and discard invalid requests so it doesn't crash
Ensure the right access controller is used
Currently, we are using a permissive access controller, we might wanna revisit that in the future
Add 3rd party providers/gateways so we don't use just our own gateway
3rd party providers:
- Cloudfare
- Infura
- More?
Add reply for store pinning
As a user, I wanna know when pinion is done pinning an orbit-db store
Add ACK for pinion requests
As a user, I wanna know if pinion got my request
"Module not found error" with `npm i -g @colony/pinion`
Node 12.10.0
Error: Cannot find module 'multicodec/src/name-table'
Require stack:
- /home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs-bitswap/src/types/message/index.js
- /home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs-bitswap/src/want-manager/index.js
- /home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs-bitswap/src/index.js
- /home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs/src/core/components/start.js
- /home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs/src/core/components/index.js
- /home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs/src/core/index.js
- /home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/lib/IPFSNode.js
- /home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/lib/Pinion.js
- /home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/lib/index.js
- /home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/bin/index.js
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:772:15)
at Function.Module._load (internal/modules/cjs/loader.js:677:27)
at Module.require (internal/modules/cjs/loader.js:830:19)
at require (internal/modules/cjs/helpers.js:68:18)
at Object.<anonymous> (/home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs-bitswap/src/types/message/index.js:10:19)
at Module._compile (internal/modules/cjs/loader.js:936:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:947:10)
at Module.load (internal/modules/cjs/loader.js:790:32)
at Function.Module._load (internal/modules/cjs/loader.js:703:12)
at Module.require (internal/modules/cjs/loader.js:830:19) {
code: 'MODULE_NOT_FOUND',
requireStack: [
'/home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs-bitswap/src/types/message/index.js',
'/home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs-bitswap/src/want-manager/index.js',
'/home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs-bitswap/src/index.js',
'/home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs/src/core/components/start.js',
'/home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs/src/core/components/index.js',
'/home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/node_modules/ipfs/src/core/index.js',
'/home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/lib/IPFSNode.js',
'/home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/lib/Pinion.js',
'/home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/lib/index.js',
'/home/mark/.nvm/versions/node/v12.10.0/lib/node_modules/@colony/pinion/bin/index.js'
]
}
Limit number of open stores (LRU)
The number of open stores should be limited to a fixed amount so we can control how many stores we are currently listening to
Open source pinion
It doesn't look like there's much that prevents us from open sourcing this (actually I can't come up with anything, really). @thiagodelgado111 what do you think?
Edit: I think we have a logo now. So left to do:
- Add CODE_OF_CONDUCT
- Add logo to README
- Use official orbit-db release
- Migrate to TS
- Make it stable
Add 'unpin' action
Add an UNPIN_HASH action or similar that calls ipfs.pin.rm(hash)
Use case:
- a user uploads an avatar
- pinion pins the avatar
- the user later removes their avatar
- pinion should unpin the avatar
Use orbit-db official release
We've been using a fork but we need to install official releases as soon as we can use them with our changes and the new ipfs-log, opening this just to track that.
Closed #7 in favour of this one
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.