Coder Social home page Coder Social logo

ltfschoen / ethquad Goto Github PK

View Code? Open in Web Editor NEW
4.0 1.0 0.0 2.55 MB

License: MIT License

HTML 3.82% CSS 1.65% JavaScript 93.79% TypeScript 0.28% Shell 0.46%
slate filecoin eth2-beacon-chain hackfs hackathon ethereum react express heroku unstoppable-domains

ethquad's Introduction


ETHQUAD

Table of Contents

HackFS Hackathon Integration

Roadmap

HackFS

  • Frontend React.js & TypeScript boilerplate (./client)
  • Backend Express.js API boilerplate (./server). Served at https://ethquad.herokuapp.com/api
    • Configure CORS to allow requests from Official IPFS Gateway
  • Local Docker container setup process for running with Textile.io storage API (PowerGate) that interacts with Lotus (Filecoin) and IPFS
  • Production Docker container running with Textile.io storage API (PowerGate) that interacts with Lotus (Filecoin) and IPFS
  • Frontend Slate Design System UI components that interact with PowerGate (./client/src/)
    • Connect and obtain Authorisation to access the Filecoin File System (FFS) API using Textile PowerGate JS Client (@textile/powergate-client)
    • Generate a Powergate token. Store or retrieve (if exists) from web browser Local Storage. Check if retrieved existing Powergate token invalid due to restarting local Docker container with Textile PowerGate
    • Create and view multiple Filecoin Addresses and associate them with PowerGate token.
    • Retrieve Pin (IPFS Hash where front-end deployed) from PinList using Express.js API endpoint that connects to Pinata (http://ethquad.herokuapp.com/api/getWebsiteIPFSHash)
    • Display latest Pin (Website IPFS Hash) in front-end
    • Use Slate's "Make a Storage Deal" code to deploy a zip file of the website to Lotus Filecoin Testnet. Only works in 'development' environment where Lotus Filecoin Testnet and PowerGate Docker instances are running (see section "Connect to Lotus (Filecoin) and IPFS" below)
    • Modify the Slate's "Make a Storage Deal" code to deploy that latest Pin to Lotus Filecoin Testnet (not just Local)
  • Frontend deployed to IPFS Address using Pinata SDK
    • Request decentralised domain name from Unstoppable Domains (https://ethquad.crypto)
    • Deployment script generate a new Pin and Unpins all previous (./scripts/pinataUploadIpfs.js)
    • Separate IPFS Address Pin for deployed frontend depending on environment (development or proudction). Used Pinata metadata to identify associated environment.
    • Preview using Official IPFS Gateway (i.e. https://ipfs.io/ipfs/<IPFS_HASH>) in development environment (yarn dev:ipfs:preview)
    • Configure ethquad.crypto domain name to redirect to an IPFS Hash (which just redirects to Heroku at ethquad.herokuapp.com), and configure ethquad.herokuapp.com to further redirects to the IPFS hash where the front-end of the website is stored.
  • Backend Express.js API connection to Infura Eth 2.0 Endpoint (https://altona.infura.io)

Optional

  • Frontend deployed to IPFS Address using Fleek Continuous Deployment using Github
  • Logo deployed using Fleek Stage App JS SDK (@fleekhq/fleek-cli). See https://docs.fleek.co/storage/fleek-storage-js/
    • Upload logo to publish it to IPFS, published to DNS, with a CDN and File Compression.
    • Reference the logo file using the Fleek Storage URL
  • Desktop Fleek Space Daemon integration to share encrypted files privately with peers using Textile/IPFS node on Local Machine
  • Textiles ThreadDB/Buckets (instead of Fleek Space Daemon) since not running private Textile/IPFS node on Local Machine

Usage

Note: Access to https://ethquad.crypto is in progress pending resolution to an IPFS Hash to direct it Note: Pending resolution of this issue #15

Development

Install dependencies

  • Install dependencies and run server and front-end. Note that Webpack is installed globally to avoid conflicts since it is installed automatically on Heroku and to match the version they use.
nvm use;
yarn global add concurrently;

Connect to Lotus (Filecoin) and IPFS

Develop website locally without deployment to IPFS

  • IMPORTANT: Wait until PowerGate is fully running before running this.
yarn dev

Preview website redirecting to deployed to IPFS

yarn dev:ipfs:preview   
  • Go to http://localhost:5000
    • Important note: Use port 5000 since only running from server instead of client with proxy.
    • Important: If the UI doesn't load, try running with yarn dev instead, since it may be caused by React.js that aren't being shown.

View information about the ethquad.crypto domain name (provided by Unstoppable Domains)

node ./scripts/unstoppableDomainsRedirect.js

Deployment (IPFS & Heroku)

Generate an IPFS Hash (for redirecting the ethquad.crypto domain name to a traditional domain). Repeat this in future if the traditional domain name (i.e. ethquad.herokuapp.com) changes.

Verify that the IPFS Hash has been created with name 'EthQuad-prod-traditional' at https://pinata.cloud/pinexplorer

Go to https://unstoppabledomains.com/manage and change the IPFS Hash value the new IPFS Hash that is output to the terminal. Check for when the .crypto domain name resolves to the new IPFS Hash by running node ./scripts/unstoppableDomainsRedirect.js (without any flags). It will redirect when a value is output for Read record of IPFS redirection hash:.

Alternatively try setting the IPFS Hash programmatically by running node ./scripts/unstoppableDomainsRedirect.js --setRedirectIpfs. However, this approach would require you to add the following to your .env file:

  • ETHEREUM_ADDRESS that owns the .crypto domain name
  • MNEMONIC phrase associate with the above Ethereum address
  • INFURA_PROJECT_ID and INFURA_PROJECT_SECRET to connect to Ethereum Mainnet using Infura Note that running node ./scripts/unstoppableDomainsRedirect.js --setRedirectTraditionalDomain to redirect the .crypto address to a Traditional Domain like https://ethquad.herokuapp.com directly is not currently supported by Unstoppable Domains.
yarn run build:release:ipfs-traditional-domain

Push changes to IPFS & Heroku

git push heroku master

Note: Heroku will build dependencies, then run the "heroku-postbuild" script in package.json. The "build:release:ipfs" script will deploy the front-end to an IPFS hash.

Configure Heroku

heroku git:remote -a ethquad

Set Heroku Environment Variables to match the contents of the .env file. Replace missing values below:

heroku config:set \
  KEY1=VALUE \
  KEY2=VALUE

heroku config --app ethquad

Troubeshooting

If you get a nodemon issue due to server already running on a port just run ps -ef | grep node and kill all associated processes kill -9 <PID>

Maintenance

MAINTENANCE.md

References

REFERENCES.md

Sia + Namebase (Handshake) "Own the Internet" Hackathon Integration

About Sia + Namebase (Handshake)

https://gitcoin.co/hackathon/own-the-internet

  • Goal:

    • Build a Skapp using Handshake and Skynet
  • Skynet

    • About
      • Offers decentralized filesharing and application hosting where the user owns the content instead of it being controled be a centralized entity (e.g. Youtube, Medium)
      • Applications hosted on Skynet are decentralized apps called "skapps"
    • Usage
      • Create application
      • Upload files or applications to Skynet so file available to all Portals
      • Skylink returned may be downloaded and content viewed through any Portal
      • Runs in browser supporting client-side applications
    • Development
      • API for web apps to upload and download data
      • SDKs supported (JS, Node.js, etc)
      • Build a Skynet Application
        • Instal Node.js, Webpack
        • Install skynet.js NPM package
    • Hosting
  • Handshake

    • Skynet Portals run Handshake Full Nodes
      • Endpoints allow users to load applications and content from Handshake domains and query Handshake domains for Skynet content
  • Examples

    • Skybin
    • Skylive - skylive.coolhd.hu
    • Instasky - Upload and share media libraries
    • iOS Uploader - Download on phone and share via Skynet
    • Dgit - dgit.dev - decentralized Git
    • SkyGallery - creates media galleries
    • hns.to
  • Docs:

Roadmap

Features

EthQuad is an open-source application that has now been extended to integrate both the Sia Skynet and Handshake ecosystems, as a step toward building a platform to help developers build on Skynet and Handshake.

It incorporates tooling to allow users to generate and upload a "redirection" page file to a Sia Skynet hash (Skylink) using the Skynet Node.js SDK to improve the developer experience. The "redirection" page may be configured to redirect to a given Handshake Domain Name (e.g. epiphysitis/) at a Skynet Portal (e.g. https://siasky.net/hns/epiphysitis/).

The tooling allows users to generate and upload a "website" from a given website folder to a different Skylink using the Skynet Node.js SDK. The "website" at EthQuad includes a React.js front-end with TypeScript support.

The tooling authenticates with the Namebase API to allow users to configure their Handshake Domain Name's Blockchain DNS records to resolve to the Skylink of that "website".

In future this tooling could be extracted into a separate application on the Skynet App Store, a CLI application, or an NPM library with configuration options to automatically generate a website using a chosen technical stack for customisation, then upload it to a Sia Skylink, and then automatically choose an freemium Handshake Domain Name and have it resolve to that Skylink.

In the process of extending EthQuad, Namebase website user experience issues were raised by ltfschoen at feedback.namebase.io, Skynet Node.js SDK issues were raised at NebulousLabs/nodejs-skynet#47 and NebulousLabs/nodejs-skynet#48.

The tooling for Sia Skynet and Handshake that was created in this project was applied to my other open-source decentralized game FlappyTips, which is built with React.js and p5.js, to deploy it to a Skylink hash (e.g. https://siasky.net/AADhTfgZns00r6rYpyThLQwLDxAea57EegdNN-leYGf07w/) and associated with a Handshake domain (e.g. https://siasky.net/hns/flappy) in this Pull Request ltfschoen/flappytips#15.

Additional functionality is currently being investigated.

Usage

Deploy Redirect using Sia Skynet and Handshake

Connect to Lotus (Filecoin) and IPFS (see section "Connect to Lotus (Filecoin) and IPFS")

Add your Handshake (HNS) Domain HNS_DOMAIN to the .env file (e.g. HNS_DOMAIN=epiphysitis)

Build the website for Development or Production. If Development is chosen the backend API is expected to be running on localhost:5000/api instead of at https://ethquad.herokuapp.com/api.

  • IMPORTANT: If using PowerGate, wait until it is fully running before running the following.

  • Development (without Skylink deployment)

    yarn dev
  • Development (using Skynet)

    yarn dev:build:sia-handlebars
  • Production (using Skynet)

    yarn build:release:sia-handlebars

When using Skynet will run the following:

Run the server

yarn dev:server:sia-handlebars

Go to https://siasky.net/<SKYLINK_WEBSITE>

Run the following to:

PUT=true node ./scripts/handshakeDomainSetSkynetPortalRecord.js

Alternatvely, manually update the Handshake domain's DNS records by going to https://www.namebase.io/domain-manager/<HANDSHAKE_DOMAIN_NAME> and adding a TXT record that points the domain to the Skylink of the Website <SKYLINK_WEBSITE>/index.html, and then waiting ~10 minutes for domain changes to propagate through Handshake nodes syncing the changes. If you change the NS record too it may take more than a day for changes to propagate.

Verify the Handshake domain's resolve configuration has been updated by either:

References:

ethquad's People

Contributors

ltfschoen avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar

ethquad's Issues

Host and connect to Docker container running Powergate, Lotus (Filecoin), and IPFS in production

If you go to https://ethquad.herokuapp.com/ and view the browser console it has errors:

Unable to connect to Powergate network
(anonymous) @ main.js:60
main.js:58 Uncaught (in promise) Error: Response closed without headers
    at Object.onEnd (main.js:58)
    at main.js:13
    at Array.<anonymous> (main.js:13)
    at e (main.js:13)
    at main.js:13

This is because here https://github.com/ltfschoen/ethquad/blob/master/client/src/App.js#L52 we're usually running a local instance of Powergate / Lotus(Filecoin) / IPFS by following this step in the README https://github.com/ltfschoen/ethquad#connect-to-lotus-filecoin-and-ipfs.

But in production we need to host that running Docker container in the cloud somewhere and connecting to it by replacing the endpoint in the code in production

Error when running dev:ipfs:preview 400

When I run yarn run dev:ipfs:preview and go to localhost:5000, it redirects to the IPFS website, but browser console error appears as shown

HEAD https://ipfs.io/ipfs/QmY5iYA2Q5z5xV9D75DxkwGu3WPtitK7GPptSEGGJFF2Wz/build/%PUBLIC_URL%/favicon.ico 400

But if you remove the %PUBLIC_URL%/ and go to that address it loads.
So need to use a different index.html template for when dealing with IPFS

Avoid global Webpack installation

Try to avoid using yarn global add webpack@^4.42.0 in package.json, which was done to overcome Heroku conflict, but it prompts you to install webpack-cli, which may be the cause of this error PinataCloud/Pinata-SDK#28 on Heroku (which doesn't happen locally)

Separate Website IPFS Address required for development and production required

At the moment, if in the 'development' environment you preview the IPFS frontend with yarn dev:ipfs:preview, it'll push the latest frontend to IPFS, and that'll become the latest Pin on Pinata. This is an issue because the frontend changes haven't even been deployed to production, and they may include new calls to endpoints that aren't yet available on server that's been deployed to Heroku

Unable to push to Heroku anymore despite Pinata SDK dependency not changing

Error as shown in latest commit d8af6f6 even though it worked previously with the same dependency and version.

~/code/src/ltfschoen/ethquad   master ●  git push heroku master      
Enumerating objects: 52, done.
Counting objects: 100% (52/52), done.
Delta compression using up to 4 threads
Compressing objects: 100% (30/30), done.
Writing objects: 100% (35/35), 8.58 KiB | 1.72 MiB/s, done.
Total 35 (delta 26), reused 5 (delta 4), pack-reused 0
remote: Compressing source files... done.
remote: Building source:
remote: 
remote: -----> Node.js app detected
remote:        
remote: -----> Creating runtime environment
remote:        
remote:        NPM_CONFIG_LOGLEVEL=error
remote:        USE_YARN_CACHE=true
remote:        NODE_ENV=production
remote:        NODE_MODULES_CACHE=true
remote:        NODE_VERBOSE=false
remote:        
remote: -----> Installing binaries
remote:        engines.node (package.json):  14.x
remote:        engines.npm (package.json):   6.x
remote:        engines.yarn (package.json):  1.x
remote:        
remote:        Resolving node version 14.x...
remote:        Downloading and installing node 14.7.0
remote:        Bootstrapping npm 6.x (replacing 6.14.7)...
remote:        npm 6.x installed
remote:        Resolving yarn version 1.x...
remote:        Downloading and installing yarn (1.22.4)
remote:        Installed yarn 1.22.4
remote:        
remote: -----> Restoring cache
remote:        Cached directories were not restored due to a change in version of node, npm, yarn or stack
remote:        Module installation may take longer for this build
remote:        
remote: -----> Installing dependencies
remote:        Installing node modules (yarn.lock)
remote:        yarn install v1.22.4
remote:        [1/4] Resolving packages...
remote:        [2/4] Fetching packages...
remote:        error An unexpected error occurred: "https://registry.yarnpkg.com/@pinata/sdk/-/sdk-1.1.10.tgz: Request failed \"404 Not Found\"".
remote:        info If you think this is a bug, please open a bug report with the information provided in "/tmp/build_e23d5753/yarn-error.log".
remote:        info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
remote: 
remote: -----> Build failed
remote:        
remote:        We're sorry this build is failing! You can troubleshoot common issues here:
remote:        https://devcenter.heroku.com/articles/troubleshooting-node-deploys
remote:        
remote:        If you're stuck, please submit a ticket so we can help:
remote:        https://help.heroku.com/
remote:        
remote:        Love,
remote:        Heroku
remote:        
remote:  !     Push rejected, failed to compile Node.js app.
remote: 
remote:  !     Push failed
remote: Verifying deploy...
remote: 
remote: !	Push rejected to ethquad.
remote: 
To https://git.heroku.com/ethquad.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://git.heroku.com/ethquad.git'

Error when running dev:ipfs:preview Mixed Content

When I run yarn run dev:ipfs:preview and go to localhost:5000, it redirects to the IPFS website, but browser console error appears as shown

Mixed Content: The page at 'https://ipfs.io/ipfs/<HASH>/build/' was loaded over HTTPS, but requested an insecure resource 'http://0.0.0.0:6002/ffs.rpc.RPCService/Create'. This request has been blocked; the content must be served over HTTPS.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.