Octopub is a Ruby on Rails application that provides a simple and frictionless way for users to publish data easily, quickly and correctly on GitHub.
More information is in the announcement blog post
The live instance of Octopub is running at http://octopub.io/
Follow the public feature roadmap for Octopub
These are the tools and services required to get Octopub fully working for development, testing and production environments. We'll explain how to set these up in the next section.
- Ruby 2.4
- PostgreSQL
- Redis/Sidekiq
- GitHub account
- AWS account
- Pusher account
- Open Data Certificates account
Sidekiq is used for managing the background proccessing of data uploads. To use Sidekiq just install Redis by following the instructions here, or if you are using Homebrew you can just do brew install redis.
For development Octopub uses the dotenv gem to load environment variables. Create a file called .env in your project root and paste in the variables below. We'll fill these in as we go along.
# GitHub App Client ID & secret
GITHUB_KEY=
GITHUB_SECRET=
# OAuth access token for GitHub API access
GITHUB_TOKEN=
S3_BUCKET=
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
PUSHER_APP_ID=
PUSHER_KEY=
PUSHER_SECRET=
PUSHER_CLUSTER=
BASE_URI=
ODC_API_KEY=
ODC_USERNAME=
# production only
SMTP_USERNAME=
SMTP_PASSWORD=
SMTP_SERVER=
Create a GitHub application:
- Log in to GitHub.
- In
Settings -> Developer settings -> OAuth applications, create a new OAuth application with a unique name. You can usehttp://octopub.iofor the homepage and for the callback URL use your local server address, i.e.http://localhost:3000. Click on your OAuth application to see yourClient IDandClient Secret, and update your.envfile:
GITHUB_KEY=<YOUR CLIENT ID>
GITHUB_SECRET=<YOUR CLIENT SECRET>
- In
Settings -> Developer settings -> Personal access tokens, generate a new token with a sensible description, e.g. octopub_dev_token, and update your.envfile:
GITHUB_TOKEN=<Your token>
Create an S3 bucket:
- In AWS go to the S3 service and create a bucket with a sensible name. Make sure the region is set to EU(Ireland) since Octopub uses this.
- Click on your bucket and go to the
Permissionstab. Click onCORS Configurationand paste in the configuration below. This will allow your local development version of Octopub to make requests to your S3 bucket.
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>http://localhost:3000</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
Grant permissions to your bucket:
- In AWS go to the IAM (Identity and Access Management page) service.
- Click
Users. - Add a new user and give it a name, e.g. octopub-development, and for Access Type select
Programmatic Access. - For permissions, select
Attach existing policies directly- this will open a new tab in your browser. - Click
create your own policyand give it a name, e.g. octopub-dev-permissions. Then for the policy document, use the following template, but add your bucket name in place of<BUCKETNAME>.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowAdminAccessToBucketOnly",
"Action": [
"s3:*"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::<YOUR BUCKET NAME>",
"arn:aws:s3:::<YOUR BUCKET NAME>/*"
]
}
]
}- Click
validate policyjust to be sure you've not made a typo, then confirm. - Back on the
Set permissions page, select the policy you've just created in the table by selecting the checkbox, then clickReviewand then clickCreate user. - Download the CSV file containing your
Access key IDandSecret access keyand update your.envfile:
AWS_ACCESS_KEY_ID=<YOUR ACCESS KEY ID>
AWS_SECRET_ACCESS_KEY=<YOUR SECRET ACCESS KEY>
S3_BUCKET=<YOUR BUCKET NAME>
- Log in to https://pusher.com or create a free account.
- Create a new application and call it something sensible.
- Select the
App Keystab and use the relevant values there to update your.envfile:
PUSHER_APP_ID=
PUSHER_KEY=
PUSHER_SECRET=
PUSHER_CLUSTER=
- Log in to https://certificates.theodi.org/ or create a free account.
- Go to your profile page, copy your API token and update your
.envfile:
ODC_API_KEY=<API TOKEN>
ODC_USERNAME=<YOUR USERNAME (email address you used when signing up)>
Assuming you have completed the setup instructions above...
- Start Redis with
redis-server. - Start Sidekiq with
bundle exec sidekiqin the application directory. - Create the postgresql databases specified in
config/database.ymland runrails db:migrate. - Start Octopub with
rails sin the application directory. - Navigate to the home page.
- Sign into octopub with your GitHub account.
Congratulations, you should be signed in! Now try adding some data.
Start a rails console session and then...
require 'sidekiq/api'
Sidekiq::Queue.new.size
Sidekiq::Queue.new.first
Octopub uses the rspec test framework and requires the presence of a .env. See earlier section for details as you can (re)use your development variables*
The test suite can be run with bundle exec rspec.
* Note - the tests use VCR or mocking to allow the tests to be run offline without interfacing with the services.
A commit to master will trigger a TravisCI run; If successful it will automatically deploy to Heroku.
The GitHub organisations are cached for the logged in user. They can be cleared from a console with Rails.cache.clear
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent