coderwassananmol / bub2

Upload books, newspapers, magazines etc. from public libraries to Internet Archive

BUB2

A book uploader bot that transfers documents from public libraries such as Google Books, Punjab Digital Library and Trove Digital Library to Internet Archive. Built for Wikimedia Tool Labs. Check out BUB2 on Wikimedia Cloud!

Table of Contents

• Getting Started
  • Prerequisites
  • Local setup
    • Clone the repo
    • Set environment variables
  • Run Redis server
    • Start the server
• Example
• Contributing
• Request to Contribute
• Keep Supporting
• License

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Prerequisites

• Node.JS ( check for installation with npm -v and node -v on terminal )
• Redis

Local setup

Clone the repo

• Clone the repository git clone https://github.com/coderwassananmol/BUB2
• Navigate to the project directory on the terminal: cd BUB2.
• For developers, checkout to develop branch: git checkout develop
• Run npm install

Set environment variables

Rename .env.example to .env. Then, to fill the credentials,

• Make an account on archive.org.

• Go to https://archive.org/account/s3.php . Generate the access and secret keys and fill them in the .env file in the corresponding fields.

• Go to Google Developers console. Make a new project to run the app. In that Google Developers project, search for 'Books API' in the Google API console, then enable the API for the project, then generate the API keys, and then copy and paste the API key in the GB_Key fields.

• Enter the redishost field. If it's hosted locally, enter 127.0.0.1, which is the default localhost.

• Enter the redisport field with 6379, which is the default port number for redis.

• You need Beta wiki OAuth 2.0 credentials to login with MediaWiki during development. Steps to obtain the OAuth credentials for the WIKIMEDIA_CLIENT_ID and WIKIMEDIA_CLIENT_SECRET fields :

  • Go to Beta-wiki and click Request a token for a new OAuth 2.0 client.
  • Enter the details (Application name, details, callback url, and applicable grants). Refer this as a sample.
  • After submitting, note the client application key and client application secret and wait for the proposed consumer to get approved.
  • Go to this file.
  • Add the following code after the clientId and clientSecret :

    token: "https://meta.wikimedia.beta.wmflabs.org/w/rest.php/oauth2/access_token",
    userinfo: "https://meta.wikimedia.beta.wmflabs.org/w/rest.php/oauth2/resource/profile",
    authorization: {
    url: "https://meta.wikimedia.beta.wmflabs.org/w/rest.php/oauth2/authorize",
    params: { scope: "" },
    }
    

• Enter the client application key and client application secret in the WIKIMEDIA_CLIENT_ID and WIKIMEDIA_CLIENT_SECRET respectively

• Enter the NEXTAUTH_URL with http://localhost:5000.

• Go to [wikisource Bot] (https://meta.wikimedia.beta.wmflabs.org/wiki/Special:BotPasswords) to generate your EMAIL_BOT_USERNAME and EMAIL_BOT_PASSWORD. When creating your bot, tick 'send email to users' under the Applicable Grants section. Go ahead and enter the generated credentials in the .env file.

• Enter the NEXT_PUBLIC_WIKIMEDIA_URL which is used to authenticate with Wikimedia environment for login, send emails etc. For example - https://meta.wikimedia.beta.wmflabs.org

• Enter the NEXT_PUBLIC_COMMONS_URL which is used to upload the files to Commons. For example - https://meta.commons.beta.wmflabs.org for local setup.

• Enter the NEXT_PUBLIC_IA_USER which is the username of Internet Archive account.

• Enter the IA_EMAIL which is the email of the Internet Archive account.

• Go to Trove API and follow the instructions on how to get a trove key. Fill in trove_key with the trove key you generated.

Run Redis server

• Refer to Redis for download and setup documentation

Start the server

• Run npm run dev for development and npm run start for production.
• Open your browser and navigate to http://localhost:5000

Contributing

Please read CONTRIBUTING.md for information on how to contribute to BUB2.

Request to Contribute

1. Fork the repository.
2. Clone the repository to your local system by using the command : git clone "https://github.com/<YOUR_USERNAME>/BUB2".
3. The issues are maintained on Phabricator.
4. Assign an issue to yourself from Phabricator.
5. Create a new branch and start working on the issue locally.
6. Create a PULL REQUEST to merge your branch with the main branch and mention the Phab task in the description.
7. The issue will be considered closed and resolved once the PR is accepted. Please read CONTRIBUTING.md for information on how to contribute to BUB2.

Contribute

We actively welcome pull requests. Learn how to contribute.

Keep Supporting

There was no Node.js wrapper available for Internet Archive, so I decided to write the Node implementation to upload books to Internet Archive. If you like this repository, show your support by starring the project. Cheers!

License

Please read license for more information.