Coder Social home page Coder Social logo

bhupesh-v / areyouok Goto Github PK

View Code? Open in Web Editor NEW
33.0 3.0 5.0 4.17 MB

A fast and easy to use URL health checker ⛑️ Keep your links healthy during tough times (Out of box support for GitHub Actions)

Home Page: https://bhupesh.me/areyouok-keep-your-urls-healthy-github-actions-golang/

License: MIT License

Go 67.66% HTML 28.95% Shell 3.39%
url url-health url-health-checker tech-blogger monitoring blogging automation actions linkchecker

areyouok's Introduction

AreYouOK?

A minimal, fast & easy to use URL health checker

cat areyouok logo

Go report card License GitHub all releases CodeQL Analysis Twitter: bhupeshimself areyouok-v1.0.0-demo

Who is AreYouOk made for ?

  • OSS Package Maintainers 📦️:
    With packages comes documentation which needs to be constantly updated & checked for dead/non-functioning URLs.
  • Tech Bloggers ✍️ :
    If you are someone who writes countless tutorials & owns the source for your website, Use areyouok to make sure your blogs don't contain any non-functioning URLs.
  • Literally anyone who wants to check a bunch of URLs for being dead ☠️ or not, SEO experts?

With time AreYouOk can evolve to analyze URLs over a remote resource as well, send your ideas ✨️ through Discussions

Installation

  • Linux

    curl -LJO https://github.com/Bhupesh-V/areyouok/releases/latest/download/areyouok-linux-amd64
  • MacOS

    curl -LJO https://github.com/Bhupesh-V/areyouok/releases/latest/download/areyouok-darwin-amd64
  • Windows

    curl -LJO https://github.com/Bhupesh-V/areyouok/releases/latest/download/areyouok-windows-amd64.exe

Check installation by running areyouok -v

$ mv areyouok-darwin-amd64 areyouok
$ areyouok -v
v1.0.0

Download builds for other architectures from releases

Usage

AreYouOk provides 3 optional arguments followed by a directory path (default: current directory)

  1. -t type of files to scan for links
  2. -i list of files or directories to ignore for links (node_modules, .git)
  3. -r type of report to generate

Some example usages:

  • Analyze all HTML files for hyperlinks. The default directory path is set to current directory.

    areyouok -t=html Documents/my-directory

    There is not limitation on file type, to analyze json files use -t=json. The default type is set to md (markdown)

  • To ignore certain directories or file-types use the ignore -i flag.

    areyouok -i=_layouts,.git,_site,README.md,build.py,USAGE.md
  • By default AreYouOk outputs analyzed data directly into console. To generate a report, use the -r flag

    areyouok -i=_layouts,.git,_site,README.md,build.py,USAGE.md -r=html ~/Documents/til/

    Currently supported report formats are: json, txt, html & github.

    Different report types provide different levels of information which is briefly summarized below:

    1. JSON (report.json)
      The JSON report can be used for other computational tasks as required (E.g emailing dead urls to yourself)

      {
         "http://127.0.0.1:8000/": {
                 "code": "",
                 "message": "Get \"http://127.0.0.1:8000/\": dial tcp 127.0.0.1:8000: connect: connection refused",
                 "response_time": ""
         },
         "http://freecodecamp.org": {
                 "code": "200",
                 "message": "OK",
                 "response_time": "5.44s"
         },
         "http://ogp.me/": {
                 "code": "200",
                 "message": "OK",
                 "response_time": "3.60s"
         },
         "http://prnbs.github.io/projects/regular-expression-parser/": {
                 "code": "200",
                 "message": "OK",
                 "response_time": "0.25s"
         },
         "https://bhupeshv.me/30-Seconds-of-C++/": {
                 "code": "404",
                 "message": "Not Found",
                 "response_time": "3.84s"
         },
         ...
      }
    2. Text (report.txt)
      The text format just lists the URLs which were not successfully fetched. Useful if you just want dead urls. Text report also puts the no.of hyperlinks analyzed along with total files & total reponse time.

      74 URLs were analyzed across 31 files in 21.69s
      
      Following URLs are not OK:
      
      http://freecodecamp.org`
      http://127.0.0.1:8000/
      https://drive.google.com/uc?export=view&id=<INSERT-ID>`
      https://drive.google.com/file/d/
      https://drive.google.com/uc?export=view&id=$get_last
      https://github.com/codeclassroom/PlagCheck/blob/master/docs/docs.md
      https://bhupeshv.me/30-Seconds-of-C++/
      

      Note that the total time would vary according to your internet speed & website latency.

    3. HTML (report.html)
      The html report is the most superior formats of all & can be used to have a visual representaion of analyzed links.
      Below is demo of how this HTML report looks like, you can see it live report-latest

    4. GitHub (report.github)
      The github report format is well suited if you are utilizing Github Actions. The format generated is largely HTML, compatible with github's commonmark markdown renderer.
      Below is a demo of a Github Action which reports the analyzed URLs through github issues. Here is a demo link

      demo-action

Development

Prerequisites

  1. Clone the repository.
    git https://github.com/Bhupesh-V/areyouok.git
  2. Run tests.
    go test -v
  3. Format & Lint the project.
    gofmt -w areyouok.go && golint areyouok.go

Projects Using AreYouOk

📝 Changelog

See the CHANGELOG.md file for details.

☺️ Show your support

Support me by giving a ⭐️ if this project helped you! or just Twitter URL

Donate using Liberapay Support on ko-fi

📝 License

Copyright © 2020 Bhupesh Varshney.
This project is MIT licensed.

👋 Contributing

Please read the CONTRIBUTING file for the process of submitting pull requests to us.

Contributors ✨

Thanks goes to these wonderful people (emoji key):


MaDDogx

🐛

This project follows the all-contributors specification. Contributions of any kind welcome!

areyouok's People

Contributors

allcontributors[bot] avatar bhupesh-v avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

areyouok's Issues

support for line numbers in github & html report format

Is your feature request related to a problem? Please describe.
Right now the links appear in list of tables for the GitHub report format.
It would be nice if we can link to the exact line number where the link exists.

Describe the solution you'd like
A new parser kinda thing will be required to find the line numbers of exact regexp matches. Its an interesting and fun problem to solve

Modularise code

Right now the logic for areyouok contains in one single big file. It would be nice to have some structure in place.
We can follow the 12 factor app pattern here

Throws a `999` Status Error Code for LinkedIn links

Describe the bug
While checking for broken links, areyouok throws a 999 Error Status code for URLs pointing towards LinkedIn.

EDIT: Just found out, even URLs pointing towards Twitter returns a 404 Not Found Error. Here's an example issue thread for reference.

To Reproduce
Steps to reproduce the behavior:

  1. Create an example README.md file with the a link to a LinkedIn profile.
  2. Run areyouok against the newly created README.md file (which contains the LinkedIn URL).
  3. Check the generated report for a 999 Error Status Code.

Expected behavior
areyouok should've returned a 200 OK Status Code.

Screenshots
image

Desktop (please complete the following information):

  • OS: Ubuntu 20.04
  • areyouok version: v1.1.0 built on (26 Jan 2021)

UTF Icons don't show up in HTML report

Describe the bug
UTF icons don't show up in the generated HTML report.

To Reproduce
Steps to reproduce the behavior:

  1. Run the binary
  2. Serve the generated HTML report.
  3. No fancy icons 😢

Expected behavior
Need muh UTF icons.

Screenshots
image

Desktop (please complete the following information):

  • OS: Windows
  • Browser: N/A
  • Version: N/A

Smartphone (please complete the following information):

  • Device: N/A
  • OS: N/A
  • Browser: N/A
  • Version: N/A

Additional context
N/A

socket: too many open files

Describe the bug
there is a limit to how many file descriptors can be open at a time.
See https://stackoverflow.com/questions/32325343/go-tcp-too-many-open-files-debug

To Reproduce
Steps to reproduce the behavior:

  1. Run Areyouok on file with greater than 200 links (For e.g awesome-go README)
  2. A lot of URLs are reported as above error

Expected behavior
All URLs should be correctly fetched

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. Windows 10]
  • areyouok version: [latest]

Additional context
Use buffered channels, and add a limit for it (use ulimit -n)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.