nobodywasishere / crystaldoc.info

Crystal Shards API Documentation Hosting

CrystalDoc.info

This is a website for hosting documentation of Crystal Shards.

Development

CrystalDoc relies on a PostgreSQL database to hold all the information about the repositories. To create a local development version using docker:

$ docker run --name crystal_doc -p 5432:5432 -e POSTGRES_USER=crystal_doc_server -e POSTGRES_PASSWORD=password -e POSTGRES_DB=crystal_doc -d postgres
$ shards run -- micrate up

Edit the config.yml

$ cp example-config.yml config.yml
$ nano config.yml # Add API keys / database URL

The server can then be run with:

$ shards run -- server # or 'searcher', 'builder', etc.

Usage

CrystalDoc.info comprises of four components that work together to provide its functionality:

• A PostgreSQL database contains all of the repositories and versions, as well as a job queue for pending doc generation
• A server that's integrated into a static file server that handles the frontend and adding new repositories
• A searcher that checks each repository daily for new versions and main branch updates, inserting new doc generation jobs as necessary
• A builder which checks for new doc generation jobs, attempts to build them, and posts the generated files to the static file portion of the site

Each component can be modified separately and scaled up or down as necessary. For instance, if there is a sudden increase of doc jobs, the number of builders can be manually increased to offset this, allowing for faster doc generation.

Deployment

CrystalDoc.info uses a PostgreSQL database to store information about each of the repositories and their versions. A database can be setup after initialization using the provided config/postgres_setup.sql script. The URL to the database is then set via the POSTGRES_DB environment variable, including options for the connection pools. See here for more information on the connection pool settings.

After the database is setup, each of the 3 services can be setup and run in their own processes, for example, using systemd daemons. It is recommended to build with --release and -D=preview_mt to provide the most performance. You can set the CRYSTAL_WORKERS environment variable to set the number of builders and searchers to run at the same time (defaults to 4).

$ shards build crystaldoc_server --release -D=preview_mt
$ ./bin/crystaldoc_server
# repeat for crystaldoc_builder and crystaldoc_searcher

Each of the services outputs are logged to their respective log file in the logs/ folder.

Contributing

1. Fork it (https://github.com/nobodywasishere/crystaldoc.info/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• Margret Riegert - creator and maintainer
• Gwen Dowling - maintainer

crystaldoc.info's Issues

Bug in rendering doc for lexbor

Hello! Thanks for creating and hosting this fantastic tool :D

I found a page in the lexbor documentation that seems to be rendering incorrectly.

Page: https://crystaldoc.info/github/kostya/lexbor/v3.1.2/Lexbor/Node.html

What I see:

I'm not seeing any missing css or failed network calls, so not sure what the culprit is.

Add README Badges

https://shields.io/

There could be a link on every doc page at the bottom Copy README Badge, or the format for specifying them could be consistent. I'm unsure if these should just be pure links, or be dynamic and return the latest docs version.

Something like this:

Add support for Fossil and Mercurial

This may require minor refactoring, but all we need to figure out to use these tools is:

• check if a given url is a valid repository of that vcs type
• get a list of all the tags and their hashes without cloning
• get the name and hash of the main branch without cloning
• cloning the repository
• forcefully checking out a specific tag

How repos are parsed will also have to be changed, as the largest fossil host doesn't follow the domain.tld/user/proj format.

The current git functionality all lives entirely within CrystalDoc::VCS

Example Crystal Fossil repo for testing:
https://chiselapp.com/user/MistressRemilia/repository/remiaudio/index

If anyone knows of an example mercurial one please let me know!

Move from firejail to docker for building docs

Docker is what docs.rs uses, and it would give a lot more flexibility towards the environment docs are generated in, allowing for using older versions of the docs generator, adding custom libs, better security, and easier setup.

docs.rs uses rustwide, a secure execution environment built on Docker. A similar shard should be developed for Crystal for safe execution.

Add a `latest` route for permalinking

If I were to create a link to the docs from a blog post or static site, it links to a specific version. Unlike the Crystal stdlib docs, I don't see a way to link to the newest version of the docs, instead of a specific static version. This could be problematic if I were to link to them from a static website.

Add `.crystaldoc.yml` config file for configuring docs generator

Add a .crystaldoc.yml file that lists config options for crystaldoc, namely what apt packages should be installed before executing, and what commands should be executed before building docs. This would allow GTK4 binding docs to be built

https://forum.crystal-lang.org/t/crystaldoc-info-hosted-shard-api-documentation/6018/5?u=nobodywasishere
https://hugopl.github.io/gtk4.cr/